Buddha Bodai made this fried tofu dish that my Ex just loved, so I reverse engineered how they made it. First, cutting them smaller than what you had done, next was to marinate them for a few hours, and then draining them, patting them down with a towel, and then putting salt on them and putting a heavy object over them for a couple of hours minimum.

Their trick was first dehydrating the Tofu, reducing the moisture content, and then they used Tempura batter. It reminded me of Sweet & Sour Pork or Chicken just without the sauce, although there is no reason why not to. You could even turn this into General Tao's Tofu.

Try this method, i guarantee that you will love it. sincerely a fellow Chef