My experience is when the tool post height controls the height of the square cutting bits, or a boring bar holder. This is the only time I had to machine material from the bottom of these holders. In this case it was a turret tool post with 4 positions, and a boring bar holder that holds different bar diameters, that is split in half, along the bar axis.
I changed over to a QCTP on both lathes (size A and CA) and never had to do anything but make a custom "T" nut to fit the compound, for the post hold down bolt.
If this is required on a QCTP to get within the adjusting range of the tool holders, then the post is probably too large for the mini lathe. I just recently have seen very small QCTP's available for this issue.
The problem with the turret tool post is that only one size of square tool bit could be used without shimming. As well not all tool bit manufactures had the brazed carbide at the exact same height, as well ground HSS tool bits having the cutting edge move, from resharpening.