I definitely agree that the inserts matter most - most toolholders are at least adequate, particularly for hobby use.
OTOH, I'd say quality is TBD. If you buy exclusively top brands, you're likely to get quality inserts (better still if you picked them according to the task at hand). OTOH, I have run across some *really* good, cheap inserts. (And quite a few that were not so good, or even good at all...) Still, the fact remains that there do exist some really good inserts for cheap. By way of qualifying that statement, when I say "really good", I'm comparing to inserts from Kennametal, Iscar, Seco, Interstate (really not that great), Hertel, and probably a few others.
GsT