We do all kinds of things to accommodate the problems with manufacturing. Imperfect chambering is why we fire form. Banana barrels or off-axis chambering is why we measure cartridge runout and index loads in the chamber.
If I had 2 minutes of inaccuracy on the firing line resulting from my hold, I'd pack it up and go home. A match winning hold is under a minute. That's why the 10-ring is 2 minutes across. You have that space to work in if your setup is perfect and you are proficient on the loading bench. In other words, there isn't more than 1/2-minute in the rifle to play with, so it needs to be right. Anything beyond that chalks up to wind reading skills and the fundamentals of marksmanship. That's why we chase the zero-tolerance dragon. Squeeze the accuracy out at every step, so that anything not hitting as called can be blamed on the shooter alone.
I don't like to think about my equipment in competition. I don't want to ponder my loads or my setup. I just want to mind my come-ups and work the wind. It should be simple as that... So we justify perfection to lay equipment thoughts to rest when the range goes hot. With that in mind, .00005 is better than .001 when doing barrel work on the lathe.