Christian, you certainly do not need both metric and imperial sets. Base your decision on whether you are commonly holding metric or imperial nominally-sized material. If you're intended use is in the lathe spindle for workholding, and you are generally starting off with imperial sized bar stock, then get the imperial sized collets. In my case, 90 percent of my work with ER40 on the lathe begins life or ends up as a metric sized bar stock for 2nd op. I also own a six ER40 collets in imperial sizes for tool holding on the mill and in the lathe tailstock.
As far as TIR is concerned, I went down this rabbit hole in 2019, performing extensive testing and talking with some pretty sophisticated tooling engineers and CNC production managers in my quest to improve tolerances. Rego-Fix (the Swiss company that invented the ER system) will guarantee their TIR specs across the full clamping range if you employ all Rego-Fix components in the ER system (collet chuck, collet, and collet nut) using a torque wrench for tightening. Their stated clamping range for ER40 is 0.040" whether it's an imperial or metric collet. But you really have to look at it at a SYSTEM, and all three of the components contribute to TIR at nominal and fully compressed limits.
Obviously, a colllet chuck that has a poorly machined taper or is off-axis to the spindle rotation is going to have runout. Same with a poorly made collet. And if the material being clamped does not extend the full length into the collet, this can also contribute to poor TIR results. But this is equally true of the collet nut which has it's own 30° taper that aligns the nose of the collet as it's driven into the collet chuck taper.
If the 30° taper inside the collet nut is not machined precisely, the nut will drive the collet slightly askew axially as it's tightened down - and the harder you tighten the nut, the more it's driven off-axis. The same holds true of the 30° taper on the nose of the collet - it must be precisely on-axis to the longer taper at the opposite end to achieve a precise alignment as the nut is tightened. Adding a bearing to the collet nut to make it easier to twist on/off can also contribute to off-axis alignment. Matching the collet size to closely the diamter of the material being held will reduce the required clamping range, which can improve TIR - just as Mark has observed.
Every element here contributes to the resulting TIR. Having a Set-True style collet chuck in the spindle provides the flexibility to adjust the collet taper on-axis to the spindle. Investing in precision collets will also improve TIR. But if you put an $80 Rego-Fix precision collet in a $800 Bison collet chuck and use a $20 ER collet nut to tighten the setup, I can speak with authority that your TIR will not be disappointing, and get worse the more you tighten down on the nut. The single best thing I have done to improve ER40 TIR is to employ a Rego-Fix collet nut. And with consistent torque applied, I see no difference in TIR across the full 0.040" clamping range with either metric or imperial ultra precision collets. With any of the other ER40 collet nuts I've tried (Maritool, Parlec, HHIP, Shars, Technicks, Accupro) the results vary considerably and are not proportional to price of the nut.
Does any of this matter in practice? Well, that has a lot to do with the kind of precision you are after, and whether you can complete all the required machining operation in one setup. If your parts are brought to final size, then parted off and flipped around in the collet chuck for a second operation (which is what I do often), it can be pretty frustrating to have to re-align the Set-True chuck individually for each of the 20 parts you just made in order to hit tolerance. Limiting the required clamping range is one strategy that helps - I actually have two half-millimeter ER40 collets (18.5 and 16.5) specifically for this reason. But investing in a decent collet nut can also really improve TIR. <end of rant>