An R8 collet with .04" slots will typically grip about .05" smaller than the nominal diameter so a set of collets by 1/32" or 1mm for metric will grip any diameter shank within the range of the set. I tend to use collets on my mill rather than a drill chuck for holding drills and I have on occasion pushed that 1/32" undersized pretty hard. I have never seen an issue with a collet as a result.
That said, a collet only grips full circle at the nominal diameter. Anything smaller results in a three point grip. Additionally, forcing a collet to grip smaller than nominal results in a distortion on=f the spring portion of the collet. In order to properly mate with the collet socket, this distortion will be S shaped. The greater the deviation from nominal diameter the more pronounced the S shape will be.
I have a set of R8 collets by 1/32nds. A near as I can tell, R8 collets are only available in 1/32" increments for inch sizes and 1mm increments for metric sizes. The implication to me is that they are designed to be used for shanks that are under nominal by that amount.