I make hundreds if rollers per year with bearing bores in each end that will be mounted on a shaft during assembly.
If possible bore soft lathe jaws to hold the parts, this has the added advantage of leaving little or no jaw marks on the part.
If possible use an adjustable chuck.
If mounting the finished product on a shaft of questionable size and roundness use what is often known as "Bearing Quality Bar" stock, turned, ground and polished, this is not to be confused with TGP material for general use.