Oh, okay. A sleeve needs a clean finish and should ideally be bored to the pre-reamer size to ensure the hole is straight before you ream it. However, I understand that boring a 3" long hole is difficult if you lack the proper bar (5/16" or 3/8" carbide bar). When I made my boring bar sleeves, I drilled and reamed it and it works just fine. And you're right; you do not need a super-precise fit because you're going to slit the side of the sleeve anyway.
Assuming you are using mild steel, then the correct pre-reamer drill would be about 31/64". The way I did mine (I've only done 4 of these sleeves but I've reamed many holes and the principles in the Yankee paper work well) was to spot drill, then pilot with a drill that spans the web of a 29/64" drill. Then I drilled with a 29/64" drill to bulk out the hole, then drilled with the 31/64" drill. This gives me a pretty clean and straight hole. Then I used a 1/2" reamer.
I have found that sulfur-based cutting oil works well. Reamers cannot take a lot of material off so the way we use them matters. What works best for me is to chamfer the entrance hole, then coat the reamer and the hole with cutting oil. I set speed at about 100 - 200 rpm and make a single pass manually so I can feel a slight resistance to feed all the way down the hole (teeth are cutting continuously) until the reamer tip emerges on the other side. I
shut the machine off and withdraw the reamer. I have tried many ways to ream - low speed, high speed, withdraw under power, etc. I have found that my reamers cut more accurately, produce a better finish and stay sharp longer by doing it as above, or at least this works best for me.
I think you're doing it right. A finely reamed finish in the holder reduces chatter in the bore so reaming the sleeve is the way to go if you cannot bore it.
I also suggest that you make two lengthwise cuts on your sleeve; one through cut and one expansion slot opposite the slit like in the pic below. This allows the sleeve to clamp easily without cracking the sleeve.
