The wiggler ball doesn't need to be drilled dead center to work. The principle of operation is that the ball is connected by a short arm to another ball, a cylinder, or a point and is free to move to a position where the far end ball, cylinder, or point is aligned with the spindle axis. The design is such that it will compensate for any runout in the chuck or end mill holder. As long as the arm is close enough to the center line of the ball so any imbalance would throw it out, it will work. For that matter, the arm could be bent at the point of entry so the axis of the arm is passing through the enter of the ball.
Thanks fellas. For some reason I never got notification of your replies--only found them by searching my subscribed threads, just now. In the interim my net search included an old PM thread http://www.practicalmachinist.com/vb/general/spotting-drill-bit-159845/ which led me to purchase a set of 120* French made cobalt spotting drills, from 5/32"-1/2" for $58, delivered. I know there's more than one way to do this, but these should increase my capability for a small investment.
I ran across this homemade tool to hold a ball center for the lathe or mill. In the lathe and using the tail stock to drill it will be center.
This is a bit big for your application but could be made smaller. Just chuck up some stock and drill the hole a bit smaller than your ball and part it apart or put two pieces of stock trued up together at the same time in the lathe so you don't have to part it.
Could use hex stock to more easily mount in the mill.