With drilling you only have rotational torque and down feed as the two main forces to deal with, A standard drill quill and MT handles these quite well. However when milling, with the exception of plunge cutting, much like drilling, you also have side force from the long or cross feed, this combined with the spiral flutes on the mill will pull the MT out of the socket. Not only will your cut get deeper you will most likely wreck the MT and socket and possibly break the cutter.
Unfortunatly most small drill presses don't have a hole down through the quill to fit a draw bar, but it's just as well because these machines also don't have the rigidity or suitable bearings in the quill to be used for milling.
However some heavier machines, sometimes known as Mill/ Drills, do have a hole for a draw bar and suitable bearings for light milling.
Many years ago I did see a device that a guy had made, where he had drilled a hole crossways through the tang and made two small blocks that fitted into the slot of the quill, they were drilled and one was tapped,
A special bolt was made with a taper on it, such that when it was fed through the blocks and the tang, then as it was tightened the taper drew the tang up tight and locked it in.
It worked but it was only for very light cuts, but it was good enough to make a few keyways etc. The process was still limited by the inadequate quill bearings.