Your opinion of "small" is subjective. I have a Colchester Chipmaster which I would consider a small lathe. I had the opportunity to own and use a much bigger lathe but passed on it. The difference being that my lathe is very light on the handles and very easy to manipulate for small work.
You miss several important points for why people choose small lathes. For one, the inrush current on a large lathe will pop most domestic fuses. Secondly, chuck changes are back breaking and difficult. Thirdly, tooling is large and can be unwieldy for smaller jobs. Fourth, large lathes tend to have a lower RPM range than smaller lathes which is not ideal for turning small parts - especially if you want to use carbide tooling on small parts.
Many hobby machinists come from a machining background. As they get older, many don't feel like trying to muscle a 200 pound carriage or slide a 100 pound tailstock up to a pen blank. There are plenty of seasoned machinists who not only like, but prefer their little lathes for the reasons I stated.
I don't know what your experience is or how long you have been machining. I would however recommend that you join a model steam club and have a look at what machines those blokes use. I think you might be surprised at the size of their favorite machines.
Paul.