Welcome, I was in your shoes about 3 years ago. My only real metal working experience was high school shop class, and my only lathe experience was a wood lathe in 9th grade. High School was some time ago.
I've got a Sherline lathe and mill which I have been very happy with, but as Mikey points out they are small so you will really need to think about the size of the things you want to make. The long bed (a $100 option that is well worth it) provides plenty of length for a small lathe, but they only have a 3 1/2" swing (theoretical maximum working diameter) which in practice means you are looking at stuff 1 1/2" diameter and smaller. I was able to make a 2" diameter fly wheel on my lathe, but that was really pushing the limits and I had to really think about how to do things.
If you feel the Sherline can handle what you want to do, they are very nice machines with a lot of support from the company. There is a huge variety of accessories available from the company. Occasionally you can find a good deal on a used one.
I think the lathe weighs about 30lbs, and the mill 50 lbs so they can easily be moved and stored in a cabinet or on a shelf out of the way and operated on a sturdy table or bench top.
Unlike a lot of the mini-machines Sherlines are basically ready to go right out of the box. Some assembly is required but I was up and running within a couple of hours of delivery. The only thing that held me back from making chips that first night was I hadn't thought ahead to buy some metal and had to wait until Monday when the local metal supply opened so I had something to practice on.
If it matters to you both Sherline and Taig are made in the USA.
Combination machines are not generally recommended but Sherline does offer a milling column for their lathe, which allows it to be converted into a basic mill for much less than the cost of a full mill. $160 vs $700 for their cheapest milling machine. Not ideal, but depending on your needs possibly a consideration.
Nobody has commented on your budget... You can probably buy a lathe and a mill for $2500 but you will find you are lacking in basic tools and tooling. The general rule of thumb seems to be to plan on spending 50-100% the cost of the machine on the basic required tooling. You will continue to find tooling that you "need" to have, it is an addiction that the people here are all to happy to feed.
Really consider the size of your projects though. I think the Sherlines are great machines, but if you buy too small you will be very frustrated. There are some nice lathes in the 8-9" size class that are still relatively portable (under 300lbs), 110v, don't take up a great deal more space and provide much more capacity. They do cost a fair bit more, but not too hard to find used.