Get a lathe first. Along with it buy a copy of South Bends "How to run a lathe" It's a book from the 30's and 40's but everything in it is still applicable and it's written before the era of all the cover your ass legal stuff. In fact buy the book first and it will help you decide if this is for you.
Every thing you will learn with your lathe you can apply to your mill.
Also get a copy of MACHINERYS HANDBOOK refer to this when starting to cut anything. If you pay attention to speeds, feed rates and lubrication/cooling you won't break many tools. You can buy a old one of these books as well and save the $$
If you have a good head on your shoulders with the help of these books and lots of hands on practice you will be the go to guy in your neiborhood. Nobody learns how to do this stuff anymore. Glad to see you are interested.