I'd work on the license first. QRZ has online practice exams with real test questions:
http://www.qrz.com/testing.html
The technician test is real easy. Small testing fee when you take the real thing. It's multiple guess, so you have a good shot...
Equipment varies depending on what you want to do. I'd pass on eBay, but you can watch some there to get an idea of what is available and pricing (be it a bit inflated).
VHF/UHF FM is the first logical stop for local communications, especially if you have repeaters in your area. I picked up a decent used dual band hand held (HT) Yaesu for $60. You can even get a new FT-60 or other brand relatively cheap. Icom has a new 2m HT for $100. Mobile rigs are usually a bit more expensive (not too much though), but offer more power. Prices range from around 100 to 600 for HTs and mobiles. Used can be as little as $20. I even picked up a channelized 2m mobile rig for $5 last swap meet. I passed on a used IC706mkIIg for $600, since they go on sale for about $100 more new.
HF is a whole new ball of wax and requires property with enough space for an antenna. A technician can talk on some of 10 meters, and use CW in several other HF bands. Talking on the lower bands requires a General class or above license. So many options here that it's best to describe what sort of experience you would like to have.
I'd personally run a simple wire dipole (actually a fan diploe) from 80 to 10 meters. It cost a few bucks for the wire and feed line (you could use coax). The antenna is around 120 feet long in an inverted V configuration.
You can also run a vertical if you want to get an antenna tuner, but that usually costs more and does not usually work better.
Then you can get into antenna towers with multi-band antennas -- sky's the limit, literally!
The radios can range from around $200 to several thousand. You might find a cheaper rig if you look real hard. I've found this to be sort of a specialized market. Used ham gear of any good quality retains its value fairly well. You usually have to settle for some old equipment or something with issues if you really want to save big. Or you get lucky. Look for estate sales.
You will usually want a 20 amp power supply at least, but it depends on the rig you get.