I was asked this question a LOT over the weekend, and had a couple of question from people here(see Bane, I remembered), so I'm going to list some of the books I've read that have good info in them. I'll break it into general smithing, and bladesmithing.
Country Blacksmithing by Charles Mcraven. Pretty good on most things, especially making tools and decorative work. Also some on making a forge.
The Complete Modern Blacksmith by Alexander Weygers. Similar to the above.
The Art of Blacksmithing by Alex Bealer. An old book that's been reprinted. The style of writing can take a bit of going through at times, but lots of nuggets of good information.
Practical Blacksmithing and Metalworking by Percy Blandford. Much like 'Art of'.
There's a book I found, early 1900's British school book for metalworking, with lots of patterns for decorative stuff. I can't @#)$*!! find my copy, I'll add the title in when I do.
Of the above, I'd say the first two are probably the best for starting off.
The Complete Bladesmith, by Jim Hrisoulis. Excellent book. Covers just about everything from steel types, heat-treating, building a forge, materials.
Wayne Goddard's $50 Knife Shop by Wayne Goddard. Working with a group putting together the simplest, cheapest way to make good blades, he actually set up a shop for $50. Good book.
Ed Fowler's Knife Talk, by Ed Fowler. Lots of stories about his knifemaking. Good read, with some good information.
These are not the only works out there. However, I'd strongly recommend Hrisoulis' work, it's very in-depth. If you're wanting to just cobble some equipment together and give it a try, Goddard's book is one to go with. Hell, read both if you can. Information from two master knifemakers- make that three, Fowler's one too- available on paper for your edification.
Materials is a different category, but I'll throw it in.
Centaur Forge is a blacksmith & farrier supply company. I'd get their catalog simply to look at everything. Tools, materials, blanks, you name it, it's probably there. Including coal for the forge. If you can find a place a reasonable drive away you can probably get it for less, especially in quantity, but for a smaller amount to try out, they've got it.
Enco Tool Supply is a good place for general tools- including sanding belts- and materials. You can get tool steel bar stock, oil, air or water-hardening for good prices if nothing else.
Depending on what you're working on, scrounging can get lots of good stuff. Broken springs, bearings, old files & horseshoe rasps, found at salvage yards or flea markets. At flea markets, keep your eyes open for odd hammers, sometimes you can find a ball peen or straight peen without a handle for cheap; or you can find a double-face that you can modify to a straight peen. For that matter, sometimes you can find tongs, anvil tools, blowers, vises(not that kind, the mechanical kind) and other stuff. A new post vise with 4.5" jaws runs about $600 last time I looked; I found one last year at a flea market for $35. It needs a new mounting bracket, can you say big frickin' deal?
If I can find/remember other books, I'll add them to this list later on. But this stuff should get you started.