Howdy, I live in the high desert where the coldest we've gotten is 4 degrees f. Summer it sticks around 100-105. My garage walls are already insulated, but the ceiling is not. We just had a long string on insulation and heating that really covered the waterfront on your questions. But in brief, it boils down to Roxol for your situation is probably best choice for walls. Does not react to moisture.
I have a roll up, steel door with about 22x48 panels. I insulated it last summer with a combination of shiny (like aluminum) surfaced bubble wrap about 1/4 thick, a layer of 1.5 inch foam at about $22 per sheet (used 4), then a second layer of bubble wrap. Total R value is close to 12-13. I then scrubbed the door steel very clean and sealed the insulation in with aluminum duct tape. That closed off air leaks and trapped all the insulation. It really helps.
I'll spare you the gory details of the ceiling, which I'm having done for a cost of about $2300 or so. It will be R38 with half inch dry wall. Its the garage, so there's a gas dryer out there I'll turn on empty to heat the place up. I only do wood prep out in the garage, so don't spend long hours there. A shed doesn't have to account for air flow for a water heater, so you won't have to leave vents in place. Although I am thinking of air source vent to the water heater through the outside, but not sure about code yet.
Without insulation, heating and an AC, you won't enjoy your shop nearly as much as with. Let's face it, this is not necessarily a cheap hobby, although you can reduce costs by careful tool selection and even buying used items. There's no rush, just get basic tools when you need them, or when you run into a great buy.
Dust collection is another thing that's easy to put off, but when you start coughing from the sawdust, it will be a little too late since the really find sawdust cannot be expelled from your lungs.
Since you're kind of new to this topic, you might find the following article helpful, It is about the 17 things that really helped me got going with woodworking over the last decade or so. Just click on the link to read it and many comments from others with their helpful ideas as well. http://www.routerforums.com/featured...ing-curve.html