Just thought I would update this with what we actually ended up doing. We got to SF on Saturday afternoon and spent a few hours walking through Chinatown after a dim sum lunch. On Sunday we walked all through Golden Gate Park, walked the Golden Gate Bridge, went to Fisherman's Wharf, rode a cable car, and were wore out by 7 pm.
We rented a car on Monday for a road trip to Napa. While we were still in the city we drove down Lombard St, The Crookedest Street, so that was pretty cool. We then drove across the Golden Gate Bridge and through Sausalito on our way to Muir Woods. We loved Muir Woods. The redwoods are amazing, but it is definitely chilly in the woods. We went to two wineries in Napa and had some pretty good wine and then drove back into SF via the Bay Bridge, so I got my engineer's fix for the week. That evening we went out to Japantown and had some sushi.
On Tuesday we went to Alcatraz. That was a great tour. It's hard to imagine how much higher the prison block is than the dock area. I think they said the walk was 1/4 of a mile, but you went up 13 stories. When we were done with the tour we walked down to the Wharf and ate some crab. Did some souvenir shopping and that pretty much finished out our trip.
We had perfect weather and I actually got sunburned the first day since I didn't bring sunscreen and I don't really have a lot of hair on my head.
Thanks to everyone for all of your suggestions!