Also, you have to keep in mind that some of the areas of San Francisco are pretty dangerous. There are a lot of homeless people there, and there has been some targeting towards people with smart phones. Two of my friends who live there have been mugged, and there were places that I've walked around where I had to keep walking without stopping to take pictures, because of the sketchy people around me.
One rule of thumb is to never walk around in downtown at night, especially by yourself; it becomes pretty much dead, minus the homeless people that will start camping out on the streets. There won't even be a lot of cars driving around, so it can make you VERY vulnerable to muggers, etc.
I made sure to be aware of all the areas to avoid in San Francisco before I went, and what I did was to basically mark all those areas on a map, which you will be able to obtain at your hotel or a visitor information centre.
There is actually very little street robbery in SF. Even in the middle of the night, most of SF is quite
safe, (even the downtown financial district - but there is no reason to go there late at night) and most of the homeless are harmless. If you are coming back to Union Square very late at night (and while Muni and BART are still operating - unlike NYC, they don't run all night) you would probably be getting off at the Powell Street Station. Keep your wits about you, and don't linger on Market Street; also don't walk West of Mason Street unless your hotel is West of Mason, and then walk only on the street the hotel is on. The only real place to avoid is the triangle formed by Mission/Market Sts., Mason/5th Sts., Polk St. on the West, and Geary on the North. This pretty much encircles "the Tenderloin." The only other really unsafe areas are the ghettos and housing projects at the south end of the city, where you would have no reason to go. Even Mission St. where it runs north/south and is its most incredibly disgusting, is only a block away from all the trendy new restaurants, and is safe - just keep looking over your shoulder.
Market St. runs to the Castro, has a streetcar line on it, and is a totally safe street for walking late at night West of Polk St.
Keep your wits about you while riding buses at any time of day - there is very little police presence, but the typical robbery is a teenager thug grabbing your cellphone or Ipod, or grabbing a wallet out of a back pocket, maybe with a slug or push to go along with it.
This next bit may sound racist, but it's merely factual - avoid the clubs south of Market that have lots of black youth in them- that's where all the shootings occur - relatively frequently.
If you are coming back to your hotel at 2 am, you might want to take a cab. There are usually cabs in the Castro. Other than in front of theatres/symphony/opera when they are leting out, the only place one can normally find a cab on the street is around Union Square.
One of the Muni lines (underground at Powell St) becomes a streetcar that runs all the way to the ocean, and is only 2 blocks South of Golden Gate Park. To the north of the park, there are bus-lines that run all the way to the ocean (or Land's End) as well.