The San Francisco Bay Area is home to some of the finest used record shopping around. These are the best sources I've found for rock 'n roll and other popular LPs.
Streetlights has a fairly small, but good selection of used LPs.
I've had good success here over the years finding things that have been hard to find elsewhere. They're slowly ditching LPs, though. The neighborhood is nice, too. One of the better used record stores in the City. Streetlights' Market Street store is a short distance around the corner, as is Tower Records.