The city is famous for its colorful houses that crowd the hills around the bay.