Burr was born on March 7, 1809, in Warren, Rhode Island. As a young man, he worked for a whaling company which sent him west. After losing his crew while docked in San Francisco—many sailors were lured away by the prospect of finding gold during the Gold Rush—Burr stayed put and opened a grocery store. His family moved to California to be with him. The venture proved so successful that he opened California's first savings union, the San Francisco Accumulating Fund Association.