Some must-visit historical sites include the War Between the States Museum, Florence Stockade Visitor Center, and the Florence National Cemetery. These sites offer a deep dive into the area's rich history.