Carswell grew up in Irwinton, Georgia, and went to Wilkinson County High School. He then attended Lincoln College of Technology in Nashville before returning to Georgia to be closer to Thompson.