Raised in Plains, Georgia, Carter graduated from the United States Naval Academy in 1946 with a Bachelor of Science degree and joined the United States Navy, where he served on submarines. After the death of his father in 1953, Carter left his naval career and returned home to Georgia to take up the reins of his family's peanut-growing business. Carter inherited comparatively little due to his father's forgiveness of debts and the division of the estate among the children. Carter inherited comparatively little due to his father's forgiveness of debts and the division of the estate among the children. Nevertheless, his ambition to expand and grow the Carters' peanut business was fulfilled. Carter had long dreamed of attending the U.S. Naval Academy.