Reavley was born in Quitman, Texas. He received a Bachelor of Arts degree from the University of Texas in 1942. While at the University of Texas, he was a member of the Tejas Club. He was in the United States Navy from 1942 to 1946. As a naval lieutenant, Reavley drove President Franklin Roosevelt to Roosevelt’s meeting with Winston Churchill and Joseph Stalin at the Yalta Conference.