A native of Johnstown, Pennsylvania, Scott began his broadcasting career on local radio in the late 1930s. (Fellow announcer Bill McColgan, in his introduction of Ray Scott for the radio broadcast of the 1957 NFL Championship Game, stated that Scott started broadcasting when he was only 17 years old.) Following a stint in the U.S. Army during World War II, he moved to Pittsburgh, where he did play-by-play for Carnegie Tech and University of Pittsburgh football and Duquesne University basketball.