He was born in Atlanta in 1925. He is the grandson of Luther Z. Rosser, Sr., who was the lead defense attorney in the Leo Frank case. Luther Rosser, Sr., died in 1923.