On 16 September 1967, McCallum married fashion model-turned-interior designer Katherine Carpenter in Valley Stream, New York. The couple met at a photo shoot for Glamour magazine in 1965, and were together for 58 years. They had a son, Peter, and a daughter, Sophie.