Ryder was born in India in 1908 to Colonel Charles Henry Dudley Ryder, Surveyor General of India, and Ida Josephine Grigg. He was a great-grandson of the Right Reverend Henry Ryder, youngest son of Nathaniel Ryder, 1st Baron Harrowby. Ryder had two brothers; both were killed in the Second World War. Lisle Charles Dudley Ryder was killed in the Le Paradis massacre of 1940 in France. Ernle Terrick Dudley Ryder died in captivity after the defence of Singapore. Ryder was educated at Hazelhurst School and Cheltenham College before he entered the Royal Navy in 1926.