Colvile married Katherine Sarah Georgina Russell, daughter of John Russell and Sophia Coussmaker. Their son Sir Henry Colvile became a major-general in the 2nd Boer War fighting at the Battle of Modder River.