Oliver was born into the prestigious and influential "gens" (clan) Rátót, as the son of Dominic I, who was killed in the Battle of Mohi in 1241. Oliver had three brothers, Roland I, Stephen I and Leustach II, and a sister, who married Maurice Pok. Oliver had four children from his unidentified wife: Reynold III, Nicholas "Vecse" – ancestor of the Putnoki family –, John and a daughter, who married powerful baron Reynold Básztély.