Eloff was the seventh of the ten children of Sarel Johannes Eloff, commander of Rustenburg from 1864 to 1889, and his wife, Susanna Cornelia Jacobsz, and a younger brother of FC (Ryk Freek) Eloff, son-in-law of President Kruger through his marriage to Paul and Gezina Kruger's daughter Elsie. Jan Eloff received his first education at home in his hometown from a private teacher and later visited the school of the Hermannsburg Missionary Society on the Morgenzon farm near Rustenburg. At Graaff-Reinet he bought wagons and returned to the Transvaal, where he disposed of them at a reasonable profit. As a young man, he served under his father in the First War of Independence (1880-81).