Von Haller was a son of Berne statesman and historian Gottlieb Emanuel von Haller, and grandson of poet and polymath Albrecht von Haller. His lineage descended from Johannes Haller (1487-1531), a Reformed preacher who died alongside Huldrych Zwingli in the Second War of Kappel.