His paternal great-grandfather, the Chevalier Josias de Robillard, Seigneur de Champagné de Torxé, Saintonge, fled France after the 1685 Edict of Fontainebleau for Holland, where he joined William of Orange's army. He married Marie de la Rochefoucauld of the noble house of the same name. Their daughter Susanne married Henri de la Motte-Fouqué, baron de Saint-Seurin et de Tonnay-Boutonne, and was mother of Heinrich August de la Motte Fouqué. Their eldest son, Josias de Robillard, Champagné's grandfather, distinguished himself at a young age in service of Major-General Isaac de Monceau de la Melonière, who commanded a regiment of exiles in William's army during the Irish campaigns.