Pándy was born on 1 June 1927, in Chop, Carpathian Ruthenia (then under Czechoslovak administration), a village just across the border from Hungary, to Hungarian parents. Pándy was a church councillor for the Reformed Church in Hungary when he met his first wife, Ilona Sőrés. Following the failed Hungarian Revolution of 1956, they fled to Belgium where Pándy became a pastor for a small Hungarian Protestant community in Brussels, and a religious teacher for the United Protestant Church. The couple had a daughter, Ágnes, the following year and two sons: Dániel (born 1961) and Zoltán (1966). Shortly after the birth of Zoltán, the couple separated when Pándy accused his wife of infidelity. Ilona moved out of the house with their sons, leaving daughter Ágnes behind with Pándy. Then 11 years old, Ágnes soon became the victim of an incestuous relationship with her father.