Ludwig Gaston was born at Ebenthal castle in Austria-Hungary, the youngest son of Prince Ludwig August of Saxe-Coburg and Gotha and Princess Leopoldina of Brazil, the second daughter of Dom Pedro II. His siblings were Princes Peter August, August Leopold and Joseph Ferdinand. Shortly after their mother's death in 1871, Ludwig and his brothers moved to Brazil, where they lived with their maternal grandfather until a military coup d'état in 1889 abolished the monarchy, forcing the imperial family into exile.