His family was politically divided during the War of the Spanish Succession, and he spent many years under house arrest in Austria. The royal family had left Brussels and returned to Munich in 1701. His father, Maximilian Emanuel, fled to the Spanish Netherlands after he had been defeated at the First Battle of Höchstäd in August 1704, and Charles and his siblings stayed with their mother, the acting queen regnant, in Munich. In May 1705, after a stay in Venice, the Austrian authorities refused to allow the queen to return to Bavaria and forced her into an exile, which lasted ten years. Maximilian Emanuel went also into exile to Compiègne after on April 29, 1706 an Imperial ban was imposed on him, as he again had been defeated at the Battle of Ramillies a few days earlier. Only in 1715 was the family reconciled. After attaining the majority age in August 1715, Charles undertook an educational tour to Italy from 3 December 1715 to 24 August 1716. In 1717, he served among Bavarian auxiliaries in the Austro-Turkish War.