From the 15th to the 17th century there were times of growth and upswing yet diminished by destruction for example through the Margrave Wars and the Thirty Years' War. In 1796 the Geuder family lost the fiefdom and the territory became Prussian, though in the Napoleonic Wars it was under French administration for four years. Ever since 1810 Heroldsberg belongs to Bavaria.