The Güssen family became too poor to support the castle, and in 1613 sold the entire village and castle to the Duchy of Württemberg. In 1617 Duke Julius Friedrich von Württemberg took the villages of Brenz and Weiltingen under his control, founding the junior Württemberg line of Württemberg-Weiltingen. Schloss Brenz was used as a temporary home for his family. However, the damaged castle was destroyed in 1634 during the Battle of Nördlingen of the Thirty Years' War. In 1672 Duke Friedrich Ferdinand had the castle rebuilt in a Renaissance style. The foundation of the old castle and some portions of the walls are still visible, but most of the castle was built new.