On 21 July 1788, during the presidency of the count of Morges, the Assembly of Vizille met in the castle's "jeu de paume" room after the 7 June Day of the Tiles in Grenoble. Pope Pius VI spent a night in the castle at the invitation of owner Claude Perier on 5 July 1799, and Napoleon stopped there during his return from the island of Elba on 7 March 1815. Used as a factory, the castle experienced a fire in the night of 9–10 November 1825 which spread to part of the city. In 1828, Adolphe Perier (son of owner Augustin Perier and grandson of Claude) married Nathalie de La Fayette (daughter of Georges Washington de La Fayette and granddaughter of the Marquis de La Fayette) in the castle. Absent from the ceremony, the Marquis de La Fayette was a guest on 19 August 1829 during a visit with Nathalie. Adolphe Perier, after the death of his father in December 1833, continued restoring the castle. In 1862, after Adolphe's bankruptcy, the Académie des Beaux-Arts classified the castle as a "monument historique" and Henry Fontenilliat (step-father of Auguste Casimir-Perier) became the new owner. Two years later, Henri Fontenilliat died; his daughter, Camille (Auguste's wife), inherited the castle.