After his retirement in 1817, through successive political upheavals, Madison lived quietly with his wife at their Montpelier, Virginia, home, where he died on this day in 1836 at age 85.