During the American Civil War, Robert and Lucy Saunders were forced to flee their home prior to the Battle of Williamsburg in 1862. The Robert Carter House was substantially looted, and Robert Saunders was criticized by Williamsburg Provost Marshal David Edward Cronin for abandoning the house full of its valuable contents of books, art, and furniture. Much of what was looted was from the family of Lucy Page Saunders: papers of Governor John Page, including documents signed by George Washington, colonial American newspapers, and a portrait by Charles Willson Peale of a man known as Selim the Algerian, the whereabouts of which are still unknown.