Chatham Manor Chatham Manor is the Georgian - style home completed in 1771 by William Fitzhugh , after about 3 years of construction , on the Rappahannock River in Stafford County , Virginia , opposite Fredericksburg . It was for more than a century the center of a large , thriving plantation . Flanking the main house were dozens of supporting structures : slave quarters , a dairy , ice house , barns , stables . Down on the river were fish traps . The estate included an orchard , mill , and a race track where Fitzhugh ' s horses vied with those of other planters for prize money . Fizhugh named the mansion after the British parliamentarian William Pitt , 1st Earl of Chatham , who championed many of the opinions held by American colonists prior to the Revolutionary War . In January 1805 , the plantation was the site of a minor slave rebellion . A number of slaves overpowered and whipped their overseer and assistants . An armed posse of white men quickly gathered to capture the slaves . They killed one slave in the attack , and two died trying to escape . The posse deported two other slaves , likely to a slave colony in the Caribbean . Slavery at Chatham . Fitzhugh owned upwards of 100 slaves , with anywhere from 60 to 90 being used at Chatham , depending on the season . Most worked as field hands or house servants , but he also employed skilled tradesmen such as miller s , carpenters , and blacksmiths . Little physical evidence remains to show where slaves lived ; until recently , most knowledge of slaves at Chatham was from written records . In January 1805 , a number of Fitzhugh ' s slaves rebelled after an overseer ordered slaves back to work at what they considered was too short an interval after the Christmas holidays . The slaves overpowered and whipped their overseer and four others who tried to make them return to work . An armed posse put down the rebellion and punished those involved . One black man was executed , two died while trying to escape , and two others were deported , perhaps to a slave colony in the Caribbean . A later owner of Chatham , Hannah Coulter , who acquired the plantation in the 1850s , tried to free her slaves through her will upon her death . Her will provided that her slaves would have the choice of being freed and migrating to Liberia , with passage paid for , or of remaining as slaves with any of her ( Coulter ' s ) family members they might choose . Chatham ' s new owner , J . Horace Lacy , took the will to court to challenge it and had it overturned . The court denied Coulter ' s slaves any chance of freedom by ruling that the 1857 " Dred Scott " decision by the U . S . Supreme Court had declared that slaves were property , and not persons with choice . Ellen Mitchell was the enslaved laundress at " Chatham " who knew of and counted on Mrs . Coulter ' s promise of manumission . When Lacy ' s court case took her freedom away , Mitchell was irate and loudly proclaimed how unfair this denial was . To be rid of her ( and the problem she represented ) , Lacy sold her to a slave trader , James Ayler , in Fredericksburg . Ellen Mitchell continued to loudly protest the unfairness of her situation . Like Lacy , Ayler was unsure what to do with Mitchell and gave her a 90 - day pass to leave Fredericksburg in early 1860 on a tour during which she attempted to raise money to buy her freedom . He sent her on her way with the understanding that she would return . She gave speeches in Washington City , Baltimore , Philadelphia , New York , and Boston , raising enough money to return to Fredericskburg and buy not only her freedom but that of her children , as well . Ayler was so impressed that he also freed Mitchell ' s mother . The Mitchell family moved to Cincinnati in the free ( i . e . slavery - prohibited ) state of Ohio . In the 1860 census , Ellen Mitchell was listed as running a laundry business . Today , some of her descendants still live in that area of Ohio . Slavery at Chatham ended in 1865 as a result of the Civil War , upon the passage of the constitutional amendment abolishing the institution . Recent research , led largely by National Park Service historians , has revealed a sketch made by a Unionist New Jersey soldier during the Civil War that showed other buildings at the Chatham site . Historians had previously thought that most slave dwellings were likely to be in the " rear " , or the field - side area of the estate . This area had been cultivated since the slave days and in the 20th century new structures were built there . The recently discovered sketch shows structures to the south side of the manor house , in an area across a ravine away from the central area of the property . Re - examination of old photographs shows the faint rooflines of structures in that area , which may indicate the location of heretofore unconfirmed slave dwellings . More of the slave - era story at Chatham may now be discovered . Antebellum . Fitzhugh was a friend and colleague of George Washington , whose family ' s farm was just down the Rappahannock River from Chatham . Washington ' s diaries note that he was a frequent guest at Chatham . He and Fitzhugh had served together in the House of Burgesses prior to the American Revolution , and they shared a love of farming and horses . Fitzhugh ' s daughter , Mary Lee , would marry the first president ' s step - grandson , George Washington Parke Custis . Their daughter wed the future Confederate General Robert E . Lee . Evidence supports that Thomas Jefferson and James Monroe also visited at Chatham , making a veritable " Who ' s Who " of important Americans who stopped in to enjoy Fitzhugh ' s hospitality . ( A letter was recently discovered among Jefferson ' s papers being catalogued at Princeton University ; in that 1792 note , TJ writes , " . . . stopped at friend Fitzhugh ' s in Fredericksburg . . . " He appeared to have been traveling between the new Capital City in Philadelphia and Monticello . ) William Henry Harrison stopped by Chatham in 1841 on his way to be inaugurated as President . The Civil War . The Civil War brought change and destruction to Chatham . At the time the house was owned by James Horace Lacy { 1823 - 1906 } , a former schoolteacher who had married Churchill Jones ' s niece . As a planter , Lacy sympathized with the South , and at the age of 37 he left Chatham to serve the Confederacy as a staff officer . His wife and children remained at the house until the spring of 1862 , when the arrival of Union troops forced them to abandon the building and move in with relatives across the river in the beleaguered city of Fredericksburg . For much of the next thirteen months , Chatham would be occupied by the Union army ; they referred to it as the " Lacy House " in their orders and reports , as well as diaries and letters . Northern officers initially used the mansion as a headquarters . In April 1862 , General Irvin McDowell brought 30 , 000 men to Fredericksburg . From his Chatham headquarters , the general supervised the repair of the Richmond , Fredericksburg and Potomac Railroad and the construction of several bridges across the Rappahannock River . Once that work was complete , McDowell planned to march south and join forces with the Army of the Potomac outside Richmond . President Abraham Lincoln journeyed to Fredericksburg to confer with McDowell about the movement , meeting with the general and his staff at Chatham . His visit gave Chatham the distinction of being one of three houses visited by both Lincoln and Washington ( the other two are Mount Vernon and Berkeley Plantation on the James River east of Richmond . ) While at Chatham , Lincoln went to Fredericksburg , walked its streets , and visited a New York regiment encamped on what would become known as " Marye ' s Heights " during the later battle . Seven months after Lincoln ' s visit to Chatham , fighting erupted at Fredericksburg . In November 1862 , General Ambrose E . Burnside brought the 120 , 000 - man Army of the Potomac to Fredericksburg . Using pontoon bridges , Burnside crossed the Rappahannock River below Chatham , seized Fredericksburg , and launched a series of bloody assaults against Lee ' s Confederates , who held the high ground behind the town . One of Burnside ' s top generals , Edwin Sumner , observed the battle from Chatham , while Union artillery batteries shelled the Confederates from adjacent bluffs . Fredericksburg was a disastrous Union defeat . Burnside suffered 12 , 600 casualties in the battle , many of whom were brought back to Chatham for care . For several days , army surgeons operated on hundreds of soldiers inside the house . Assisting them were volunteers , including the poet Walt Whitman and Clara Barton , who later founded the American chapter of the International Red Cross . Whitman came to Chatham searching for a brother who was wounded in the fighting . He was shocked by the carnage . Outside the house , at the foot of a tree , he noticed " a heap of amputated feet , legs , arms , hands , etc . - about a load for a one - horse cart . Several dead bodies lie near , " he added , " each covered with its brown woolen blanket . " In all , more than 130 Union soldiers died at Chatham and were buried on the grounds . After the war , their bodies were removed to the Fredericksburg National Cemetery . Years later when three additional bodies were discovered , the remains were buried at Chatham , in graves marked by granite stones lying flush to the ground . In the winter following the battle , the Union army camped in Stafford County , behind Chatham . The Confederate army occupied Spotsylvania County , across the river . Opposing pickets patrolled the riverfront , keeping a wary eye on their foe . Occasionally the men would trade newspapers and other articles by means of miniature sailboats . When not on duty , Union pickets slept at Chatham ; Dorothea Dix of the United States Sanitary Commission operated a soup kitchen in the house . As the winter progressed and firewood became scarce , some soldiers tore paneling from the walls for fuel , exposing the underlying plaster . Some of the soldiers ' pencil graffiti is still visible , with additional scrawls being deciphered by Park Service staff . Dr . Mary Edwards Walker served the wounded at Chatham . Walker was awarded the Medal of Honor , the only woman from the Civil War to be so recognized , for her meritorious service to the wounded during several battles . When the law for the Medal of Honor was changed to restrict the medal to combat veterans , the US government asked her to return hers . She refused and died with the medal in her possession . Her family continued to petition for the full restoration of the honor . In 1977 , then - President Jimmy Carter signed the Congressional bill into law that restored Dr . Walker ' s medal . Military activity resumed in the spring . In April , the new Union commander , General Joseph Hooker , led most of the army upriver , crossing behind Lee ' s troops . Other portions remained in Stafford County , including John Gibbons ' division at Chatham . The Confederates marched out to meet Hooker ' s main force and for a week fighting raged around a country crossroad known as Chancellorsville . At the same time , Union troops crossed the Rappahannock at Fredericksburg and drove a Confederate force off Marye ' s Heights , behind the town . Many of 1 , 000 casualties suffered by the Union army in that 1863 engagement were sent back to Chatham which again served as a hospital . Postwar years . By the time the Civil War ended in 1865 , Chatham was desolate and severely damaged . Blood stains spotted the floors , graffiti marred its bare plaster walls and sections of the interior wood paneling had been removed for firewood . In addition to the damaged house , the grounds had suffered . The surrounding forests had been cut down for fuel , the gardens and several of the outbuildings where damaged or destroyed , and the lawn had been used as a graveyard . In 1868 the Lacys returned to their home . Unable to maintain it properly , they moved to their house known as " Ellwood " and sold Chatham in 1872 . The property had a succession of owners until the 1920s , when Daniel and Helen Devore undertook its restoration ( and made significant changes ) . They literally saved the house . During the restoration , they re - oriented the house away from the west front on the river , as it was no longer the main transportation route ; they made the east entrance the main entrance , easily reached by the automobile . They also added a large , walled English - style garden designed by the noted landscape architect Ellen Biddle Shipman on the east side . Shipman completed re - making of the estate ' s grounds . As a result of the DeVores ' efforts , Chatham regained its place among Virginia ' s finest homes . Today the house and the of surrounding grounds are open to the public . The last private owner , John Lee Pratt , purchased the larger Chatham estate from the Devores in 1931 for $ 150 , 000 cash and used the estate as a retirement home and working farm . Chatham ' s distinction as a destination of note continued during his ownership . Serving as one of President Roosevelt ' s " dollar - a - year " men , Pratt met and had as visitors Generals George Marshall and Dwight Eisenhower . Upon Pratt ' s death in 1975 , his will bequeathed land for parks to Stafford County and Fredericksburg , as well as a large section to the region ' s YMCA . This also left the manor house and approximately surrounding it to the National Park Service , which uses it as the headquarters facility for the Fredericksburg and Spotsylvania National Military Park . Five of the rooms are open as a museum facility , and the grounds are open to the public . The remainder of the house and outbuildings are used as offices for administrative and maintenance facilities . Additional support for the site is provided by the Friends of Chatham Manor group formed in 2012 to emphasize the need to maintain the historic building and its grounds .