Fort Wool Fort Wool was a seacoast fortification located in the mouth of Hampton Roads approximately one mile south of Fort Monroe . The island fortress was designed by Brigadier General of engineers Simon Bernard , an expatriated Frenchman who had served under Napoleon as his chief engineer , was one of more than forty forts started after the War of 1812 when the British boldly sailed up the Chesapeake Bay to burn the Capital . Started upon a shoal of ballast stones that were dumped as sailing ships entered Hampton ' s harbor called Rip Raps , the fort was to have three tiers of casemate s and a parapet mounting a total of 232 muzzle - loading cannons . Originally named Castle Calhoun for the Secretary of War , John C . Calhoun , Fort Wool was built to maintain a crossfire with Fort Monroe , located directly across the channel , thereby protecting the entrance to the harbor . In 1902 , as a result of the Endicott Board ' s findings , all of the original fort , except 8 casemates , was demolished and new fortifications were constructed . The new armament mounted on five batteries of two to four guns remained in place for decades , although modifications were made from time to time . Only six of the original three - inch guns remained in 1942 , when two were sent to Fishermens Island . A modern battery of two new long - range six - inch guns was constructed on top of one of the old Endicott period batteries during World War II . The outmoded fort was finally abandoned by the military in 1953 . History . Brigadier - general of engineers Simon Bernard was tasked by Secretary of War John C . Calhoun to create or improve fortifications for the protection of vital US ports . Bernard ' s plan was to build more than forty new forts , including Fort Wool , which he had named Fort Calhoun . The fort was to have three tiers of casemate s and a parapet with a total of 232 muzzle - loading cannons mounted , and was to be manned by a garrison of 1 , 000 soldiers . It was to built on a 15 acre ( 61 , 000 m² ) artificial island southeast of Old Point Comfort in Hampton , Virginia . Construction got underway in 1819 when crews started dumping granite boulders into the water . It took four years to bring the rock pile up to the a 6 - foot tall island called for in the plans . A controversy soon arose over the stones purchased for the island . A nineteen page report sent May 7 , 1822 to the House of Representatives from the Military Affairs committee included depositions from individuals alleging that the contract to deliver 150 , 000 perche s of stone at $ 3 . 00 per perch awarded to Elijah Mix on July 25 , 1818 was fraudulent . Before awarding the contract , inquiries had been made at several quarries , and it was no secret that the government needed lots of stone for the project . The allegation was that the contract was awarded to Mix by General Joseph Swift of the War Department without advertising for bids . While other contracts had been awarded without advertising for bids as was standard procedure before April 1818 , in April 1818 the newly reorganized United States Army Corps of Engineers required public notice be given for every contract after that date . Two proposals were received before the contract was awarded to Mix , and two more afterwards . All four of these proposals came in with higher bids than the one Mix offered . Many knowledgeable men agreed that the government had made a good deal , and the contractor had made a bad one for himself . In 1819 , the controversy increased when Mix assigned half his contract to Major Christopher Van Deventer , who was chief clerk in the War Department . Major Van Deventer and Mix were married to the daughters of Major Samuel Cooper , a noted Revolutionary War officer . Van Deventer later sold half of his interest to his father - in - law , Major Cooper . Secretary of War Calhoun had advised him that while what was done was not illegal , it “ might expose Van Deventer to improper insinuations . ” Major Van Deventer deposed that he had no interest in the contract when it was first negotiated nor did he have any influence when the contract was awarded . General Swift deposed that he felt Major Van Deventer had no interest in the contract when it was originally negotiated . Major Cooper and Major Van Deventer sold their shares to Howes Goldsborough & Co . on July 1 , 1820 . Construction of the fort began in 1826 , and after considerable delays caused by subsidence of the island , the first level of casemates was finally completed in 1830 . Only ten guns were mounted . Construction continued through 1834 , and only half of the second tier was completed . It was then found that Fort Calhoun ' s foundations had continued settling . A young second lieutenant and engineer in the U . S . Army , Robert E . Lee was transferred there to assist Captain Andrew Talcott , the US Army engineer in charge of the construction Fort Wool and its larger companion Fort Monroe , across the channel on the mainland . Lee was given the task of stabilizing the island as his first independent command . He found that the island wouldn ' t hold the weight of the two tiers of casemates and brought more stone in to stabilize it , but the fort never reached its intended size . Lee found the stone foundation under the fort was the problem , and that it would never support the weight of three tiers and parapet of the completed fort . Fort Wool also has a little - known association with presidents . President Andrew Jackson , broken hearted after the death of his wife and in frail health , came to Fort Wool in the late 1820s and the 1830s . Jackson made the fort his " White House . " Jackson built a hut and would watch ships from on the island . He even made key policy decisions from the fort with cabinet advisers . Ironically President Jackson ' s Secretary of War John C . Calhoun had become the president ' s arch rival by this stage , by threatening to pull South Carolina out of the union . Later President John Tyler took sanctuary on the island after the death of his wife . Abraham Lincoln also visited the fort . Fort Wool even has an association with the actor Sir Alec Guinness , who was grounded in a minefield off the fort in World War II . The comedian Red Skelton also showed up at Fort Wool during the war to entertain troops . Civil War . In 1862 , with the Civil War underway , a name change was in order for Fort Calhoun . The fort was first named after John C . Calhoun , President Monroe ' s Secretary of War who was a Southern politician of secessionist tendencies , it was decided that it would be named after Maj . Gen . John Ellis Wool , a Mexican War hero and commander at Fort Monroe . A very powerful experimental cannon , the Sawyer Gun , was installed at Fort Wool during the Civil War . The range of this weapon extended all the way to Sewell ' s Point , more than three miles away ( where the Norfolk Naval Base is now located , the site of a Confederate earthen fort with bastion s and a redan and three artillery batteries and total of 45 guns . The Battle of Hampton Roads took place off Sewells Point on March 8 – 9 , 1862 . USS Monitor of the Union Navy faced CSS Virginia of the Confederate States Navy during the Battle of the Ironclads in 1862 . The Sawyer Gun also fired at the CSS Virginia , although it did no damage to the ironclad ' s armor . Endicott batteries . Five gun batteries were constructed after 1902 , when funding became available to implement the recommendations of the Endicott board . World War I . In 1917 and 1918 , all of the six - inch guns and disappearing carriages were removed . World War II . Additional armament and other changes were made after 1942 : Decommissioned . The outmoded fort was finally abandoned by the military in 1953 . After decommissioning , it was given to the Commonwealth of Virginia . In the 1950s , the southern man - made island of the Hampton Roads Bridge - Tunnel was constructed next to Fort Wool , and used as the southernmost anchor for the tunnels . A small earthen causeway connected the man - made island with that of Fort Wool . The bridge - tunnel opened to traffic in 1957 . In 1967 and again in 1970 , the City of Hampton developed it into a park . The Fort Wool passenger ferry , " Miss Hampton II " , allows tourists boarding in Hampton to visit the island during most of the year , but it can also be briefly glimpsed by passengers in westbound vehicles prior to entering the southern end of the tunnel portion of the Hampton Roads Bridge - Tunnel , which carries Interstate 64 across the mouth of the harbor . The island , now called Rip Raps , continues to settle in modern times , and occasionally the casemates of the original fortress are put off - limits for safety reasons . It remains a major draw for tourists , who usually include it in a visit to Fort Monroe . During the summer months , it is served by various harbor tour boats .