Fort Amsterdam Fort Amsterdam ( subsequently named Fort James , Fort Willem Hendrick , Fort James ( again ) , Fort William Henry , Fort Anne and Fort George ) was a fort on the southern tip of Manhattan that was the administrative headquarters for the Dutch and then British rule of New York from 1625 until being torn down in 1790 after the American Revolution . The fort was to change hands eight times in various battles including the Battle of Long Island in the American Revolution when volleys were exchanged between the fort and British emplacements on Governor ' s Island . The construction of the fort marked the official founding date of New York City as recognized by the Seal of New York City . In October 1683 what would become the first session of the New York legislature convened at the fort . Guns at the fort formed the original battery that is today called Battery Park ( New York ) . The fort ' s site is now occupied by the Alexander Hamilton U . S . Custom House , which currently houses the George Gustav Heye Center , part of the National Museum of the American Indian . History . Dutch Rule ( 1625 – 1664 ) . The site of the fort is now the Alexander Hamilton U . S . Custom House , which prior to landfilling and the creation of Battery Park was at the shoreline . The fort was the nucleus of the New Amsterdam settlement with a mission of protecting New Netherland colony operations in the Hudson River against attack from the English and the French . Around 1620 , the Dutch East India Company contacted the English architect Inigo Jones asking him to design a fortification for the harbor . Jones responded in a letter with a plan for a star - shaped fortification made of stone and lime and surrounded by a moat and defended with cannon s . Jones advised the company against constructing a timber fort out of haste . The building of the fort commenced in 1625 , under the direction of Willem Verhulst , the second director of the New Netherland colony and his chief engineer Cryn Fredericks . By the end of the year , Fredericks had surveyed the site . He returned to the Dutch Republic in November 1626 . At the time , Manhattan was only lightly settled , as most of the Dutch West India Company operations were upriver along the Hudson in order to conduct trading operation for beaver pelts . Despite Jones ' plea in his letter , the plan for the masonry fortification was abandoned , however , out of the need for a hasty completion . This was due primarily to : British Rule ( 1664 – 1673 ) . No shots were fired on August 27 , 1664 , when the Dutch surrendered the fort and Manhattan in what amounted to one of the skirmishes in the bigger Second Anglo – Dutch War . The fort was renamed Fort James in honor of James II of England . New Amsterdam was renamed New York in recognition of James ' s title as Duke of York . Dutch Rule ( 1673 – 1674 ) . In August 1673 , the Dutch brought in a fleet of 21 ships and recaptured Manhattan . The fort was renamed Fort Willem Hendrick in honor of the Dutch leader who was Stadtholder and Prince of Orange . New York was renamed New Orange . The Dutch attack was part of the bigger Third Anglo - Dutch War . In 1674 the fort and New Orange was turned back over to the British in the Treaty of Westminster ( 1674 ) which ended the war ( the Dutch got Suriname ) . British Rule ( 1674 – 1689 ) . The British once again named the area " New York " and returned the Fort James name . During this period , the Thomas Dongan , 2nd Earl of Limerick who was the royal governor of New York convened the first legislature of New York in October 1683 for a meeting at the fort . Dongan was also the first to establish batteries of cannon just to the south of the fort . Colonist Rule ( 1689 – 1691 ) . In 1689 German born colonist Jacob Leisler seized the fort in what was called Leisler ' s Rebellion and enacted a government of direct popular representation . By some counts , he also moved to redistribute wealth to the poor . British Rule ( 1691 – 1775 ) . Leisler ' s rule ended in 1691 when British sovereign William III of England sent troops to evict him . The fort had earlier been named for Willem when he was head of the Dutch government . He was to become the sovereign of the English government by the overthrow of James in the Glorious Revolution . The fort was then renamed with a new Anglicized Fort William Henry name . The fort was to continued be named for subsequent British sovereigns : Fort Anne ( for Anne , Queen of Great Britain ) and Fort George for the succession of George monarchs ( George I of Great Britain , George II of Great Britain , George III of the United Kingdom ) . 92 guns were added to the battery in 1756 . Colonist / American Rule ( 1775 – 1776 ) . The fort became the target of American rioting following the Stamp Act 1765 in which the guns at the battery were spiked . In the American Revolution , the colonists under George Washington seized the fort in 1775 , before the colonies formally seceded from the British Empire . During the Battle of Long Island , guns from the fort engaged British frigates starting on July 12 . Between September 2 and 14 , the fort and British guns on Governors Island exchanged volleys . British Rule ( 1776 – 1783 ) . The British recaptured the fort along with lower Manhattan in September and were to rule New York from the fort throughout the war . American Rule ( 1783 – 1790 ) . The Americans took over the fort in Manhattan on the Evacuation Day , November 25 , 1783 . The fort was torn down in 1790 and the Government House , intended as the presidential residence , was built on the site . However , the need for new fortifications soon became apparent and in 1798 guns were placed in temporary fortifications on the Battery . Eventually a new fort Castle Clinton would be built just before the War of 1812 .