The original section of the main house was built 1811–1815. Construction was interrupted by the War of 1812 when John Armstrong Jr. (1758-1843), the owner, served as a Brigadier General, Minister to France, and later as US Secretary of War under James Madison. When the British burned Washington DC in 1814, Armstrong received much of the blame, as he had previously insisted that the British would not attack Washington and failed to properly provide for the defense of the city; he consequently retired to finish building his estate on the Hudson River in 1815. The Armstrongs originally called their home "La Bergerie," French for "the sheepfold," as they were raising a large herd of Merino sheep which had been a gift from Napoleon Bonaparte.