Upon their return, the U.S. was in the throes of the War of 1812. Consequently, they had to follow a circuitous route back to Portsmouth from their entry in Virginia. Under James Madison, he was then appointed as a secretary to the War Department and moved to a location a few blocks from the White House. While serving in this post, the British attacked and burned the city.