Following his removal from "Sully" to Washington, DC in 1815, Richard, along with John Peter Van Ness and Tench Ringgold, was appointed by longtime friend President James Madison as one of three Commissioners to superintend the reconstruction of the Federal buildings damaged by British troops in the attack on Washington DC on August 24, 1814 during the War of 1812. Following expiration of this commission in 1816 President Madison intended to nominate Lee to the new position of "Commissioner of Public Buildings". However, after learning the Senate would not confirm any of the three members of the previous commission because of their perceived ill treatment of Benjamin Latrobe who had been appointed Architect of the Capitol in order to aid in its reconstruction, Richard was instead appointed by President Madison as a commissioner to adjudicate claims arising out of the loss or destruction of property during the War of 1812. Lee occupied this position for two contentious years, during which his decisions came in for harsh criticism by Congress. John Randolph of Virginia accused Lee of "malfeasance," and congress finally appointed a "Committee of Claims" to look into the operation of Lee's office. While Lee himself was exonerated of any corruption, the committee noted that many "claimants had perpetrated an extensive 'system of fraud, forgery, and perhaps perjury." With his power severely curtailed, Lee, despondent over his treatment, left his position and for a short time seriously considered moving his family to Kentucky. Realizing his prospects would be limited there, and that his wife opposed the move, he decided to stay in Washington DC. In 1819 he was appointed by President Monroe as a judge of the Orphans' Court of the District of Columbia, a position he held until his death on March 12, 1827.