Samuel Phillips Lee Samuel Phillips Lee ( February 13 , 1812 — June 7 , 1897 ) was a Rear Admiral of the United States Navy . He commanded the North Atlantic Blockading Squadron from 4 September 1862 to 12 October 1864 . His flagship was . Life and career . Lee was born at " Sully " in Fairfax County , Virginia to Francis Lightfoot Lee II and Jane Fitzgerald . He was the grandson of Richard Henry Lee , great - nephew of Francis Lightfoot Lee I , brother - in - law of Francis Preston Blair , Jr . , and of Montgomery Blair , and was third cousin of Robert E . Lee . He was appointed a midshipman in the U . S . Navy in November 1825 and subsequently saw extensive service at sea , including combat action during the Mexican – American War and exploration , surveying and oceanographic duty . At the outbreak of the American Civil War in 1861 , he held the rank of commander and was captain of the sloop of war in the East Indies , sailing her home on his own initiative to join the blockade of the Southern coast . Commander Lee commanded the new steam sloop during the New Orleans campaign and subsequent operations on the Mississippi River in the first half of 1862 . Lee became well known in Washington society due to the influence of his wife , the former Elizabeth Blair , of Maryland . When asked about his loyalty , Lee famously replied " When I find the word Virginia in my commission I will join the Confederacy . " This quote is often referenced by historians in contrast to the actions of his cousin Robert E . Lee , to show how the war divided families . In September 1862 , Lee was placed in command of the North Atlantic Blockading Squadron with the rank of Acting Rear Admiral . His flagship at this point was the . He led this force for over two years , during which it was responsible for the blockade of the North Carolina coast and operations on North Carolina and Virginia inland waters , all areas of very active combat between Union and Confederate forces . Acting Rear Admiral Lee transferred to the command of the Mississippi River Squadron in October 1864 and led it to the end of the Civil War in 1865 . His flagship during his time as commander of the Mississippi River Squadron was the . Reverting to his permanent rank of captain after the Civil War , Lee extensively served in the Washington , D . C . area . He was promoted to rear admiral in 1870 and retired from active service in February 1873 . Residence . In 1859 , Lee ' s father - in - law , Francis Preston Blair , built a house for Lee and his wife ( Francis ' s daughter Elizabeth Blair ) next door to his own . These two houses , within a block of the White House in Washington , D . C . , were later combined into one house and became the property of the U . S . government . Today they are the Blair - Lee House , used by the President as his guest house . Upon retirement he moved to the family home in Silver Spring , Maryland , where he died on 7 June 1897 . In memoriam . Two U . S . Navy ships have been named in honor of Rear Admiral Samuel Phillips Lee , the destroyer , which was commissioned in October 1920 and lost by stranding in September 1923 , and the research vessel ( ex - AGS - 31 ) , which was in naval service between 1968 and 1974 .