Elmer E . Ellsworth Elmer Ephraim Ellsworth ( April 11 , 1837 – May 24 , 1861 ) was a law clerk and soldier , best known as the first conspicuous casualty of the American Civil War . Early life . Ellsworth was born in Malta , New York , grew up in Mechanicville , New York , and lived in New York City . In 1854 , he moved to Rockford , Illinois , where he worked for a patent agency . In 1859 , he became engaged to Carrie Spafford , the daughter of a local industrialist and city leader . Carrie ' s father demanded that he find more suitable employment , so he moved to Chicago , to study law and work as a law clerk . In 1860 , Ellsworth went to Springfield , Illinois , to work with Abraham Lincoln . He studied law in Lincoln ' s office and helped Lincoln with his 1860 campaign for president . Ellsworth was only 5 ' 6 " tall , but Lincoln called Ellsworth " the greatest little man I ever met . " He accompanied Lincoln to Washington , D . C . in 1861 . Military career . Ellsworth became drillmaster of the " Rockford Greys " , the local militia company , in 1857 . He studied military science in his spare time . After some success with the Greys , he helped train militia units in Milwaukee and Madison . When he moved to Chicago he became colonel of Chicago ' s National Guard Cadets . Ellsworth had studied the Zouave soldiers , French colonial troops in Algeria , and was impressed by their reported fighting quality . He outfitted his men in gaudy Zouave - style uniforms , and modeled their drill and training on the Zouaves . Ellsworth ' s unit eventually became a nationally famous drill team . The Civil War broke into open warfare in April , and on April 15 , 1861 , Lincoln called for 75 , 000 troops to put down rebellion . Ellsworth helped recruit these soldiers : he raised the 11th New York Volunteer Infantry Regiment ( the " Fire Zouaves " ) from New York City ' s volunteer firefighting companies , and returned to Washington as their colonel . Death . Ellsworth died shortly after returning to Washington . On May 24 , 1861 ( the day after Virginia ' s secession was ratified by referendum ) , President Lincoln looked out from the White House across the Potomac River , and saw a large Confederate flag prominently displayed over the town of Alexandria , Virginia . Ellsworth immediately offered to retrieve the flag for Lincoln . He led the 11th New York across the Potomac and into the streets of Alexandria uncontested . He detached some men to take the railroad station , while he led others to secure the telegraph office and get the Confederate flag , which was flying above the Marshall House Inn . Ellsworth and four men went upstairs and cut down the flag . As Ellsworth came downstairs with the flag , the owner , James W . Jackson , killed him with a shotgun blast to the chest . Corporal Francis E . Brownell , of Troy , New York , immediately killed Jackson . Brownell was later awarded a Medal of Honor for his actions . Lincoln was deeply saddened by his friend ' s death and ordered an honor guard to bring his friend ' s body to the White House , where he lay in state in the East Room . Ellsworth was then taken to the City Hall in New York City , where thousands of Union supporters came to see the first man to fall for the Union cause . Ellsworth was then buried in his hometown of Mechanicville , in the Hudson View Cemetery . Thousands of Union supporters rallied around Ellsworth ' s cause and enlisted . " Remember Ellsworth " was a patriotic slogan : the 44th New York Volunteer Infantry Regiment called itself the " Ellsworth Avengers " , as well as " The People ' s Ellsworth Regiment . " Honored as a hero . Relics associated with Ellsworth ' s death became prized souvenirs . The Smithsonian Institution and Bates College ' s Special Collections Library have pieces of the Confederate flag that Ellsworth had when he was shot — in 1894 , Brownell ' s widow was offering to sell small pieces of the flag for $ 10 and $ 15 each . The New York State Military Museum and Veterans Research Center in Saratoga Springs has most of the flag itself and Ellsworth ' s uniform , showing the hole from the fatal shot . The Fort Ward Museum in Alexandria dedicates a section of their museum to Ellsworth , displaying the kepi he wore when he was killed , patriotic envelopes bearing his image , a piece of the Confederate Flag ( on which Ellsworth ' s blood is visible ) , and the " O " from the Marshall House sign that a soldier took as a souvenir . In 1862 , the newly established county seat of Pierce County , Wisconsin , located in the undeveloped center of the county to settle the controversy between two established cities , was named Ellsworth , Wisconsin in his honor . The 1876 Centennial Exposition in Philadelphia had a major exhibit devoted to Col . Ellsworth . In addition , Ellsworth , Michigan , Ellsworth , Wisconsin , and Fort Ellsworth were named in his honor . He is a character in the 2012 film " Saving Lincoln " , in which his death is portrayed .