Kelly resumed his role as colonel of his regiment as more senior officers returned to the brigade. However, with the death of Col. Richard Byrnes at the Battle of Cold Harbor in 1864, Kelly again commanded the brigade. At the age of 42, Kelly died during the Siege of Petersburg when he was shot through the head while leading the Irish Brigade forward against Confederate earthworks. His body was recovered and sent back to New York for his funeral. He was buried in First Calvary Cemetery in Woodside, New York.