In 1728 he married Elizabeth, the wealthy young widow of Sir John Dickson Haman, 1st and last Baronet. She was the daughter of Sir Thomas Prendergast, 1st Baronet, who was killed at the Battle of Malplaquet, and the sister of Sir Thomas Prendergast, 2nd Baronet. Charles and Elizabeth's children included Thomas Smyth and John Prendergast Smyth, 1st Viscount Gort, both of whom followed their father into politics. Their grandson Charles Vereker later inherited the Gort viscountcy.