Hoff married Ann Eliza van Rensselaer, member of a prominent political family, shortly after his move. Their son, John van Rensselaer Hoff, was born in 1848. John became a military surgeon, too. He founded the Army Hospital Corps in 1898, served as Deputy Surgeon General, returned for service during World War I and was posthumously awarded the Distinguished Service Cross for service at Wounded Knee. Two years after the birth of John, in 1850, their daughter Caroline Clay Hoff was born. In 1870 she married Lieutenant, later Colonel, Edward Hunter while both were in California. Henry and Ann had another daughter, Harriet L. Hoff, who died in 1915.