Gifford was born on 3 August 1889, the youngest of 12 children of Frederick and Isabella Gifford. Isabella Gifford, née Burton, was a niece of the artist Frederic William Burton, and was raised with her siblings in his household after the death of her father, Robert Nathaniel Burton, a rector, during the Famine.