His brother John became the founder of the Princeton Preparatory School, while one of his sisters, May Margaret, became the founder of Miss Fine's School. Lambert Fine died in 1869 when Henry was ten years old, and his mother moved to Ogdensburg, New York, for a period and then to Winona, Minnesota, and finally in 1875 she came with her children to Princeton so they might have a good education:
Ang kanyang kuya John ay naging tagapagtatag ng Princeton-umpisa ng paaralan, habang ang isa sa kanyang mga madre, May Margaret, naging tagapagtatag ng Miss's Fine School. Lambert Fine namatay sa 1869 kapag Henry ay sampung taong gulang, at ang kanyang ina inilipat sa Ogdensburg, New York, para sa isang panahon at pagkatapos ay sa Winona, Minnesota, at sa wakas sa 1875 siya dumating sa kanyang mga anak sa Princeton kaya sila ay maaaring magkaroon ng isang mahusay na edukasyon: