The house was built in 1743 by Nathaniel Tolman, around the time of his marriage. Tolman's son John was prominent in the local militia, serving in the American Revolutionary War in the Battles of Lexington and Concord, the Siege of Boston, and the Saratoga campaign. He remained active in the militia after the war, and was activated for the War of 1812 but saw no service. Tolman was also active in civic affairs, serving as assessor and as an agent in negotiations over the separation of Needham's West Parish (now Wellesley). The house's next owner was Jonathan Gay, a military compatriot of Tolman's who was also from an old Needham family. Gay was a schoolteacher and served in the state legislature. The house remained in the Gay family until 1923.