He was born in Windsor, Nova Scotia, the son of Winckworth Tonge and Martha Grace Cottnam. He was trained as an attorney and served as deputy to his father, Nova Scotia's naval officer; he succeeded his father in that post in 1792. Tonge also was named a justice of the peace and served as judge in the Inferior Court of Common Pleas for Hants County. In 1793, he married Elizabeth Bonnell. Tonge suffered financial losses and was forced to sell much of the family property. Although he was able to secure some military contracts, his attempts to improve his finances through patronage were blocked by the lieutenant governor John Wentworth. In 1799, Tonge was elected in a Halifax County seat as well as in Hants County; his election was overturned because he held no property there and Michael Wallace, a supporter of Wentworth, was declared elected. Tonge became the centre of a dispute between the lieutenant governor and the assembly over control of expenditures in the province in the legislative session that followed. In 1805, he became speaker following the resignation of Richard John Uniacke from that post. In 1806, Wentworth dismissed Tonge from his post as provincial naval officer. Tonge's attempts to have himself reinstated were unsuccessful. However, the conflicts between Wentworth and the assembly led to the lieutenant governor being replaced in 1808. Later that year, George Prevost, the new lieutenant governor, named Tonge deputy commissary general for the Invasion of Martinique (1809). He later settled in Georgetown, Guyana, where he died at the age of 68.