Tupper was born in Halifax, Nova Scotia, the son of Charles Tupper (who later served as Premier of Nova Scotia from 1863 to 1867, and Prime Minister of Canada in 1896) and Frances Amélia Morse. He was named in honour of his father's mentor James William Johnston. He was educated at Upper Canada College and Harvard Law School but returned to Nova Scotia to practice law, being called to the bar in 1885. Later in the same year, Tupper enlisted as a private in the Canadian army to assist in putting down the North-West Rebellion, and remained in Manitoba afterwards. He was called to the Manitoba Bar in 1886, and worked in a Winnipeg law firm with Hugh John Macdonald, son of Prime Minister John A. Macdonald. In 1887, he married Margaret, the daughter of James McDonald. He was named a King's Counsel in 1912. Tupper also served as president of the Army and Navy Veterans in Canada.