With the defeat of the Tories in the 1896 election, Foster retained his seat and joined the Opposition. He was a prominent supporter of Canada's involvement in the Anglo-Boer War from 1899 to 1901. He lost his seat in the 1900 election but returned to parliament in 1904, this time representing the riding of Toronto North in Ontario. He remained an Opposition MP until his party returned to government in the 1911 federal election under Sir Robert Borden and he continued in the government under Arthur Meighen.