After Confederation, he represented the riding of Charlotte, New Brunswick in the House of Commons of Canada from 1874 to 1896. He was a strong candidate for the office of Lieutenant-Governor of New Brunswick in 1899, being strongly endorsed by his former political opponents as well as supporters, but did not secure the appointment. He was subsequently appointed Canadian commissioner to the World's Fair at Paris. He was appointed to the Senate by Sir Wilfrid Laurier in 1900, where he served until his death in 1903, when he died on a train in Mattawamkeag, Maine while en route to Ottawa after a return trip to his hometown of St. George. Mr. Gillmor is buried in the St. George Rural Cemetery, in St. George, New Brunswick.