Originally held by the Iroquois, one of the first Europeans to take notice of the island was Pierre François Xavier de Charlevoix, in 1720. He called it "Isle aux Chevreuils", the Island of Roe Bucks, and wrote in 1721 that its bays could be useful. A 1779 map of the Province of New York indicates that the island was at one time called Isle a la Biche, French for Doe Island. It was also referred to as Buck Island or Deere Island. The island was renamed Carleton Island after Major General Sir Guy Carleton, Governor of the Province of Quebec. It was one of several islands in the area that were named by John Graves Simcoe after General James Wolfe's adjutants in the Battle of the Plains of Abraham. The largest is Wolfe island, while the others are Howe Island, Amherst Island, and Gage Island (now Simcoe Island).