Born in Epping, New Hampshire, Plumer took part in the California Gold Rush. He then moved to Saint Louis, Missouri. In 1854, Plumer moved to Wausau, Wisconsin and was involved in the lumber business. Plumer also farmed and raised cattle. In 1866, Plumer served in the Wisconsin State Assembly. His brother was Daniel L. Plumer, who also served in the Wisconsin State Assembly. He committed suicide with a firearm in Wausau, Wisconsin.