In 1850, Penniman defeated incumbent Democrat Alexander W. Buel to be elected as a Whig from Michigan's 1st congressional district to the Thirty-second Congress, serving from March 4, 1851 to March 3, 1853. He was the first Plymouth citizen elected to the United States Congress. He was not a candidate for renomination in 1852. He was a member of the convention that met under the oaks at Jackson, Michigan, July 6, 1854, at the organization of the Republican Party in Michigan. He was a delegate to 1856 Republican National Convention from Michigan.