Born in Newport, New Hampshire, Belknap was appointed a Midshipman in 1847. He commanded the monitor during the attacks on Battle of Fort Fisher, and the sloop-of-war during the Formosa Expedition of 1867. He was the senior officer present during the riots following David Kalākaua's election as the King of Hawaii in 1874.