Recognizing Japan's need to learn from the Western powers, Inoue joined the Chōshū Five and was smuggled out of Japan to study at University College, London in England in 1863. When he returned with Itō Hirobumi, he unsuccessfully tried to prevent war (the Battle of Shimonoseki) between Chōshū and the Western naval powers over the closing of the Straits of Shimonoseki to foreign shipping. Later, he fought against the forces of the Tokugawa shogunate in the 1864 First Chōshū Expedition, during which he was severely wounded by the attack of the assassins, received a near-fatal injury, appealing to Inoue's elder brother for beheading because of the unbearable pain and finally Ikutaro Tokoro () who was then in hiding from the pursuit of Tokugawa shogunate with Prince Sanjō Sanetomi and rushed to Inoue pulled him through this by putting about 50 stitches of tatami needle in the wounds on the whole body without anesthesia because of emergency during the domestic war time (The story that Inoue's mother holding bloody Inoue then dissuaded her elder son from beheading was introduced in the National Japanese text book of the 5th period as the power of mother). He later played a key role in the formation of the Satchō Alliance against the Tokugawa shogunate.