In early 1865, Abe went to Kyoto at the head of an army of 4000 troops to arrange for the meeting between Shōgun Tokugawa Iemochi and Emperor Kōmei and to suppress pro-"sonnō jōi" "rōnin" in the area. He returned again three months later as part of the train of Shōgun Tokugawa Iemochi, and was joined by 1200 troops from Shirakawa, who arrived by sea and were garrisoned in Osaka. Although it was widely expected that he would led these forces in the Second Chōshū expedition, Abe came under pressure from the United Kingdom, France and the Netherlands for the opening of a port and foreign settlement at Hyōgo, as per the terms of treaties already signed. The opening of Hyōgō was vehemently opposed by Emperor Kōmei and the anti-treaty faction within the shogunate, and after Abe gave in to the foreign demands, he was fired from his office by Iemochi under pressure from Emperor Kōmei. Abe returned to Edo and from there to Shirakawa, where he was placed under house arrest and order to turn the post of "daimyō" to his son, Abe Masakiyo. The following year, he was punished further when the Abe clan was ordered to relocate to Tanagura Domain. The clan stalled for a year, and even after moving to Tanagura, managed to recover Shirakawa for a month before the start of the Boshin War. Both Shirakawa and Tanagura were members of the "Ōuetsu Reppan Dōmei", but both domains were quickly overrun by the troops of the new Meiji government. Abe died in Tokyo in 1887 and his grave is at the Tama Cemetery in Fuchū, Tokyo.