As power shifted from the Edo bakufu to the Imperial Court, Yoshikatsu spent most of his time in Kyoto. In the spring of 1863, Emperor issued "Order to expel barbarians" (攘夷実行の勅命, jōi jikkō no chokumei), and on the last day of the year formed the conference of councillors (Sanyo Kaigi (参預会議))), a panel of powerful daimyo to serve as his military arm. Yoshikatsu was invited to join, but declined. After the Kinmon incident in the summer, in which samurai from the Chōshū Domain attempted a coup against the Tokugawa Bakufu, Yoshitaka served as the military commander during the punitive First Chōshū expedition, in September–November 1864, with Saigō Takamori as second-in-command. The campaign was considered a success, but, when Chōshū again challenged Bakufu authority, Yoshikatsu refused to participate in the Second Chōshū expedition, which ended in a loss of prestige for the Bakufu and a secret alliance between the Satsuma and Chōshū domains, an alliance that soon overthrew the shogunate.