Following the communist coup in 1978, Ahmadzai returned to Afghanistan to join the mujahideen. He was a close associate of Burhanuddin Rabbani, being deputy of his Jamiat-e Islami party, but then left and joined Abdul Rasul Sayyaf's Islamic Dawah Organisation of Afghanistan party in 1992, the year that communist rule ended. He served as a minister in the post-communist Afghan government, variously as interior, construction and education minister, and later became prime minister between 1995 and 1996, although he has downplayed the import of these posts given the chaos due to the fighting at that time.