Khairallah joined the Iraqi branch of the Ba'ath Party in 1956. He was arrested in 1959 following an by the party to assassinate then-Prime Minister Abd al-Karim Qasim. Khairallah would later go on to play a role in the 1963 and 1968 coups that brought the Ba'ath party to power.