Yazid was a member of the Banu Sulaym tribe, which had participated in the Muslim conquest of Armenia and been settled by the caliphs in the western Armenian borderlands with the Byzantine Empire. Yazid's father, Asid (or Usayd) ibn Zafir al-Sulami, had been a general in the service of the Umayyads, and served under the last Umayyad caliph, Marwan ibn Muhammad, and his father, Muhammad ibn Marwan, in their campaigns in the region. Al-Baladhuri further reports that his mother was a Christian, the daughter of the "Patrician of Siwnik", who had been taken captive during Muhammad ibn Marwan's campaigns.