First Bulgarian Empire
The Bulgarians took the castle in the 9th century but with the end of the First Bulgarian Empire in the beginning of the 11th century the Byzantines conquered it.