The Byzantines recaptured the city in 1261, but never regained their former strength