It is likely that it was at this time, with the construction of the castle of Castel Rosso (some 4 km from the modern town, at the modern village of Myloi) and the rise of piracy, that the town was moved from its coastal location to the inland around Castel Rosso. The town remained an episcopal see under Latin rule, with the Greek bishop remaining in office; in 1222 however it was merged with the Bishopric of Euripos (Chalcis). In 1276/7 it was reconquered by the Byzantines under Licario and held until 1296, when it was recovered by Boniface of Verona. In 1318 it passed into Catalan hands as part of the dowry of Marulla of Verona for her marriage with Alfonso Fadrique. The town was coveted by the Venetians, who already in 1339 offered to purchase it; negotiations dragged on until 1365, however, and only in 1366 did Karystos finally pass into Venetian control when Boniface Fadrique sold the barony to the Republic. After the conquest of Euboea by the Ottoman Empire in 1470, the local Orthodox see was reactivated as part of the Metropolis of Euripos.