achimrabus/trocr-glagolitic-handwritten

TrOCR Fine-tuned on Croatian Glagolitic Handwriting

This model is a fine-tuned version of microsoft/trocr-large-handwritten for recognizing 14th–15th century Croatian Glagolitic handwritten text. It was trained by Achim Rabus (Slavic Department, University of Freiburg) on the same dataset used for the Transkribus model glagolitic-handwritten-14th-and-15th-century.

Important: This model reads Glagolitic handwriting and outputs Latin script transliteration, not Glagolitic Unicode characters. It handles ligatures and resolves the most common abbreviations.

Model Description

TrOCR (Transformer-based OCR) is a vision-to-text model using a ViT encoder and a causal language model decoder. This version is fine-tuned specifically on medieval Croatian Glagolitic handwriting.

• Base model: microsoft/trocr-large-handwritten
• Intended use: OCR/HTR of Croatian Glagolitic historical manuscripts
• Output: Latin script transliteration (UTF-8)

Training Data

Trained on Glagolitic handwriting images transcribed and exported from Transkribus (see the corresponding Transkribus model page). The dataset covers 14th–15th century Croatian Glagolitic handwriting.

Source manuscripts:

• Cod. Vind. Slav. 3 (Breviary of Vid of Omišalj)
• II. beramski brevijar

Ground truth data was kindly provided by Sanja Zubčić (Rijeka) and Jagoda and Guido Kappel (Vienna). Model trained and curated by Achim Rabus (Slavic Department, University of Freiburg). The Transkribus collection comprises 531 training pages and 31 validation pages (~31,035 lines in total). This TrOCR model was trained on 23,203 lines (training) and 1,361 lines (validation) from this export.

• Preprocessing: resize to 128 px height, aspect ratio preserved (LANCZOS); no background normalization
• Validation: in-training evaluation on the validation lines

Performance

Metric	Value
CER (validation)	6.42%

Note: A CNN + BiLSTM + CTC ("CRNN-CTC") model trained on the same Glagolitic data reaches a lower CER on this validation set (5.33%, see achimrabus/crnn-ctc-glagolitic). The two models are best seen as complementary — it is worth comparing both on your own material rather than relying on the validation CER alone.

Training Details

Parameter	Value
Base model	microsoft/trocr-large-handwritten
Optimizer	AdamW
Learning rate	3e-5
Per-device batch size	16 (gradient accumulation 2)
Epochs	10 (best checkpoint at step 9000)
FP16	Yes
Augmentation	Rotation ±2°, brightness/contrast ±0.2
Generation max length	96
Framework	HuggingFace Transformers, Seq2SeqTrainer

How to Use

from transformers import TrOCRProcessor, VisionEncoderDecoderModel
from PIL import Image

processor = TrOCRProcessor.from_pretrained("achimrabus/trocr-glagolitic-handwritten")
model = VisionEncoderDecoderModel.from_pretrained("achimrabus/trocr-glagolitic-handwritten")

# The model was trained on line images normalized to 128 px height with aspect
# ratio preserved. Reproduce that before the processor (which otherwise squashes
# directly to 384x384), to match the training distribution.
def resize_to_height(img, height=128):
    w, h = img.size
    return img.resize((max(1, round(w * height / h)), height), Image.LANCZOS)

image = resize_to_height(Image.open("line_image.png").convert("RGB"))
pixel_values = processor(images=image, return_tensors="pt").pixel_values

generated_ids = model.generate(pixel_values)
text = processor.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(text)  # Output: Latin script transliteration

Note: Input should be a single text line image, not a full page.

Recommended: Polyscriptor

For real-world use on whole manuscript pages, this model is best run through Polyscriptor, a multi-engine HTR training and comparison tool. Polyscriptor handles automatic line segmentation, full-page batch processing, PAGE XML import/export, and lets you compare this TrOCR model directly against the CRNN-CTC model and other engines on your own material. It is available as a browser UI, a PyQt UI, and a command-line interface.

Intended Use

• Transcription of 14th–15th century Croatian Glagolitic manuscripts
• Digital humanities research on medieval Croatian texts

Limitations

• Trained on two manuscript sources (Cod. Vind. Slav. 3 and II. beramski brevijar); may underperform on other hands or periods
• Output is Latin script transliteration, not Glagolitic Unicode
• Full-page segmentation quality depends on the segmentation method used upstream

Acknowledgements

Ground truth data kindly provided by Sanja Zubčić (Rijeka) and Jagoda and Guido Kappel (Vienna). Model trained and curated by Achim Rabus, Slavic Department, University of Freiburg.

Citation

If you use this model in your research, please cite the publication describing the training data and recognition system, the TrOCR architecture, and this model:

@article{rabus2022glagolitic,
  title   = {Handwritten Text Recognition for Croatian Glagolitic},
  author  = {Rabus, Achim},
  journal = {Slovo: časopis Staroslavenskoga instituta u Zagrebu},
  volume  = {72},
  pages   = {181--192},
  year    = {2022},
  doi     = {10.31745/s.72.5},
  url     = {https://doi.org/10.31745/s.72.5}
}

@article{li2021trocr,
  title   = {TrOCR: Transformer-based Optical Character Recognition with Pre-trained Models},
  author  = {Li, Minghao and Lv, Tengchao and Chen, Jingye and Cui, Lei and Lu, Yijuan and
             Florencio, Dinei and Zhang, Cha and Li, Zhoujun and Wei, Furu},
  journal = {arXiv preprint arXiv:2109.10282},
  year    = {2021}
}

@misc{rabus2026polyscriptor,
  title  = {Polyscriptor: Multi-Engine HTR Training \& Comparison Tool},
  author = {Rabus, Achim},
  year   = {2026},
  url    = {https://github.com/achimrabus/polyscriptor}
}