trocr-large-uk-handwritten

A TrOCR-Large (VisionEncoderDecoderModel, ~558M params) recognizer for line-level handwritten Ukrainian text, trained on a mix of real and synthetic Cyrillic handwriting corpora.

This checkpoint is a general-purpose handwriting reader, meant as a starting point for downstream HTR fine-tunes. A real-data-only refinement of this checkpoint is published separately as Hukyl/trocr-large-uk-handwritten-real.

TL;DR

	value
Architecture	TrOCR-Large (ViT-L/16 encoder @ 384px + TrOCR decoder), VisionEncoderDecoderModel
Parameters	~558M
Init from	Kansallisarkisto/cyrillic-htr-model
Tokenizer	extended Cyrillic vocab, size 50336 (ships with this repo)
Languages	Ukrainian
In-mix held-out CER / WER	0.0173 / 0.0768
Input	a single cropped text line (RGB)
Output	the transcribed string

Intended use

Recognizing single cropped lines of handwritten Ukrainian text, downstream of a line/region detector or on pre-segmented lines. Useful directly as a general handwriting reader, or as an init for fine-tuning on a target domain.

How to use

Note: load the processor from this repo, not from microsoft/trocr-large-handwritten — the tokenizer here is an extended Cyrillic vocabulary (50336 vs stock 50265).

from PIL import Image
from transformers import VisionEncoderDecoderModel, TrOCRProcessor

repo = "Hukyl/trocr-large-uk-handwritten"
processor = TrOCRProcessor.from_pretrained(repo)          # load from THIS repo
model = VisionEncoderDecoderModel.from_pretrained(repo).eval()

crop = Image.open("line_crop.png").convert("RGB")          # one cropped text line
pixel_values = processor(images=crop, return_tensors="pt").pixel_values
generated = model.generate(pixel_values, max_new_tokens=256, num_beams=1)  # greedy
text = processor.batch_decode(generated, skip_special_tokens=True)[0]
print(text)

Lineage & training curriculum

Full fine-tune, optimizer: AdamW via the 🤗 Seq2SeqTrainer, cosine LR decay, max_target_length 256, online augmentation, seed 42, single L40S.

1. Base — Kansallisarkisto/cyrillic-htr-model (Apache-2.0): microsoft/trocr-large-handwritten fine-tuned on ~30k historical Cyrillic rows by the Finnish National Archives; extended tokenizer (vocab 50336).
2. Mixed pretrain (6 ep, LR 4e-5, warmup 0.1): UkrHandwritten (oversampled ×3) + Cyrillic Handwriting Dataset + pumb-ai/synthetic-cyrillic-large capped at 555k, for a synthetic:real ratio ≈ 3:1 (≈579k crops per epoch). The published weights are the best-val-CER epoch (6 of 6).

Hyperparameters (as launched)

hyperparameter	value
epochs	6
batch size	32
learning rate	4e-5
warmup ratio	0.1
weight decay	0.0
max target length	256 tokens
optimizer / schedule	AdamW, cosine decay
online augmentation	on (default profile)
eval	greedy (num_beams=1), eval batch 32
precision / device	bf16, CUDA (single L40S)
seed	42

Training ran 115,662 steps in ~15.9h.

Online data augmentation

Online augmentation was applied during training. Only training crops were augmented (no augmented validation was measured).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. All training crops are handwritten lines, so a single handwriting-oriented transform set was used.

group	operations (ranges)
geometric (one)	margin pad 2–15% / trim 1–5%, rotation ±1–5°, elastic distortion (α=25, σ=5), baseline warp (amplitude 2–8 px, frequency 0.5–2.0)
photometric (one or two)	paper-colour shift (LAB a±10 / b±15), Gaussian noise (σ 5–15), JPEG recompression (quality 30–65), contrast/gamma (0.7–1.3 / 0.6–1.5), morphological erode/dilate (kernel 2)

These simulate scanner/paper variation, ink thinning/bleed, and natural handwriting deformation, widening the range of appearances beyond the raw training crops.

Results

In-mix held-out validation

Held-out 0.3% of the training mix (~2k lines drawn from UkrHandwritten + Cyrillic Handwriting + the synthetic corpus, same distribution as training). Greedy decode.

metric	value
CER	0.0173
WER	0.0768

This split mirrors the 3:1 synthetic:real training ratio, so the absolute numbers are not comparable to real-document CER — they measure fit to the pretraining distribution.

Out-of-domain transfer probe (zero-shot)

The same checkpoint evaluated zero-shot on a held-out page-level split of the Rukopys dataset (handwritten Ukrainian document pages). 1000-sample draw, seed 42, greedy decode, max_new_tokens=512.

class	CER	WER	accuracy	n
overall	0.1925	0.4860	0.1990	1000
handwritten	0.1795	0.4771	0.1939	954
annotation	0.2880	0.7027	0.3704	27
printed	0.3950	0.6489	0.2500	16
table	0.8850	1.0417	0.0000	3

The gap between in-mix (0.0173) and zero-shot documents (0.1925) is the expected domain shift from clean line corpora to document crops; a fine-tune on in-domain document data closes most of it. We also acknowledge that printed/table/annotation n is quite small, so measuring CER against them is quite noisy.

Limitations & biases

• General, not specialized: expect higher error on noisy archival scans until fine-tuned (see the zero-shot transfer probe).
• A large share of the training mix is synthetic; for a real-data-only consolidation see Hukyl/trocr-large-uk-handwritten-real.
• Single seed and validation split — no across-run variance estimate.

Training data & attribution

dataset	source	license	role
UkrHandwritten	Kaggle annyhnatiuk/ukrainian-handwritten-text	CC BY-SA 4.0	mixed pretrain (×3)
Cyrillic Handwriting Dataset	Kaggle constantinwerner/cyrillic-handwriting-dataset	CC0	mixed pretrain
pumb-ai/synthetic-cyrillic-large	HF	Apache-2.0	mixed pretrain (capped 555k)

Attribution is mandatory for the UkrHandwritten dataset (CC BY-SA 4.0) — please keep this credit if you redistribute or build on this model.