crnn-rukopys

A compact CRNN + CTC line recognizer for handwritten Ukrainian document text, trained from scratch on the Rukopys dataset. At ~6.25M parameters with greedy CTC decode it is CPU-friendly — a lightweight alternative to transformer recognizers.

The training data included handwritten, printed and annotation region crops from Rukopys.

TL;DR

	value
Architecture	CRNN: 6-layer CNN → BiLSTM (hidden 256) → linear CTC head
Parameters	~6.25M
Trained	from scratch (no pretrained backbone), silver→gold curriculum
Alphabet	340 characters + CTC blank
Handles	handwritten, printed, annotation
Gold-val CER / WER	0.1539 / 0.4188
Input	a single grayscale line crop, height 32 px, width ≤ 512 px, scaled to [-1, 1]
Output	the transcribed string (metric-normalized character set)

Intended use

Recognizing cropped line/region images from handwritten Ukrainian documents, downstream of a layout detector — where a small, fast, CPU-capable recognizer is preferred over a transformer one.

For stronger recognition quality at ~90× the parameter count, see Hukyl/trocr-large-rukopys.

Architecture

CNN feature extractor → BiLSTM sequence model → linear CTC head. No pretrained backbone — trained end-to-end on Rukopys crops.

• CNN: 6 conv layers (1→64→128→256→256→512→512, 3×3), BatchNorm after the 3rd and 5th conv, MaxPool to height 1 (pools 2,2 · 2,2 · (2,1) · (2,1) so width resolution is preserved for the sequence).
• RNN: single bidirectional LSTM, hidden size 256 (512-d output).
• Head: Linear(512 → 256) → log-softmax → CTC (blank index 0).
• Alphabet: 340 characters + CTC blank (341 classes total), built from the silver+gold training vocabulary. Covers Ukrainian Cyrillic (incl. і ї є ґ ў ѣ), Latin, digits, punctuation, and a long tail of typographic / mathematical symbols (–—«»“”№∑∫≈≤≥… etc.).
• Input: grayscale, height-normalised to 32 px, width capped at 512 px, scaled to [-1, 1].

How to use

Note: resize crops with cv2.INTER_AREA — training used it, and INTER_LINEAR aliases on the heavy downscale to 32 px height.

import cv2
import torch
import torch.nn as nn
from huggingface_hub import hf_hub_download

IMG_HEIGHT, IMG_MAX_WIDTH = 32, 512


class CRNN(nn.Module):
    def __init__(self, num_classes: int, rnn_hidden: int = 256) -> None:
        super().__init__()
        self.cnn = nn.Sequential(
            nn.Conv2d(1, 64, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2, 2),
            nn.Conv2d(64, 128, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2, 2),
            nn.Conv2d(128, 256, 3, padding=1), nn.BatchNorm2d(256), nn.ReLU(),
            nn.Conv2d(256, 256, 3, padding=1), nn.ReLU(), nn.MaxPool2d((2, 1)),
            nn.Conv2d(256, 512, 3, padding=1), nn.BatchNorm2d(512), nn.ReLU(),
            nn.Conv2d(512, 512, 3, padding=1), nn.ReLU(), nn.MaxPool2d((2, 1)),
        )
        self.rnn = nn.LSTM(512, rnn_hidden, bidirectional=True, batch_first=False)
        self.fc = nn.Linear(rnn_hidden * 2, num_classes)

    def forward(self, x):
        feat = self.cnn(x)
        _b, _c, h, _w = feat.shape
        feat = feat.squeeze(2) if h == 1 else feat.mean(dim=2)
        feat = feat.permute(2, 0, 1)
        out, _ = self.rnn(feat)
        return torch.log_softmax(self.fc(out), dim=-1)


ckpt = torch.load(hf_hub_download("Hukyl/crnn-rukopys", "best.pt"), weights_only=False)
chars = ckpt["chars"]
model = CRNN(len(chars), ckpt["config"]["rnn_hidden"])
model.load_state_dict(ckpt["model_state"])
model.eval()

gray = cv2.imread("line_crop.jpg", cv2.IMREAD_GRAYSCALE)
new_w = max(4, min(int(gray.shape[1] * IMG_HEIGHT / gray.shape[0]), IMG_MAX_WIDTH))
resized = cv2.resize(gray, (new_w, IMG_HEIGHT), interpolation=cv2.INTER_AREA)
tensor = (torch.from_numpy(resized).float().unsqueeze(0) / 127.5 - 1.0).unsqueeze(0)

with torch.inference_mode():
    log_probs = model(tensor)[:, 0, :]

# greedy CTC decode (collapse repeats, drop blank=0)
idx, prev, out = log_probs.argmax(-1).tolist(), None, []
for i in idx:
    if i != prev and i != 0:
        out.append(i)
    prev = i
print("".join(chars[i] for i in out))

The model expects single-line / single-region crops, not full pages.

Training curriculum

Two-stage silver→gold curriculum, trained from scratch. CTC loss (blank=0, zero_infinity), AdamW, OneCycleLR schedule, online augmentation, best checkpoint selected by lowest validation CER.

1. Silver pretrain (20 ep, early stopping): CTC pretrain from scratch on the Rukopys silver split (auto-labeled crops): 119,346 train / 21,061 val.
2. Gold fine-tune (30 ep): warm-start from the stage-1 checkpoint, with the alphabet grown to cover gold-only characters, on the human-labeled gold train split: 18,948 train / 3,345 val crops, stratified at crop level. The published weights are the best-val-CER epoch (29 of 30).

Hyperparameters (as launched)

hyperparameter	value
epochs	20 (silver) + 30 (gold), early stopping with patience 5
batch size	128
learning rate	1e-3
weight decay	1e-4
optimizer / schedule	AdamW, OneCycleLR
gradient clipping	5.0
loss	CTC (blank=0, zero_infinity)
online augmentation	on (default profile, at working resolution)
eval	greedy CTC decode, full val each epoch
precision / device	fp16 mixed precision, CUDA
seed	42

Online data augmentation

Online augmentation was applied during training, at working resolution: each crop was first downscaled with INTER_AREA to a 64 px working height (2× the model's 32 px input), augmented, then resized to the final 32 px tensor. Only training crops were augmented (no augmented validation was measured).

Each crop was transformed once per epoch by one geometric + one or two photometric operations at random. Each class received a separate augmentation profile that was selected to minimize the distribution shift.

region type	geometric (one)	photometric (one or two)
handwritten, annotation	margin pad 2–15% / trim 1–5%, rotation ±1–5°, elastic distortion (α=25, σ=5), baseline warp (amp 2–8 px, freq 0.5–2.0)	paper-colour shift (LAB a±10 / b±15), Gaussian noise (σ 5–15), JPEG recompression (q 30–65), contrast/gamma (0.7–1.3 / 0.6–1.5), morphological erode/dilate (kernel 2)
printed	margin pad 2–15% / trim 1–5%, rotation ±1–3°	paper-colour shift (a±5 / b±10), Gaussian noise (σ 3–10), JPEG recompression (q 40–70), contrast/gamma (0.8–1.2 / 0.8–1.3), morphological erode/dilate (kernel 2)

These simulate scanner/paper variation, ink thinning/bleed, and natural handwriting deformation, widening the range of appearances beyond the raw training crops.

Results

Protocol: full held-out gold validation split filtered to the model's classes (3,345 crops, no subsampling), greedy CTC decode, INTER_AREA resize (training parity). Labels are metric-normalized so CER/WER reflect the scored character set.

Overall

metric	value
CER	0.1539
WER	0.4188
exact-match accuracy	0.2706
n_samples	3,345

Per class

class	n	CER	WER	accuracy
handwritten	3,223	0.1507	0.4132	0.2696
annotation	79	0.4496	0.7578	0.3924
printed	43	0.2716	0.6267	0.1163

We also acknowledge that printed/annotation n is quite small, so measuring CER against them is quite noisy.

Files

file	description
best.pt	lowest-val-CER checkpoint (gold fine-tune, epoch 29)

The checkpoint is a self-contained .pt payload: model weights, the alphabet, the architecture config (rnn_hidden, img_height, img_max_width), and the training-time metrics.

Limitations & biases

• Handwritten Ukrainian archival document material only; expect degradation on other scripts, languages, or modern born-digital text.
• The character set and normalization are tuned to the document-OCR metric — outputs are normalized text, not faithful transcription.
• Greedy CTC decode only — no language model or beam search.
• The train/val split is stratified at crop level, not page level — crops from the same page (and writer) appear on both sides, so the reported val CER is optimistic compared to a page-level protocol.
• annotation and printed classes have small training support and higher CER; formula and table are unsupported.
• Single training run — no across-run variance estimate.

Training data & attribution

dataset	source	license	role
Rukopys	UkrainianCatholicUniversity/rukopys	CC BY 4.0	silver pretrain (auto-labeled) + gold fine-tune