Thareah/ThaOCR-models

ThaOCR models — Khmer + English document OCR (ONNX, CPU / phone size)

Five-stage document OCR pipeline. Every model is plain ONNX (opset 17) and runs with onnxruntime on CPU, in the browser (ONNX Runtime Web) or on a phone (ORT Mobile). models.json is the manifest the ThaOCR backend reads: file/vocab are relative to the folder where all stage repos are downloaded, defaults names the model used for each stage.

Versions

release	git tag	what is in it
v3 (latest)	v3	layout v4 (default, plus int8) · textdet v2 · rec v2.3 · font v2 · structure v1
v2	–	first publication: layout v3 default, same textdet / rec / font / structure

Pin a release in every repo with --revision v3; without it you get the latest commit.

Where the files are

The model files live in one repo per stage. This repo is the index: it holds this model card and the full models.json manifest.

stage	repo
layout	Thareah/ThaOCR-layout
textdet	Thareah/ThaOCR-textdet
rec	Thareah/ThaOCR-rec
font	Thareah/ThaOCR-font
structure	Thareah/ThaOCR-structure

To get a working set, download all five into one folder so the relative paths in models.json resolve:

for r in layout textdet rec font structure; do hf download Thareah/ThaOCR-$r --local-dir ThaOCR; done
hf download Thareah/ThaOCR-models models.json --local-dir ThaOCR

(Each stage repo also has a models.<stage>.json listing only its own models; models.json here covers all five stages.)

stage	default model	file	size	what it does
layout	layout_yolo11n_v4_22cls (_int8: 3.1 MB, less exact)	layout/layout_v4_n_22cls.onnx	10.9 MB	22 region classes: text, title, section_header, list, table, figure, caption, header, footer, page_number, footnote, formula, signature, stamp, logo, qr_code, barcode, photo, handwriting, checkbox, form_field, decoration. Input [1,3,1024,1024].
textdet	db_ppocrv5_khmer_v2	textdet/db_v2.onnx	4.8 MB	DB text-line detector (PP-OCRv5 mobile, fine-tuned for Khmer). Input [1,3,H,W], H and W multiples of 32.
rec	crnn_m_v2_stamp2 / _int8	rec/crnn_m_v2_stamp2*.onnx + .vocab.json	19 MB / 4.8 MB	CRNN-CTC line recogniser, Khmer + English, 195 symbols, line height 48, dynamic width [N,1,48,W].
font	font_cnn_v2	font/font_cnn_v2.onnx	1.9 MB	per line: font family (29 Khmer/Latin families), weight, slant, em size. Input 32x256 grey.
structure	struct_v1	structure/struct_v1.onnx	4.0 MB	small Transformer over the page's lines: role, paragraph start, list level, bullet, reading order.

Older versions kept for comparison: layout v1 (YOLO11s, 12 classes; .onnx + .pt), layout v2 (YOLO11s round 2, 21 classes), v2 YOLO11n and v3 YOLO11n, textdet v1 and the original PP-OCRv5 detector, recogniser v1, v2, v2.1 (_hw), v2.2 (_stamp), v2.4 (_stamp3), font v1.

How the models were trained

Layout detector (Ultralytics YOLO11, imgsz 1024, AdamW, cosine LR, mosaic 0.6, no horizontal flip)

1. v1 — YOLO11s, 12 classes, trained in the earlier Khm-Dataset-Creation repo on synthetic Khmer pages, real Khmer PDF pages with heuristic labels and DocLayNet.
2. v2 round 1 (layout_r1) — YOLO11s from v1 weights, 21-class taxonomy, 30 epochs (stopped at 26 by the time budget), lr0 1e-3, batch 8. Val mAP50 0.964, mAP50-95 0.834.
3. v2 round 2 (layout_r2, layout_v2_r2_21cls.onnx) — continued from round 1 on the round-2 set (32,290 train / 923 val pages), lr0 5e-4. mAP50 0.962, mAP50-95 0.834.
4. v2 nano (layout_v2_n_21cls.onnx) — YOLO11n (2.6 M params) from the COCO-pretrained yolo11n.pt on the same round-2 set, 31 epochs, lr0 2e-3, batch 16. mAP50 0.956, mAP50-95 0.798.
5. v3 nano (layout_v3_n_22cls.onnx, default) — fine-tune of v2 nano on layout_v3 (18,000 train / 1,100 val: 10,000 new docgen v3 pages with the decoration class + 8,000 pages replayed from round 2), 37 epochs, lr0 6e-4. mAP50 0.972, mAP50-95 0.842.
6. v4 nano (layout_v4_n_22cls.onnx, default) — fine-tune of v3 nano on layout_v4 (19,000 train / 1,200 val: 9,000 new docgen v4 pages — Khmer government notices and letters with emblem letterheads and mottos, handwritten reference numbers and dates, label-value blocks, tables with spanning group rows, stamps printed over signatures and dates — + 10,000 pages replayed from layout_v3), 57 epochs in a 3 h budget, lr0 0.0003. On the v4 validation set: mAP50 0.974, mAP50-95 0.850 (v3 on the same set: 0.935 / 0.802). Largest gains: handwriting 0.42 → 0.75, header 0.70 → 0.93, text 0.72 → 0.87, stamp 0.76 → 0.82, signature 0.66 → 0.73 (mAP50-95). layout_v4_n_22cls_int8.onnx is the same network after onnxruntime dynamic weight quantisation (3.1 MB); boxes are visibly less tight on tables and forms, so the app does not pick it by default.

Layout training data (round 2 set): docgen v2 synthetic pages (12,000 Cambodian + US/English documents: letters, invoices, forms, ID cards, exam sheets, receipts, reports…), DocLayNet v2 (6,895 train / 647 val real pages, 11 classes + page_number, plus degraded scan/photo copies), real Khmer PDF pages with silver heuristic labels (548 train / 173 val, each also degraded), SignverOD signatures and stamp scans (round 3 of the v1 repo), D4LA real US scans (2,834 train, only form/invoice/letter/memo; missing classes filled by a teacher model, training/layout/pseudo_label.py), CommonForms (1,517 pages, checkbox boxes only), docgen exam answer sheets (3,000 pages). Taxonomy in training/taxonomy.py.

Text-line detector (DB, PP-OCRv5 mobile backbone, 1.16 M params)

det/train_textdet.py converts the original PP-OCRv5 mobile detector ONNX to PyTorch (onnx2torch) and fine-tunes the probability map with DB shrunk kernels (r = 0.4) so the original unclip post-processing still works.

• v1 — fine-tuned on synthetic Khmer pages + real Khmer PDF pages + DocLayNet text-layer line boxes.
• v2 (db_v2.onnx, default) — 20,000 iterations, batch 16, --robust augmentation, from the textdet_robust checkpoint; loss weights synth 0.45 / Khmer PDF 0.20 / DocLayNet 0.35. Data: 15,000 synthetic train pages (docgen v2 + exam sheets + handwriting lines) / 750 val, DocLayNet 6,895 / 647, Khmer PDF 548 / 173. Final F1: synthetic 0.894, DocLayNet 0.880, Khmer PDF 0.697 (0.690 low-res).

Text recogniser (CRNN-CTC, arch "M": conv 48-96-192-256, proj 320, BiLSTM 2x256, 4.76 M params, line height 48)

Trainer: training/rec/train.py (the Khm-Dataset-Creation trainer + packed line sets), --aug strong (GPU warp, slant, low resolution, polarity), greedy CTC decoding. Vocabulary 195 symbols (Khmer, Latin, digits, punctuation), text normalised with normalize_khmer. Lineage (each step initialised from the previous best checkpoint):

1. v1 (crnn_m_v1) — 6 epochs on mix_print_scene: synthetic print sections S01–S08 (200k lines sampled; news corpus rendered with 21 open-licence Khmer fonts + Khmer OS fonts, print/scan effects) + synthetic scene/signboard sets (dataset_v2 scene/heavy/medium/clean, nf_mix, hard, nf_hard, long). 739k lines/epoch.
2. expA — 2 epochs adding real English lines from DocLayNet text layers (rec_doclaynet, 90,860 train lines).
3. v2 (crnn_m_v2) — 6 epochs, lr 1e-3, on final_v2_hand: everything above + docgen printed lines (99,869) + synthetic handwriting lines from docgen exam sheets (69,081, Khmer handwriting fonts) + docgen form handwriting (11,905). 1.008 M lines/epoch. Val CER: synthetic print 0.22 %, English real 0.30 %, synthetic handwriting 4.8 %.
4. v2.1 (crnn_m_v2_hw) — 3 epochs, lr 5e-4, on final_v2_hand_real: + real Khmer handwriting from Thareah/khmer-handwriting (2,311 train lines, x8 oversampled) and SoyVitou/khmer-handwritten-dataset-4.2k (3,549, x4). 1.041 M lines/epoch.
5. v2.2 (crnn_m_v2_stamp) — 2 epochs with STAMP_AUG=0.12: training/rec/stamp_aug.py draws grey ring arcs and stamp strokes over 12 % of the crops so text under official red stamps is still read.
6. v2.3 (crnn_m_v2_stamp2, default) — 4 epochs, lr 1.2e-4, same data and stamp augmentation, from v2.1. Val CER: synthetic print 0.216 %, English real 0.29 %, docgen 7.5 %, synthetic exam handwriting 4.5 %, real handwriting Thareah 21.3 % / SoyVitou 9.8 %, owner's real document dev set 12.7 %.
7. v2.4 (crnn_m_v2_stamp3) — 4 more epochs, lr 1e-4. Slightly better on every val set (print 0.213 %, real_dev 12.6 %); not yet made the default. int8 variants are dynamic-quantised exports of the same checkpoints (training/rec/export.py).

Font classifier (font/font_cnn_v2.onnx, FontNet, ~0.45 M params)

Trained entirely on synthetic line crops rendered on the fly (training/font/gen.py: Khmer news corpus + English sentences, PIL + HarfBuzz shaping, 29 font families: all Khmer OS faces, Noto Sans/Serif Khmer, Hanuman, Kantumruy Pro, Nokora, Odor Mean Chey, Chenla, Preahvihear, Suwannaphum, the Khmer display fonts (Koulen, Dangrek, Bayon, Angkor, Taprom) and Latin serif / sans; synthetic bold by dilation and italic by shear; neighbour-line intrusion, blur, noise, JPEG, paper tint). v1: 40 epochs x 400 steps x batch 256. v2: 24 epochs from v1 adding the em-size regression head. Val (8,000 held-out crops): family 92.0 %, weight 98.0 %, slant 99.9 %, size error median 1.7 %.

Structure model (structure/struct_v1.onnx, StructureNet, ~0.6 M params)

4-layer Transformer (d 160) over per-line geometric + text features, with a pointer head for reading order. Trained 30 epochs, batch 16, on docgen v3 pages with struct labels (10,000 train / 500 val synthetic pages). Val: role accuracy 0.988, paragraph-start F1 0.971, successor accuracy 0.964, Kendall tau 0.943.

Datasets

data	type	used by	licence
Khmer news corpus (5,601 articles, 257k lines; scraped by the owner for research) rendered with open Khmer fonts	synthetic	rec, font	text: research use; fonts OFL / LGPL
docgen v2 / v3 / v4 / exam synthetic pages (Project Gutenberg English, Faker identities, procedural logos/stamps/figures)	synthetic	layout, textdet, rec, structure	generated
DocLayNet v2	real	layout, textdet, rec (English lines)	CDLA-Permissive 1.0
D4LA (form/invoice/letter/memo)	real	layout	Apache-2.0 (mirror), page content third-party
CommonForms (checkbox boxes)	real	layout	Apache-2.0
Real Khmer PDF pages (government/NGO PDFs crawl, heuristic labels)	real	layout, textdet	research use
SignverOD signatures, mapo80 stamps	real	layout	Apache-2.0 / CC-BY-4.0
Thareah/khmer-handwriting, SoyVitou/khmer-handwritten-dataset-4.2k	real	rec	MIT
FUNSD, KhmerEduDoc	real	evaluation only (non-commercial)	NC

Base weights: Ultralytics YOLO11 (AGPL-3.0 — the layout models inherit it), PaddleOCR PP-OCRv5 mobile detector (Apache-2.0). The owner's private exam photos were never used for training.

Licence

The repo mixes licences, so the Hub tag is license: other. Per stage:

Stage	Files	Licence
Layout	layout/*.onnx, layout/layout_v1_12cls.pt	AGPL-3.0 — fine-tuned from Ultralytics YOLO11 weights and inherits that licence
Text detection	textdet/db_v1.onnx, textdet/db_v2.onnx, textdet/db_ppocrv5_orig.onnx	Apache-2.0 — derived from (or identical to, for db_ppocrv5_orig) the PaddleOCR PP-OCRv5 mobile detector
Recognition	rec/*	owner's own models, trained from scratch
Font	font/*	owner's own models, trained from scratch on synthetic data
Structure	structure/struct_v1.onnx	owner's own model, trained from scratch on synthetic data

Training data licences are listed in the Datasets table above. FUNSD and KhmerEduDoc were used for evaluation only (non-commercial); no model weights were trained on them.

Use

import onnxruntime as ort, json
sess = ort.InferenceSession("rec/crnn_m_v2_stamp2_int8.onnx")
vocab = json.load(open("rec/crnn_m_v2_stamp2.vocab.json"))["itos"]
# grey line crop, height 48, shape [1,1,48,W] float32 in 0..1 -> greedy CTC over the output, blank id 0

Pre/post-processing (letterbox + NMS, DB unclip, width bucketing + CTC) is small numpy/OpenCV code in the ThaOCR backend (backend/app/services/stages/).