khattat-hybrid-ocr · v3

Hybrid, 100% free & offline OCR pipeline for the khattat dataset (fa / ar / en handwriting + math formulas + shapes) — now with a Persian upgrade (FA-Boost) and full-paper (page-level) inference.

What's new in v3

Component	Method (all free/offline)
FA-Boost candidate pool	EasyOCR fragment-joined + paragraph mode, natural + 2× upscale, Tesseract-5 tessdata_best fas/ara on Otsu and adaptive views
Char-LM rescoring	order-4 character n-gram LM trained on the khattat train texts (10.5k fa / 6.6k ar / 4.6k en lines)
Word-fragment repair	lexicon-driven rejoin + word-boundary shift + fuzzy word fix (سفیدر ود → سفیدرود)
Blind fa/ar duel	both normalisations scored, the LM decides
Full-page pipeline	adaptive page ink mask → border cleanup → shape contours → line segmentation (horizontal morphology + projection split) → per-region dispatch (text / formula via math_score → pix2tex / shapes) → caption linking → structured JSON + colour overlay

Results

Persian line-level upgrade (20 local test lines, blind)

system	pooled CER
CRNN (own fine-tune, v1 repo)	0.68
hybrid v2 (confidence ensemble)	0.353
v3 FA-Boost	0.275

(with the oracle best-of-pool ceiling at 0.255)

Full-page inference — 19 test pages, own segmentation, no GT hints

metric	value
text line detection recall / precision	0.928 / 0.928
text CER (matched lines)	0.275
formula recall	0.885
formula CER	0.478
shape recall	0.792
shape name accuracy	0.62
shape caption CER (linked from page)	0.59

English full pages reach CER ≈ 0.01–0.05 with the dedicated en reader + LM.

Demo

• demo/khattat_hybrid_demo_v3.mp4 — before/after Persian lines, full-page inference with overlays (fa mixed page with formulas+shapes, fa text page, en page), page metrics.
• demo/pages_eval/*_overlay.png — per-page detection overlays.

Repo layout

hybrid_ocr/
  page.py        full-page pipeline (segmentation + dispatch + caption link)
  faboost.py     char-LM + lexicon + fragment rejoin + fa/ar duel + fusion
  pipeline.py    HybridOCR.read_line (boost path) / read_formula / read_shape
  engines.py     EasyOCR + Tesseract wrappers, script detection, normalisation
  preprocess.py  OpenCV pre-processing (upscale, deskew, Otsu/adaptive)
  formula.py     pix2tex + LaTeX cleanup + math_score
  shapes.py      contour shape classification + caption band
demo/
  evaluate_pages.py   resumable 19-page evaluation
  make_video_v3.py    demo renderer

Notes

• Engines: EasyOCR (Apache-2.0), Tesseract 5 tessdata_best (Apache-2.0), pix2tex/LaTeX-OCR (MIT), OpenCV (Apache-2.0) — no paid APIs, no TrOCR.
• For Arabic pages pass lang_hint="ar"; blind fa/ar line input is decided by the LM duel.
• Dataset: saeidseyfi/khattat