genshiai-daichi/baberu-ocr

Baberu OCR

English · 日本語

Baberu OCR reads the text inside a manga speech bubble. One 115M model, three languages. On the public Japanese benchmark it beats the specialist it was distilled from (manga-ocr); on Chinese and English it goes toe-to-toe with a general model 8× its size — all measured on independent ground truth.

Most manga OCR is Japanese-only and assumes clean horizontal lines. Real manga bubbles are not that. They run vertical, pack in sound effects, switch scripts mid-sentence, and mix full-width and half-width characters. Baberu was trained for exactly that, in Japanese, Chinese, and English at once.

At a glance

• Three languages, one checkpoint (ja / zh / en). Lowest character-error-rate of every model compared on Japanese; on Chinese and English it matches a model 8× its size on the typical bubble, and is more robust.
• 115M parameters, about one-eighth the size of the 0.9B vision-language models it goes toe-to-toe with.
• Character-level (14,630-symbol vocab): sound effects, rare kanji, and full/half-width mixing stay one token each, with no subword fragmentation.
• Local and small. Apache-2.0, shipped as a 121 MB or 242 MB ONNX build that needs only onnxruntime, numpy, and pillow.

>>> from inference import BaberuOCR
>>> ocr = BaberuOCR(".")
>>> ocr(Image.open("bubble.jpg"))
'私がこの手で啓太を守る！！'

How it works

DINOv2 looks, a small decoder writes.

The decoder is built from scratch: 6 layers, grouped-query attention, SwiGLU, RMSNorm sandwich, 1D RoPE, tied embeddings. Nothing exotic. What mattered was the training order.

Freeze the vision encoder, train the decoder, then unfreeze the encoder and keep training. The accuracy jumps in that second stage, and it saturates fast: around 30k steps, roughly a tenth of an epoch. Character-level output fits the task — the target is a sequence of characters — and avoids the uneven, ballooning tokenizations that subword vocabularies produce on sound effects and mixed scripts (manga-ocr is character-level for the same reason).

Benchmarks

Teacher labels would grade their own homework, so every number here is measured against independent ground truth, on images the model never trained on. Japanese uses the public human-labeled Manga109-v2026. For Chinese and English we built a fresh held-out set from galleries published after the training cutoff (IDs above the training maximum — zero overlap) and re-annotated every crop by hand, with the teacher output hidden. The metric is lCER (lenient character-error-rate: NFKC, whitespace stripped, decorative symbols dropped, dot-runs unified), lower is better.

Japanese, Manga109-v2026 (n=2000)

model	size	lCER ↓	nCER ↓
Baberu OCR	115M	0.0345	0.0871
PaddleOCR-VL	0.9B	0.0368	0.0808
manga-ocr (its teacher)	110M	0.0422	0.0954
PP-OCRv5	~10M	0.6317	0.6491

On Japanese, Baberu beats its actual teacher, manga-ocr, on every metric, and edges the 0.9B on lenient CER. The 0.9B stays ahead on the stricter nCER and on exact-match — it is eight times larger. PaddleOCR-VL-For-Manga posts a lower Japanese number but trained on Manga109, so that score is a data leak and sits out.

Chinese and English, hand-annotated held-out

Two ways to count, because they disagree and the disagreement is the result. Per-bubble averages the error over each bubble (the unit a bubble reader delivers); char-weighted weights by total characters (long passages dominate). PaddleOCR-VL is Baberu's Chinese/English teacher.

metric	zh: Baberu 115M	zh: PaddleOCR-VL 0.9B	en: Baberu 115M	en: PaddleOCR-VL 0.9B
per-bubble lCER ↓	0.052	0.077	0.062	0.081
char-weighted lCER ↓	0.043	0.030	0.033	0.028
exact-match ↑	0.85	0.83	0.82	0.79
n	246		237

Per bubble, the 115M student wins both languages and gets more bubbles exactly right; it also never melted into a repetition loop, while the 0.9B did on two short English bubbles. Per character, the 0.9B wins both — it reads long passages better. Call it toe-to-toe with a model 8× its size: ahead and more reliable on the typical bubble, behind on long text.

Where it loses

Public, never-seen gold labels from COMICS Text+ (1940s Western comics, n=480):

model	size	lCER ↓
PaddleOCR-VL	0.9B	0.0256
PP-OCRv5	~10M	0.1332
Baberu OCR	115M	0.1488

On vintage Western comics it has never seen, the general 0.9B model reads cleaner. Baberu is tuned for modern manga, and this gap marks the edge of that focus rather than hiding it.

Quick start

The input is one speech-bubble crop (run a text detector upstream), not a full page.

ONNX, the small path (no PyTorch):

pip install onnxruntime numpy pillow
huggingface-cli download genshiai-daichi/baberu-ocr --local-dir baberu-ocr
cd baberu-ocr

python onnx_infer.py --image bubble.jpg                          # 242 MB, lossless
python onnx_infer.py --vision vision_int4.onnx --image bubble.jpg # 121 MB, smallest

onnx_infer.py carries its own preprocessing, KV-cache decode loop, and the published decode settings (greedy, repetition_penalty=1.2, and a symbol-aware content-run cap), so its output matches the PyTorch model.

PyTorch, for fine-tuning or transformers:

pip install torch transformers safetensors pillow
huggingface-cli download genshiai-daichi/baberu-ocr --local-dir baberu-ocr
cd baberu-ocr
python inference.py --model . --image bubble.jpg

To fine-tune the released weights on your own bubbles, see TRAINING.md: train_ocr.py --finetune-from . continues training with a fresh optimizer (add --unfreeze-vision to adapt the encoder too). The full from-scratch recipe (train_text.py → train_ocr.py) is included as well.

Sizes and quantization (ONNX)

component	fp16	int8	int4
vision (86M)	173 MB	92 MB	52 MB
decoder (prefill+step)	n/a	69 MB	(89 MB)

Two tiers are recommended, both with the decoder at int8:

tier	files	size	nCER (ja)
lossless	vision_fp16.onnx + decoder_*_int8.onnx	242 MB	0.0867
smallest	vision_int4.onnx + decoder_*_int8.onnx	121 MB	0.0893

The measured sweep (quantization-results.md) found two things worth knowing. Decoder int8 is free here, 0.0867 against fp32's 0.0871, because greedy decoding shrugs off weight-quantization noise. The cost lives entirely in vision, and even there int4 weight-only beats int8-dynamic on both size and accuracy. Both tiers still clear the teacher manga-ocr (0.0954). The dominated variants (vision int8-dynamic, decoder int4) are not shipped.

Limitations

• One bubble crop in, text out. You supply the text-detection stage. This is not page-level OCR.
• It confuses dakuten and handakuten (ポ vs ボ), misses some rare kanji, and drops the occasional small kana.
• On print far outside its domain, a general large VLM reads better.

Training data and license

Trained on an independently collected corpus of manga speech-bubble crops, with text labels distilled from manga-ocr (Japanese) and PaddleOCR-VL (Chinese, English). The training images are not redistributed; only the weights and code ship here. License is Apache-2.0 for the weights and code, matching the upstream components (DINOv2, manga-ocr, and PaddleOCR are all Apache-2.0).

Acknowledgements

• DINOv2 (Meta), the vision encoder
• manga-ocr, the de-facto Japanese manga OCR baseline and the Japanese teacher
• PaddleOCR-VL, the Chinese and English teacher