subhashhhhhh/lumen-ppocrv6-small

PP-OCRv6 small for Lumen

On-device text reading for Lumen, a local-first Android gallery: it finds the words in photos and screenshots so they can be searched. Lumen downloads these files on request from its Settings; nothing leaves the phone.

These are PaddlePaddle's PP-OCRv6 small text detection and recognition models (50 languages: Chinese, English, Japanese, Latin scripts), Apache 2.0, plus an int8 line reader and GPU conversions:

• Detection: PaddlePaddle/PP-OCRv6_small_det_onnx, commit 28fe5895c24fd108c19eb3e8479f4ab385fbfc62
• Recognition: PaddlePaddle/PP-OCRv6_small_rec_onnx, commit b8f84f0b80c529de40b4fbb3544b84fa7233a513

Files

File	What	Changes from PaddlePaddle's ONNX export
det.onnx	Text detection, input x [1, 3, H, W] (multiples of 32) → probability map [1, 1, H, W]	none (inference.onnx, renamed)
rec.onnx	Text recognition, input x [N, 3, 48, W] → [N, W/8, 18710] per-step character scores (CTC)	none (inference.onnx, renamed)
rec_int8.onnx	Text recognition, as rec.onnx, with int8 convolutions; what Lumen uses	Static per-channel int8 quantization of the convolutions only (ONNX Runtime QDQ, percentile calibration on text lines from phone screenshots), opset 17. ~1.45x faster line reading on a Snapdragon 7 Gen 1 class CPU; the same words on 30 test screenshots, small changes in spaces and stray symbols. 13 MB
rec.yml	Recognition config; character_dict is the character list (index 0 is CTC blank, a space follows the list)	none (inference.yml, renamed)
det_gpu.tflite	Text detection for LiteRT (GPU or CPU), fp32, input [1, 960, 960, 3] NHWC → text probability map [1, 960, 960, 1]	HardSigmoid rewritten as relu6(6αx + 6β) / 6 (LiteRT's GPU has no RELU_0_TO_1), then converted with onnx2tf at a fixed 960×960 input. Matches the ONNX model to within 1.4e-5. Run the GPU at fp32
rec_gpu_320.tflite, rec_gpu_640.tflite, rec_gpu_1280.tflite	Text recognition for LiteRT (GPU or CPU), fp32, 8 lines per run, input x [8, 48, W, 3] NHWC → raw character scores in two halves, logits_0 and logits_1, each [8, W/8, 9355] (classes 0–9354, then 9355–18709)	Same HardSigmoid rewrite; attention's 5-D query/key/value split replaced by slices of the 4-D tensor; the classifier (MatMul to 18,710 classes) split in two, wider than the GPU's 16,384 texture limit, and its Softmax left to the app. Converted with onnx2tf at three fixed widths (LiteRT's GPU can't resize). Character choices identical to the ONNX model; probabilities within 1.2e-4. Run the GPU at fp32. Not used by Lumen: fixed shapes (8 lines at one of three widths) cost as much padding as the GPU saved on real screenshots

SHA256SUMS lists each file's checksum; Lumen verifies them after download.

Using it

Lumen runs det.onnx and rec_int8.onnx on ONNX Runtime, and det_gpu.tflite instead of det.onnx where the phone's GPU passes a check.

Detection on the GPU (det_gpu.tflite):

• Scale the image so its long side is at most 960 px, keeping its shape, and place it in the top-left corner of a 960×960 input; leave the rest at 0.
• BGR channel order, (x / 255 - mean) / std with mean (0.485, 0.456, 0.406) and std (0.229, 0.224, 0.225) applied to B, G, R in that order. NHWC.
• Post-process as PaddleOCR's DBPostProcess: threshold 0.2, box score 0.45, unclip ratio 1.4.

Recognition on the GPU (rec_gpu_*.tflite): batches of up to 8 lines, padded with 0 (after normalizing) on the right to the smallest of the three widths that fits the widest line, and unused lines left at 0. Lines wider than 1280 px stay on rec.onnx. Join the two halves of scores per step; the best class is the character, softmax over both halves its probability.

Recognition: crop each box straightened, resize to 48 px high keeping its shape, BGR, (x / 255 - 0.5) / 0.5, then greedy CTC decoding against rec.yml's character list.

The rewrite script (gpu_friendly.py), the conversion notes and a Python reference pipeline (ocr_reference.py) are in Lumen's repository under tools/ocr/.

License

Apache License 2.0, as the original. See LICENSE and NOTICE.