ciddwd/manga-ocr-2025-decoder-int8-pc

Manga OCR 2025 — INT8 decoder (per-channel)

This is a per-channel, dynamically quantized INT8 derivative of the FP32 decoder from l0wgear/manga-ocr-2025-onnx, intended for Japanese manga OCR experiments. No additional training or fine-tuning was performed.

This repository provides only the decoder, not a complete OCR model. A matching encoder, vocabulary, and preprocessing configuration are required. This is an independent release and does not imply endorsement by the upstream authors.

Files and checksum

File	Size	Purpose
decoder_model.int8_pc.onnx	45,275,652 bytes (approximately 45.3 MB)	Quantized decoder
LICENSE	—	Upstream Apache License 2.0 text
NOTICE	—	Attribution and modification notice
PROVENANCE.md	—	Sources, pinned revisions, quantization settings, and licensing notes
SHA256SUMS	—	Model checksum
quantize_decoder.py	—	Script documenting the quantization procedure

SHA-256 of the quantized model:

7648719ca8870795b584819f190c102b8042567bc8a9db6103b4693a6a1b1c83

The original FP32 decoder is 118,053,454 bytes. This file is approximately 38.4% of its size; not every operator or tensor has been converted to INT8.

Sources

1. Original model and project: kha-white/manga-ocr-base and kha-white/manga-ocr.
2. 2025 fine-tuned model: jzhang533/manga-ocr-base-2025.
3. Direct ONNX input: l0wgear/manga-ocr-2025-onnx.

See PROVENANCE.md for the full source and licensing record. The original model and the 2025 model declare Apache-2.0. The intermediate ONNX repository did not provide a separate license declaration or LICENSE file at the reviewed revision. This repository retains the upstream model's Apache-2.0 declaration on the basis of that provenance chain; it does not claim additional written permission from the ONNX conversion author.

Quantization

The decoder was quantized with ONNX Runtime quantize_dynamic:

quantize_dynamic(
    input_model,
    output_model,
    op_types_to_quantize=["MatMul"],
    weight_type=QuantType.QInt8,
    per_channel=True,
)

Quantization environment: ONNX Runtime 1.20.0, ONNX 1.23.2, and Python 3.13.4. Dynamic quantization does not require calibration images. The test manga pages were not used to retrain the weights.

Interface and intended use

ONNX opset: 16. The model is self-contained and does not require an additional ONNX external-data file.

Name	Type	Shape
input_ids	int64	[batch_size, decoder_sequence_length]
encoder_hidden_states	float32	[batch_size, encoder_sequence_length, 192]
logits	float32	[batch_size, decoder_sequence_length, 6144]

Use the matching encoder and vocabulary from the source ONNX repository. Similar filenames do not establish compatibility with older manga-ocr encoders or other models. Not all batch sizes, inference backends, or devices have been validated.

Two decoding approaches are possible:

• Direct INT8 decoding: this decoder generates the text directly. Its output may differ from FP32.
• Speculative decoding: this decoder proposes tokens, and the matching FP32 decoder verifies and corrects them. The FP32 decoder is still required. Downloading this file alone does not enable speculative decoding; the application must implement it.

Local device experiment

The following results summarize one Android debug A/B experiment: 13 runs and 220 crops, reusing the same FP32 encoder output for each crop. The decoding paths were executed in a fixed order: FP32 → INT8 → speculative. This was not a randomized, interleaved benchmark.

Path	Cumulative decoding time	Agreement with FP32 reference
FP32 greedy	31.083 seconds	Reference
Direct INT8 greedy	11.170 seconds	208 / 220 (text)
INT8 draft + FP32 verification	16.163 seconds	220 / 220 (token sequences)

In this experiment, speculative decoding reduced cumulative decoding time by approximately 48.0%, corresponding to a 1.92× speedup over the FP32 reference. Its timing includes drafting and verification, but excludes page detection, encoding, background repair, translation, and text layout.

The decoder benchmark used batch=1; the application's normal batch=4 recognition path was not measured by this table. The device was Android model 23116PN5BC, whose CPU reported asimddp and i8mm support. The application used ONNX Runtime Android 1.20.0. The actual low-level instruction path was not confirmed through kernel-level profiling.

These figures measure agreement with FP32, not accuracy against human annotations. Errors made by FP32 may be preserved. These results do not guarantee identical output on arbitrary images or a 1.92× end-to-end OCR speedup.

Limitations

• Direct INT8 decoding can change dialogue, sound effects, and other text. It should not be treated as a token-equivalent replacement for FP32.
• When drafts are repeatedly rejected, speculative decoding can be slower than FP32 for individual crops. One crop in this experiment took approximately 443 ms with FP32 and 1,586 ms with speculative decoding.
• The model may produce meaningless text from artwork, interface elements, or low-quality input.
• This repository does not distribute the manga images used for testing or grant rights to those images.

License and notices

The upstream model's Apache-2.0 declaration is retained. See LICENSE, NOTICE, and PROVENANCE.md. This declaration does not expand any upstream permission or warrant third-party rights in the upstream materials.