Unlimited-OCR-2 — LiteRT-LM (OCR)

On an iPhone 18 Pro this file reads a page in about 11 s on the CPU, and on a Galaxy S26 in 19–21 s, engine creation included (2026-11-04). This repository is a conversion of baidu/Unlimited-OCR-2 to LiteRT-LM (.litertlm), made with hf-to-litertlm for Google's LiteRT-LM 0.19.0.

Unlimited-OCR-2 (Baidu, 2026-10-27) is a 3.3B document-parsing model: a vision encoder (DeepEncoder: SAM ViT-B and CLIP-L, with a projector; 401M parameters) and a 12-layer DeepSeek-V2 text decoder (2.9B parameters; the first layer dense, then 11 mixture-of-experts layers of 64 routed experts, 6 per token, and 2 shared) with DASH, a block-sparse attention: for each query a router picks 12 blocks of 32 positions to attend to. It writes a page as its layout blocks, each with a label (title, text, table, equation, …), a box and the block's content (tables as HTML, formulas as LaTeX). The original can read many pages in one request. This file reads one page per request, as one 1,024 × 1,024 view of 273 image tokens: the view the original takes of each page of a multi-page document.

File	What is inside	Size	sha256
Unlimited-OCR-2_int8.litertlm	text decoder with int8 weights in its linear layers (the DASH router's query and key projections and the position table stay float), fp32 activations declared for the GPU; int8 embedding table; vision encoder and projector with fp16 weights	3.81 GB	12012b6d8be7820553d5eb01baebba03869fbb40c60fe04f60bf1bdfefd3784a

Which backend. The file holds two models: a vision encoder that reads the picture and a text decoder that writes the text. Each can run on the CPU or the GPU.

• Use the CPU on Android. On the Galaxy S26 a page took 19–25 s on the CPU against 25–31 s with the decoder on the GPU. With the decoder and the vision encoder both on the Galaxy S26's GPU, the available memory fell to 0.82 GiB and our memory guard stopped the process (see Known runtime issues). With the vision encoder on the CPU, the GPU decoder missed the receipt photo and the dense-equation page, which the CPU read like the original (see Correctness).
• Use the CPU on the iPhone. On an iPhone 18 Pro the CPU was faster than the GPU (Metal) once the runtime's cache files were written (with an empty cache the CPU was slower: 14.5 s against 13.8 s for a paragraph) and used half the memory (see Performance). Each missed one photo that the other read like the original: the GPU the receipt photo, the CPU the form photo (see Correctness).
• Use the GPU on the Mac: it passes 18 of our 19 tests and decodes a page at 98–99 tok/s, against 17 tests and 55–56 tok/s on the CPU.
• 8 GB Android phones: we could not run an earlier build of this file at a usable speed on a Pixel 8a (see Known runtime issues).

How to use

ocr_unlimited.py pads your pages and runs the file with litert-lm:

pip install litert-lm pillow huggingface_hub
hf download litert-community/Unlimited-OCR-2 --include "Unlimited-OCR-2_int8.litertlm" --include "samples/*" --local-dir uocr
curl -LO https://raw.githubusercontent.com/john-rocky/hf-to-litertlm/main/unlimitedocr2_work/ocr_unlimited.py
python ocr_unlimited.py uocr/Unlimited-OCR-2_int8.litertlm uocr/samples/page_en.png
python ocr_unlimited.py uocr/Unlimited-OCR-2_int8.litertlm uocr/samples/table.png uocr/samples/formula_2.png --backend gpu

The first request on a device writes the runtime's cache files next to the model (about 4.4 GB for the CPU, 2.8 GB for the GPU) and takes the (cold) times in Performance; later requests are faster.

Why the padding: LiteRT-LM resizes every picture to 1,024 × 1,024 without keeping its aspect ratio and without reading the EXIF orientation. So the script applies the EXIF orientation and then centres the page on a gray (127, 127, 127) square, as the original does with each page of a multi-page document (the original's code reads the pixels as stored, without the EXIF orientation). Without the padding a non-square page is stretched to a square; our tests were not run that way.

The model takes no prompt of your choosing: the file's template writes <PAGE>, the page and the model's instruction Multi page parsing.. --backend gpu puts the decoder and the vision encoder on the GPU (--vision-backend cpu keeps the vision encoder on the CPU), --max-tokens caps each page's answer, --cache-dir moves the runtime's cache files and --no-cache writes none (see Known runtime issues). A page's prompt takes 279 of the 4,096 tokens the file holds, so a page's answer can be up to 3,817 tokens long (the longest answers in our tests, on the dense-equation page, were about 1,600 tokens). Several pages are read one request each, in order, and their texts are printed one after the other, each starting with <PAGE>.

The output is the model's raw text with its tags: <PAGE>, then for each block <BLOCK>label[x1, y1, x2, y2]<FORK>content. The box is on a 0–999 scale of the page as you gave it, before the padding (checked on our drawn page and table, where we know where the text was drawn).

From your own code, pad the page the same way (page_picture in the script), send it alone in one user message, start a new conversation for every page, and decode as the script does: the most likely token at each step (the file sets no sampler, so LiteRT-LM 0.19.0 takes top-p with k = 1) with NoRepeatNgramConfig(35, 128), the original's no-repeat setting for one page.

Correctness

The bar: for each test case, the answer must match what the fp32 original model writes from the same page (transformers, greedy, one 1,024 × 1,024 view per page, the original's own code). Match means: character error rate ≤ 0.02 for text (≤ 0.05 for photos and pages), at least 18 of 20 table cells, the same LaTeX, every receipt field, and for the 3-page document three <PAGE> marks and a character error rate ≤ 0.05 over the whole text. There are 19 cases. 14 are regions: a text line, a title, three paragraphs (plain, small, photographed), a Chinese paragraph, a table, two formulas, a drawn receipt, a receipt photo counted twice (once read as text, once checked for its fields), a calligraphy sample and a second receipt photo. 4 are pages: a drawn page, a dense-equation page, a handwritten derivation and a form photo. The last case is 3 consecutive pages of a MarathonDocBench document. The receipt photo, the dense-equation page, the handwritten derivation and the calligraphy sample are the example pictures of the Xiaomi-OCR-0 card; the second receipt photo and the form photo come from the transformers documentation.

Device	Backend	Regions that pass (of 14)	Pages that pass (of 4)	3-page document
Mac M4 Max	GPU	13	4	pass
Mac M4 Max	CPU	13	3	pass
Galaxy S26	CPU	13	3	pass
Galaxy S26	GPU decoder, CPU vision encoder	8 of the 10 we ran	2 of the 4 we ran	not run
iPhone 18 Pro	CPU	13	2	pass
iPhone 18 Pro	GPU (Metal)	12	3	pass

LiteRT-LM 0.19.0 everywhere (the Python API on the Mac, the PyPI Android wheel on the Galaxy S26, the iOS framework of the 0.19.0 release on the iPhone), one process per case, and the 3-page document read one page per request.

Each miss starts at a token where the original itself is nearly undecided: its two best candidates are 0.0075 to 0.25 logit (raw score) apart, and the file writes another token there. From that token on, the answer takes another path through the page. All but two of the misses start in the first block's label or box.

• The calligraphy sample is a near tie. For the first block's label the original writes title, 0.056 logit ahead of text. The Mac GPU, the Galaxy S26 CPU and GPU decoder, and the iPhone 18 Pro CPU and GPU write text and then read the page differently (character error rate 0.51 to 1.57 against the original). The Mac CPU writes title and leaves the original at its 101st token, where the margin is 0.15 (character error rate 0.45).
• The handwritten derivation. For the first number of the first box the original writes 78, 0.0075 logit ahead of 76. The Mac CPU and the iPhone 18 Pro CPU write 76; the iPhone 18 Pro GPU and the Galaxy S26 GPU decoder write 75. The Galaxy S26 CPU follows the original to the box's last number and writes 188 where the original writes 190, 0.16 ahead of 191. The Mac GPU passes it (character error rate 0.03 against the original).
• The receipt photo on the phone GPUs. For the first number of the first box the original writes 272, 0.25 logit ahead of 269. The iPhone 18 Pro GPU and the Galaxy S26 GPU decoder both write 269, and then the same block texts as each other (character error rate 0.13 against the original). The Mac GPU and every CPU we ran it on (Mac, Galaxy S26, iPhone 18 Pro) write 272.
• The form photo on the iPhone 18 Pro CPU. After Name/Phone Ext.: the original writes M, 0.20 logit ahead of a space. The iPhone 18 Pro CPU writes the space and then wraps that field and three other filled-in fields in LaTeX underlines ($ \underline{\text{…}} $). Without that markup its words are the original's, in the same 12 blocks with boxes within 1 of the original's on the 0–999 scale; with it the character error rate is 0.0975 against the bar's 0.05.
• The dense-equation page on the Galaxy S26 GPU decoder misses the bar by a hair: character error rate 0.0505 against 0.05. For the second number of the first box it writes 8 where the original writes 6, 0.17 logit ahead of 7.

Performance

One request = one process (one app launch on the iPhone) with a new Engine, timed inside the process from engine creation to the last token: engine creation, vision encode, prefill and decode. "Paragraph" = a drawn English paragraph (279 tokens in, 110 out). "Page" = the drawn English page of samples/page_en.png (279 tokens in, 256 out). First token = from the request to the first token of the answer (vision encode and prefill). Three requests per cell (two for the Galaxy S26 GPU), fastest first; (cold) = the runtime's cache directory was empty. Bold = the fastest form on that device.

Galaxy S26 (SM8850)	Paragraph request (s)	Page request (s)	Engine (s)	First token (s)	Decode (tok/s)	Peak (GiB)
CPU	12.4 / 12.6 / 17.9 (cold)	18.9 / 20.7 / 24.8 (cold)	1.64–8.28	4.2–6.1	19.3–20.5	4.4–6.1
GPU decoder, CPU vision encoder	—	24.8 / 30.8 (cold)	6.89–12.16	3.5–4.2	17.3–17.4	3.5–4.0

iPhone 18 Pro	Paragraph request (s)	Page request (s)	Engine (s)	First token (s)	Decode (tok/s)	Peak (GiB)
CPU	6.3 / 6.8 / 14.5 (cold)	10.5 / 10.7 / 11.6	0.15–4.40	2.3–6.6	29.7–31.7	2.1–2.3
GPU (Metal)	7.8 / 7.9 / 13.8 (cold)	12.5 / 12.6 / 13.4	0.31–5.04	4.1–5.4	28.0–32.0	4.6–4.7

On the Galaxy S26, Peak is the process's peak resident memory (VmHWM); on the GPU row it counts only the part of the GPU's memory that the Adreno driver maps into the process. The phone was at Android thermal status 1 (light) for the first request and 2 (moderate) for the others, with the CPU clocks capped from the second request on. On the iPhone, Peak is the app's lifetime peak phys_footprint as iOS reports it, and the phone stayed at the nominal thermal state.

On a Mac M4 Max (LiteRT-LM 0.19.0 Python API, the runtime's cache in place) a page request took 4.3–4.4 s on the GPU (decode 98–99 tok/s) and 6.3 s on the CPU (decode 55–56 tok/s), two processes each.

Conversion notes

• Source: baidu/Unlimited-OCR-2 at revision 04809669 (3,343,978,497 parameters in the checkpoint: 401,372,417 in the vision encoder and projector, 2,942,606,080 in the text decoder).
• Text decoder: 2 signatures, the graph's entry points: a prefill signature of 280 positions for the 279-token one-page prompt, and decode for one token at a time. A 4,096-token context. The embedding table in its own graph, for LiteRT-LM's fast_vlm path. DASH (blocks of 32 positions; a router that keeps 12 blocks) is written out for the one-page layout: every query has at most 9 candidate blocks, fewer than the 12 the router keeps, so the selection drops nothing and the graph needs no top-k or gather. The KV cache has 416 slots: positions 0–287 stay, and a ring holds the last 128 positions, because one-page decoding reads nothing else; the caches are declared local, the runtime's type for sliding-window and ring caches (a global cache of 416 slots would cap the context at 416), so the runtime keeps the 4,096-token context. Each mixture-of-experts layer computes all 64 routed experts as two fully connected ops and keeps the 6 the router picks (the others get weight 0). Dynamic int8, channelwise, on the linear layers (the DASH router's query and key projections and the position table stay float) and on the embedding; fp32 activations declared for the GPU. Toolchain: litert-torch 0.9.3, litert-converter 0.4.0, ai-edge-quantizer 0.9.0, ai-edge-litert 2.2.0, torch 2.13.0.
• Vision: the encoder and projector re-authored for a fixed 1,024 × 1,024 picture, exported with the same toolchain, weights cast to fp16. On 7 pictures the fp32 export's output correlates at 0.9999999997 or higher with the checkpoint's own modules, and the fp16 copy's at 0.99999993 or higher with the fp32 export.
• Template: <PAGE>, the picture and Multi page parsing., the original's multi-page prompt for one page; start token <｜begin▁of▁sentence｜>, stop token <｜end▁of▁sentence｜>. The tokenizer is the repository's own tokenizer.json.
• Packing: litert-lm-builder 0.19.0.
• Reproduction: REPRODUCE.md · machine-readable facts: litertlm_manifest.json.

Known runtime issues

• One page per request. The graph is built for one page's fixed prompt (start token, <PAGE>, 273 image tokens, 4 instruction tokens), so several pages are several requests and the model does not see the other pages. The original reads a document's pages in one request and can join blocks that continue across pages. On our 3-page document, reading page by page gave the original's 3-page answer within a character error rate of 0.0014.
• One conversation per page. The graph is built for a page's prompt as the first turn of a conversation. From the second turn on, LiteRT-LM also puts the previous answer's stop token into the KV cache with a zero embedding, because this file holds the embedding table in its own section and the decoder takes embeddings, so later answers in a conversation can differ from the answer the model gives to the same messages in one prompt (read in the source of LiteRT-LM 0.16.0 to 0.19.0; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on this file). Start a new conversation for every page, as the script does.
• Not the original's single-page mode. For a single page the original reads tiles by default (its gundam mode). This file always reads one 1,024 × 1,024 view per page, the original's mode for multi-page documents, and our bar compares against that mode.
• All 64 experts on every token. LiteRT's MoE operation, which computes only the chosen experts, does not yet take this model's SiLU experts (LiteRT #11192 and #11188, open on 2026-11-04). So the decoder computes all 64 and weights them by the router: the same answer, more work per token.
• The runtime's cache files are large. The first run writes them next to the model: about 4.4 GB for the CPU and 2.8 GB for the GPU (about 7.2 GB if you run both). Keep them: without the cache (--no-cache) one page peaked at a 9.1 GB memory footprint on the Mac CPU instead of 2.6 GB, and at 7.9 GB on the Mac GPU instead of 5.9 GB.
• 8 GB Android phones. On a Pixel 8a (8 GB), with an earlier build of this file (the same weights with a 4,096-slot KV cache), the phone's available memory ran out as soon as the decoder was on the GPU, and the CPU decoded at 0.30 tok/s while the phone swapped.
• Galaxy S26 GPU. With the decoder and the vision encoder both on the GPU, the phone's available memory fell from 7.4 GiB at the start of the process to 0.82 GiB, about 3 GiB of it while the vision encoder was being created, and our memory guard stopped the process. With the vision encoder on the CPU it runs. In one of our 32 GPU processes on the Galaxy S26 (2026-11-04) the Adreno driver ended the process while it submitted GPU work (os_exit in cl_a8x_cmdbuf_mgr_submit_ibs).
• No NPU path. On the Galaxy S26, with the earlier build, the Qualcomm compiler refused the decoder's KV cache writes (24 DynamicUpdateSlice ops not supported, 24 ElementWiseSelect ops failing QNN validation with error 3110), and this file carries no ahead-of-time NPU section.
• Tool calls are not parsed. The template renders no tool definitions, and this file's metadata carries no tool-call config.

License and changes

MIT, inherited from baidu/Unlimited-OCR-2. Changes from the original work: exported to the LiteRT-LM format for one 1,024 × 1,024 page per request; DASH written out for that layout, with a KV cache of 288 kept and 128 ring slots; each mixture-of-experts layer computed over all experts with the router's weights; the text decoder's linear layers and the embedding quantized to int8; the vision encoder and projector cast to fp16; a prompt template for one page. The sample pictures in samples/ were drawn with the fixture script of hf-to-litertlm (DejaVu fonts). Problems with these files: open a discussion in this repo.