MNN-Pocket-0.3B — LiteRT-LM

This repository is a conversion of taobao-mnn/MNN-Pocket-0.3B to LiteRT-LM for LiteRT-LM 0.19.1: on a Galaxy S26, after a 1,024-token prompt, the SM8850 file (the NPU file compiled ahead of time for that phone's chip) decodes 157.7 tokens/s and creates its engine in 0.27 s, and the NPU file decodes 147.3 tokens/s (2026-11-07); the 2,048-token file decodes 120.0 tokens/s on the GPU and 87.4 on the CPU (2026-11-06).

MNN-Pocket-0.3B is the MNN team's general-purpose instruction-tuned model for on-device use: about 300M parameters in 28 layers (hidden size 896, 14 attention heads sharing 2 key-value heads), a 65,536-token Chinese and English vocabulary, and tied input and output embeddings. It chats in a ChatML format with tool definitions and has no separate thinking block. The source card describes it as intended for on-device assistants, data extraction and lightweight agentic pipelines, and not recommended for reasoning-heavy work such as advanced mathematics or long-horizon code generation. The authors also publish an int4 build for their MNN runtime (MNN-Pocket-0.3B-MNN), GGUF files and a technical report.

Files

File	What is inside	Size	sha256
MNN-Pocket-0.3B_int8_ekv2048.litertlm	int8 weights (one scale per output channel) with float activations, fp16 on the GPU; KV cache of 2,048 tokens; prefill signatures of 128 and 1,024 tokens	320,025,488 B	3d4710d406870fd0a65169503851335256445580999f56a9c219da0e51e49315
MNN-Pocket-0.3B_int8.litertlm	The same with a KV cache of 4,096 tokens	320,025,488 B	092a838999e99f4ab03239ca8de1c0caba01d1772d7a22909e76988b0b3dc1e9
MNN-Pocket-0.3B_qualcomm_npu_ekv2048.litertlm	For the NPU of Snapdragon phones: int8 weights (one scale per output channel) with 16-bit integer activations; KV cache of 2,048 tokens; a prefill signature of 128 tokens	374,213,768 B	e8af457e3d2840a1c5b35f66e7e8bf17d1c88212608f2ec0c374ef01966060a0
MNN-Pocket-0.3B_qualcomm_sm8850_ekv2048.litertlm	The NPU file compiled ahead of time for the Snapdragon SM8850 only (the Galaxy S26's chip), so the phone does not compile it	380,374,152 B	ab3aa3aa48a27f2170caf025deb59dc5f2e6c392ff6968d73100f3a02c178138
litertlm_manifest.json	Machine-readable description of the files	39,485 B

The two int8 files hold the same weights and differ only in the KV cache's length. The 2,048-token file decodes faster everywhere but on the Mac GPU, where the two tie (Performance); take the 4,096-token file when a conversation needs more than 2,048 tokens, prompt and reply together. On the phone CPUs the 4,096-token file decodes 2 % (Galaxy S26) and 7 % (Pixel 11 Pro) slower than the first upload's file, re-measured in the same session, and its peak memory is about 0.76 GB higher.

The NPU file is for the NPU of Snapdragon phones, and we ran it on one, the Galaxy S26. The ranges of its 16-bit activations were measured on 48 prompts: 32 questions from GSM8K's training split, in the 4-shot form of the GSM8K check, and the first 16 of litert-torch's sample calibration prompts. The phone compiles the file for its NPU each time an engine is created. LiteRT-LM 0.19.1's CPU backend does not open it. The SM8850 file is the same file compiled ahead of time for the Snapdragon SM8850 with QAIRT 2.47.0.260601: the phone loads the compiled code, so an engine is created in 0.27 s instead of 27 to 29 s. It is compiled for that chip only, and we have run it only on that chip; a phone with another Snapdragon chip should take the NPU file.

The two int8 files carry the source repository's tokenizer.json byte for byte, the source's chat template with two changes (see Conversion notes), the stop tokens <|im_end|> (65512) and <|endoftext|> (65510) from the source's generation_config.json, the sampling settings of the source card's example (top-k 20, top-p 0.8, temperature 0.8), and the model type generic_model. The input embedding and the output layer share one int8 table, as they share one matrix in the source. The NPU file carries the same tokenizer, chat template, stop tokens and sampling settings, but declares the model type qwen2p5 and holds the int8 table twice, with the same values: once for the input embedding and once for the output layer. The SM8850 file holds every section of the NPU file byte for byte except the main graph, which it holds compiled.

Usage

One question with the LiteRT-LM command-line tool (uvx comes with uv; with pip, pip install litert-lm==0.19.1 and run litert-lm run … instead):

pip install -U huggingface_hub
hf download litert-community/MNN-Pocket-0.3B MNN-Pocket-0.3B_int8_ekv2048.litertlm --local-dir .
uvx --from litert-lm==0.19.1 litert-lm run MNN-Pocket-0.3B_int8_ekv2048.litertlm --backend cpu --prompt "Give me three tips for writing a clear email." < /dev/null

The same from Python (pip install litert-lm==0.19.1):

import litert_lm
from litert_lm import interfaces

engine = litert_lm.Engine("MNN-Pocket-0.3B_int8_ekv2048.litertlm", backend=interfaces.Backend.CPU())
conversation = engine.create_conversation()
reply = conversation.send_message("Give me three tips for writing a clear email.")
print("".join(part["text"] for part in reply["content"] if part["type"] == "text"))

--backend gpu (or interfaces.Backend.GPU()) runs either int8 file on the GPU. Both pass the checks under Accuracy on the Mac's GPU and on the GPUs of the two phones we tested (the 4,096-token file's phone GPU checks ran on a build with the same four sections).

The NPU file runs with LiteRT-LM's command-line tool for Android. LiteRT-LM 0.19.1 releases no Android build of the tool or of its NPU libraries, so build them from the v0.19.1 tag as on LiteRT-LM's NPU page, with Qualcomm's compiler plugin added (this file is compiled on the phone), and take the QNN libraries from Qualcomm's QAIRT 2.47.0.260601, the zip that ai-edge-litert-sdk-qualcomm 2.2.0 downloads. What we ran for the Galaxy S26, on a Mac (Apple silicon) with Bazelisk, Git LFS and ANDROID_NDK_HOME set to the Android NDK 29.0.13113456 (build, download the model, push, run):

curl -L -o v2.47.0.260601.zip https://softwarecenter.qualcomm.com/api/download/software/sdks/Qualcomm_AI_Runtime_Community/All/2.47.0.260601/v2.47.0.260601.zip
unzip -q v2.47.0.260601.zip
export QAIRT_ROOT=$PWD/qairt/2.47.0.260601
git lfs install
git clone --branch v0.19.1 https://github.com/google-ai-edge/LiteRT-LM.git
cd LiteRT-LM
bazel build --config=android_arm64 --repo_env=LITERT_QAIRT_SDK=$QAIRT_ROOT \
  //runtime/engine:litert_lm_main \
  @litert//litert/vendors/qualcomm/dispatch:dispatch_api_so \
  @litert//litert/vendors/qualcomm/compiler:qnn_compiler_plugin_so

hf download litert-community/MNN-Pocket-0.3B MNN-Pocket-0.3B_qualcomm_npu_ekv2048.litertlm --local-dir .

export DEVICE_FOLDER=/data/local/tmp/mnn_pocket
adb shell mkdir -p $DEVICE_FOLDER
adb push MNN-Pocket-0.3B_qualcomm_npu_ekv2048.litertlm $DEVICE_FOLDER/model.litertlm
adb push $QAIRT_ROOT/lib/aarch64-android/libQnnHtp*Stub.so $DEVICE_FOLDER
adb push $QAIRT_ROOT/lib/aarch64-android/libQnnHtp.so $DEVICE_FOLDER
adb push $QAIRT_ROOT/lib/aarch64-android/libQnnSystem.so $DEVICE_FOLDER
adb push $QAIRT_ROOT/lib/aarch64-android/libQnnHtpPrepare.so $DEVICE_FOLDER
adb push $QAIRT_ROOT/lib/hexagon-*/unsigned/libQnnHtp*Skel.so $DEVICE_FOLDER
adb push bazel-bin/external/litert/litert/vendors/qualcomm/*/*.so $DEVICE_FOLDER
adb push prebuilt/android_arm64/*.so $DEVICE_FOLDER
adb push bazel-bin/runtime/engine/litert_lm_main $DEVICE_FOLDER

adb shell LD_LIBRARY_PATH=$DEVICE_FOLDER ADSP_LIBRARY_PATH=$DEVICE_FOLDER \
  $DEVICE_FOLDER/litert_lm_main \
  --backend=npu \
  --model_path=$DEVICE_FOLDER/model.litertlm \
  --input_prompt="'What is 17 + 25? Answer briefly.'"

Each new engine compiles the file for the NPU before the first reply: 27 to 29 s on a Galaxy S26 whose CPU was not capped when the engine started, longer once earlier compiles have capped it. LiteRT-LM 0.19.1 keeps no compiled copy from one process to the next. On the NPU, LiteRT-LM 0.19.1 picks the most likely token at every step, so the file's sampling settings do not apply there.

The phone does not compile the SM8850 file, and it runs without the compiler plugin. After the lines above, on a phone with the Snapdragon SM8850:

hf download litert-community/MNN-Pocket-0.3B MNN-Pocket-0.3B_qualcomm_sm8850_ekv2048.litertlm --local-dir .
adb push MNN-Pocket-0.3B_qualcomm_sm8850_ekv2048.litertlm $DEVICE_FOLDER/model_sm8850.litertlm
adb shell rm $DEVICE_FOLDER/libLiteRtCompilerPlugin_Qualcomm.so
adb shell LD_LIBRARY_PATH=$DEVICE_FOLDER ADSP_LIBRARY_PATH=$DEVICE_FOLDER \
  $DEVICE_FOLDER/litert_lm_main \
  --backend=npu \
  --model_path=$DEVICE_FOLDER/model_sm8850.litertlm \
  --input_prompt="'What is 17 + 25? Answer briefly.'"

Removing the compiler plugin is optional: with it left in the folder, the SM8850 file gave the same reply, but the engine took 0.37 s instead of 0.27 s while the plugin tried to compile the small graphs for the NPU, and QNN printed errors (one run each). Push the compiler plugin back before running the NPU file again.

On Android, MNN-Pocket-0.3B_int8_ekv2048.litertlm is in our model list for OlliteRT, an Android app that serves a model on the phone over an OpenAI-compatible API. In OlliteRT 0.9.7-beta.1, which we ran on a Pixel 11 Pro, it runs on the CPU only: the app's LiteRT-LM 0.16.1 does not run this file on the GPU.

Performance

Phones, LiteRT-LM 0.19.1 (the litert-lm-api 0.19.1 Android wheel from PyPI, its Benchmark class): a forced prefill of 1,024 tokens, then 256 decoded tokens, context 2,048, three runs per process. Each cell is the mean of runs 2 and 3 in two processes that had loaded the runtime's cache; peak memory is the process's VmHWM (KiB, as /proc reports it); 2026-11-06 (Galaxy S26) and 2026-11-07 (Pixel 11 Pro).

Phone	File	Backend	Prefill (tokens/s)	Decode (tokens/s)	Time to first token	Peak memory
Galaxy S26	2,048-token	GPU	5,984	120.0	0.18 s	0.39 GB (384,616 KiB)
Galaxy S26	2,048-token	CPU	697	87.4	1.50 s	1.49 GB (1,454,428 KiB)
Galaxy S26	4,096-token	GPU	5,960	105.6	0.18 s	0.37 GB (365,752 KiB)
Galaxy S26	4,096-token	CPU	333	53.2	3.16 s	2.43 GB (2,375,828 KiB)
Pixel 11 Pro	2,048-token	CPU	247	76.4	4.19 s	1.33 GB (1,300,808 KiB)
Pixel 11 Pro	2,048-token	GPU	2,864	33.3	0.39 s	0.29 GB (287,840 KiB)
Pixel 11 Pro	4,096-token	CPU	163	45.4	6.48 s	2.16 GB (2,107,928 KiB)
Pixel 11 Pro	4,096-token	GPU	2,781	28.3	0.40 s	0.29 GB (285,700 KiB)

On the Galaxy S26 the GPU is the faster backend; on the Pixel 11 Pro the CPU decodes faster and the GPU prefills faster. Both phones lowered their CPU clocks within the first run of every CPU process, so the CPU figures are taken at those lowered clocks. On the Pixel 11 Pro the third run of every 4,096-token CPU process, our first upload's file included, decoded 26 to 40 % slower than the second, so the four runs behind the 45.4 tokens/s cell span 35 to 59 tokens/s.

The NPU file and the SM8850 file on the Galaxy S26 (Snapdragon SM8850), 2026-11-07: the same 1,024 prompt tokens and 256 decoded tokens, context 2,048, three runs in each of two processes. Each cell is the mean of runs 2 and 3; engine creation is the mean of the two processes' engine set-up, the NPU file's compile included; peak memory is the larger VmHWM. The NPU file's processes started with no CPU clock cap; the phone capped the CPU clocks during the compile and lowered them further while the three runs went on, and the embedding lookup and the small graphs named below the table run on the CPU. The SM8850 row's session is described below the table. The tool is litert_lm_advanced_main, the variant of the command-line tool in the same v0.19.1 tag that takes the benchmark flags (add //runtime/engine:litert_lm_advanced_main to the bazel line above), with the Qualcomm libraries above and these flags: --backend=npu --model_path=$DEVICE_FOLDER/model.litertlm --input_prompt_file=$DEVICE_FOLDER/bench.txt --max_num_tokens=2048 --max_output_tokens=256 --async=false --use_session=false --benchmark --benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256 --num_iterations=3 (bench.txt is a one-line text file; the benchmark flags set the 1,024 prompt tokens).

Phone	File	Backend	Prefill (tokens/s)	Decode (tokens/s)	Time to first token	Engine creation	Peak memory
Galaxy S26	NPU	NPU	10,802	147.3	0.10 s	29.2 s	3.43 GB (3,346,984 KiB)
Galaxy S26	SM8850	NPU	11,199	157.7	0.10 s	0.27 s	0.17 GB (162,468 KiB)

The SM8850 row comes from a later session on the same phone, with the same tool and flags (the SM8850 file's processes without the compiler plugin), where the two files ran two processes each in the order SM8850, NPU, NPU, SM8850. There the NPU file decoded 147.5 tokens/s, prefilled 10,994 and created its engine in 27.6 s. The NPU file's processes ran with the CPU clocks capped from the first seconds of the compile, down to 2.50 of 3.63 GHz and 2.44 of 4.74 GHz during the runs; the SM8850 file's were capped less, down to 3.40 and 3.76 GHz. So part of the difference in decode speed may come from the CPU, which runs the embedding lookup and the small graphs, and part from the two compiles, which differ (Conversion notes); we have not measured either. Peak memory is the process's own: the phone's available memory (MemAvailable) stayed at 6.68 GB (6,519,664 KiB) or more during the SM8850 file's runs, and fell to 4.12 GB (4,025,640 KiB) while the NPU file was compiled (5.82 GB, 5,682,232 KiB, during its runs).

The main graph runs whole on the NPU: all 1,477 operations of the 128-token prefill signature and all 1,376 of decode. The embedding lookup and the small graphs for the rotary position, the attention mask and the cache update run on the CPU.

Mac M4 Max (128 GB), LiteRT-LM 0.19.1 from PyPI, litert-lm benchmark <file> -p 1024 -d 256 --max-num-tokens 2048 --runs 3 --cache no --backend <cpu|gpu>: a warm-up and then 3 runs in each of two processes; each cell is the mean of the two, 2026-11-06.

File	Backend	Prefill (tokens/s)	Decode (tokens/s)	Time to first token
2,048-token	GPU	18,269	490.9	0.0581 s
2,048-token	CPU	1,115	116.0	1.43 s
4,096-token	GPU	18,233	490.5	0.0582 s
4,096-token	CPU	608	95.9	1.81 s

The two CPU processes disagreed (the 2,048-token file: prefill 1,291 and 939 tokens/s, decode 121.6 and 110.5, time to first token 0.80 and 2.05 s; the 4,096-token file: prefill 546 and 671 tokens/s), so read the Mac CPU rows as rough; the two GPU processes of each file agree within 0.2 %.

Accuracy

GSM8K, the first 100 questions of the test split, 4-shot chain of thought (the technical report's setting; the exemplars are the first four of lm-evaluation-harness gsm8k-cot), greedy decoding, at most 1,024 new tokens. The source model runs in transformers; the files run in LiteRT-LM 0.19.1 on a Mac M4 Max, 3 times in fresh processes:

Model	Tested on	Correct of 100	Mean
Source, bfloat16 (transformers)	Mac CPU	58	58
MNN-Pocket-0.3B_int8_ekv2048.litertlm	Mac GPU	64, 62, 63	63.0
MNN-Pocket-0.3B_int8_ekv2048.litertlm	Mac CPU	59, 58, 61	59.3
MNN-Pocket-0.3B_int8.litertlm	Mac GPU	58, 63, 61	60.7
MNN-Pocket-0.3B_int8.litertlm	Mac CPU	56, 55, 58	56.3

The bar, fixed as a rule before the files existed, was the source model's score on this run minus 5 (58 − 5 = 53). For scale: the source model in float32 answered 65 of the same 100 (one run).

On each phone each int8 file answered 8 short questions and the first 20 of the GSM8K questions above (prompts of several hundred tokens), greedy, and each answer was compared with the Mac's on the same backend:

• CPU, Galaxy S26 and Pixel 11 Pro: for both files every reply is the Mac CPU's reply word for word (28 of 28 for each file on each phone).
• GPU, Galaxy S26 and Pixel 11 Pro: 16 or 17 of the 20 GSM8K answers equal the Mac GPU's, under our bar of 18. Where a reply differs, the token the phone picked at the first difference is at most 2 logits behind the first choice of a reference run of the same file. Our rule for that case is a run of all 100 GSM8K questions on the phone, against the bar of 53: 59 (Galaxy S26) and 63 (Pixel 11 Pro) with the 2,048-token file, 61 and 64 with the 4,096-token file. The 4,096-token file's GPU checks ran on a build with the same four sections.

There is no Mac reply to match word for word: LiteRT-LM 0.19.1's CPU backend does not open the NPU file (Files), and no Mac backend computes its 16-bit integer graph the way the phone's NPU does. On the Galaxy S26 NPU it answered 57 of the 100 GSM8K questions in one pass (greedy; the bar of 53; the source model 58), run through LiteRT-LM 0.19.1's Android Python package (litert-lm-api). Two command-line tools built from the v0.19.1 tag, with the same Qualcomm libraries, gave the same replies character for character: litert_lm_main, the tool under Usage, to 8 short questions, and litert_lm_advanced_main, the build of the speed row under Performance, to those 8, to the first 40 of the GSM8K questions and to two three-turn conversations. All 8 short questions stopped on their own after 8 to 21 tokens, and in both conversations the third reply recalled what the first turn had said.

The SM8850 file gave the NPU file's replies character for character (blank lines at the end of a reply not compared) to the same 8 short questions, the two conversations and the first 40 GSM8K questions (litert_lm_advanced_main, without the compiler plugin), and the same reply to the Usage prompt through litert_lm_main (17 + 25 = 42.). We did not run the other 60 GSM8K questions on it.

Limitations

• Tool calls come back as text. Given tool definitions, the model writes the call as a bare JSON object ({"name": …, "arguments": …}) without the <tool_call> tags its chat template describes, and the source model in transformers does the same on our test prompt. These files carry no tool-call configuration, so LiteRT-LM returns the JSON as the reply's text: parse it in your app. A configuration for the template's tags would not change that, because LiteRT-LM 0.19.1 parses only what stands between two tags: a copy of the 2,048-token file with one for <tool_call> and </tool_call> parsed none of 24 calls (six requests with tools, greedy and with the sampling settings our OlliteRT list gives this file, top-k 40, top-p 0.95 and temperature 0.7, at three seeds; Mac CPU), all of them bare JSON. In 9 of those 24 the model gave the arguments as a list instead of an object, so check them against your tool's parameters too.
• No int4 file. None of the int4 builds we made passed our Mac checks on both the CPU and the GPU, including one that carries the authors' quantization-aware int4 values.
• On the GPU these files need LiteRT-LM 0.19.0 or newer, the first release whose GPU kernels run these attention composites: with 0.18.1 the Mac GPU reports a shader error and the reply is a string of symbols, while its CPU runs them (we did not run 0.18.1 on the phones).
• Context is 2,048 or 4,096 tokens, prompt and reply together, by file (the source model supports 32K in deployment).
• On LiteRT-LM 0.18.1 and 0.19.1, once a message fails because it does not fit in the context window, every later message in that conversation fails as well (the next one with Task failed with state: 7). Start a new conversation after such an error.
• We ran the NPU file on one phone, a Galaxy S26 (Snapdragon SM8850). The file holds no compiled NPU code, so on another Snapdragon phone the engine compiles it for that NPU when it starts, but we have not run it on one. It needs Android binaries built from source (Usage), and each new engine compiles it first.
• The NPU file has one prefill signature, of 128 tokens, so a longer prompt is prefilled 128 tokens at a time. So does the SM8850 file.
• The SM8850 file is compiled for the Snapdragon SM8850 with QAIRT 2.47.0.260601. We ran it only on the Galaxy S26, with that version's QNN libraries. Like the NPU file, it needs Android binaries built from source (Usage).

Conversion notes

• Source: taobao-mnn/MNN-Pocket-0.3B at revision 087a52fd.
• Runtime: every measurement of the files on this card used LiteRT-LM 0.19.1, which runs the graphs on LiteRT, except the GPU runtime check under Limitations (0.18.1 and 0.19.0, on the Mac) and the OlliteRT run under Usage (0.16.1, the app's runtime, on a Pixel 11 Pro). The NPU file ran in two ways on the Galaxy S26, with the same Qualcomm libraries (the dispatch and compiler-plugin libraries built from the v0.19.1 tag and the QNN libraries of QAIRT 2.47.0.260601): through the command-line tools built from the v0.19.1 tag (litert_lm_main for the Usage block and the 8 short questions; litert_lm_advanced_main for the speed row, the 8 short questions, the first 40 GSM8K questions and the two conversations), and through the litert-lm-api 0.19.1 Android wheel (all 100 GSM8K questions, and the replies that ran to the limit before the rotary-position change). The SM8850 file ran through the same two command-line tools with the dispatch library and the QNN libraries, without the compiler plugin.
• Export: litert-torch 0.10.0 export_hf (fused QKV and gate/up projections, GPU composites, and the attention composites --use_qkv_norm_rope_composite=True --use_sdpa_composite=True), prefill signatures 128 and 1,024, KV cache 2,048 or 4,096 (--cache_length), quantization recipe dynamic_wi8_afp32. Packed with litert-lm-builder 0.18.1: litert-torch 0.10.0 calls a builder API that litert-lm-builder 0.19 no longer has.
• NPU file: litert-torch at commit 6d4c622, npu_export in three steps: the int8 weights of dynamic_wi8_afp32, activation ranges from 48 prompts (32 from GSM8K's training split, 16 from litert-torch's sample calibration prompts), then static quantization of the activations to 16-bit integers; prefill signature 128, KV cache 2,048; packed with litert-lm-builder 0.17.0. We changed two things in what npu_export gives. The prefill attention mask is broadcast over the query heads instead of repeated for each, so the mask the NPU adds stays under 1 MiB (557,056 bytes instead of 3,899,392): on the SM8850 NPU with QAIRT 2.47, the version under Usage, an int16 mask concatenated to more than 1 MiB was added to the wrong rows (litert-torch#1184; QAIRT 2.50 adds the issue's two-operation case correctly). And the 11 ranges in the two rotary-position graphs, which ended at the largest positions the calibration reached (511 in prefill, 527 in decode), are raised to 2,047.9375, the top of a 16-bit range in steps of 1/16, so that every position of the 2,048-token cache falls inside them. With the measured ranges the cosines and sines were off by up to 1.99 past the calibrated positions, and on the Galaxy S26 NPU 31 of the first 40 GSM8K replies ran to the 1,024-token limit (through LiteRT-LM 0.19.1's Android Python package), against 10 with the raised ones.
• SM8850 file: the NPU file compiled with the NPU compiler of litert-torch at commit 6d4c622 (compile_litertlm, backend qualcomm, SoC SM8850, the small graphs left to the phone's CPU), in a container with ai-edge-litert 2.2.0 and the QNN libraries of QAIRT 2.47.0.260601 (ai-edge-litert-sdk-qualcomm 2.2.0): optimization level O3 and one compiled context of 319,000,576 bytes for both signatures, with their weights shared (the phone's own compile of the NPU file makes two contexts, of 243,429,376 and 309,768,192 bytes).
• Model type generic_model. LiteRT-LM 0.19.1 gives a file that declares any other model type different GPU settings; one of them, allow_src_quantized_fc_conv_ops, is on for every type except generic_model. Our first upload was typed qwen2p5. On the Galaxy S26 GPU it failed our answer checks; on the Pixel 11 Pro GPU 14 of its 20 GSM8K answers equaled the Mac GPU's, under our bar of 18, and we did not run the 100-question check for it. A copy of it with only the type changed to generic_model passed on both phone GPUs, and so do these files (Accuracy); how much of the difference the type itself makes, we have not measured.
• Chat template: the source's chat_template.jinja with two changes. LiteRT-LM hands each message's content to the template as a list of text parts, and the source template keeps only string content, so it rendered every user message as empty: the bundled template reads both forms. It also renders with an empty message list, where the source template stops with an error. On 7 test conversations (with and without a system message, tools, tool calls and tool results, several turns), each rendered with and without the generation prompt, it gives the same text, byte for byte, as transformers' apply_chat_template with the source template.
• Tokenizer: the source's tokenizer.json, unchanged. Stop tokens 65512 and 65510.
• Reproduce: REPRODUCE.md in hf-to-litertlm; its commands, run from a clean clone on 2026-11-07, rebuilt files whose sections are identical to the two int8 files. The NPU file: its entry; its commands, run from a clean clone on 2026-11-07, rebuilt a file whose sections are identical to the NPU file's. The SM8850 file: its entry; its commands, run from a clean clone on 2026-11-07, rebuilt a file whose sections, the compiled main graph included, are identical to the SM8850 file's.

Updates

2026-11-07, SM8850 file. MNN-Pocket-0.3B_qualcomm_sm8850_ekv2048.litertlm is added: the NPU file compiled ahead of time for the Snapdragon SM8850. On the Galaxy S26 it creates its engine in 0.27 s, against 27 to 29 s for the NPU file, and decodes 157.7 tokens/s after a 1,024-token prompt (Performance). Its replies equal the NPU file's on everything we compared: the 8 short questions, the two conversations and the first 40 GSM8K questions (Accuracy). The other three files are unchanged. sha256 ab3aa3aa… (380,374,152 B).

2026-11-07, NPU file. MNN-Pocket-0.3B_qualcomm_npu_ekv2048.litertlm is added, for the NPU of Snapdragon phones. On the Galaxy S26 it decodes 147.3 tokens/s after a 1,024-token prompt, against 120.0 for the 2,048-token file on the GPU, and answers 57 of the 100 GSM8K questions (Accuracy). It runs with LiteRT-LM binaries built from the v0.19.1 tag (Usage). The other two files are unchanged. sha256 e8af457e… (374,213,768 B).

2026-11-07. MNN-Pocket-0.3B_int8.litertlm is replaced by a file that passes our answer checks on the phone GPUs (Accuracy), and MNN-Pocket-0.3B_int8_ekv2048.litertlm is added, which decodes about 1.6 times as fast as the previous file on the phone CPUs. On the phone CPUs the replacement decodes up to 7 % slower than the previous file re-measured in the same session (on the Pixel 11 Pro 45.4 against 49.0 tokens/s; the 54.1 on the earlier card was measured the day before), and its peak memory is about 0.76 GB higher. The replacement turns on the attention composites in the export and declares the model type generic_model instead of qwen2p5; the int8 weights, the tokenizer and the chat template are unchanged. On the GPU the new files need LiteRT-LM 0.19.0 or newer. sha256: MNN-Pocket-0.3B_int8.litertlm 7a62836b… (320,836,208 B) → 092a8389… (320,025,488 B); MNN-Pocket-0.3B_int8_ekv2048.litertlm new, 3d4710d4… (320,025,488 B). Checked with LiteRT-LM 0.19.1 on a Mac M4 Max, a Galaxy S26 and a Pixel 11 Pro, CPU and GPU, greedy, as under Accuracy. On the Galaxy S26 GPU the 2,048-token file decodes 120.0 tokens/s and the replacement 105.6. Procedure: Reproduce under Conversion notes.

License

Apache License 2.0, the same license as the source model. The files here are converted from taobao-mnn/MNN-Pocket-0.3B: the weights are quantized and the chat template is changed as described above.