experimentalmachines/Qwen2.5-0.5B-ExecuTorch

Qwen2.5-0.5B for ExecuTorch

ExecuTorch exports of Qwen/Qwen2.5-0.5B (revision 060db6499f32) for on-device inference with the openweights Android app or any ExecuTorch 1.4.0 runtime.

Files

Every backend is exported at every context window the runner could build (2k to 32k). The window is fixed inside the file: the runtime allocates the whole KV cache at load, so pick the largest window the device can hold (fits_phone_budget in each folder's config.json is the estimate against a 5 GB budget).

Backend	Target	File	Window	Size	Smoke test
XNNPACK (CPU)	any arm64	xnnpack/Qwen2.5-0.5B-8da4w-2k.pte	2,048 tokens	0.42 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/Qwen2.5-0.5B-8da4w-4k.pte	4,096 tokens	0.42 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/Qwen2.5-0.5B-8da4w-8k.pte	8,192 tokens	0.42 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/Qwen2.5-0.5B-8da4w-16k.pte	16,384 tokens	0.43 GB	passed ("Paris")
Qualcomm QNN (HTP)	SM8650 (Snapdragon 8 Gen 3)	qnn/sm8650/Qwen2.5-0.5B-qnn-hybrid-2k.pte	2,048 tokens	0.58 GB	structure checked (no host NPU runtime)
Qualcomm QNN (HTP)	SM8750 (Snapdragon 8 Elite)	qnn/sm8750/Qwen2.5-0.5B-qnn-hybrid-2k.pte	2,048 tokens	0.58 GB	structure checked (no host NPU runtime)
Qualcomm QNN (HTP)	SM8750 (Snapdragon 8 Elite)	qnn/sm8750/Qwen2.5-0.5B-qnn-hybrid-4k.pte	4,096 tokens	0.60 GB	structure checked (no host NPU runtime)
Qualcomm QNN (HTP)	SM8750 (Snapdragon 8 Elite)	qnn/sm8750/Qwen2.5-0.5B-qnn-hybrid-8k.pte	8,192 tokens	0.61 GB	structure checked (no host NPU runtime)

Tokenizer: tokenizer.json, copied unchanged from the source repo. Each backend folder has a config.json listing every window as a variant with the metadata the .pte reports, and an export-report-<window>.json per file with the full export record.

Memory

• XNNPACK (CPU) at 2,048 tokens: the KV cache costs 24,576 bytes per token (fp32), 50,331,648 bytes for the whole window, allocated in full when the model loads.
• XNNPACK (CPU) at 4,096 tokens: the KV cache costs 24,576 bytes per token (fp32), 100,663,296 bytes for the whole window, allocated in full when the model loads.
• XNNPACK (CPU) at 8,192 tokens: the KV cache costs 24,576 bytes per token (fp32), 201,326,592 bytes for the whole window, allocated in full when the model loads.
• XNNPACK (CPU) at 16,384 tokens: the KV cache costs 24,576 bytes per token (fp32), 402,653,184 bytes for the whole window, allocated in full when the model loads.

How it was made

• XNNPACK (CPU) any arm64: ExecuTorch 1.4.0 export_llm: 8-bit dynamic activations and 4-bit weights in groups of 32, int8 per-channel embeddings, XNNPACK with extended ops, prefill chunk 2048, fp32 KV cache. Built by run 1.
• Qualcomm QNN (HTP) SM8650 (Snapdragon 8 Gen 3): ExecuTorch 1.4.0 Qualcomm static LLM (examples/qualcomm/oss_scripts/llama, --decoder_model qwen2_5-0_5b): the quantization recipe ExecuTorch registers for this model, calibrated on wikitext (1 sample), hybrid prefill (128 tokens per step) and decode graphs compiled with QAIRT 2.37.0.250724 for SM8650 (Snapdragon 8 Gen 3). Built by run 1.
• Qualcomm QNN (HTP) SM8750 (Snapdragon 8 Elite): ExecuTorch 1.4.0 Qualcomm static LLM (examples/qualcomm/oss_scripts/llama, --decoder_model qwen2_5-0_5b): the quantization recipe ExecuTorch registers for this model, calibrated on wikitext (1 sample), hybrid prefill (128 tokens per step) and decode graphs compiled with QAIRT 2.37.0.250724 for SM8750 (Snapdragon 8 Elite). Built by run 1.

License

A quantized derivative of Qwen/Qwen2.5-0.5B, distributed under the same terms (apache-2.0). The upstream license files are included unchanged: LICENSE.

The qnn/ folders hold QNN HTP context binaries compiled with the Qualcomm AI Runtime SDK (QAIRT) 2.37.0.250724 from Qualcomm Technologies, Inc., used under its AI Stack License. No Qualcomm SDK or runtime library is included; running them needs the matching QNN runtime (for example executorch-android-qnn 1.4.0, which depends on qnn-runtime 2.37.0).