Qwen3-1.7B for ExecuTorch (XNNPACK), several context windows
The same weights exported 2 times, differing only in the context window compiled into
the static KV cache. Exported with ExecuTorch 1.4.0's own recipe for the family
(examples/models/qwen3, extension.llm.export.export_llm): int8 dynamic activations,
int4 weights in groups of 32 (the layout Arm KleidiAI kernels take in the XNNPACK runtime),
int8 embedding table, prefill chunk 2048 tokens. Runs in any app on the ExecuTorch 1.4.0
Android or iOS LLM runtime; built for OpenWeights.
| File | Window (tokens) | Size | KV cache at load (fp32) |
|---|---|---|---|
qwen3_1_7b_8da4w_2k.pte |
2048 | 1285 MB | about 448 MB |
qwen3_1_7b_8da4w_32k.pte |
32768 | 1348 MB | about 7168 MB |
Pick by memory, not speed: the runtime attends over filled positions only, so a larger
window costs memory at load and nothing per token (measured: every window within about 1% of
the smallest on prefill and decode in a paired probe). The 32k file needs about 7 GB of KV cache on top of the weights; on 12 GB phones (Galaxy S24+, S25 Ultra, Poco X8 Pro) the process was killed after a few prompts, by Samsung's Heimdall guard and by MIUI's ActivityManager alike, both at a 6 GB threshold; only a 16 GB Pixel 10 Pro XL ran it. Use 2k on a 12 GB phone. Measured on a Dimensity 9400
(Poco X8 Pro), awake, median of six turns: see docs/research/window-matrix.md. The full window matrix on four
chips is in the OpenWeights repository under docs/research/window-matrix.md.
tokenizer.json is the base model's. EOS 151645. Chat template:
the base model's ChatML-style template (<|im_start|>role ... <|im_end|>).
Weights are Alibaba Cloud's Qwen3-1.7B, Apache-2.0; this repository only changes their packaging.
- Downloads last month
- 34