ASID/Spark-X2.6-4B-LiteRT

Spark-X2.6-4B — LiteRT-LM

XHToken/Spark-X2.6-4B converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime. Built and tested on litert-lm 0.18.0.

Spark-X2.6 is the SparkLLM Team's (iFLYTEK) August 2026 refresh of the Spark-X2.5 series. It keeps exactly the same architecture as Spark-X2.5-4B — a 4.11B dense decoder (36 layers, hidden 2560, 16 query / 4 KV heads of 256, sliding-window(512) x3 : full x1 hybrid attention, per-head sigmoid attention-output gates, exact-GELU gated MLPs, partial rotary 0.25 on the full layers, 131,072-entry tied vocab) — and gains its capability from post-training. The delta that matters for this bundle is the vendor chat template: X2.6 adds a thinking_effort (reasoning_effort = high by default, low optional) instruction block to the default system prompt, which this bundle carries verbatim.

It is a reasoning model: it works problems inside <think>…</think> before answering. The bundle declares the thought channel, so the runtime routes reasoning out of the streamed answer and a thinking budget (--thinking-budget, ThinkingConfig) is effective. Apache-2.0.

File	Recipe	Size
Spark-X2.6-4B_int4.litertlm	int4 blockwise-128 + OCTAV on linears, int8 embedding (externalized), fp32 GPU activations declared	2490 MB

The single int4 file is the recommended one, matching the shipped Spark-X2.5-4B recipe: on that bundle int4 matched int8 on the 8-question gate (8/8 both) and on GSM8K (94 vs 94) while being 41% smaller, and it was the faster GPU file on Apple silicon.

Correctness

• 8-question sanity gate (Apple M5, litert-lm 0.18.0 CLI, one process per question, greedy, thinking on, KV 4096): GPU 8/8 (131 s) and CPU 8/8 (165 s). Every answer arrived as a clean final answer through the thought channel — no reasoning leakage, no degeneration, every think block closed.
• Tokenizer parity: the bundle's tokenizer section (the upstream tokenizer.json, byte-level BPE) encodes 234 of 234 probe rows — every added token alone and mid-string, the role-marker turn, Latin-1 / Extended-A, emoji, CJK, digits, whitespace — to the same ids as the tokenizers reading of the upstream file.
• Load / echo gate: scripts/check_bundle.py PASS (loads, echoes pineapple, answers 42 to 17 + 25).
• Thinking switch: --thinking-budget 0 yields a direct answer with no thought block; --thinking-budget 24 closes the trace and answers. The bundle declares the channel; litert-lm describe prints Supports Thinking: NO because the exporter leaves the separate metadata boolean unset — the same metadata shape as the shipped Spark-X2.5-4B bundle, whose budget also works.
• GSM8K was not re-run for X2.6 (the architecture and recipe are identical to Spark-X2.5-4B, which scored 94% int4 vs 94% bf16 on the same harness).

Usage

litert-lm run ./Spark-X2.6-4B_int4.litertlm --prompt "What is 17 + 25? Answer briefly." --thinking true

# GPU
litert-lm run ./Spark-X2.6-4B_int4.litertlm --backend gpu --cache no --thinking true --prompt "..."

Notes for a reasoning model:

• Give it a generous output budget (≥ 2048 tokens; 3584 for math). The model thinks before it answers; truncated mid-thought it produces no final answer at all.
• The bundle declares the thought channel (<think> … </think>), so runtimes that expose ThinkingConfig / a thinking budget can cap or read the reasoning separately; the streamed answer contains only the final response. enable_thinking: false renders the vendor's no-think form (<|Bot|></think>).
• The bundle carries the vendor prompt format as a Jinja template: a default system block (<|System|>\nyou are a helpful assistant. plus X2.6's thinking_effort instructions, high by default; a user system prompt appended after it), every message wrapped in <｜start▁of▁sentence｜> … <｜end▁of▁sentence｜>, generation prompt <|Bot|><think>; stop token <｜end▁of▁sentence｜>. Tool-call formatting is not carried. KV budget 4096 tokens (the original's 1M context does not apply on-device), six prefill signatures (1024, 256, 64, 16, 4, 1).
• The vendor's recommended sampling is temperature 1.0 / top-p 0.95; the numbers on this card are greedy.

Performance

Apple M5 (MacBook Pro, 32 GB; litert-lm benchmark 0.18.0, -p 256 -d 256 --runs 3 --cache no, quiet machine, background services idle):

Backend	Prefill (256)	Decode	TTFT	Init
GPU (Metal)	231.4 tok/s	18.2 tok/s	1.16 s	4.2 s
CPU	93.9 tok/s	12.0 tok/s	2.83 s	5.3 s

No phone was measured for this card. Expect a flagship Android GPU to be several times slower in decode than this Mac; the Spark-X2.5-4B card's Galaxy S26 rows (~5 tok/s on every path) are the closest reference for the same architecture and quantization.

Conversion notes

Converted with litert-torch 0.9.4 / transformers 5.14.1 / litert-lm 0.18.0, released wheels, through the Spark recipe of hf-to-litertlm (spark_work/convert_spark.py + spark_work/patch_modeling.py).

• The vendor modeling code is patched for export, not re-implemented. modeling_spark.py computes attention through its own eager function and ignores config._attn_implementation; the export copy dispatches through the registered attention interface (so litert-torch's transposed KV cache is used), threads the per-call kwargs, declares the attention-backend capability flags, and applies the per-head sigmoid output gate in the interface's layout. The upstream config.json, modeling_spark.py and tokenizer.json at commit 66e3ae7bee54e0a523d692cb8ef841f35409807b are byte-identical to Spark-X2.5-4B's, so the proven X2.5 patch applies unchanged.
• X2.6's template delta is carried. The bundle's Jinja is the vendor chat_template.jinja for the plain-chat shapes, byte-identical over 60 shape × flag combinations (jinja2 and the runtime's minijinja 2.14): the default system block now includes the thinking_effort (high / low) instructions, and the generation prompt is the vendor's <|Bot|><think> opener. The literal <think> is what makes litert-torch declare the thought channel. reasoning_effort defaults to high, the vendor default; the runtime exposes no effort knob, only the thinking switch/budget.
• No start token in the metadata. The tokenizer declares <｜start▁of▁sentence｜> as BOS but never prepends it (add_bos_token: false); the template carries its own per message, so the exporter's start_token write was suppressed.
• int4 externalizes the embedding table. The vocab is tied; asking int4 for the lm_head and int8 for the embedding makes the quantizer copy the 131,072-row table once per signature. The embedder lives in its own 346.6 MB section instead.
• GPU activations are declared fp32 in the bundle (prefer_activation_type = "fp32", a metadata-only repack; weights byte-identical). This is the setting under which the same weights scored GSM8K 94 on Spark-X2.5-4B (fp16 GPU activations scored 84).
• Tokenizer embedded as the upstream tokenizer.json (HF tokenizer section), not a SentencePiece conversion.
• Quantization at export time: int4 blockwise-128 + OCTAV on linears, int8 embedding (externalized). No post-processing.
• Bundle sections: LlmMetadata 5532 B, ExecutorMetadata 19411 B, HF tokenizer 2157741 B, text decoder 2141454112 B, embedder 346567008 B. File sha256 e0e85a90b480089672d0ea95abc0087c71aaf053e48814a502f0a0fd6aaad926.

License and changes

Distributed under Apache-2.0 (inherited from the base model; the upstream LICENSE file is included). Changes from the original work: weights converted from safetensors bf16 to LiteRT flatbuffers and quantized as described above; tokenizer and chat template repackaged into the .litertlm bundle (thinking pre-fill and thought-channel metadata as described); tool-calling template branches not carried. No training or fine-tuning. This repository is a community conversion and is not affiliated with iFLYTEK or the SparkLLM Team.