mlx-community/Spark-X2.6-4B-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs

A mixed-precision MLX quant of XHToken/Spark-X2.6-4B, the on-device agent model that follows Spark-X2.5 with improved post-training for coding, tool use, long context and reasoning. 2.8 GB on disk, down from 7.7 GB bf16. See the original card for what the model is trained to do and how XHToken evaluates it.

Spark is a new architecture for MLX, and mlx-optiq adds support for it, so import optiq before mlx_lm.load: stock mlx-lm does not know the spark2_5 model type.

What it is

Property	Value
Base	XHToken/Spark-X2.6-4B (same spark2_5 architecture as Spark-X2.5-4B)
Layers	108 at 8-bit, 109 at 4-bit (217 total)
Group size	64
KV cache	kv_config.json included (mixed-precision KV for optiq serve --kv-config)
On disk	2.8 GB

Following llama.cpp's naming for mixed quants, the "4bit" label denotes the predominant precision, not the weighted average.

How the bit-widths were chosen

The per-layer allocation was not measured on this model. It was transferred from mlx-community/Spark-X2.5-4B-OptiQ-4bit, whose allocation came from a KL-divergence sensitivity sweep over a six-domain calibration mix. Spark-X2.6 keeps exactly the same architecture and modeling code as X2.5, and all 217 tensors matched with none unmatched.

Checked after conversion

Run through this quant and its bf16 parent on Apple Silicon with greedy decoding:

Prompt	This quant	bf16 parent
240 km in 3 h, average speed with steps	80 km/h, same three steps	80 km/h
Capital of Australia, and why not Sydney	Canberra, a planned federal capital	Canberra, a planned federal capital
is_palindrome(s) ignoring case and non-alphanumerics	correct function	correct function
Author and year of Pride and Prejudice	Jane Austen, 1813	Jane Austen, 1813
Call get_weather for Paris, JSON only	{"tool_name": "get_weather", "params": {"location": "Paris"}}	same call, city as the key
240 km in 3 h, thinking on	short think, then 80 km/h	same think and answer
17 × 23, number only	thinking off: 491 (wrong); thinking on: 391	391 both ways

The one miss is bare mental arithmetic with thinking off, where the quant answers in a single token with no working; the Spark-X2.5 quant makes the same slip. With thinking on, the default, it gets it right.

Usage

pip install mlx-optiq   # brings mlx-lm and the spark2_5 architecture

import optiq  # registers spark2_5 with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Spark-X2.6-4B-OptiQ-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain quantum computing in simple terms."}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=300))

Serving, with the mixed-precision KV cache, tool calling and the OpenAI, Responses and Anthropic Messages APIs:

optiq serve --model mlx-community/Spark-X2.6-4B-OptiQ-4bit --kv-config kv_config.json

Thinking and sampling

Thinking is on by default. The chat template takes reasoning_effort of high (the default) or low to set how much it thinks, or enable_thinking: false to turn it off; pass them as chat_template_kwargs at the endpoint or as keywords to apply_chat_template. Through optiq serve, the standard reasoning_effort: "none" also turns it off. XHToken's sampling defaults, temperature 1.0 and top_p 0.95, are bundled in generation_config.json.

See the Spark family guide for serving flags and family caveats.

Links

• Project website: mlx-optiq.com
• Spark family guide: mlx-optiq.com/docs/spark
• PyPI: pypi.org/project/mlx-optiq
• Base model: XHToken/Spark-X2.6-4B