Jakevin/spark-x2.6-4b-mixed-mlx

Spark-X2.6-4B mixed48 (MLX, 4-bit GPTQ + 8-bit sensitive layers)

MLX quantization of XHToken/Spark-X2.6-4B for Apple Silicon.

• All 180 linear layers quantized with GPTQ: 167 at 4-bit, 13 most sensitive projections at 8-bit (group size 64). Embedding / tied LM head: 4-bit. Norms and per-head attention output gates: bf16.
• File size 2.48 GB (bf16 original: 8.22 GB). About 4.85 bits per weight as stored by MLX.

Results (measured on an M4 Pro Mac mini, 64 GB)

Model	Size	Mean KL vs bf16 (nats/token)	Top-1 agree	MMLU-Pro (700-question holdout)	Decode tok/s
bf16 original	8.22 GB	0	100%	36.4%	27.5
MLX 4-bit RTN (standard)	2.31 GB	0.212	82.7%	34.7%	85.6
this model (mixed48)	2.48 GB	0.050	88.1%	37.1%	81.1

• KL / top-1: 16 held-out sequences, 4,617 tokens.
• MMLU-Pro: a fixed 700-question subset (questions 300–999 of our 1,000-question sample) not used for any build decision; 10-choice, single run, no confidence interval reported. Accuracy is within noise of bf16, but only 81.6% of the questions bf16 answers correctly are still answered correctly (answers shift on low-confidence questions), so per-question behaviour is not identical to bf16.
• Peak memory during generation: 3.96 GB.

Usage

spark2_5 is a custom architecture not yet in mlx-lm, so this repo ships the MLX model file spark2_5.py.

pip install mlx-lm
hf download Jakevin/spark-x2.6-4b-mixed-mlx --local-dir spark-mixed48
python spark-mixed48/run_mlx.py "Hello"

run_mlx.py registers spark2_5.py with mlx-lm and calls mlx_lm.load / generate. The original modeling_spark.py is kept for reference (PyTorch, trust_remote_code).

Limitations

• Evaluated only on MMLU-Pro subsets and KL on short text; no long-context, coding, or chat-quality evaluation.
• The holdout has 700 questions, fewer than the 1,000 we originally targeted.
• Sliding-window (512) + full attention 3:1 layout is reproduced in spark2_5.py; greedy outputs of the bf16 MLX port matched the reference transformers implementation token-for-token on 5 short prompts.

License

Apache-2.0, same as the base model (see LICENSE).

Comparison chart (all MLX builds of this base model)

Revision	GiB	KL DIV	PPL	MMLU-Pro (700)	Decode tok/s
bf16	7.66	0.0000	12.43	36.4%	27.5
MLX 4-bit RTN	2.15	0.2125	14.23	34.7%	85.6
GPTQ 4-bit	2.16	0.0509	12.51	37.7%	85.2
mixed48	2.31	0.0496	12.80	37.1%	81.1
tern48 (unreleased)	2.31	0.1416	13.30	37.1%	79.5
mixed 3 bpw (unreleased)	1.50	1.3204	31.22	—	99.4
ternary 3 bpw (unreleased)	1.50	1.0873	29.29	—	96.7

KL and PPL: 16 held-out sequences (4,601 predicted positions, full vocabulary) against the MLX bf16 model. These numbers are not comparable with KL/PPL reported by other tools (llama.cpp, exllamav3), which use different chunking and reference text.

For reference, on the same 700 MMLU-Pro questions with the same prompt and scoring, llama.cpp GGUF Q4_K_M builds scored 34.1–35.4% (XHToken official 34.1%, bartowski 35.4%) and GGUF bf16 scored 36.6%. With 700 questions the standard error of a single score is roughly ±1.8 points, so these MMLU-Pro differences are within noise; no paired significance test was run.