0x36/Spark-X2.6-4B-mlx-4bit

Spark-X2.6-4B — MLX (4bit)

MLX conversion of XHToken/Spark-X2.6-4B for Apple Silicon.

• Weights: 4-bit affine, group size 64 (4.501 bits per weight — recommended for most Macs)
• Download size: 2.2 GB
• Format: MLX safetensors, model_type: spark2_5
• Architecture, tokenizer and chat template are identical to the original; only the weight layout and precision changed.

spark2_5 is not part of mlx-lm yet, so this repo ships its own implementation (spark2_5_mlx.py) and points to it from config.json via model_file. No patched mlx-lm is required — stock mlx-lm loads it directly.

Usage

pip install -U mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("0x36/Spark-X2.6-4B-mlx-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Write a Python function that reverses a linked list."}],
    add_generation_prompt=True,
    tokenize=False,
    enable_thinking=False,  # True for reasoning / thinking mode
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

CLI:

mlx_lm.generate --model 0x36/Spark-X2.6-4B-mlx-4bit \
  --prompt "What is the capital of Anhui Province?" --max-tokens 256

Notes

• Hybrid attention: 36 layers, sliding-window layers (window 512) alternating with full-attention layers (partial RoPE, 25%, theta 5e6). Generation uses RotatingKVCache on the sliding layers.
• Sampling from the original generation_config.json: temperature=1.0, top_p=0.95.
• Tied input/output embeddings (no separate lm_head).
• Long context (up to 1M positions per config) is supported, but KV memory grows with the full-attention layers.

Conversion

mlx_lm.convert --hf-path XHToken/Spark-X2.6-4B --mlx-path ./4bit --dtype bfloat16 -q --q-bits 4

Other precisions: bf16 · 8bit · 4bit