WinterCharm/Qwen3.6-35B-A3B-wMix58

Qwen3.6-35B-A3B — WinterMix58 (MLX)

A 24 GiB mixed-precision MLX quantization of Qwen/Qwen3.6-35B-A3B — built for 32 GB Macs, and the first MLX build of this project to beat the source Dynamic GGUF's imatrix rounding at short context.

Native MLX, no strings attached. Standard MLX affine layout — bit-identical packing to mx.quantize, no custom kernels, no forked runtime. Loads in anything that speaks MLX (mlx-vlm, mlx-lm-based servers, LM Studio) at exactly the speed of any stock-converted model. The vision tower is fully functional.

Method (brief)

Winter-Mix: sensitivity-informed mixed-precision allocation with hard pins on routing-critical tensors, calibrated error-compensated rounding (GPTQ-family, independent MLX-native implementation), and a diverse long-context calibration mixture engineered for full expert coverage — including multilingual content so language-specialist experts are calibrated rather than silently degraded. Validated against RTN controls, an 8-bit reference, and cross-engine baselines. The full pipeline is not released.

Measured quality

One scoring rule for every row — NLL over the second half of each context window, token-aligned across engines (llama.cpp's native rule). Comparison rows are benchmarked reference points from the same harness, not published artifacts.

model	GiB	short-2K ppl	long-16K ppl
8-bit MLX reference	35	6.0275	4.6008
WinterMix58	24	6.0420	4.6167
Unsloth UD-Q5_K_XL GGUF (llama.cpp)	24.8	6.0504	4.6060
oQ5 (oMLX)	24	6.0806	4.6350
oQ4 (oMLX)	20	6.1221	4.6934

• Beats the source GGUF at short context (−0.14%) at a slightly smaller size; the GGUF keeps a small 16K edge (+0.23%).
• Beats its exact size-peer oQ5 in both bands (−0.63% short, −0.39% long).
• Sits just +0.24% / +0.35% behind the full 8-bit reference — at 11 GiB less.
• Long-context checked to 98K tokens with no depth collapse; multilingual (8-script) parity verified.

Beyond perplexity

Selected using three instruments — NLL, seeded behavioral probes, and direct reading of reasoning traces. On a 30-step state-tracking benchmark with knife-edge conditional rules (5 seeded runs, 50K-token context), WinterMix58 delivered a correct final state in 5 of 5 runs — the only build in the evaluation family to do so, including the 8-bit reference — with zero fabricated figures and zero truncations. Its reasoning traces show scheduled verification passes that resolve forward and catch-and-correct behavior before delivery. Measured generation speed during these runs: ~70 tok/s at 50K context on an M5 Max.

Running on 32 GB Macs

The model occupies 24 GiB; this architecture's KV cache is unusually light (~2 GB per 100K-token session — only 10 of 40 layers are full attention). On a 32 GB machine, raise the GPU wired limit, e.g.:

sudo sysctl iogpu.wired_limit_mb=28672

leaving room for a 100K+ context session. On 48 GB+ machines it runs with no configuration at all.

Provenance & credits

• Base model: Qwen/Qwen3.6-35B-A3B — quantized directly from the BF16 release; Apache 2.0 inherited.
• Bit allocation informed by analysis of the tensor-type metadata in Unsloth's published Dynamic GGUF (metadata analysis only; no weights or code reused).
• Built on MLX / mlx-vlm.
• Quantized end-to-end on a single M5 Max MacBook Pro.

Sibling releases for other Apple-silicon memory tiers: Qwen3.5-122B-A10B-wMix58 / -wMix48 / -wMix68.

Want a Winter-Mix quant of another model? Open a request in the Community tab.