mlx-community/GLM-5.3-DQ4-Q5-q8

This model mlx-community/GLM-5.3-DQ4-Q5-q8 was converted to MLX format from zai-org/GLM-5.3-BF16 using mlx-lm version 0.31.3 (with PR #1410).

Note that this quant is using the GLM-5.3-BF16 as base.

This is created for people using a single Apple Mac Studio M3 Ultra with 512 GB. The 4-bit version of GLM-5.3 fits comfortably. But we can do better. Using research results, we aim to get better results from a slightly larger and smarter quantization. It should also not be so large that it leaves no memory for a useful context window.

You can find more similar MLX model quants for Apple Mac Studio with 512 GB at https://huggingface.co/bibproj

pip install mlx-lm

mlx_lm.generate --model mlx-community/GLM-5.3-DQ4-Q5-q8 --prompt "Hi"

---

What is this DQ4-Q5-q8?

In the Arxiv paper Quantitative Analysis of Performance Drop in DeepSeek Model Quantization the authors write,

We further propose DQ3_K_M, a dynamic 3-bit quantization method that significantly outperforms traditional Q3_K_M variant on various benchmarks, which is also comparable with 4-bit quantization (Q4_K_M) approach in most tasks.

and

dynamic 3-bit quantization method (DQ3_K_M) that outperforms the 3-bit quantization implementation in llama.cpp and achieves performance comparable to 4-bit quantization across multiple benchmarks.

The resulting multi-bitwidth quantization has been well tested and documented.

---

Recipe

The recipe is a bit different from that of a normal DQ3_K_M. To make to the quant perform better under stress, only the up and gate expert tensors are quantized to 4-bit, and the down expert to 5-bit. All the other tensors are kept at 8-bit. You could say that this quant has an 8-bit "brain" and 4-bit/5-bit experts.

In the convert.py file of mlx-lm on your system ( you can see the original code here ), replace the code inside def mixed_quant_predicate() with something like

        # Build a mixed quant like "DQ4-Q5-q8" similar to the "DQ3" of Arxiv paper https://arxiv.org/abs/2505.02390
        #    Quantitative Analysis of Performance Drop in DeepSeek Model Quantization
        q_bits = 8
        if "switch_mlp.up_proj" in path:
           q_bits = 4
        if "switch_mlp.gate_proj" in path:
           q_bits = 4
        if "switch_mlp.down_proj" in path:
           q_bits = 5
        print("path:", path, "index:", index, "q_bits:", q_bits)
        return {"group_size": group_size, "bits": q_bits, "mode": mode}

Then create your GLM-5.3-DQ4-Q5-q8 quant with

mlx_lm.convert --hf-path zai-org/GLM-5.3-BF16 --mlx-path GLM-5.3-DQ4-Q5-q8 -q --quant-predicate mixed_4_6

---

Enjoy!