hamuz/Nanbeige4.3-33B-MLX

Nanbeige4.3-33B · MLX

Every MLX quantization of Nanbeige/Nanbeige4.3-33B in one repo, measured on real Apple Silicon.

bf16 · 8-bit · 6-bit · 5-bit · 4-bit · 3-bit · 2-bit

---

About the model

Nanbeige4.3-33B is the flagship teacher model of the Nanbeige4.3 series by the Nanbeige team (BOSS Zhipin). It is a dense 33B Llama-architecture model built for agentic coding and professional cowork tasks, with a thinking mode, XML/JSON tool calling and a 512K-token context window. The official model card reports state-of-the-art agentic results in the ~30B class (Terminal-Bench 2.1: 80.9, DeepSWE: 65.5, AIME 2026: 95.2).

This repository packages the model for MLX so it runs natively on Apple Silicon Macs. Each variant lives in its own folder, so you only download the one you need.

Variants & measured results

Variant	Folder	Size	Bits/weight	Perplexity ↓	Δ vs 8-bit	Prompt tok/s	Gen tok/s	Peak RAM	Min. Mac RAM	Notes
BF16	bf16/	66.2 GB	16.00	not tested¹	—	—	—	—	96 GB	Reference (unquantized)
8-bit	8bit/	35.2 GB	8.50	8.951 ± 0.237	baseline	50.9	5.4	35.6 GB	64 GB	Near-lossless
6-bit	6bit/	26.9 GB	6.50	8.943 ± 0.238	-0.09%	58.5	7.7	27.4 GB	48 GB	Excellent
5-bit	5bit/	22.8 GB	5.50	8.903 ± 0.234	-0.54%	55.8	8.1	23.3 GB	36 GB	Very good
4-bit	4bit/	18.6 GB	4.50	8.656 ± 0.221	-3.30%	53.5	9.8	19.3 GB	32 GB	⭐ Best balance
3-bit	3bit/	14.5 GB	3.50	9.822 ± 0.246	+9.73%	62.6	11.4	15.2 GB	24 GB	Compact, small quality drop
2-bit	2bit/	10.3 GB	2.50	64.686 ± 1.860	+622.67%	55.4	13.2	11.0 GB	24 GB	⚠️ Broken output, not recommended²

_{¹ BF16 needs 66 GB just for weights and was not run on the 64 GB test machine. It is included as the full-precision MLX reference (for example to re-quantize or fine-tune on larger Macs).
² Plain 2-bit affine quantization is too aggressive for this model: perplexity jumps ~7× and generation degenerates into repeated words. It is kept only for completeness and research. Use 3bit or larger.
Reading the perplexity column: 4-, 5- and 6-bit measure within ~1–1.5 standard errors of 8-bit (slightly lower is sampling noise on 32 samples, not a real gain). Treat 4-bit through 8-bit as equivalent on this metric. The real cliffs are at 3-bit (+10%) and 2-bit.
Perplexity: mlx_lm.perplexity, allenai/tulu-3-sft-mixture, 32 samples × 512 tokens, seed 123 (lower is better, same data for all variants). Speed: mlx_lm.benchmark, 512-token prompt / 128 generated tokens, mean of 3 trials. Min. Mac RAM: weights + ~3 GB headroom must fit inside the default ~75% GPU memory limit (see Memory tips).
Hardware: Apple M1 Max · 64 GB unified memory · macOS 27.2 · mlx 0.32.4 · mlx-lm 0.32.2. Raw logs are in results/.}

Which one should I pick?

Your Mac	Recommended	Why
128 GB+	8bit (or bf16 on 192 GB+)	Practically indistinguishable from full precision
64 GB	8bit or 6bit	Highest quality that still leaves room for long context
48 GB	6bit / 5bit	Very close to 8-bit quality
32–36 GB	4bit	The sweet spot of quality, speed and size
24 GB	3bit	Fits with a modest context window; small quality drop
16 GB	—	Not recommended: only 2bit fits, and it produces broken output. Consider a smaller model

Generation on Apple Silicon is memory-bandwidth bound, so smaller variants are also faster.

Quickstart

1. Install

pip install -U mlx-lm "huggingface_hub[hf_xet]"

2. Download one variant

# pick one of: bf16 8bit 6bit 5bit 4bit 3bit 2bit
hf download hamuz/Nanbeige4.3-33B-MLX --include "4bit/*" --local-dir ./Nanbeige4.3-33B-MLX

ℹ️ Because all variants share this repo, point MLX at the local folder (./Nanbeige4.3-33B-MLX/4bit) instead of the repo id.

3. Run

# One-shot generation (thinking mode is on by default)
mlx_lm.generate --model ./Nanbeige4.3-33B-MLX/4bit \
  --prompt "Which number is bigger, 9.11 or 9.8?" \
  --max-tokens 4096 --temp 0.6 --top-p 0.95

# Disable thinking for quick answers
mlx_lm.generate --model ./Nanbeige4.3-33B-MLX/4bit \
  --prompt "Write a haiku about Apple Silicon." \
  --chat-template-config '{"enable_thinking": false}'

# Interactive chat
mlx_lm.chat --model ./Nanbeige4.3-33B-MLX/4bit --max-tokens 8192

Python

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("./Nanbeige4.3-33B-MLX/4bit")

messages = [{"role": "user", "content": "Which number is bigger, 9.11 or 9.8?"}]
prompt = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    enable_thinking=True,      # False = non-thinking mode
    preserve_thinking=False,   # True for multi-turn agent / tool-use sessions
)

text = generate(
    model, tokenizer, prompt,
    max_tokens=8192,
    sampler=make_sampler(temp=0.6, top_p=0.95),
    verbose=True,
)

OpenAI-compatible server (agents, tool calling)

mlx_lm.server --model ./Nanbeige4.3-33B-MLX/4bit --port 8080

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "What is the weather in Istanbul?"}],
  "tools": [{"type": "function", "function": {
    "name": "get_weather",
    "description": "Get the current weather for a city",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}
  }}],
  "temperature": 1.0, "top_p": 0.95, "max_tokens": 4096,
  "chat_template_kwargs": {"preserve_thinking": true}
}'

The chat template emits tool calls in the XML <tool_call><function=...> format, which the official card recommends for best tool-calling accuracy (tool_call_format="json" is also supported via chat_template_kwargs).

LM Studio

Download a single folder (as above) into your LM Studio models directory, for example ~/.lmstudio/models/hamuz/Nanbeige4.3-33B-MLX-4bit/, and it will show up as an MLX model.

Recommended settings

From the official model card:

Scenario	Temperature	Top-p	Max new tokens	Template flags
Agentic & tool-use tasks	1.0	0.95	65,536	preserve_thinking=True
Reasoning & chat	0.6	0.95	262,144	preserve_thinking=False

• enable_thinking (default True): generate a <think>…</think> block before answering.
• preserve_thinking: keep reasoning from earlier assistant turns. Use True for multi-turn tool use, office tasks and coding agents.
• tool_call_format: "xml" (default, recommended) or "json".

Memory & long context

The model uses grouped-query attention (52 layers, 8 KV heads × 128), so the KV cache costs ~0.21 MB per token on top of the weights:

Context	KV cache (fp16)	--kv-bits 8	--kv-bits 4
8K	1.7 GB	0.9 GB	0.5 GB
32K	7.0 GB	3.7 GB	2.0 GB
128K	27.9 GB	14.8 GB	7.9 GB
256K	55.8 GB	29.7 GB	15.7 GB
512K	111.7 GB	59.3 GB	31.4 GB

• Quantize the cache: mlx_lm.generate --kv-bits 8 (or 4) and mlx_lm.server --kv-bits 8.
• Cap the cache with a rotating window: --max-kv-size 32768.
• macOS limits GPU-wired memory to ~75% of RAM by default. On a dedicated machine you can raise it, for example on 64 GB: sudo sysctl iogpu.wired_limit_mb=57344 (resets on reboot).

How these were made

# full precision
mlx_lm.convert --hf-path Nanbeige/Nanbeige4.3-33B --mlx-path bf16 --dtype bfloat16
# quantized (affine, group size 64), N in 8 6 5 4 3 2
mlx_lm.convert --hf-path Nanbeige/Nanbeige4.3-33B --mlx-path Nbit -q --q-bits N --q-group-size 64

• Source: Nanbeige/Nanbeige4.3-33B, all shards verified against their SHA-256 hashes.
• Quantization: standard MLX affine quantization, group size 64, applied to all linear and embedding layers. RMSNorm weights are kept in bf16.
• Chat template, tokenizer and special tokens are copied unchanged from the original repo.
• Converted and evaluated with mlx 0.32.4 / mlx-lm 0.32.2 on Apple M1 Max · 64 GB unified memory · macOS 27.2.

Original benchmarks

Scores are from the original BF16 model as reported by the Nanbeige team. Quantization quality for each variant is reported above as perplexity.

License & credits

• Model weights © Nanbeige Team (BOSS Zhipin), released under Apache-2.0. This repo only contains format conversions and quantizations.
• MLX conversion by hamuz. Issues with the quantizations go to this repo's Community tab; issues with the model itself go to the original repo.

@misc{nanbeige43chat2026,
  title        = {Nanbeige4.3-33B},
  author       = {Nanbeige Team},
  year         = {2026},
  publisher    = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Nanbeige/Nanbeige4.3-33B}}
}