fcmeyer/Nanbeige4.3-33B-MLX-8bit

Nanbeige4.3-33B MLX 8bit

MLX conversion of Nanbeige/Nanbeige4.3-33B, by the Nanbeige Team. Source revision: c1093c501cc6460427c0ac634db7223d0450c0ee. Affine quantization: 8 base bits, group size 64.

Usage

from mlx_lm import load, generate
model, tokenizer = load("fcmeyer/Nanbeige4.3-33B-MLX-8bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Hello!"}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=256))

The upstream chat template is preserved. enable_thinking controls current reasoning; preserve_thinking=True retains previous reasoning for agent workflows. Upstream recommends tool_call_format="xml"; JSON is also supported. Template support does not certify any server's tool parser.

Validation

Weight files: 35,189,757,302 bytes. Tensor count: 1203. English/Chinese generation, finite smoke logits, tokenizer equivalence, and eight thinking/tool template cases passed. Small fixed synthetic diagnostic corpus (563 scored tokens): mean token NLL 2.074798, perplexity 7.962938. NLL difference versus BF16: +0.001105. Teacher-forced fidelity versus BF16 on the same 563 tokens: mean KL(BF16 || quant) 0.002034 nats/token; top-1 token agreement 98.76%. Lower KL means closer output distributions on these inputs. This does not establish downstream capability equivalence. This is a conversion regression check, not a standardized capability benchmark or a claim of benchmark equivalence. See the included validation JSON for outputs and per-category measurements. The source's 524,288-token context setting is retained; full 512K context has not been validated. Performance results, when included, apply only to the stated hardware and tested lengths.

Performance

Measured on Apple M5 Max with 128 GB unified memory, unquantized KV cache, 512-token prefill steps, 256 output tokens; one warm-up and three measured trials per length. Prompts are seeded synthetic token sequences. Values are medians.

Input tokens	Prefill tok/s	Decode tok/s	Peak GB
512	501.70	14.99	35.63
4096	540.49	14.22	36.44
16384	468.49	13.47	39.04

Peak memory is MLX's reported peak allocation, excluding other applications and OS memory. See environment.json for OS/runtime details.

Reproduction

Use the included requirements.lock.txt, conversion.json, evaluation.json, and scripts/release.py. From the repository root:

uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python -r requirements.lock.txt
.venv/bin/hf download Nanbeige/Nanbeige4.3-33B --revision c1093c501cc6460427c0ac634db7223d0450c0ee --local-dir source
.venv/bin/python scripts/release.py convert bf16
.venv/bin/python scripts/release.py validate bf16

For quantized variants, then run .venv/bin/python scripts/release.py convert 8bit and .venv/bin/python scripts/release.py validate 8bit. All quantized variants derive independently from BF16.

Attribution

Original model: Nanbeige Team, Nanbeige4.3-33B (2026), Apache-2.0. Conversion and measurements: fcmeyer. No claim is made that this is universally the best quantization.