Nanbeige4.3-33B-MLX-4bit

4-bit (group size 64, ~4.5 bits/weight) MLX conversion of Nanbeige/Nanbeige4.3-33B, sized for 32-48 GB Apple Silicon Macs.

Measured on an M4 Pro, 48 GB (mlx-lm 0.29.1):


Disk	17 GB
Peak memory	18.8 GB
Generation	~13.7 tok/s

Usage:

pip install mlx-lm
mlx_lm generate --model <this-repo> --prompt "Hello" --temp 0.6 --top-p 0.95

Recommended settings follow the original card (agentic: temp 1.0, top_p 0.95; chat/reasoning: temp 0.6, top_p 0.95). The model thinks by default; enable_thinking=False turns it off.

Quantized with a streaming shard-by-shard script (all 2-D linear and embedding weights to 4-bit, norms in bf16). Quality was spot-checked only; benchmarks are those of the original model, not re-run on this quant. License is Apache 2.0, inherited from the base model.

Downloads last month: 99

Safetensors

Model size

33B params

Tensor type

U32

BF16

MLX

Hardware compatibility

4-bit

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for killen8/Nanbeige4.3-33B-MLX-4bit

Base model

Nanbeige/Nanbeige4.3-33B-Base

Finetuned

Nanbeige/Nanbeige4.3-33B

Quantized

(86)

this model