mlx-community/Nex-N2.5-mini-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs

A 4-bit mixed-precision MLX quant of nex-agi/Nex-N2.5-mini, Nex's agentic fine-tune of Qwen3.5-35B-A3B (35B total parameters, 3B active, 256 experts, image input). Sensitive layers are kept at 8-bit and robust ones at 4-bit, so the on-disk size stays within a few percent of a stock uniform 4-bit quant while the parts that matter keep their precision.

Nex-N2.5-mini is built for long-horizon agent work: terminal use, coding agents, computer and browser use. Upstream reports 73.4 on Terminal-Bench 2.1 and 43.8 on SWE-Bench Pro for the bf16 model (their numbers, their harness). It is the model behind OptiQ Code's local-agent setup on a 36 GB Mac.

Quantization details

Property	Value
Predominant precision	4-bit
Layers at 8-bit (sensitive)	399
Layers at 4-bit (robust)	113
Total quantized layers	512
Group size	64
Per-layer allocation	reused from mlx-community/Qwen3.5-35B-A3B-OptiQ-4bit, the base model's measured recipe
Vision tower	bf16, bundled as optiq/optiq_vision.safetensors (0.9 GB)
KV cache	kv_config.json, per-layer mixed precision for optiq serve --kv-config
MTP head	none (the upstream fine-tune ships without one, so --mtp has nothing to load)
Weights on disk	22.2 GB (+0.9 GB vision sidecar)

A fine-tune keeps its base architecture, and which layers tolerate fewer bits is a property of that architecture, so this quant reuses the bit map measured on Qwen3.5-35B-A3B with a KL-divergence sensitivity pass over a six-domain calibration mix rather than re-measuring it. The Capability Score was not re-run for this variant; the base model's score (77.42 against 76.92 for uniform 4-bit) is the reference for what the allocation buys.

We follow the same naming convention llama.cpp uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is for the predominant precision, not the weighted average.

Usage

Text, with mlx-lm:

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Nex-N2.5-mini-OptiQ-4bit")
response = generate(model, tokenizer, prompt="Explain quantum computing in simple terms.", max_tokens=200)

Text and images, tool calling, and the mixed-precision KV cache, with mlx-optiq:

pip install mlx-optiq
optiq serve --model mlx-community/Nex-N2.5-mini-OptiQ-4bit --kv-config kv_config.json

The server speaks the OpenAI chat and Responses APIs and the Anthropic Messages API; image parts in a message are routed through the bundled vision tower. optiq code uses the same server as a local coding agent.

Sampling

Upstream evaluates with temperature 0.7, top_p 0.95, top_k 40. Thinking is controlled with reasoning_effort (none for a direct answer); the chat template is the upstream one and is bundled as chat_template.jinja.

See the Qwen3.5 family guide for serving flags, training recipes, and family caveats.

Links

• Project website: mlx-optiq.com
• Qwen3.5 family guide: mlx-optiq.com/docs/qwen3.5
• PyPI: pypi.org/project/mlx-optiq
• Calibration mix: mlx-optiq.com/blog/calibration-mix
• Base model: nex-agi/Nex-N2.5-mini
• Recipe source: mlx-community/Qwen3.5-35B-A3B-OptiQ-4bit

Quantize your own

This quant was produced by mlx-optiq. Point it at any Hugging Face model to get the same sensitivity-aware mixed precision:

pip install mlx-optiq
optiq convert <hf-model-id> --target-bpw 5.0 --candidate-bits 4,8
optiq lab   # full local workbench: chat, compare, quantize, fine-tune

License

Apache 2.0, inherited from the base model.