Ornith-1.0-35B-UD-Q2_K_XL-MLX

An MLX quantization of deepreinforce-ai/Ornith-1.0-35B for running locally on Apple Silicon — the default model for chad, a Claude-Code-style local coding agent.

chad runs one Ornith model, picked by your RAM: this 35B on ≥24 GB Macs, and auto-falls back to the smaller Ornith-1.0-9B-UD-Q4_K_XL-MLX on 16/18 GB Macs.

Quant

• Params: 35B (MoE, 256 experts / top-8, ~3B active per token)
• Scheme: 2-bit experts / 6-bit backbone / 8-bit router, AWQ-calibrated (35/40 layers kept activation-aware scales)
• Footprint: ~12.2 GB peak, ~71 tok/s on a 24 GB M4 Pro
• RAM: Best on ≥24 GB. Tight on 16/18 GB Macs (raise the Metal wired limit or use the 9B).

The naming follows Unsloth's Dynamic 2.0 convention (UD-Q2_K_XL = a dynamic quant that spends extra bits on the layers that matter) so the scheme is recognizable at a glance. It is not literally a llama.cpp k-quant — this is an MLX group-64 affine quant produced by our own per-module predicate (the bulk low-bit, sensitive layers high), then AWQ-calibrated block-by-block with a guaranteed-no-regression revert (a layer keeps AWQ scales only if they lower its quant error, else it falls back byte-for-byte to plain quant).

Use it

With chad (auto-downloads this model on first run):

uvx --from git+https://github.com/nathansutton/chad chad

Or directly with mlx-lm:

from mlx_lm import load, generate
model, tok = load("nathansutton/Ornith-1.0-35B-UD-Q2_K_XL-MLX")
print(generate(model, tok, "Write a haiku about quantization.", max_tokens=64))

License

Inherits the license of the base model deepreinforce-ai/Ornith-1.0-35B. Review it before use or redistribution.