ToPo-ToPo/Ornith-1.5-122B-mlx-4bit

MLX 4bit conversion of ornith-ai/Ornith-1.5-122B for Apple Silicon (mlx-vlm), with the vision tower kept.

It uses a mixed-precision recipe: the routed experts are 4bit and every other linear layer is 8bit. This is the same recipe as ToPo-ToPo/Ornith-1.5-35B-A3B-mlx-4bit.

Provenance (self-converted from official weights)

• Source: ornith-ai/Ornith-1.5-122B (license: mit)
• Tool: mlx-vlm 0.7.6 (mlx 0.32.3), mlx_vlm.convert(..., quantize=True, q_bits=4, q_group_size=64, quant_predicate=predicate) with the predicate below
• Quantization (affine, group size 64):
  • Routed experts (mlp.switch_mlp.{gate,up,down}_proj, 144 layers): 4bit
  • All other linear layers (470 layers): 8bit. This covers lm_head, embed_tokens, full attention, Gated DeltaNet projections, the shared expert and the routers.
  • Vision tower: not quantized (bf16)
• Size: about 72.7 GB of weights. The official 4bit is 69.6 GB but has no vision tower; the 8bit is 130.6 GB.
• Vision: the source checkpoint is Qwen3_5MoeForConditionalGeneration and contains the vision tower (333 tensors) and the image/video processor configs. This conversion keeps them, so the model accepts image input. The official ornith-ai/Ornith-1.5-122B-MLX-4bit is a text-only conversion without the vision tower.
• The MTP (multi-token prediction) weights of the source are not included. A separately extracted drafter is at ToPo-ToPo/Ornith-1.5-122B-MTP-bf16.

from mlx_vlm.utils import skip_multimodal_module

def predicate(path, module):
    if not hasattr(module, "to_quantized") or skip_multimodal_module(path):
        return False
    if ".switch_mlp." in path:
        return {"group_size": 64, "bits": 4}
    return {"group_size": 64, "bits": 8}

Why mixed precision

The official MLX 4bit uses the default uniform 4bit conversion, with only the routers and shared-expert gates kept at 8bit. Only 8 of the 256 experts run for each token, while the non-expert layers run for every token. Keeping the non-expert layers at 8bit costs little size and reduced quantization errors in our tests.

Small in-house evaluation, thinking off, 5 samples per case: 34 short Japanese/English probes and 20 tool-calling scenarios, each in Japanese and English. All models were served directly with mlx-vlm (no MTP) on the same Apple Silicon Mac.

Metric	8bit	official 4bit	this repo
Japanese tool-calling accuracy	98%	94%	99%
Valid tool-call format (Japanese)	98.6%	98.5%	100%
English tool-calling accuracy	99%	100%	100%
Responses with Simplified Chinese / Hangul (tool calling)	0%	3%	0%
Same, probes	2.4%	3.5%	3.5%
Thinking leaked with thinking off (probes)	0%	2.9%	0%
Probe accuracy	97.4%	100%	98.3%
Generation speed (no MTP, 400 tokens)	46.7 tok/s	60.9 tok/s	52.3 tok/s
Peak memory	130.9 GB	72.9 GB	72.9 GB

• The official 4bit failed one Japanese tool-calling scenario (drafting a meeting-reschedule email) in all 5 samples. It called an unrelated calendar tool or wrote the email without the tool. The 8bit and this repo passed it.
• Ornith-1.5-122B switches to Chinese in some Japanese idiom explanations even at 8bit. This repo's remaining probe cases are the same idioms, so they look like a habit of the model rather than quantization damage.
• The differences in probe accuracy are within one or two samples. The suite is small and does not cover long agentic tasks.

Validation

• Image input tested with mlx-vlm. The model correctly named the colors, shapes and positions in a synthetic test image.
• Tokenization and the chat template give the same token IDs as the source checkpoint on Japanese, English, code, tool-call markup and combining characters, with and without enable_thinking=False.

Usage

from mlx_vlm import load, generate
model, processor = load("ToPo-ToPo/Ornith-1.5-122B-mlx-4bit")