Lernex/Metis-1.6-Logos-MLX-8bit

Metis-1.6 Logos MLX 8-bit

Lernex/Metis-1.6-Logos for Apple silicon, in MLX at 8 bits. It is nearly as close to the original as the BF16 weights themselves, for Macs with the memory to hold it.

Parameters	10.2B (latent mixture of experts, 192 experts, top 4)
Size	11.1 GB (8.69 bits per weight)
Context	128k tokens
Runs on	a Mac with 24 GB or more

Run it

pip install -U mlx-lm
mlx_lm.chat --model Lernex/Metis-1.6-Logos-MLX-8bit --trust-remote-code

--trust-remote-code is needed because this repository carries the model's MLX implementation, metis16.py, which mlx-lm runs in place of a built-in one until it ships Metis-1.6 itself. Read it before you trust it: it is the whole model in one file.

From Python:

from mlx_lm import load, generate

model, tokenizer = load("Lernex/Metis-1.6-Logos-MLX-8bit", trust_remote_code=True)
messages = [{"role": "user", "content": "What is 17 x 23?"}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, enable_thinking=True
)
print(generate(model, tokenizer, prompt=prompt, max_tokens=2048, verbose=True))

The model reasons inside <think> before it answers; enable_thinking=False (or --chat-template-config '{"enable_thinking": false}' with mlx_lm.generate) asks for a direct answer. The mlx-lm command line has no repetition penalty, so think at temperature 1.0, top-p 0.95, top-k 20 (--temp 1.0 --top-p 0.95 --top-k 20 for mlx_lm.generate; mlx_lm.chat takes --temp and --top-p only). For direct answers, use temperature 0.3.

mlx-lm 0.32.2 and earlier do not recognize this model's thinking markers, so they start in direct mode unless you pass enable_thinking explicitly, as above, and leave the reasoning in the reply. Later versions think by default and keep the reasoning apart.

As a server

mlx_lm.server --model Lernex/Metis-1.6-Logos-MLX-8bit --trust-remote-code

The endpoint is OpenAI-compatible; chat_template_kwargs sets enable_thinking per request. On mlx-lm newer than 0.32.2 the reasoning arrives in reasoning, apart from the answer. Tool calls arrive as text in the reply until mlx-lm includes this model's tool-call parser, and from then on as typed tool_calls.

Quality

Measured against FP32 logits from the native training runtime on the release's reference prompts (a 16-token thinking chat, a 176-token chat with tools, 300 tokens of prose): mean KL divergence over the top 20 tokens.

Weights	Thinking chat	Chat with tools	Prose
MLX 8-bit (this repository)	0.0043	0.0123	0.0044
MLX 4-bit	0.0197	0.0674	0.0175

For scale, the original BF16 weights score 0.0025, 0.013 and 0.0037 against the same reference, so the 8-bit model is close to the noise of BF16 itself.

How it is quantized

Every linear layer goes to 8 bits in groups of 64, except the parts this architecture is sensitive to:

• Unquantized: the Mamba-2 output projections, the expert routers, the per-pass attention adapters, the depth memory and the depth head.
• FP32: the routers' balance bias and the SSM A, D and dt_bias, as the reference keeps them.

The recipe is Model.quant_predicate in metis16.py, which mlx-lm applies whenever it quantizes this model. The same parts are kept wide in the GGUF and FP8 releases of this model.

Other formats

• Metis-1.6-Logos: BF16, Transformers
• Metis-1.6-Logos-MLX-4bit
• Metis-1.6-Logos-GGUF: llama.cpp, Ollama, LM Studio
• Metis-1.6-Logos-FP8: vLLM and SGLang

License

Apache 2.0, the same as the original model.