mlx-community/Tess-4-27B-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs

A 4-bit mixed-precision MLX quant of migtissera/Tess-4-27B, an agentic reasoning model built on Qwen3.6-27B. Sensitive layers are kept at 8-bit and robust ones at 4-bit.

52 GB of bf16 weights become 19 GB, which fits a 24 GB Mac.

Image input works. The vision tower is kept at bf16 in a sidecar, so this quant takes images as well as text.

Quantization details

Property	Value
Predominant precision	4-bit
Layers at 8-bit (sensitive)	220
Layers at 4-bit (robust)	276
Total quantized layers	496
Achieved bits per weight	4.769
Group size	64
Vision tower	bf16, 333 tensors, in optiq/optiq_vision.safetensors
Bundled MTP head	optiq/mtp.safetensors (4-bit projections, BF16 norms)
Size on disk	19 GB, from a 52 GB bf16 base

We follow the same naming convention llama.cpp uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is the predominant precision, not the weighted average.

How the bit-widths were chosen

The per-layer allocation is transferred from mlx-community/Qwen3.6-27B-OptiQ-4bit, where it was derived by a KL-divergence sensitivity sweep against the bf16 reference on a six-domain calibration mix.

Tess-4-27B is a fine-tune of Qwen/Qwen3.6-27B and its architecture is unchanged (every field of the text config matches), so all 496 quantizable layers map across exactly and the allocation lands at the same 4.769 bits per weight when recomputed against Tess's own tensors.

These are measured bit-widths, not a static rule-of-thumb recipe. But they were measured on the base model, not on this fine-tune. Fine-tuning shifts weights, so Tess's own per-layer sensitivities could differ somewhat from the base's. Which layers are fragile is mostly a property of the architecture, so the transfer is sound, but it is a transfer and you should know that.

Only the language tower is quantized. The vision tower stays at bf16, which is how every OptiQ VLM ships.

Usage

Text

The sidecars live in an optiq/ subfolder, so a stock *.safetensors glob ignores them and mlx-lm sees a clean language model.

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Tess-4-27B-OptiQ-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain the tradeoffs of optimistic concurrency control."}],
    add_generation_prompt=True, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))

This is a reasoning model: it thinks inside <think>...</think> before answering, so give it enough max_tokens to finish.

Images

Image input needs mlx-optiq, which loads the bf16 vision sidecar and feeds the merged embeddings to the quantized language tower:

pip install mlx-optiq

from PIL import Image
from optiq.runtime.engine import OptiqEngine

engine = OptiqEngine("mlx-community/Tess-4-27B-OptiQ-4bit")
answer = engine.generate("What is in this image?",
                         images=[Image.open("photo.jpg")], max_tokens=512)
print(answer.text)

Or serve it over an OpenAI-compatible endpoint that accepts image content parts:

optiq serve --model mlx-community/Tess-4-27B-OptiQ-4bit

Speculative decoding (MTP)

The base ships a Multi-Token Prediction head, bundled here as optiq/mtp.safetensors:

optiq serve --model mlx-community/Tess-4-27B-OptiQ-4bit --mtp

Verification

Text, arithmetic reasoning, and image understanding were all exercised on the finished artifact before release.

The quantization was also checked numerically: dequantizing individual layers out of the artifact and comparing them against the bf16 checkpoint gives 0.7% mean relative error on the 8-bit layers and 9.8% on the 4-bit layers, which is what each bit-width should cost.

No task benchmarks were run on this quant; for measured quality numbers on the base architecture, see the Qwen3.6-27B OptiQ card.

Quantization does not change the behaviour or alignment of the base model. Use it under the same terms as the original.