teex-pt/AMALIA-9B-0626-DPO-MLX-4bit

AMALIA-9B-0626-DPO — MLX 4-bit

4-bit MLX quantization of amalia-llm/AMALIA-9B-0626-DPO, the Portuguese (pt-PT) 9B model presented on 1 July 2026, for native Apple Silicon inference.

This is the speed build: ~3.4× faster than BF16 (55–59 tok/s on an M5 Pro) in only ~6 GB of memory — it runs on an 8 GB M-series Mac. The cost is measurable but small: +4.3% perplexity and occasional factual slips (details below). For maximum fidelity use the 8-bit build, which is lossless in practice.

Usage

pip install mlx-lm
mlx_lm.chat --model teex-pt/AMALIA-9B-0626-DPO-MLX-4bit --max-tokens 1000

Prompt it in European Portuguese — that is what the model is tuned for.

Default system prompt

When you don't supply a system message, the chat template injects a factual self-presentation (extended from the original model's one-liner): it tells the model it is Amália, a 9B open-source (Apache 2.0) Portuguese model named after Amália Rodrigues, presented on 1 July 2026, and instructs it to answer in the user's language (primarily pt-PT) and not to invent details about its origin. Without this, the base model confabulates its identity (we observed invented personas and funding programmes). Supplying your own system message fully overrides it.

Findings: how quantization affects AMALIA-9B

We converted and benchmarked five variants of this model (MLX and GGUF, 4-bit and 8-bit) on an Apple M5 Pro (48 GB, macOS 25.5, mlx-lm 0.31.3, llama.cpp b9850). All quality tests used greedy decoding (temp 0) with fixed pt-PT prompts, so outputs are exactly reproducible and diffable across variants.

Variant	Size	Gen speed	Perplexity Δ vs BF16¹	Verdict
BF16 (original)	17 GB	16–17 tok/s	—	reference
MLX 8-bit	9.1 GB	30–32 tok/s	+0.3%	quality-free speedup
MLX 4-bit (this repo)	4.8 GB	55–59 tok/s	+4.3%	fastest, small slips
GGUF Q8_0	9.1 GB	30 tok/s	−0.1% (noise)	Q8 is free here too
GGUF Q4_K_M	5.2 GB	48 tok/s	+2.7%	best 4-bit fidelity

¹ Deltas measured within each runtime family against its own BF16 baseline: MLX via a fixed pt-PT text under mlx-lm; GGUF via llama-perplexity over ~16.5k tokens of Portuguese Wikipedia prose. Absolute values are not comparable across runtimes; deltas are.

What 4-bit costs, concretely. In greedy side-by-side comparison against BF16 this variant:

• hallucinated nonexistent/misattributed works in a Luís de Camões question where BF16 and 8-bit answered correctly;
• leaked one English word into an otherwise-correct pt-PT JSON answer ("regiao": "Coastal");
• chose a more verbose path on a multi-step arithmetic problem, needing more tokens to reach the (correct) method.

Grammar correction, translation, summarization, JSON structure, long-context retrieval (needle at ~2k tokens) and overall pt-PT fluency remained solid. For chat and drafting, the 3.4× speedup is usually worth it; for factual or precision-critical work, prefer 8-bit.

Note for cross-platform users: at the same ~4-bit budget, the GGUF Q4_K_M build measures less degradation (+2.7% vs +4.3%) thanks to its mixed-precision K-quant layout — MLX 4-bit wins on raw Mac speed (55 vs 48 tok/s).

Raw benchmark data (bench-*.json) and the benchmark script (bench.py) are included in this repo for reproduction.

Conversion command (mlx-lm 0.31.3):

mlx_lm.convert --hf-path amalia-llm/AMALIA-9B-0626-DPO -q --mlx-path amalia-mlx-4bit

(4 bits per weight, group size 64, round-to-nearest.)

Related repos

• teex-pt/AMALIA-9B-0626-DPO-MLX-8bit — near-lossless, recommended for quality (30 tok/s, 11 GB)
• teex-pt/AMALIA-9B-0626-DPO-GGUF — Q4_K_M + Q8_0 for Ollama/LM Studio/llama.cpp on any platform
• github.com/teex-pt/pt-amalia — benchmark scripts, conversion pipelines, findings, improvement plan

Attribution

All credit for the model goes to the AMALIA team — these repos are format conversions plus benchmark documentation. Original model: amalia-llm/AMALIA-9B-0626-DPO (Apache 2.0). EuroLLM-based, llama architecture, 32k context.