teex-pt/AMALIA-9B-0626-DPO-MLX-8bit

AMALIA-9B-0626-DPO — MLX 8-bit

8-bit MLX quantization of amalia-llm/AMALIA-9B-0626-DPO, the Portuguese (pt-PT) 9B model presented on 1 July 2026, for native Apple Silicon inference.

TL;DR of our measurements: 8-bit quantization of this model is free. This build is statistically indistinguishable from BF16 (+0.3% perplexity) while generating ~1.9× faster in 55% of the memory. If you run AMALIA on a Mac and care about output quality, this is the variant to download.

Usage

pip install mlx-lm
mlx_lm.chat --model teex-pt/AMALIA-9B-0626-DPO-MLX-8bit --max-tokens 1000

Prompt it in European Portuguese — that is what the model is tuned for. ~11 GB RAM needed; any M-series Mac with 16 GB+ works.

Default system prompt

When you don't supply a system message, the chat template injects a factual self-presentation (extended from the original model's one-liner): it tells the model it is Amália, a 9B open-source (Apache 2.0) Portuguese model named after Amália Rodrigues, presented on 1 July 2026, and instructs it to answer in the user's language (primarily pt-PT) and not to invent details about its origin. Without this, the base model confabulates its identity (we observed invented personas and funding programmes). Supplying your own system message fully overrides it.

Findings: how quantization affects AMALIA-9B

We converted and benchmarked five variants of this model (MLX and GGUF, 4-bit and 8-bit) on an Apple M5 Pro (48 GB, macOS 25.5, mlx-lm 0.31.3, llama.cpp b9850). All quality tests used greedy decoding (temp 0) with fixed pt-PT prompts, so outputs are exactly reproducible and diffable across variants.

Variant	Size	Gen speed	Perplexity Δ vs BF16¹	Verdict
BF16 (original)	17 GB	16–17 tok/s	—	reference
MLX 8-bit (this repo)	9.1 GB	30–32 tok/s	+0.3%	quality-free speedup
MLX 4-bit	4.8 GB	55–59 tok/s	+4.3%	fastest, small slips
GGUF Q8_0	9.1 GB	30 tok/s	−0.1% (noise)	Q8 is free here too
GGUF Q4_K_M	5.2 GB	48 tok/s	+2.7%	best 4-bit fidelity

¹ Deltas measured within each runtime family against its own BF16 baseline: MLX via a fixed pt-PT text under mlx-lm; GGUF via llama-perplexity over ~16.5k tokens of Portuguese Wikipedia prose. Absolute values are not comparable across runtimes; deltas are.

Key findings:

1. 8-bit is lossless in practice — in both runtimes the perplexity delta is within measurement noise, and greedy outputs are near-identical to BF16 across culture, grammar-correction, summarization, translation and JSON-formatting prompts.
2. 4-bit is cheap but not free — at 4-bit we observed occasional factual slips absent from BF16/8-bit: the MLX 4-bit hallucinated a nonexistent Camões work and leaked an English word ("Coastal") into a pt-PT JSON answer.
3. K-quants beat round-to-nearest at 4-bit — GGUF Q4_K_M degrades measurably less (+2.7%) than plain RTN 4-bit (+4.3%) at nearly the same size.
4. Generation speed tracks bytes, not runtime — MLX and llama.cpp are equally fast at equal bit-width on Apple Silicon (both ~30 tok/s at 8-bit); speed scales with memory bandwidth.
5. Long-context retrieval survives quantization — all variants passed a needle-in-haystack test at ~2k tokens.
6. All variants kept natural European Portuguese; quantization did not degrade fluency, only (at 4-bit) factual precision.

Raw benchmark data (bench-*.json) and the benchmark script (bench.py) are included in this repo for reproduction.

Detailed MLX numbers (this repo vs siblings)

Metric	BF16	Q8 (this repo)	Q4
Load time	3.9 s	1.1 s	0.7 s
Generation speed	16–17 tok/s	30–32 tok/s	55–59 tok/s
Prompt processing (~2k tok)	1,156 tok/s	1,190 tok/s	1,270 tok/s
Peak memory	19.5 GB	10.7 GB	6.2 GB
Perplexity (fixed pt-PT text)	4.321	4.334	4.508

Conversion command (mlx-lm 0.31.3):

mlx_lm.convert --hf-path amalia-llm/AMALIA-9B-0626-DPO -q --q-bits 8 --mlx-path amalia-mlx-8bit

Related repos

• teex-pt/AMALIA-9B-0626-DPO-MLX-4bit — fastest on Apple Silicon (55+ tok/s, 6 GB)
• teex-pt/AMALIA-9B-0626-DPO-GGUF — Q4_K_M + Q8_0 for Ollama/LM Studio/llama.cpp on any platform
• github.com/teex-pt/pt-amalia — benchmark scripts, conversion pipelines, findings, improvement plan

Attribution

All credit for the model goes to the AMALIA team — these repos are format conversions plus benchmark documentation. Original model: amalia-llm/AMALIA-9B-0626-DPO (Apache 2.0). EuroLLM-based, llama architecture, 32k context.