RoLlama-3.2-1B-mlx-8bit

An 8-bit MLX quantization of a RoLlama 3.2 1B checkpoint — a Romanian-adapted Llama 3.2 1B — for fast on-device inference on Apple Silicon.

• Architecture: Llama (LlamaForCausalLM), 16 layers, hidden size 2048, vocab 128256
• Quantization: MLX affine, 8-bit, group size 64
• Format: safetensors (single shard, ~1.59 GB), MLX-loadable
• Language: Romanian (ro)

Intended use

Built for Cloudless Voice (formerly Onit Dictate) as a language-specialized smart-cleanup / perplexity-scoring model. The default English Llama 3.2 1B scores Romanian text poorly; this model gives more natural scoring for Romanian dictation cleanup. In the app: Settings → Smart Formatting → Language model (or add it via "Add a custom model (Pro)" by pasting getonit/RoLlama-3.2-1B-mlx-8bit).

Usage (mlx-lm)

from mlx_lm import load, generate

model, tokenizer = load("getonit/RoLlama-3.2-1B-mlx-8bit")
print(generate(model, tokenizer, prompt="Salut, ce mai faci?", max_tokens=64))

License

Inherits the Llama 3.2 Community License from the base model, plus any terms of the RoLlama checkpoint it was adapted from.