Agisight/tyv-gemma4-e4b-mlx-8bit

Gemma 4 E4B — Russian ↔ Tuvan translation, MLX 8-bit

google/gemma-4-E4B-it fine-tuned for bidirectional Russian ↔ Tuvan translation (LoRA adapter: Agisight/tyv-gemma4-e4b-lora), merged into the base weights and quantized to 8 bits for Apple Silicon with mlx-vlm. Runs fully offline on a Mac.

• Size: 8.4 GB (9.0 bits per weight on average; some layers are kept at higher precision).
• Speed on a MacBook Air M4 (16 GB): 0.89 s per sentence (median), 8.9 GB peak memory.
• Other variant: Agisight/tyv-gemma4-e4b-mlx-4bit.

Results

Clean held-out test set, 1,369 pairs (NLLB v3 test split of Agisight/tyv-rus-200k with training duplicates removed). chrF++, greedy decoding.

Model	ru→tyv	tyv→ru
NLLB v3 (Agisight/nllb-rus-tyv-v3-dict_16.5k)	49.2	49.1
Fine-tuned Gemma 4 E4B, bf16 (adapter)	50.5	50.9
This 8-bit MLX model (full clean test, 1,369 pairs)	50.3	50.9

Lossless, but superseded for most uses. On the full clean test this full-vocabulary 8-bit model matches the bf16 model (50.3 / 50.9 vs. 50.5 / 50.9). For translation, use Agisight/tyv-gemma4-e4b-pruned-mlx-6bit instead: same quality (50.2 / 50.8), 3.2 GB instead of 8.4 GB, text-only with a pruned vocabulary. Keep this version if you need Gemma's vision or audio towers.

The bf16 adapter significantly outperforms NLLB v3 in both directions (paired bootstrap, p < 0.05) and also on BLEU, normalized chrF++ and COMET — see the adapter card for the full table.

Prompt format

Single-turn chat message with a short prefix, exactly as in training:

• Russian → Tuvan: ru→tyv: <Russian text>
• Tuvan → Russian: tyv→ru: <Tuvan text>

Usage

pip install -U mlx-vlm
mlx_vlm.generate --model Agisight/tyv-gemma4-e4b-mlx-8bit \
  --prompt "ru→tyv: Завтра я поеду в Кызыл к родителям." --max-tokens 64 --temperature 0.0

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("Agisight/tyv-gemma4-e4b-mlx-8bit")
prompt = apply_chat_template(processor, model.config, "ru→tyv: Завтра я поеду в Кызыл к родителям.")
print(generate(model, processor, prompt, max_tokens=64, temperature=0.0))

Notes

• The vision and audio towers of Gemma 4 are kept, but the model was trained for text translation only.
• On iPhone, 8.4 GB exceeds Apple's ~2 GB guidance for on-device models; a pruned-vocabulary version for iOS is planned.

Code

Training notebook (Colab), data preparation, conversion and evaluation scripts: Agisight/tyv-translation-mlx.

Credits

• Base model: Google, Gemma 4 E4B.
• Data, fine-tuning, evaluation and MLX conversion: Ali Kuzhuget.