mlx-community/translategemma-12b-it-6bit

This model mlx-community/translategemma-12b-it-6bit was converted to MLX format from google/translategemma-12b-it using mlx-lm version 0.31.3.

6-bit (6.50 bits per weight, 9.6 GB) fills the gap between the 8-bit conversion (12.5 GB - slightly too large for comfortable use on 16 GB unified-memory Macs) and the 4-bit conversion (6.7 GB), while 6-bit affine quantization stays close to 8-bit output quality: in a 34-translation battery across 6 target languages plus reverse directions, this conversion's greedy outputs were byte-identical to the 8-bit conversion on 28 of 34 items; five of the six divergences are equal-quality paraphrases, and one (an idiom into Japanese) produced a correct first rendering followed by two unrequested alternatives.

Model Description

TranslateGemma 12B is Google's translation-specialized Gemma 3 model supporting 55 languages. It uses a structured translation message rendered through the model's chat template.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/translategemma-12b-it-6bit")

messages = [{
    "role": "user",
    "content": [{
        "type": "text",
        "source_lang_code": "en",
        "target_lang_code": "pl",
        "text": "The city council approved the new bike path along the river.",
    }],
}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=prompt, verbose=True)

License

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms.