mlx-community/translategemma-27b-it-6bit

This model mlx-community/translategemma-27b-it-6bit was converted to MLX format from google/translategemma-27b-it (via mlx-community/translategemma-27b-it-bf16) using mlx-lm version 0.30.4.

6-bit (6.50 bits per weight, 22.0 GB) targets 32 GB unified-memory Macs: the 8-bit conversion needs more memory than a 32 GB machine can dedicate to inference, while 6-bit affine quantization stays close to 8-bit output quality. For smaller machines see the 4-bit conversion.

Model Description

TranslateGemma 27B is Google's translation-specialized Gemma 3 model supporting 55 languages. It uses a structured translation message rendered through the model's chat template.

Use with mlx

pip install mlx-lm

from mlx_lm import load, generate

model, tokenizer = load("mlx-community/translategemma-27b-it-6bit")

messages = [{
    "role": "user",
    "content": [{
        "type": "text",
        "source_lang_code": "en",
        "target_lang_code": "pl",
        "text": "The city council approved the new bike path along the river.",
    }],
}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)

response = generate(model, tokenizer, prompt=prompt, verbose=True)

License

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms.