Menterium/translategemma-4b-it-onnx-int4-cpu

TranslateGemma 4B — ONNX Runtime GenAI, int4, CPU

ONNX Runtime GenAI build (int4 weights, fp32 activations, CPU execution provider) of the text component of google/translategemma-4b-it, Google's translation-specialized Gemma 3 model covering 55 languages.

Built for local, fully offline translation in Cortility (Menterium IT Solutions GmbH) on machines without a DirectX-12-capable GPU; usable with any onnxruntime-genai client. A GPU variant is published separately as translategemma-4b-it-onnx-int4-dml.

Conversion notes

• Text decoder extracted from the multimodal checkpoint (Gemma3ForCausalLM, model type gemma3_text); the vision tower is not included.
• Converted with the onnxruntime-genai model builder (python -m onnxruntime_genai.models.builder -p int4 -e cpu).
• genai_config.json ships with eos_token_id: [1, 106] so that <end_of_turn> terminates generation (the stock config only stopped on <eos>).

Prompt format

Gemma requires the <bos> token at the start of the prompt — without it the output degenerates. The tokenizer of onnxruntime-genai does not add it automatically:

<bos><start_of_turn>user
Translate the following text into English. Reply with ONLY the translation.

Text:
Wie heißt die Hauptstadt von Frankreich?<end_of_turn>
<start_of_turn>model

License

Gemma is provided under and subject to the Gemma Terms of Use found at ai.google.dev/gemma/terms. Use of this model is also subject to the Gemma Prohibited Use Policy. This repository redistributes a quantized derivative of TranslateGemma; all rights in the underlying model remain with Google.