Instructions to use ShinePixelOrg/translategemma-27b-it-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ShinePixelOrg/translategemma-27b-it-NVFP4 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ShinePixelOrg/translategemma-27b-it-NVFP4")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ShinePixelOrg/translategemma-27b-it-NVFP4")
model = AutoModelForMultimodalLM.from_pretrained("ShinePixelOrg/translategemma-27b-it-NVFP4", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ShinePixelOrg/translategemma-27b-it-NVFP4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ShinePixelOrg/translategemma-27b-it-NVFP4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ShinePixelOrg/translategemma-27b-it-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ShinePixelOrg/translategemma-27b-it-NVFP4

SGLang

How to use ShinePixelOrg/translategemma-27b-it-NVFP4 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ShinePixelOrg/translategemma-27b-it-NVFP4" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ShinePixelOrg/translategemma-27b-it-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ShinePixelOrg/translategemma-27b-it-NVFP4" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ShinePixelOrg/translategemma-27b-it-NVFP4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ShinePixelOrg/translategemma-27b-it-NVFP4 with Docker Model Runner:
```
docker model run hf.co/ShinePixelOrg/translategemma-27b-it-NVFP4
```

Access Gemma on Hugging Face

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

To access Gemma on Hugging Face, you’re required to review and agree to Google’s usage license. To do this, please ensure you’re logged in to Hugging Face and click below. Requests are processed immediately.

TranslateGemma 27B IT NVFP4

NVFP4 (W4A4) quantization of google/translategemma-27b-it, produced with llm-compressor GPTQ and saved in compressed-tensors nvfp4-pack-quantized format.

These weights are a derivative of the official Google TranslateGemma 27B instruction-tuned checkpoint. Use is governed by the same Gemma Terms of Use as the official weights.

Terms: https://ai.google.dev/gemma/terms
Prohibited Use Policy: https://ai.google.dev/gemma/prohibited_use_policy
Official model card: https://huggingface.co/google/translategemma-27b-it
Technical report: https://arxiv.org/pdf/2601.09012

Access stays gated. Acknowledge Google’s Gemma license before downloading, the same way you do for google/translategemma-27b-it.

What changed

Text Linear layers (attention Q/K/V/O and MLP gate/up/down) are GPTQ-quantized to NVFP4. Everything else stays BF16:

Precision	Modules
NVFP4 W4A4	Language-model `Linear` layers
BF16	Vision tower (SigLIP), multimodal projector, token embeddings, norms

lm_head is tied to the embeddings in this checkpoint, so there is no separate output-head tensor. The saved ignore list also names lm_head.

NVFP4 settings from config.json / recipe.yaml:

format: nvfp4-pack-quantized
weights and input activations: 4-bit float, symmetric, group size 16
scales: float8_e4m3fn
GPTQ: block_size 128, dampening_frac 0.01, static activation order, Hessians offloaded
compressed-tensors 0.14.0.1

Safetensors weight files shrink from 51.10 GiB (54,864,980,440 bytes) to 16.83 GiB (18,067,502,952 bytes). That is 3.04× smaller, and 67.1% of the original bytes are removed. In vLLM 0.33.0 the loaded weight memory was 51.54 GiB for BF16 and 17.42 GiB for NVFP4 (2.96×). Vision, embeddings, and the projector stay BF16, so the whole checkpoint does not reach a pure 4× cut.

Architecture is unchanged from the base Gemma 3 27B IT translation model: 62 decoder layers, hidden size 5376, 32 query heads / 16 KV heads, sliding window 1024 with every 6th layer full attention, vocabulary 262208, SigLIP vision tower at 896×896 (256 tokens per image). max_position_embeddings is 131072. Google’s model card describes the intended translation context as about 2K tokens.

Calibration

GPTQ used 64 samples built by repeating 16 short translation prompts (Chinese, English, Japanese, German, French, Spanish, Korean, Russian, Arabic, Portuguese, Italian, Vietnamese, Thai, Indonesian, Polish, Dutch). max_seq_length was 512, batch size 1, sequential over Gemma3DecoderLayer. This is a small calibration set. The MetricX / COMET / MQM numbers in the official card are for the BF16 TranslateGemma models. The comparison below is a separate WMT23 sample scored on this machine.

Translation quality

Both checkpoints were served with the existing image vllm/vllm-openai:v0.33.0 (--enforce-eager, --max-model-len 2048, --chat-template-content-format openai). Decoding was greedy (temperature=0, max_tokens=512). The official TranslateGemma chat template supplied every prompt.

Test set: WMT23 news, 8 directions, 40 segments each (320 total). Segments were drawn with seed 2026 from sources of 40–400 characters. The reference is WMT23 refA. Scores are corpus chrF++ (primary; character n-grams, same settings for every language) and corpus BLEU (zh tokenization for Chinese, character BLEU for Japanese, 13a otherwise). Higher is better. Δ is NVFP4 minus BF16.

Direction	n	chrF++ BF16	chrF++ NVFP4	Δ	BLEU BF16	BLEU NVFP4	Δ
de→en	40	63.49	61.32	−2.17	37.69	35.33	−2.36
en→de	40	64.03	58.70	−5.33	39.09	32.20	−6.89
en→ja	40	24.93	20.22	−4.72	30.77	28.23	−2.54
en→ru	40	51.64	47.92	−3.72	25.39	20.40	−4.99
en→zh	40	28.05	23.32	−4.73	33.11	26.61	−6.50
ja→en	40	47.77	46.79	−0.98	20.37	18.71	−1.66
ru→en	40	59.27	57.57	−1.70	32.99	28.76	−4.23
zh→en	40	49.32	47.61	−1.72	21.29	19.56	−1.73
Macro average	320	48.56	45.43	−3.13	30.09	26.23	−3.86

Every request finished with stop. The drop is smaller when translating into English (chrF++ about −1.0 to −2.2) than when translating out of English (about −3.7 to −5.3). This is a 40-sentence sample of WMT23, not the full WMT24++ MetricX / COMET evaluation in the TranslateGemma technical report.

Generation speed

Single-request speed on one NVIDIA RTX PRO 6000 Blackwell (96 GB), measured with vllm/vllm-openai:v0.33.0 after other GPU services were stopped. Both checkpoints used the same flags: compilation mode 3, CUDA graph capture size 1, --max-num-seqs 1, --max-model-len 2048, --gpu-memory-utilization 0.62, --chat-template-content-format openai. Eight WMT23 prompts (one per direction above, 95–130 prompt tokens) each generated 128 tokens with temperature=0 and ignore_eos. One warmup request was discarded. Decode speed is (128 − 1) / (end − first token).

	BF16	NVFP4	NVFP4 / BF16
Decode	27.0 tok/s	73.5 tok/s	2.72×
Time to first token	73 ms	65 ms	1.12×

Across the eight prompts the decode rate stayed inside 26.9–27.1 tok/s for BF16 and 73.5–73.6 tok/s for NVFP4. NVFP4 used FlashInferCutlassNvFp4LinearKernel. A 128-token reply therefore takes about 4.8 s on BF16 and 1.8 s on NVFP4.

Serving check

Local load was checked with vLLM 0.33.0 on one NVIDIA GPU:

quantization backend: compressed-tensors
GEMM kernel: FlashInferCutlassNvFp4LinearKernel
checkpoint shards loaded (4/4)
GET /health returned 200
launch used --max-model-len 4096

Requests that do not follow TranslateGemma’s chat schema are rejected by the template (HTTP 400). vLLM 0.33.0 auto-detected this template as string content. If you see that warning, start the server with --chat-template-content-format openai so content stays a one-item list.

vllm serve ShinePixelOrg/translategemma-27b-it-NVFP4 \
  --max-model-len 8192 \
  --gpu-memory-utilization 0.85 \
  --chat-template-content-format openai

Text translation body:

{
  "model": "ShinePixelOrg/translategemma-27b-it-NVFP4",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "source_lang_code": "zh",
          "target_lang_code": "en",
          "text": "合同将于下月一日生效，付款期限为三十天。"
        }
      ]
    }
  ],
  "max_tokens": 256
}

Transformers

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "ShinePixelOrg/translategemma-27b-it-NVFP4"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id,
    device_map="auto",
    dtype=torch.bfloat16,
)

messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "text",
                "source_lang_code": "cs",
                "target_lang_code": "de-DE",
                "text": "V nejhorším případě i k prasknutí čočky.",
            }
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device, dtype=torch.bfloat16)
input_len = inputs["input_ids"].shape[-1]

with torch.inference_mode():
    generation = model.generate(**inputs, do_sample=False, max_new_tokens=200)

print(processor.decode(generation[0][input_len:], skip_special_tokens=True))

Image translation uses the same message shape with "type": "image" and a "url" (or image) field instead of "text". See the official model card for that example. The chat template in this repo is the official one: user content must be a list of exactly one item, with type of text or image, plus source_lang_code and target_lang_code.

TranslateGemma targets 55 languages. Codes the template does not know raise an error. Supported forms are ISO 639-1 codes (en) and regional codes (en-US or en_US).

Citation

@article{gemmatranslate2026,
  title={{TranslateGemma Technical Report}},
  url={https://arxiv.org/pdf/2601.09012},
  publisher={Google DeepMind},
  author={{Google Translate Research Team} and Finkelstein, Mara and Caswell, Isaac and Domhan, Tobias and Peter, Jan-Thorsten and Juraska, Juraj and Riley, Parker and Deutsch, Daniel and Dilanni, Cole and Cherry, Colin and Briakou, Eleftheria and Nielsen, Elizabeth and Luo, Jiaming and Agrawal, Sweta and Xu, Wenda and Kats, Erin and Jaskiewicz, Stephane and Freitag, Markus and Vilar, David},
  year={2026}
}

Downloads last month: 4

Safetensors

Model size

16B params

Tensor type

BF16

Model tree for ShinePixelOrg/translategemma-27b-it-NVFP4

Base model

google/translategemma-27b-it

Quantized

(25)

this model

Paper for ShinePixelOrg/translategemma-27b-it-NVFP4

TranslateGemma Technical Report

Paper • 2601.09012 • Published Jan 13 • 22