Instructions to use nvidia/Riva-Translate-4B-Instruct-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nvidia/Riva-Translate-4B-Instruct-v2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/Riva-Translate-4B-Instruct-v2")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nvidia/Riva-Translate-4B-Instruct-v2")
model = AutoModelForCausalLM.from_pretrained("nvidia/Riva-Translate-4B-Instruct-v2", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use nvidia/Riva-Translate-4B-Instruct-v2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nvidia/Riva-Translate-4B-Instruct-v2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Riva-Translate-4B-Instruct-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nvidia/Riva-Translate-4B-Instruct-v2

SGLang

How to use nvidia/Riva-Translate-4B-Instruct-v2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nvidia/Riva-Translate-4B-Instruct-v2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Riva-Translate-4B-Instruct-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nvidia/Riva-Translate-4B-Instruct-v2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nvidia/Riva-Translate-4B-Instruct-v2",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nvidia/Riva-Translate-4B-Instruct-v2 with Docker Model Runner:
```
docker model run hf.co/nvidia/Riva-Translate-4B-Instruct-v2
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Riva-Translate-4B-Instruct-v2

Model Overview

The Riva-Translate-4B-Instruct-v2 Neural Machine Translation model translates text in English and 36 non-English languages. The supported languages are: English(en), Czech(cs), Danish(da), German(de), Greek(el), European Spanish(es-ES), LATAM Spanish(es-US), Finnish(fi), French(fr), Hungarian(hu), Italian(it), Lithuanian(lt), Latvian(lv), Dutch(nl), Norwegian(no), Polish(pl), European Portuguese(pt-PT), Brazilian Portuguese(pt-BR), Romanian(ro), Russian(ru), Slovak(sk), Swedish(sv), Simplified Chinese(zh-CN), Traditional Chinese(zh-TW), Japanese(ja), Hindi(hi), Korean(ko), Estonian(et), Slovenian(sl), Bulgarian(bg), Ukrainian(uk), Croatian(hr), Arabic(ar), Vietnamese(vi), Turkish(tr), Indonesian(id), Thai(th). It supports both sentence- and document-level translation. The model surpasses all in-house NMT models we've built so far.

Model Developer: NVIDIA

Model Dates: Riva-Translate-4B-Instruct-v2 was trained between Nov 2025 and May 2026.

License

GOVERNING TERMS: Use of the model is governed by the NVIDIA Open Model License Agreement ADDITIONAL INFORMATION: Apache License, Version 2.0.

Quick Start Guide

How to Choose the Language Pair

To select a language pair for translation, include one of the following tags in the system prompt:

en-zh-cn or en-zh: English to Simplified Chinese
en-zh-tw: English to Traditional Chinese
en-ar: English to Arabic
en-bg: English to Bulgarian
en-cs: English to Czech
en-da: English to Danish
en-de: English to German
en-el: English to Greek
en-es or en-es-es: English to European Spanish
en-es-us: English to Latin American Spanish
en-et: English to Estonian
en-fi: English to Finnish
en-fr: English to French
en-hi: English to Hindi
en-hr: English to Croatian
en-hu: English to Hungarian
en-id: English to Indonesian
en-it: English to Italian
en-ja: English to Japanese
en-ko: English to Korean
en-lt: English to Lithuanian
en-lv: English to Latvian
en-nl: English to Dutch
en-no: English to Norwegian
en-pl: English to Polish
en-pt or en-pt-pt: English to European Portuguese
en-pt-br: English to Brazilian Portuguese
en-ro: English to Romanian
en-ru: English to Russian
en-sk: English to Slovak
en-sl: English to Slovenian
en-sv: English to Swedish
en-th: English to Thai
en-tr: English to Turkish
en-uk: English to Ukrainian
en-vi: English to Vietnamese
zh-en or zh-cn-en: Simplified Chinese to English
zh-tw-en: Traditional Chinese to English
ar-en: Arabic to English
bg-en: Bulgarian to English
cs-en: Czech to English
da-en: Danish to English
de-en: German to English
el-en: Greek to English
es-en or es-es-en: European Spanish to English
es-us-en: Latin American Spanish to English
et-en: Estonian to English
fi-en: Finnish to English
fr-en: French to English
hi-en: Hindi to English
hr-en: Croatian to English
hu-en: Hungarian to English
id-en: Indonesian to English
it-en: Italian to English
ja-en: Japanese to English
ko-en: Korean to English
lt-en: Lithuanian to English
lv-en: Latvian to English
nl-en: Dutch to English
no-en: Norwegian to English
pl-en: Polish to English
pt-en or pt-pt-en: European Portuguese to English
pt-br-en: Brazilian Portuguese to English
ro-en: Romanian to English
ru-en: Russian to English
sk-en: Slovak to English
sl-en: Slovenian to English
sv-en: Swedish to English
th-en: Thai to English
tr-en: Turkish to English
uk-en: Ukrainian to English
vi-en: Vietnamese to English

Use it with Transformers

from transformers import AutoTokenizer, AutoModelForCausalLM


tokenizer = AutoTokenizer.from_pretrained("nvidia/Riva-Translate-4B-Instruct-v2")
model = AutoModelForCausalLM.from_pretrained("nvidia/Riva-Translate-4B-Instruct-v2")

messages = [
    {
        "role": "system",
        "content": "en-zh-cn",
    },
    {"role": "user", "content": "The GRACE mission is a collaboration between the NASA and German Aerospace Center.?"},
 ]
tokenized_chat = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to(model.device)
outputs = model.generate(tokenized_chat,  max_new_tokens=128, pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(outputs[0]))

Use it with vLLM

To install vllm, use the following pip command in a terminal within a supported environment.

pip install vllm: 0.19.1

Launch a vLLM server using the below python command. In this example, we use a context length of 8k as supported by the model.

FLASHINFER_DISABLE_VERSION_CHECK=1 python3 -m vllm.entrypoints.openai.api_server \
    --model nvidia/Riva-Translate-4B-Instruct-v2 \
    --dtype bfloat16 \
    --gpu-memory-utilization 0.95 \
    --max-model-len 8192 \
    --host 0.0.0.0 \
    --port 8000 \
    --tensor-parallel-size 1 \
    --served-model-name Riva-Translate-4B-Instruct-v2

Alternatively, you can use Docker to launch a vLLM server.

docker run --runtime nvidia --gpus all \
           -v ~/.cache/huggingface:/root/.cache/huggingface \
           -p 8000:8000 \
           --ipc=host \
           vllm/vllm-openai:v0.5.3.post1 \
           --model nvidia/Riva-Translate-4B-Instruct-v2 \
           --dtype bfloat16 \
           --gpu-memory-utilization 0.95 \
           --max-model-len 8192 \
           --host 0.0.0.0 \
           --port 8000 \
           --tensor-parallel-size 1 \
           --served-model-name Riva-Translate-4B-Instruct-v2

If you are using DGX Spark or Jetson Thor, please use this vllm container. On Jetson Thor, be sure to include --runtime nvidia when running the Docker container.

# On DGX SPark or Jetson Thor
docker run \
  --runtime nvidia \ # Remove this on DGX Spark
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -p 8000:8000 \
  --ipc=host \
  nvcr.io/nvidia/vllm:25.12.post1-py3 \
  vllm serve nvidia/Riva-Translate-4B-Instruct-v2 \
    --dtype bfloat16 \
    --gpu-memory-utilization 0.95 \
    --max-model-len 8192 \
    --host 0.0.0.0 \
    --port 8000 \
    --tensor-parallel-size 1 \
    --served-model-name Riva-Translate-4B-Instruct-v2

On Jetson Thor, the previous vLLM cache is not currently cleaned automatically, so it must be cleared manually. Always run this command on the host before serving any model on Jetson Thor.

sudo sysctl -w vm.drop_caches=3

Here is an example client code for vLLM.

curl http://localhost:8000/v1/chat/completions \
-H "Content-Type: application/json"
-d '{
"model": "Riva-Translate-4B-Instruct-v2",
"messages": [
      {"role": "system", "content": "en-zh"},
      {"role": "user", "content": "The GRACE mission is a collaboration between the NASA and German Aerospace Center.?"}
    ]
}'

Chat Template Structure

{%- set language_pairs = {
  'en-zh-cn': {'source': 'English', 'target': 'Simplified Chinese'},
  'en-zh': {'source': 'English', 'target': 'Simplified Chinese'},
  'en-zh-tw': {'source': 'English', 'target': 'Traditional Chinese'},
  'en-ar': {'source': 'English', 'target': 'Arabic'},
  'en-de': {'source': 'English', 'target': 'German'},
  'en-es': {'source': 'English', 'target': 'European Spanish'},
  'en-es-es': {'source': 'English', 'target': 'European Spanish'},
  'en-es-us': {'source': 'English', 'target': 'Latin American Spanish'},
  'en-fr': {'source': 'English', 'target': 'French'},
  'en-ja': {'source': 'English', 'target': 'Japanese'},
  'en-ko': {'source': 'English', 'target': 'Korean'},
  'en-ru': {'source': 'English', 'target': 'Russian'},
  'en-pt': {'source': 'English', 'target': 'Brazilian Portuguese'},
  'en-pt-br': {'source': 'English', 'target': 'Brazilian Portuguese'},
  'en-pt-pt': {'source': 'English', 'target': 'European Portuguese'},
  'zh-en': {'source': 'Simplified Chinese', 'target': 'English'},
  'zh-cn-en': {'source': 'Simplified Chinese', 'target': 'English'},
  'zh-tw-en': {'source': 'Traditional Chinese', 'target': 'English'},
  'ar-en': {'source': 'Arabic', 'target': 'English'},
  'de-en': {'source': 'German', 'target': 'English'},
  'es-en': {'source': 'European Spanish', 'target': 'English'},
  'es-es-en': {'source': 'European Spanish', 'target': 'English'},
  'es-us-en': {'source': 'Latin American Spanish', 'target': 'English'},
  'fr-en': {'source': 'French', 'target': 'English'},
  'ja-en': {'source': 'Japanese', 'target': 'English'},
  'ko-en': {'source': 'Korean', 'target': 'English'},
  'ru-en': {'source': 'Russian', 'target': 'English'},
  'pt-en': {'source': 'Brazilian Portuguese', 'target': 'English'},
  'pt-br-en': {'source': 'Brazilian Portuguese', 'target': 'English'},
  'en-it': {'source': 'English', 'target': 'Italian'},
  'it-en': {'source': 'Italian', 'target': 'English'},
  'en-nl': {'source': 'English', 'target': 'Dutch'},
  'nl-en': {'source': 'Dutch', 'target': 'English'},
  'en-pl': {'source': 'English', 'target': 'Polish'},
  'pl-en': {'source': 'Polish', 'target': 'English'},
  'en-cs': {'source': 'English', 'target': 'Czech'},
  'cs-en': {'source': 'Czech', 'target': 'English'},
  'en-sv': {'source': 'English', 'target': 'Swedish'},
  'sv-en': {'source': 'Swedish', 'target': 'English'},
  'en-da': {'source': 'English', 'target': 'Danish'},
  'da-en': {'source': 'Danish', 'target': 'English'},
  'en-fi': {'source': 'English', 'target': 'Finnish'},
  'fi-en': {'source': 'Finnish', 'target': 'English'},
  'en-no': {'source': 'English', 'target': 'Norwegian'},
  'no-en': {'source': 'Norwegian', 'target': 'English'},
  'en-hu': {'source': 'English', 'target': 'Hungarian'},
  'hu-en': {'source': 'Hungarian', 'target': 'English'},
  'en-ro': {'source': 'English', 'target': 'Romanian'},
  'ro-en': {'source': 'Romanian', 'target': 'English'},
  'en-bg': {'source': 'English', 'target': 'Bulgarian'},
  'bg-en': {'source': 'Bulgarian', 'target': 'English'},
  'en-uk': {'source': 'English', 'target': 'Ukrainian'},
  'uk-en': {'source': 'Ukrainian', 'target': 'English'},
  'en-sk': {'source': 'English', 'target': 'Slovak'},
  'sk-en': {'source': 'Slovak', 'target': 'English'},
  'en-hr': {'source': 'English', 'target': 'Croatian'},
  'hr-en': {'source': 'Croatian', 'target': 'English'},
  'en-sl': {'source': 'English', 'target': 'Slovenian'},
  'sl-en': {'source': 'Slovenian', 'target': 'English'},
  'en-et': {'source': 'English', 'target': 'Estonian'},
  'et-en': {'source': 'Estonian', 'target': 'English'},
  'en-lv': {'source': 'English', 'target': 'Latvian'},
  'lv-en': {'source': 'Latvian', 'target': 'English'},
  'en-lt': {'source': 'English', 'target': 'Lithuanian'},
  'lt-en': {'source': 'Lithuanian', 'target': 'English'},
  'en-el': {'source': 'English', 'target': 'Greek'},
  'el-en': {'source': 'Greek', 'target': 'English'},
  'en-tr': {'source': 'English', 'target': 'Turkish'},
  'tr-en': {'source': 'Turkish', 'target': 'English'},
  'en-id': {'source': 'English', 'target': 'Indonesian'},
  'id-en': {'source': 'Indonesian', 'target': 'English'},
  'en-vi': {'source': 'English', 'target': 'Vietnamese'},
  'vi-en': {'source': 'Vietnamese', 'target': 'English'},
  'en-th': {'source': 'English', 'target': 'Thai'},
  'th-en': {'source': 'Thai', 'target': 'English'},
  'en-hi': {'source': 'English', 'target': 'Hindi'},
  'hi-en': {'source': 'Hindi', 'target': 'English'} 
} -%}

{%- set system_message = '' -%}
{%- set source_lang = '' -%}
{%- set target_lang = '' -%}

{%- if messages[0]['role'] == 'system' -%}
  {%- set lang_pair = messages[0]['content'] | trim -%}
  {%- set messages = messages[1:] -%}
  {%- if lang_pair in language_pairs -%}
    {%- set source_lang = language_pairs[lang_pair]['source'] -%}
    {%- set target_lang = language_pairs[lang_pair]['target'] -%}
    {%- set system_message = 'You are an expert at translating text from ' + source_lang + ' to ' + target_lang + '.' -%}
  {%- else -%}
    {%- set system_message = 'You are a translation expert.' -%}
  {%- endif -%}
{%- endif -%}

{{- '<s>System\n' + system_message + '</s>\n' -}}

{%- for message in messages -%}
  {%- if (message['role'] in ['user']) != (loop.index0 % 2 == 0) -%}
    {{- raise_exception('Conversation roles must alternate between user and assistant') -}}
  {%- elif message['role'] == 'user' -%}
    {%- set user_content = (
          target_lang
          and 'What is the ' + target_lang + ' translation of the sentence: ' + message['content'] | trim
          or message['content'] | trim
        ) -%}
    {{- '<s>User\n' + user_content + '</s>\n' -}}
  {%- elif message['role'] == 'assistant' -%}
    {{- '<s>Assistant\n' + message['content'] | trim + '</s>\n' -}}
  {%- endif -%}
{%- endfor -%}

{%- if add_generation_prompt -%}
  {{ '<s>Assistant\n' }}
{%- endif -%}

Inference

Engine: HF, vLLM
Test Hardware: NVIDIA A100, H100 80GB, Jetson Thor, DGX Spark

Ethical Considerations

NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal model team to ensure this model meets requirements for the relevant industry and use case and addresses unforeseen product misuse.

We advise against circumvention of any provided safety guardrails contained in the Model without a substantially similar guardrail appropriate for your use case. For more details: Safety and Explainability Subcards.

For more detailed information on ethical considerations for this model, please see the Model Card++ Bias, and Privacy Subcards.

Please report model quality, risk, security vulnerabilities or NVIDIA AI Concerns here.

Technical Limitations & Mitigation:

Accuracy varies based on the characteristics of input (Domain, Use Case, Noise, Context, etc.). Grammar errors and semantic issues may be present. As a potential mitigation, the user can change the prompt to get a better translation.

Downloads last month: 180

Safetensors

Model size

4B params

Tensor type

BF16

Model tree for nvidia/Riva-Translate-4B-Instruct-v2

Base model

nvidia/Mistral-NeMo-12B-Base

Finetuned

(4)

this model

Quantizations

2 models