AIOKiet/envit5-base-iwslt2015-en-vi

EnViT5-Base Fine-tuned for English-to-Vietnamese Translation

This model is a fine-tuned version of VietAI/envit5-base for English-to-Vietnamese neural machine translation.

The model was fine-tuned exclusively on the IWSLT2015 English-Vietnamese dataset as part of a controlled comparison with mBART-50 and NLLB-200-Distilled-600M.

Model Details

• Base model: VietAI/envit5-base
• Task: English-to-Vietnamese machine translation
• Architecture: Encoder-decoder Transformer
• Source language: English
• Target language: Vietnamese
• Framework: Hugging Face Transformers
• Training hardware: 1 × NVIDIA Tesla T4
• Precision: FP16
• Training epochs: 2
• Training steps: 33,330
• Random seed: 42
• Training time: approximately 4 hours 32 minutes
• Checkpoint in this repository: Final checkpoint after 2 epochs

This repository contains the final checkpoint after two training epochs.

It does not contain the intermediate checkpoint with the highest observed validation BLEU because checkpoint saving was disabled during this experimental run.

Training Data

The model was fine-tuned using:

nguyenvuhuy/iwslt2015-en-vi

Dataset splits:

Split	Sentence pairs
Train	133,317
Validation	1,268
Test	1,268

No additional parallel corpus, back-translation data, data augmentation, or synthetic data was used for this baseline.

Preprocessing

EnViT5 uses explicit language prefixes.

Source sentences were formatted as:

en: <English sentence>

Target sentences were formatted as:

vi: <Vietnamese sentence>

Maximum sequence lengths:

Parameter	Value
Maximum source length	128
Maximum target length	128

Target padding token IDs were replaced with -100 during training so that padding positions were ignored when computing the loss.

Training Hyperparameters

Hyperparameter	Value
Batch size	8
Gradient accumulation steps	1
Effective batch size	8
Learning rate	5e-5
Optimizer	AdamW
Weight decay	0.01
Number of epochs	2
Learning-rate scheduler	Linear
Warmup ratio	0.0
Maximum gradient norm	1.0
Mixed precision	FP16
Gradient checkpointing	Enabled
Random seed	42
Data seed	42
Evaluation interval	5,000 steps
Beam size	5
Maximum generation length	128

Validation Results

Validation performance observed during training:

Step	Training Loss	Validation Loss	BLEU ↑	chrF++ ↑	TER ↓
5,000	1.5481	1.372407	33.8194	51.6843	49.6093
10,000	1.4626	1.320226	35.5434	53.2691	45.5709
15,000	1.4020	1.283764	36.4253	54.0865	44.7437
20,000	1.1957	1.277159	36.6341	54.3397	44.5147
25,000	1.1858	1.259284	37.2454	54.8320	43.8307
30,000	1.1693	1.246683	37.1260	54.9289	44.0110

The highest observed validation BLEU was 37.2454 at step 25,000.

The lowest observed validation TER was 43.8307 at step 25,000.

The highest observed validation chrF++ was 54.9289 at step 30,000.

Because checkpoint saving was disabled for this run, the model hosted in this repository corresponds to the final checkpoint at step 33,330 rather than the intermediate checkpoint at step 25,000.

Test Results

The final checkpoint is evaluated on the IWSLT2015 test set using the same decoding configuration used for the other baseline models.

Evaluation configuration:

• Beam size: 5
• Maximum generation length: 128
• Test samples: 1,268
• BLEU implementation: SacreBLEU
• Additional metrics: chrF++ and TER
• COMET: Evaluated separately

Final test results will be added after completing the evaluation.

Metric	Score
BLEU ↑	TBD
chrF++ ↑	TBD
TER ↓	TBD
COMET ↑	TBD

Usage

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_name = "AIOKiet/envit5-base-iwslt2015-en-vi"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)

device = "cuda" if torch.cuda.is_available() else "cpu"

model = model.to(device)
model.eval()

text = "en: Artificial intelligence is changing the world."

inputs = tokenizer(
    text,
    return_tensors="pt",
    truncation=True,
    max_length=128
).to(device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        num_beams=5,
        max_length=128
    )

translation = tokenizer.decode(
    outputs[0],
    skip_special_tokens=True
)

print(translation)

The generated translation may include the vi: prefix because this prefix is part of the EnViT5 translation format.

For downstream evaluation, the prefix can be removed before computing translation metrics.

Intended Use

This model is intended for:

• English-to-Vietnamese machine translation experiments
• Neural machine translation research
• Controlled baseline comparisons
• Comparison with multilingual encoder-decoder models
• Parameter-efficient fine-tuning research
• English-Vietnamese translation experiments on IWSLT2015

Out-of-Scope Use

This model has not been evaluated for:

• Medical translation
• Legal translation
• Safety-critical translation
• Professional certified translation
• High-stakes production deployment

Additional evaluation is required before using the model in such settings.

Limitations

The model was fine-tuned only on the IWSLT2015 English-Vietnamese corpus, which primarily contains TED-talk-style parallel data.

Performance on other domains, including medical, legal, conversational, e-commerce, scientific, and technical text, has not been established.

The current repository contains the final two-epoch checkpoint rather than the best intermediate validation checkpoint.

Translation quality may also vary depending on sentence length, domain, terminology, and linguistic structure.

Evaluation Metrics

The following metrics are used in this experiment:

• BLEU: Higher is better
• chrF++: Higher is better
• TER: Lower is better
• COMET: Higher is better

BLEU, chrF++, and TER are computed using SacreBLEU.

COMET is evaluated separately for the final baseline comparison.

Experimental Context

This model is one of three controlled English-to-Vietnamese baselines:

Model	Role
mBART-50	Multilingual sequence-to-sequence baseline
NLLB-200-Distilled-600M	Multilingual machine translation baseline
EnViT5-Base	English-Vietnamese specialized baseline

The experiments use the same IWSLT2015 dataset and a controlled training protocol to support a fair comparison.

The main shared settings include:

• 2 training epochs
• Batch size of 8
• Learning rate of 5e-5
• AdamW optimizer
• Weight decay of 0.01
• Maximum source length of 128
• Maximum target length of 128
• FP16 training
• Beam size of 5
• Maximum generation length of 128
• Random seed of 42

Model-specific language handling is preserved according to the requirements of each pretrained architecture.

Reproducibility

Main experimental configuration:

Dataset: nguyenvuhuy/iwslt2015-en-vi

Train samples: 133317
Validation samples: 1268
Test samples: 1268

Epochs: 2
Training steps: 33330

Batch size: 8
Gradient accumulation steps: 1
Effective batch size: 8

Learning rate: 5e-5
Optimizer: AdamW
Weight decay: 0.01
LR scheduler: Linear
Warmup ratio: 0.0
Maximum gradient norm: 1.0

Maximum source length: 128
Maximum target length: 128

FP16: True
Gradient checkpointing: True

Beam size: 5
Maximum generation length: 128

Seed: 42
Data seed: 42

Compute Infrastructure

Hardware

• GPU: NVIDIA Tesla T4
• Number of GPUs: 1
• Training precision: FP16

Training Time

The two-epoch fine-tuning run required approximately:

4 hours 31 minutes 47 seconds

Checkpoint Selection

This experiment used a fixed training budget of two epochs.

Checkpoint saving was disabled during the run. Therefore, the model in this repository represents the final model after 33,330 optimization steps.

The best intermediate validation BLEU was observed at step 25,000, but the corresponding model weights were not retained.

For this baseline experiment, the final checkpoint is used to maintain the same checkpoint-selection protocol across the compared models.

Citation

A citation for the associated study will be added after publication.

When using this model, please also cite the original EnViT5 work and the IWSLT2015 dataset where appropriate.

Model Card Author

This model card documents an EnViT5-Base baseline fine-tuned for controlled English-to-Vietnamese neural machine translation experiments.