Driw0x/my_awesome_opus_books_model

T5 Small fine-tuned on OPUS Books for English-to-French Translation

Model Description

This model is a fine-tuned version of google-t5/t5-small for English-to-French translation.

It was fine-tuned on the English-French configuration of the OPUS Books dataset and evaluated with SacreBLEU.

The model was created as a practical exercise while following the Hugging Face LLM Course.

• Developed by: Driw0x
• Model type: T5 encoder-decoder Transformer
• Languages: English → French
• Base model: google-t5/t5-small
• Task: Machine translation / sequence-to-sequence generation

Training and Evaluation Data

The model was fine-tuned on:

Helsinki-NLP/opus_books

Configuration:

en-fr

The original training split is divided into a new 80/20 train/evaluation split:

books = load_dataset("Helsinki-NLP/opus_books", "en-fr")
books = books["train"].train_test_split(test_size=0.2)

Because the generated evaluation split is used during model development, it should not be considered a completely untouched final benchmark.

Preprocessing

T5 is prompted explicitly for the translation task:

translate English to French: <text>

The preprocessing configuration is:

• source language: English (en);
• target language: French (fr);
• maximum tokenized length: 128;
• truncation enabled;
• dynamic sequence-to-sequence padding with DataCollatorForSeq2Seq.

Training Procedure

The model was fine-tuned with Seq2SeqTrainer.

Training Hyperparameters

Hyperparameter	Value
Learning rate	2e-5
Train batch size	16
Evaluation batch size	16
Number of epochs	2
Weight decay	0.01
Seed	42
Evaluation strategy	epoch
LR scheduler	linear
Mixed precision	FP16 / Native AMP
Predict with generate	True
Save total limit	3
Optimizer	AdamW (ADAMW_TORCH_FUSED)

Evaluation uses SacreBLEU on generated translations.

Training Results

Epoch	Training Loss	Validation Loss	BLEU	Generated Length
1	1.7705	1.5670	5.9461	17.5581
2	1.7483	1.5517	6.0284	17.5540

Final reported evaluation results:

• Validation loss: 1.5517
• SacreBLEU: 6.0284
• Mean generated length: 17.554

Usage

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

model_id = "Driw0x/my_awesome_opus_books_model"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSeq2SeqLM.from_pretrained(model_id)

text = "translate English to French: Legumes share resources with nitrogen-fixing bacteria."
inputs = tokenizer(text, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=40,
    do_sample=True,
    top_k=30,
    top_p=0.95,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Intended Uses

This model is primarily intended for:

• learning sequence-to-sequence translation with Transformers;
• experimenting with T5 fine-tuning;
• reproducing a Hugging Face translation workflow;
• educational English-to-French translation experiments.

Limitations

Important limitations include:

• the model is based on the relatively small t5-small checkpoint;
• the reported BLEU score is modest and the model should not be treated as a production translation system;
• OPUS Books is a literary-domain dataset and may not generalize well to conversational, technical, legal, or specialized text;
• sampled generation can produce different outputs across runs;
• biases and errors inherited from the base model and training corpus may remain.

This model was created as a course exercise and has not been validated for production or high-stakes translation.

Framework Versions

• Transformers 5.17.0
• PyTorch 2.11.0+cu130
• Datasets 4.8.5
• Tokenizers 0.23.2

Training Source

The complete training procedure is available in:

Driw0x/hf-ai-courses

Notebook:

llm-course/1-transformer-models/notebooks/translation.ipynb