mt5.balanced.old / README.md
samzirbo's picture
End of training
3597a41 verified
metadata
base_model: samzirbo/mT5.en-es.pretrained
tags:
  - generated_from_trainer
metrics:
  - bleu
model-index:
  - name: mt5.balanced
    results: []

mt5.balanced

This model is a fine-tuned version of samzirbo/mT5.en-es.pretrained on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.4858
  • Bleu: 39.6397
  • Meteor: 0.6687
  • Chrf++: 61.2886

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0005
  • train_batch_size: 64
  • eval_batch_size: 64
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_steps: 1000
  • training_steps: 30000

Training results

Training Loss Epoch Step Validation Loss Bleu Meteor Chrf++
4.0381 0.5398 3000 2.0786 30.3181 0.59 53.6607
2.3046 1.0795 6000 1.8303 34.3855 0.6263 57.2186
2.0531 1.6193 9000 1.6969 36.2615 0.643 58.7366
1.9175 2.1591 12000 1.6209 37.2847 0.6507 59.5299
1.814 2.6988 15000 1.5698 38.2556 0.6599 60.3273
1.746 3.2386 18000 1.5331 38.7361 0.6629 60.6706
1.6921 3.7783 21000 1.5058 39.1633 0.6658 60.9563
1.6484 4.3181 24000 1.4933 39.4633 0.6677 61.1991
1.6291 4.8579 27000 1.4869 39.5943 0.6687 61.2864
1.6185 5.3976 30000 1.4858 39.6397 0.6687 61.2886

Framework versions

  • Transformers 4.40.1
  • Pytorch 2.2.1+cu121
  • Datasets 2.19.0
  • Tokenizers 0.19.1