prekrasnypok's picture
Pronkin/model_name
a36ed3b
metadata
license: apache-2.0
base_model: google/mt5-small
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: mt5-small-finetuned-amazon-en-de
    results: []

mt5-small-finetuned-amazon-en-de

This model is a fine-tuned version of google/mt5-small on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 2.6824
  • Rouge1: 16.5188
  • Rouge2: 9.9087
  • Rougel: 16.3497
  • Rougelsum: 16.3207

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5.6e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
8.4221 1.0 651 3.1302 13.8418 6.2062 13.6147 13.7041
4.1085 2.0 1302 2.8969 13.842 7.0502 13.6309 13.7681
3.7329 3.0 1953 2.8285 13.4412 6.5045 13.2123 13.1854
3.5489 4.0 2604 2.7547 16.8572 9.781 16.8349 16.8095
3.4223 5.0 3255 2.7334 16.7217 9.9946 16.5297 16.5576
3.3509 6.0 3906 2.6994 16.8925 10.2889 16.7603 16.7358
3.2895 7.0 4557 2.6871 16.4238 9.974 16.3198 16.2857
3.281 8.0 5208 2.6824 16.5188 9.9087 16.3497 16.3207

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu121
  • Datasets 2.15.0
  • Tokenizers 0.15.0