greyfoss's picture
Training complete
796b2fe
metadata
license: apache-2.0
base_model: google/mt5-small
tags:
  - summarization
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: mt5-small-finetuned-xlsum-en-zh
    results: []

mt5-small-finetuned-xlsum-en-zh

This model is a fine-tuned version of google/mt5-small on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 3.2988
  • Rouge1: 12.7272
  • Rouge2: 2.4338
  • Rougel: 10.5647
  • Rougelsum: 10.5889

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5.6e-05
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum
6.215 1.0 1000 3.5021 10.9361 1.9845 9.1348 9.0929
4.6051 2.0 2000 3.4190 11.6653 2.0884 9.6847 9.7251
4.3735 3.0 3000 3.3685 12.1941 2.2109 10.2818 10.237
4.2439 4.0 4000 3.3417 12.6308 2.4293 10.5036 10.5079
4.1552 5.0 5000 3.3148 12.5122 2.2873 10.3496 10.3545
4.0853 6.0 6000 3.3112 12.6426 2.3154 10.5514 10.5622
4.0508 7.0 7000 3.3048 12.843 2.3893 10.7232 10.7357
4.0293 8.0 8000 3.2988 12.7272 2.4338 10.5647 10.5889

Framework versions

  • Transformers 4.35.2
  • Pytorch 2.1.0+cu118
  • Datasets 2.15.0
  • Tokenizers 0.15.0