Edit model card

mT5-TextSimp-LT-BatchSize8-lr1e-4

This model is a fine-tuned version of google/mt5-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 0.0826
  • Rouge1: 0.6956
  • Rouge2: 0.532
  • Rougel: 0.6875
  • Sacrebleu: 41.0349
  • Gen Len: 38.0501

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 500
  • num_epochs: 8

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Sacrebleu Gen Len
22.5133 0.96 200 14.4822 0.0057 0.0 0.0056 0.0013 512.0
1.0276 1.91 400 0.7352 0.022 0.0005 0.0215 0.0232 41.4702
0.6477 2.87 600 1.5193 0.1021 0.012 0.0954 0.0573 83.3723
0.1784 3.83 800 0.1149 0.6014 0.4222 0.5898 32.2723 38.0501
0.158 4.78 1000 0.0930 0.6546 0.4822 0.6463 37.3842 38.0501
0.1059 5.74 1200 0.0884 0.6714 0.4983 0.6635 39.0129 38.0501
0.1542 6.7 1400 0.0830 0.688 0.5184 0.6803 40.419 38.0501
0.1206 7.66 1600 0.0826 0.6956 0.532 0.6875 41.0349 38.0501

Framework versions

  • Transformers 4.33.0
  • Pytorch 2.1.2+cu121
  • Datasets 2.14.4
  • Tokenizers 0.13.3
Downloads last month
1

Finetuned from