T5_base_title_v3 / README.md
Swarnava's picture
End of training
9441fcc verified
metadata
license: apache-2.0
base_model: t5-base
tags:
  - generated_from_trainer
metrics:
  - rouge
model-index:
  - name: T5_base_title_v3
    results: []

T5_base_title_v3

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.8544
  • Rouge1: 0.4104
  • Rouge2: 0.212
  • Rougel: 0.3579
  • Rougelsum: 0.3576
  • Gen Len: 16.585

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 16
  • eval_batch_size: 16
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 20
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Rouge1 Rouge2 Rougel Rougelsum Gen Len
2.2216 1.0 500 1.9241 0.3884 0.1992 0.3375 0.3371 16.168
1.9818 2.0 1000 1.8699 0.391 0.2002 0.342 0.341 16.386
1.8876 3.0 1500 1.8377 0.3972 0.2033 0.3447 0.3434 16.635
1.805 4.0 2000 1.8202 0.3981 0.2061 0.3482 0.3477 16.213
1.7422 5.0 2500 1.8180 0.395 0.2051 0.345 0.3445 16.74
1.6919 6.0 3000 1.8154 0.4042 0.2091 0.3526 0.3519 16.197
1.6426 7.0 3500 1.8160 0.4048 0.2094 0.3509 0.3506 16.546
1.5982 8.0 4000 1.8224 0.4088 0.2131 0.3556 0.3552 16.516
1.5615 9.0 4500 1.8219 0.4079 0.2121 0.3549 0.3545 16.474
1.5304 10.0 5000 1.8235 0.4059 0.2128 0.3548 0.3547 16.498
1.4996 11.0 5500 1.8299 0.4098 0.211 0.3569 0.3564 16.378
1.4735 12.0 6000 1.8349 0.4108 0.2129 0.3576 0.3572 16.672
1.4502 13.0 6500 1.8373 0.4103 0.2132 0.3566 0.3565 16.68
1.426 14.0 7000 1.8434 0.4092 0.2104 0.3559 0.3554 16.669
1.4137 15.0 7500 1.8442 0.4117 0.212 0.358 0.3576 16.547
1.401 16.0 8000 1.8503 0.4109 0.2126 0.3569 0.3567 16.552
1.3901 17.0 8500 1.8517 0.4115 0.2146 0.3601 0.36 16.553
1.3817 18.0 9000 1.8539 0.4104 0.2118 0.3572 0.3573 16.588
1.3696 19.0 9500 1.8553 0.4103 0.2126 0.3581 0.3578 16.568
1.369 20.0 10000 1.8544 0.4104 0.212 0.3579 0.3576 16.585

Framework versions

  • Transformers 4.37.2
  • Pytorch 2.1.2
  • Datasets 2.1.0
  • Tokenizers 0.15.1