t5-small-finetuned

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4379
• Rouge1: 90.6092
• Rouge2: 83.4758
• Rougel: 90.5852
• Rougelsum: 90.5876
• Gen Len: 14.4013

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.7983	1.0	2615	0.5940	87.8038	78.5894	87.7459	87.7514	14.4888
0.6489	2.0	5230	0.4997	89.5362	81.5387	89.5084	89.5107	14.4385
0.6058	3.0	7845	0.4607	90.2817	82.8752	90.2513	90.2539	14.4189
0.5656	4.0	10460	0.4433	90.5289	83.3057	90.503	90.5057	14.4028
0.5558	5.0	13075	0.4379	90.6092	83.4758	90.5852	90.5876	14.4013

Framework versions

• Transformers 4.39.3
• Pytorch 2.1.2
• Datasets 2.1.0
• Tokenizers 0.15.2