t5-small-finetuned-xlsum-with-multi-news-10-epoch

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.2332
• Rouge1: 31.4802
• Rouge2: 9.9475
• Rougel: 24.6687
• Rougelsum: 24.7013
• Gen Len: 18.8025

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
2.7314	1.0	20543	2.3867	29.3997	8.2875	22.8406	22.8871	18.8204
2.6652	2.0	41086	2.3323	30.3992	8.9058	23.6168	23.6626	18.8447
2.632	3.0	61629	2.3002	30.8662	9.2869	24.0683	24.11	18.8122
2.6221	4.0	82172	2.2785	31.143	9.5737	24.3473	24.381	18.7911
2.5925	5.0	102715	2.2631	31.2144	9.6904	24.4419	24.4796	18.8133
2.5812	6.0	123258	2.2507	31.3371	9.7959	24.5801	24.6166	18.7836
2.5853	7.0	143801	2.2437	31.3593	9.8156	24.5533	24.5852	18.8103
2.5467	8.0	164344	2.2377	31.368	9.8807	24.6226	24.6518	18.799
2.5571	9.0	184887	2.2337	31.4356	9.9092	24.6543	24.6891	18.8075
2.5563	10.0	205430	2.2332	31.4802	9.9475	24.6687	24.7013	18.8025

Framework versions

• Transformers 4.13.0
• Pytorch 1.13.1+cpu
• Datasets 2.8.0
• Tokenizers 0.10.3