t5-small-finetuned-xsum

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6799
• Rouge1: 16.5588
• Rouge2: 10.1416
• Rougel: 15.5658
• Rougelsum: 15.5525
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.002
• train_batch_size: 10
• eval_batch_size: 10
• seed: 42
• gradient_accumulation_steps: 5
• total_train_batch_size: 50
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	180	2.0606	12.0435	4.427	10.6651	10.6054	18.888
No log	2.0	360	1.4788	17.2418	9.6974	16.1178	16.1121	19.0
2.5647	3.0	540	1.2028	16.3251	9.0308	15.2903	15.2937	19.0
2.5647	4.0	720	1.0332	16.3718	9.5348	15.3831	15.3778	19.0
2.5647	5.0	900	0.9030	16.5137	9.7914	15.5124	15.5142	19.0
1.1692	6.0	1080	0.8346	16.82	10.2316	15.7513	15.7496	19.0
1.1692	7.0	1260	0.7406	16.6103	9.8786	15.5361	15.5297	19.0
1.1692	8.0	1440	0.6799	16.5588	10.1416	15.5658	15.5525	19.0

Framework versions

• Transformers 4.27.0.dev0
• Pytorch 1.13.1+cu116
• Datasets 2.9.0
• Tokenizers 0.13.2