t5-small-informal

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7186
• Rouge1: 80.8589
• Rouge2: 67.0673
• Rougel: 80.7564
• Rougelsum: 80.7571
• Gen Len: 14.9451
• Accuracy Log Reg: 0.8694

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len	Accuracy Log Reg
0.9513	1.0	5229	0.8179	79.3053	64.409	79.1848	79.1797	14.975	0.8545
0.8649	2.0	10458	0.7577	80.2047	66.004	80.1014	80.0958	14.9712	0.8665
0.8209	3.0	15687	0.7333	80.6018	66.664	80.5042	80.4997	14.9507	0.8681
0.801	4.0	20916	0.7219	80.7869	66.9538	80.6835	80.6831	14.9521	0.8693
0.7953	5.0	26145	0.7186	80.8589	67.0673	80.7564	80.7571	14.9451	0.8694

Framework versions

• Transformers 4.40.0
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.19.1