no-sum-t5-3

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1815
• Rouge1: 33.2039
• Rouge2: 12.1054
• Rougel: 32.7253
• Rougelsum: 32.7032
• Gen Len: 19.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	139	1.6711	22.764	3.4862	22.3717	22.3978	19.0
No log	2.0	278	1.3811	31.9748	10.5421	31.4438	31.4451	19.0
No log	3.0	417	1.2981	32.0918	11.2326	31.4844	31.5121	19.0
1.9326	4.0	556	1.2542	32.0848	11.1865	31.6965	31.669	19.0
1.9326	5.0	695	1.2250	32.6516	11.7983	32.117	32.1061	19.0
1.9326	6.0	834	1.2063	33.0203	11.973	32.5571	32.5407	19.0
1.9326	7.0	973	1.1932	33.1756	11.7729	32.5844	32.6138	19.0
1.3969	8.0	1112	1.1886	33.0571	11.792	32.5059	32.4676	19.0
1.3969	9.0	1251	1.1825	33.2126	12.1217	32.6524	32.6517	19.0
1.3969	10.0	1390	1.1815	33.2039	12.1054	32.7253	32.7032	19.0

Framework versions

• Transformers 4.38.2
• Pytorch 2.1.0+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2