T5_base_title_v4

This model is a fine-tuned version of t5-base on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.6697
• Rouge1: 0.4305
• Rouge2: 0.2304
• Rougel: 0.3728
• Rougelsum: 0.3729
• Gen Len: 16.6586

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.9653	1.0	2019	1.7927	0.4092	0.2145	0.3528	0.3528	16.6021
1.828	2.0	4038	1.7374	0.4148	0.217	0.3557	0.3558	16.7601
1.7597	3.0	6057	1.7053	0.4183	0.2199	0.3595	0.3594	16.8878
1.6787	4.0	8076	1.6875	0.4221	0.224	0.3649	0.3647	16.6098
1.6361	5.0	10095	1.6730	0.4227	0.2229	0.3655	0.3657	16.6044
1.6032	6.0	12114	1.6679	0.4266	0.227	0.3696	0.3697	16.4617
1.5701	7.0	14133	1.6657	0.4265	0.2273	0.3694	0.3692	16.4184
1.5359	8.0	16152	1.6677	0.4273	0.2274	0.3695	0.3695	16.5704
1.5136	9.0	18171	1.6639	0.4271	0.2278	0.3697	0.3697	16.5989
1.4776	10.0	20190	1.6641	0.4291	0.2297	0.3723	0.3722	16.5137
1.4507	11.0	22209	1.6650	0.4307	0.2303	0.372	0.3718	16.5868
1.437	12.0	24228	1.6654	0.4277	0.2274	0.3711	0.3711	16.7277
1.4428	13.0	26247	1.6689	0.4296	0.2287	0.3714	0.3715	16.7078
1.4183	14.0	28266	1.6697	0.4307	0.2301	0.3726	0.3725	16.6979
1.4244	15.0	30285	1.6697	0.4305	0.2304	0.3728	0.3729	16.6586

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.2
• Datasets 2.1.0
• Tokenizers 0.15.1