t5-small-task2

This model is a fine-tuned version of google-t5/t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5527
• Rouge L: 27.6493
• Bleu-4: 9.2240
• Bertscore F1: 86.3135

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge L	Bleu-4	Bertscore F1
0.4432	1.0	492	0.5810	26.8512	8.4811	85.8446
0.5458	2.0	984	0.5578	27.9861	9.1750	86.0499
0.5367	3.0	1476	0.5536	26.9743	8.6872	86.0634
0.5053	4.0	1968	0.5521	27.1588	8.8376	86.0979
0.5277	5.0	2460	0.5512	27.8130	9.4552	86.2264
0.4125	6.0	2952	0.5517	27.9565	9.0676	86.4354
0.4427	7.0	3444	0.5507	27.9484	9.4518	86.4661
0.4019	8.0	3936	0.5511	27.2611	9.2863	86.0906
0.4356	9.0	4428	0.5503	27.1845	9.3854	86.3148
0.3828	10.0	4920	0.5504	27.5384	9.5138	86.1584

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cu121
• Datasets 3.3.1
• Tokenizers 0.21.0