my_t5_small_test

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.4891
• Bleu: 0.0
• Gen Len: 6.8662

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 25
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	200	2.3870	0.0	6.8375
No log	2.0	400	2.2439	0.0	6.9825
2.7045	3.0	600	2.1468	0.0	6.8488
2.7045	4.0	800	2.0665	0.0	7.1575
2.2912	5.0	1000	2.0073	0.0	7.0238
2.2912	6.0	1200	1.9477	0.0	7.3162
2.2912	7.0	1400	1.8906	0.0	7.1425
2.1524	8.0	1600	1.8395	0.0	7.1438
2.1524	9.0	1800	1.7949	0.0	7.1825
2.032	10.0	2000	1.7475	0.0	7.0612
2.032	11.0	2200	1.7084	0.0	7.0075
2.032	12.0	2400	1.6749	0.0	6.9975
1.9334	13.0	2600	1.6440	0.0	6.8962
1.9334	14.0	2800	1.6142	0.0	7.0388
1.8689	15.0	3000	1.5895	0.0	6.9475
1.8689	16.0	3200	1.5692	0.0	6.8938
1.8689	17.0	3400	1.5509	0.0	6.8538
1.8137	18.0	3600	1.5340	0.0	6.8862
1.8137	19.0	3800	1.5244	0.0	6.8412
1.7782	20.0	4000	1.5137	0.0	6.8712
1.7782	21.0	4200	1.5053	0.0	6.8775
1.7782	22.0	4400	1.4980	0.0	6.8512
1.7453	23.0	4600	1.4930	0.0	6.8638
1.7453	24.0	4800	1.4901	0.0	6.8788
1.7352	25.0	5000	1.4891	0.0	6.8662

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1