fine_tuned_t5_small_model-naive-approach

This model is a fine-tuned version of t5-small on the None dataset. It achieves the following results on the evaluation set:

• Loss: 3.3579
• Rouge1: 0.3553
• Rouge2: 0.1154
• Rougel: 0.2155
• Rougelsum: 0.2154
• Gen Len: 130.1211
• Bert F1: 0.8401

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len	Bert F1
4.3358	2.1053	200	3.5813	0.3207	0.1049	0.1965	0.1964	112.5737	0.8379
3.6728	4.2105	400	3.4776	0.3307	0.1098	0.2012	0.2007	120.2947	0.8382
3.5819	6.3158	600	3.4250	0.3422	0.114	0.2086	0.2084	122.5947	0.8399
3.5355	8.4211	800	3.3926	0.345	0.1142	0.2106	0.2106	125.2474	0.8398
3.5078	10.5263	1000	3.3709	0.3475	0.113	0.2118	0.2117	128.4211	0.8386
3.4899	12.6316	1200	3.3615	0.3538	0.1145	0.2157	0.2155	130.8632	0.8396
3.4672	14.7368	1400	3.3579	0.3553	0.1154	0.2155	0.2154	130.1211	0.8401

Framework versions

• Transformers 4.45.1
• Pytorch 2.4.0
• Datasets 3.0.1
• Tokenizers 0.20.0