t5-small-hblastKP

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.0138
• Rouge1: 56.1916
• Rouge2: 46.6889
• Rougel: 56.0783
• Rougelsum: 56.1408
• Gen Len: 6.6737

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 8

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.2249	1.0	6222	1.0369	56.5144	47.7112	56.4169	56.5046	7.3609
1.027	2.0	12444	1.0361	55.904	46.7622	55.7977	55.9268	6.9696
0.9344	3.0	18666	1.0188	58.2277	49.2274	58.112	58.1932	7.1203
0.8994	4.0	24888	1.0171	57.8553	48.3674	57.7255	57.8761	6.7841
0.8559	5.0	31110	1.0138	56.1916	46.6889	56.0783	56.1408	6.6737
0.7633	6.0	37332	1.0178	58.1465	48.3511	57.9589	58.0973	6.8882
0.7444	7.0	43554	1.0221	58.295	48.8287	58.1788	58.2618	6.8316
0.7256	8.0	49776	1.0322	58.1448	48.5088	58.0255	58.1326	6.9370

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.1+cu121
• Datasets 2.18.0
• Tokenizers 0.15.2