t5-small-finetuned-DEPlain

This model is a fine-tuned version of t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.4349
• Rouge1: 55.9974
• Rouge2: 33.5645
• Rougel: 49.3408
• Rougelsum: 50.3503
• Gen Len: 16.7644

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.8141	1.0	667	1.5924	55.8422	33.3789	49.0964	50.0345	16.7644
1.7476	2.0	1334	1.5489	55.8013	33.356	48.9789	49.9383	16.8058
1.6973	3.0	2001	1.5193	55.7584	33.2723	48.9591	49.8935	16.7725
1.6513	4.0	2668	1.4988	55.9388	33.5848	49.2591	50.1911	16.7823
1.6271	5.0	3335	1.4846	55.8441	33.4064	49.2314	50.2123	16.7994
1.6048	6.0	4002	1.4735	55.9061	33.4165	49.207	50.1571	16.8107
1.5856	7.0	4669	1.4647	55.9145	33.4539	49.2251	50.1857	16.7953
1.5711	8.0	5336	1.4548	55.9216	33.4538	49.2822	50.2536	16.7628
1.5586	9.0	6003	1.4504	55.9937	33.5651	49.2948	50.2935	16.7807
1.548	10.0	6670	1.4442	55.9368	33.5696	49.2953	50.292	16.7506
1.5394	11.0	7337	1.4409	56.0439	33.6125	49.3406	50.3633	16.7628
1.5358	12.0	8004	1.4380	56.0279	33.6056	49.3376	50.3537	16.7579
1.5252	13.0	8671	1.4357	55.9468	33.4637	49.2525	50.2542	16.7571
1.5225	14.0	9338	1.4353	55.9919	33.5532	49.3214	50.3302	16.766
1.523	15.0	10005	1.4349	55.9974	33.5645	49.3408	50.3503	16.7644

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.17.0
• Tokenizers 0.15.2