flan-t5-base-srbd

This model is a fine-tuned version of google/flan-t5-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2005
• Rouge1: 73.7352
• Rouge2: 65.6464
• Rougel: 72.5004
• Rougelsum: 73.5633
• Gen Len: 16.2793

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.4137	0.85	200	0.2799	71.5649	62.1166	70.2942	71.4139	16.4072
0.2614	1.7	400	0.2634	73.0795	63.9831	71.8233	72.9432	16.2217
0.2396	2.55	600	0.2147	73.0266	64.1461	71.8748	72.8353	16.2516
0.2128	3.4	800	0.2130	73.3643	64.7286	72.1957	73.2775	16.2473
0.188	4.26	1000	0.2106	73.632	64.8732	72.3219	73.4369	16.2751
0.1737	5.11	1200	0.2027	73.2334	64.2004	72.084	73.1345	16.2068
0.1649	5.96	1400	0.2037	73.7358	64.8953	72.5475	73.6218	16.2345
0.1543	6.81	1600	0.2012	73.7211	65.1263	72.527	73.6729	16.2665
0.1485	7.66	1800	0.2032	73.7563	65.5173	72.6785	73.6162	16.2473
0.1482	8.51	2000	0.2010	73.9481	65.4549	72.8475	73.7871	16.2921
0.1471	9.36	2200	0.2005	73.7352	65.6464	72.5004	73.5633	16.2793

Framework versions

• Transformers 4.33.2
• Pytorch 2.0.1
• Datasets 2.14.5
• Tokenizers 0.13.3