bart-text-simplification_1e4_adafactor_biendata

This model is a fine-tuned version of facebook/bart-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.7599
• Rouge1: 29.7176
• Rouge2: 10.9512
• Rougel: 25.5101
• Rougelsum: 25.526
• Gen Len: 15.2029

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
No log	1.0	232	0.5813	30.604	12.4253	26.5172	26.4807	15.2241
No log	2.0	464	0.5739	31.9076	12.798	27.4728	27.4929	15.2241
1.0176	3.0	696	0.5700	31.3776	12.2852	27.1116	27.0878	15.6459
1.0176	4.0	928	0.5762	30.8731	12.3014	26.9196	26.8301	14.6353
0.4798	5.0	1160	0.5863	29.927	11.7166	25.9447	25.921	14.4297
0.4798	6.0	1392	0.6003	29.9528	11.2098	25.6908	25.7209	14.7414
0.3855	7.0	1624	0.6179	30.1161	11.2863	26.1433	26.1519	15.1698
0.3855	8.0	1856	0.6290	29.5566	11.1229	25.6003	25.5754	14.87
0.3092	9.0	2088	0.6538	29.7844	11.2434	25.8222	25.8067	14.9708
0.3092	10.0	2320	0.6698	28.9941	10.6603	25.0054	25.0198	15.0239
0.247	11.0	2552	0.6906	28.732	10.4525	24.8897	24.8953	14.9721
0.247	12.0	2784	0.7023	29.0609	10.4762	24.9678	24.9893	15.317
0.198	13.0	3016	0.7200	29.9516	11.2397	25.7347	25.7489	15.1485
0.198	14.0	3248	0.7263	29.1565	10.7363	25.2238	25.203	14.9761
0.198	15.0	3480	0.7376	30.0068	11.2078	26.0012	26.0235	14.9589
0.1602	16.0	3712	0.7489	29.8747	11.0555	25.7321	25.7543	15.2931
0.1602	17.0	3944	0.7487	29.6901	10.8692	25.5467	25.5808	15.2798
0.1342	18.0	4176	0.7553	29.5496	10.8611	25.2895	25.3218	15.3156
0.1342	19.0	4408	0.7590	29.7733	11.1577	25.671	25.6883	15.1313
0.1184	20.0	4640	0.7599	29.7176	10.9512	25.5101	25.526	15.2029

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3