bart-text-simplification_1e4_adafactor_newsela

This model is a fine-tuned version of facebook/bart-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5221
• Rouge1: 53.696
• Rouge2: 36.5456
• Rougel: 50.0629
• Rougelsum: 50.0673
• Gen Len: 18.558

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.7479	1.0	803	0.3428	55.7433	39.7505	52.5585	52.6043	18.5474
0.2505	2.0	1606	0.3552	54.8713	38.517	51.9121	51.9413	18.4364
0.213	3.0	2409	0.3733	55.0367	38.8217	51.5907	51.6237	18.8225
0.167	4.0	3212	0.3933	55.0962	38.7575	51.9311	51.9376	18.7433
0.1412	5.0	4015	0.4097	54.8308	38.2353	51.5186	51.5117	18.611
0.1193	6.0	4818	0.4258	53.8669	37.2692	50.4845	50.4928	18.6443
0.1039	7.0	5621	0.4395	54.1498	37.7107	50.9405	50.9451	18.5728
0.0928	8.0	6424	0.4502	53.9131	37.1201	50.6696	50.6776	18.5488
0.0801	9.0	7227	0.4594	53.8123	37.0674	50.4964	50.4957	18.4986
0.0734	10.0	8030	0.4733	53.8377	36.8825	50.3857	50.3775	18.4569
0.0648	11.0	8833	0.4747	53.3192	36.0006	49.724	49.7651	18.4844
0.0601	12.0	9636	0.4888	54.0952	36.8581	50.6073	50.6233	18.5714
0.0558	13.0	10439	0.4903	53.2469	36.1195	49.7181	49.7835	18.4123
0.0506	14.0	11242	0.4987	53.3193	36.3095	49.7999	49.8537	18.4958
0.0484	15.0	12045	0.5051	53.297	36.1379	49.5479	49.5797	18.4144
0.0444	16.0	12848	0.5134	53.696	36.768	50.0134	50.0706	18.5813
0.042	17.0	13651	0.5162	53.4729	36.5564	49.8635	49.8709	18.5269
0.0404	18.0	14454	0.5165	53.5562	36.4654	49.9419	49.9367	18.524
0.0376	19.0	15257	0.5195	53.3768	36.359	49.7394	49.7357	18.5877
0.0365	20.0	16060	0.5221	53.696	36.5456	50.0629	50.0673	18.558

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3