pegasus-text-simplification_1e4_adafactor_wikilarge_20epici

This model is a fine-tuned version of google/pegasus-x-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4783
• Rouge1: 72.0576
• Rouge2: 53.6457
• Rougel: 66.9028
• Rougelsum: 66.9302
• Gen Len: 21.9895

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
0.5187	1.0	1163	0.4502	73.6687	54.9499	68.2544	68.2568	23.6492
0.4798	2.0	2326	0.4472	73.8465	55.5415	68.5292	68.5457	23.4241
0.4542	3.0	3489	0.4456	74.0031	55.521	68.7023	68.6871	23.0838
0.4376	4.0	4652	0.4476	73.4239	55.0591	68.1931	68.1833	22.9529
0.4226	5.0	5815	0.4484	73.1859	54.7558	67.8843	67.9234	22.6702
0.4108	6.0	6978	0.4517	72.8602	54.3813	67.5761	67.6266	22.356
0.3993	7.0	8141	0.4532	73.0413	54.7254	67.7802	67.8065	22.5131
0.391	8.0	9304	0.4590	72.9448	54.3763	67.689	67.6712	22.5079
0.3844	9.0	10467	0.4579	72.6204	54.107	67.3903	67.3863	22.2775
0.3773	10.0	11630	0.4606	72.2651	53.6432	66.9958	67.0199	22.2251
0.3671	11.0	12793	0.4633	72.7308	54.2544	67.3945	67.4726	22.0
0.3605	12.0	13956	0.4675	72.5548	53.834	67.2447	67.2855	22.555
0.3588	13.0	15119	0.4686	71.9681	53.2995	66.7298	66.779	22.089
0.353	14.0	16282	0.4694	71.9544	53.4137	66.7585	66.7915	22.1099
0.3474	15.0	17445	0.4738	72.0006	53.3351	66.7153	66.7157	22.1728
0.3441	16.0	18608	0.4749	72.0188	53.5242	66.7556	66.7977	22.0733
0.3431	17.0	19771	0.4756	71.91	53.0955	66.5612	66.5038	22.1361
0.3385	18.0	20934	0.4769	71.9702	53.5832	66.915	66.9436	22.0157
0.3384	19.0	22097	0.4778	72.0563	53.7153	66.9998	67.061	22.0314
0.3368	20.0	23260	0.4783	72.0576	53.6457	66.9028	66.9302	21.9895

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3