saeedehj/led-base-finetune-cnn

led-base-16384-finetune-cnn

This model is a fine-tuned version of allenai/led-base-16384 on the cnn_dailymail dataset. It achieves the following results on the evaluation set:

• Loss: 3.2020
• Rouge1: 24.2258
• Rouge2: 9.0151
• Rougel: 19.0336
• Rougelsum: 22.2604
• Gen Len: 20.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
1.8988	1.0	2000	2.0031	25.1709	10.0426	20.1311	23.1639	20.0
1.6038	2.0	4000	2.0314	25.0213	9.8701	19.8987	23.0129	20.0
1.3352	3.0	6000	2.1124	24.99	9.905	19.9566	23.0973	20.0
1.1173	4.0	8000	2.2055	25.0568	10.0949	19.9602	23.18	20.0
0.9566	5.0	10000	2.3262	24.941	9.5856	19.6285	23.042	20.0
0.7986	6.0	12000	2.4489	24.4114	9.2808	19.3296	22.5481	20.0
0.6685	7.0	14000	2.5211	24.467	9.5124	19.2685	22.5624	20.0
0.5601	8.0	16000	2.6299	24.6939	9.6533	19.4627	22.8048	20.0
0.4757	9.0	18000	2.7185	24.2098	9.1232	19.0181	22.4085	20.0
0.3926	10.0	20000	2.7947	24.5092	9.3964	19.2593	22.5592	20.0
0.3391	11.0	22000	2.8626	24.4731	9.3634	19.2966	22.5688	20.0
0.2872	12.0	24000	2.9175	24.5587	9.3888	19.3335	22.6443	20.0
0.2479	13.0	26000	2.9658	24.2983	9.1038	19.019	22.3675	20.0
0.213	14.0	28000	3.0273	24.4196	9.1481	19.0458	22.5135	20.0
0.1828	15.0	30000	3.0751	24.3283	9.2334	18.9771	22.3322	20.0
0.1608	16.0	32000	3.1185	24.3965	9.2047	19.0899	22.4666	20.0
0.1442	17.0	34000	3.1494	24.3832	9.1915	19.077	22.4366	20.0
0.1293	18.0	36000	3.1738	24.3796	9.1132	19.1015	22.3862	20.0
0.1165	19.0	38000	3.2073	24.2804	9.1018	19.0692	22.3023	20.0
0.1118	20.0	40000	3.2020	24.2258	9.0151	19.0336	22.2604	20.0

Framework versions

• Transformers 4.30.2
• Pytorch 2.0.1+cu118
• Datasets 2.13.1
• Tokenizers 0.13.3