exp2-led-risalah_data_v1

This model is a fine-tuned version of silmi224/finetune-led-35000 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.5716
• Rouge1: 19.6052
• Rouge2: 10.044
• Rougel: 14.481
• Rougelsum: 18.8794

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 150
• num_epochs: 30
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.3238	1.0	10	2.7939	8.2919	2.3936	6.4452	7.8858
3.0941	2.0	20	2.5393	8.884	2.3029	6.6763	8.2192
2.7749	3.0	30	2.2953	11.5346	3.61	7.9843	10.5072
2.5095	4.0	40	2.1271	13.3139	4.2267	9.3106	11.9731
2.3044	5.0	50	2.0140	14.8318	5.4652	10.4052	13.8404
2.1532	6.0	60	1.9280	15.6855	6.4587	10.7669	14.5093
2.032	7.0	70	1.8598	14.9367	5.7627	10.481	14.0571
1.9376	8.0	80	1.8049	14.8866	6.1106	10.0165	14.4686
1.8459	9.0	90	1.7491	13.6909	5.6398	9.2128	12.9399
1.7765	10.0	100	1.7213	16.7363	7.2146	11.2988	16.0402
1.704	11.0	110	1.6857	18.4687	8.7089	12.9138	17.8621
1.6542	12.0	120	1.6610	19.2238	8.9265	13.1614	17.6696
1.5957	13.0	130	1.6335	19.6057	9.8766	13.6908	18.6659
1.5413	14.0	140	1.6145	19.2875	9.7272	14.3241	17.7305
1.496	15.0	150	1.6232	18.1669	8.857	13.5735	17.1252
1.4535	16.0	160	1.6036	19.3501	10.1008	14.5871	18.4397
1.4204	17.0	170	1.5954	19.4201	10.3577	14.2019	18.4312
1.3829	18.0	180	1.5794	18.4944	9.4098	13.9	17.3891
1.3535	19.0	190	1.5814	19.9886	11.1416	15.0161	19.122
1.3328	20.0	200	1.5758	20.2011	10.5645	14.7218	19.1219
1.3063	21.0	210	1.5722	20.7308	10.834	15.3016	19.8805
1.2858	22.0	220	1.5745	19.648	10.77	14.0294	19.0395
1.2726	23.0	230	1.5651	20.4129	10.8196	15.0054	19.7253
1.2557	24.0	240	1.5709	18.6308	9.3525	13.8142	18.1621
1.2456	25.0	250	1.5659	19.6106	10.4499	14.4439	18.9271
1.233	26.0	260	1.5702	19.1583	9.7391	14.1738	18.5077
1.2267	27.0	270	1.5651	18.7654	9.8637	13.7809	18.2034
1.2203	28.0	280	1.5703	19.9698	10.4741	14.3559	19.389
1.2147	29.0	290	1.5739	19.9054	10.0052	14.6427	19.2278
1.2124	30.0	300	1.5716	19.6052	10.044	14.481	18.8794

Framework versions

• Transformers 4.41.2
• Pytorch 2.1.2
• Datasets 2.19.2
• Tokenizers 0.19.1