indobert-base-uncased-risalah-cobain

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 5.2899
• Rouge1: 23.1385
• Rouge2: 7.3334
• Rougel: 15.7735
• Rougelsum: 21.6953

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
3.5115	1.0	3	5.2203	15.5242	4.2398	11.374	14.7883
3.3129	2.0	6	5.0227	16.7735	5.198	13.2395	15.8504
2.9808	3.0	9	5.1890	16.1631	5.247	12.1408	15.1207
2.8024	4.0	12	5.1468	19.0466	4.9413	13.2276	17.4135
2.6153	5.0	15	5.1368	20.9181	5.2618	13.8707	19.0082
2.4717	6.0	18	5.2183	19.2699	5.9044	14.158	17.9402
2.3641	7.0	21	5.2604	20.0623	6.3966	14.585	18.4161
2.2708	8.0	24	5.2560	19.1485	5.3841	14.2997	18.5492
2.1554	9.0	27	5.2796	19.0462	5.7227	14.2268	17.6771
2.0751	10.0	30	5.2899	23.1385	7.3334	15.7735	21.6953

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1