indobert-base-uncased-risalah

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 4.7608
• Rouge1: 17.5948
• Rouge2: 5.7046
• Rougel: 12.9364
• Rougelsum: 16.8985

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
6.6456	1.0	3	6.5623	12.0228	1.4627	8.9826	11.5534
6.2195	2.0	6	6.2558	13.4413	2.6947	9.9762	13.1906
5.7034	3.0	9	5.9070	13.8874	2.3833	9.7477	11.8108
5.3584	4.0	12	5.7837	9.2202	2.2314	8.233	8.7991
5.039	5.0	15	5.5776	8.9092	3.0916	7.5057	8.2755
4.732	6.0	18	5.4388	10.5968	4.4625	9.1293	10.316
4.5451	7.0	21	5.3076	14.2127	5.5899	11.5194	13.6803
4.2744	8.0	24	5.2115	13.6695	5.553	10.7011	12.5259
4.0553	9.0	27	5.1073	14.9044	3.3316	10.5591	13.8547
3.8821	10.0	30	5.0516	13.1847	3.1547	9.6358	12.4631
3.7057	11.0	33	4.9741	15.6141	4.9602	11.9355	15.416
3.5819	12.0	36	4.9127	16.3652	5.6285	13.2311	15.5054
3.387	13.0	39	4.8841	16.9982	5.048	11.5842	16.5562
3.3165	14.0	42	4.8235	15.1943	4.7871	11.6678	14.7917
3.1875	15.0	45	4.7965	16.6478	4.882	12.4033	16.2249
3.0969	16.0	48	4.7910	16.8959	4.2552	11.4183	16.1743
2.9983	17.0	51	4.7859	16.4024	5.1662	11.7594	15.972
2.958	18.0	54	4.7787	17.5239	5.4397	12.9231	17.1452
2.9241	19.0	57	4.7644	17.5575	5.5904	12.0571	16.6271
2.892	20.0	60	4.7608	17.5948	5.7046	12.9364	16.8985

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1