indobart-indonlg-nusax-500-jv-id

This model is a fine-tuned version of indobenchmark/indobart-v2 on IndoNLG, NusaX, and manual translation dataset. It achieves the following results on the evaluation set:

• Loss: 0.6148
• Bleu: 20.3132
• Gen Len: 19.3294

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

Javanese Most Common Words

Indonesian Most Common Words

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
No log	1.0	114	0.8182	12.2063	19.4575
No log	2.0	228	0.7390	14.2482	19.3932
No log	3.0	342	0.6975	15.9974	19.4014
No log	4.0	456	0.6706	16.578	19.3601
0.849	5.0	570	0.6527	17.2313	19.3713
0.849	6.0	684	0.6392	17.9	19.3477
0.849	7.0	798	0.6285	18.0166	19.3495
0.849	8.0	912	0.6242	18.2424	19.3347
0.4461	9.0	1026	0.6170	18.6378	19.3648
0.4461	10.0	1140	0.6148	19.0513	19.3365
0.4461	11.0	1254	0.6106	19.5335	19.3383
0.4461	12.0	1368	0.6097	19.2886	19.3353
0.4461	13.0	1482	0.6079	19.5712	19.3347
0.3344	14.0	1596	0.6067	19.4256	19.3684
0.3344	15.0	1710	0.6078	19.7062	19.3483
0.3344	16.0	1824	0.6052	19.6353	19.3506
0.3344	17.0	1938	0.6072	19.8745	19.3318
0.2674	18.0	2052	0.6076	20.0834	19.3318
0.2674	19.0	2166	0.6078	20.082	19.3506
0.2674	20.0	2280	0.6098	20.1934	19.3117
0.2674	21.0	2394	0.6094	20.1326	19.3453
0.225	22.0	2508	0.6109	20.2045	19.3329
0.225	23.0	2622	0.6133	20.0595	19.3495
0.225	24.0	2736	0.6117	20.2089	19.3442
0.225	25.0	2850	0.6141	20.2566	19.3264
0.225	26.0	2964	0.6143	20.3092	19.3329
0.199	27.0	3078	0.6134	20.3808	19.33
0.199	28.0	3192	0.6141	20.412	19.34
0.199	29.0	3306	0.6148	20.2613	19.3359
0.199	30.0	3420	0.6148	20.3132	19.3294

Framework versions

• Transformers 4.33.1
• Pytorch 2.1.0+cu118
• Datasets 2.15.0
• Tokenizers 0.13.3