Mikask/indobart-altp-en-id

indobart-en-id

This model is a fine-tuned version of indobenchmark/indobart-v2. It achieves the following results on the evaluation set:

• Loss: 0.4035
• Bleu: 20.4747
• Gen Len: 19.134

Model description

Trained on the Asian Language Treebank Project Corpus (en-id). The goal of the model is to translate from English to Indonesian.

Training and evaluation data

The ALT Project Corpus has over 20,000 entries of English to Indonesian sentences. The training and evaluation data is split 80/20.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.914	1.0	503	0.5118	14.8205	19.2874
0.5805	2.0	1006	0.4606	17.2483	19.236
0.4971	3.0	1509	0.4370	18.4965	19.2449
0.4444	4.0	2012	0.4251	19.3336	19.1765
0.4037	5.0	2515	0.4152	19.6643	19.1648
0.3732	6.0	3018	0.4103	20.0701	19.1987
0.3503	7.0	3521	0.4066	20.1264	19.1258
0.3325	8.0	4024	0.4040	20.5349	19.1571
0.3207	9.0	4527	0.4041	20.4371	19.1348
0.3116	10.0	5030	0.4035	20.4747	19.134

Framework versions

• Transformers 4.33.3
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.13.3