mBART_translator_kobart_2

This model is a fine-tuned version of hyunwoongko/kobart on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0971
• Bleu: 34.7406
• Gen Len: 19.863

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
0.7514	1.0	1912	0.3009	33.1304	19.8465
0.4925	2.0	3824	0.2213	33.7817	19.8541
0.3893	3.0	5736	0.1789	34.1249	19.8577
0.3328	4.0	7648	0.1555	34.3074	19.8598
0.2896	5.0	9560	0.1375	34.4127	19.8585
0.2614	6.0	11472	0.1240	34.5061	19.8604
0.2292	7.0	13384	0.1116	34.6476	19.8632
0.2098	8.0	15296	0.1050	34.6956	19.863
0.1967	9.0	17208	0.0995	34.7304	19.863
0.1812	10.0	19120	0.0971	34.7406	19.863

Framework versions

• Transformers 4.23.0
• Pytorch 1.12.1+cu113
• Datasets 2.5.2
• Tokenizers 0.13.1