Edit model card

koja_mbartLarge_55p_run1

This model is a fine-tuned version of facebook/mbart-large-50-many-to-many-mmt on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.9008
  • Bleu: 8.9861
  • Gen Len: 17.9149

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Bleu Gen Len
1.0254 0.48 8000 1.0065 7.1252 18.5403
0.9374 0.97 16000 0.9200 8.2047 18.2091
0.7429 1.45 24000 0.9086 8.4622 17.9606
0.7039 1.93 32000 0.9008 8.9198 17.8841
0.5616 2.42 40000 0.9210 8.7938 17.78
0.5355 2.9 48000 0.9400 8.6506 17.9146
0.4194 3.38 56000 1.0054 8.4235 17.6759
0.3852 3.86 64000 1.0154 8.5886 17.7885

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.1.0+cu121
  • Datasets 2.14.6
  • Tokenizers 0.14.1
Downloads last month
1

Finetuned from