Edit model card

jako_mbartLarge_6p_run1

This model is a fine-tuned version of facebook/mbart-large-50-many-to-many-mmt on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.1539
  • Bleu: 26.2658
  • Gen Len: 19.3961

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 300
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Bleu Gen Len
1.4641 0.48 1000 1.3276 21.6162 19.4434
1.2615 0.96 2000 1.1866 24.346 19.4734
0.9103 1.44 3000 1.1638 25.4249 19.0086
0.8285 1.92 4000 1.1539 26.2658 19.3961
0.5977 2.4 5000 1.1978 25.5651 19.6248
0.5423 2.88 6000 1.1830 26.8441 19.1349
0.3816 3.36 7000 1.2670 26.1301 19.1207
0.3412 3.84 8000 1.2870 26.7783 19.2417

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.1.0+cu121
  • Datasets 2.14.6
  • Tokenizers 0.14.1
Downloads last month
1

Finetuned from