alwaysaditi's picture
End of training
99f5521 verified
metadata
base_model: google/pegasus-x-base
tags:
  - generated_from_trainer
model-index:
  - name: google/pegasus-x-base
    results: []

google/pegasus-x-base

This model is a fine-tuned version of google/pegasus-x-base on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0135

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 8
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 5
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss
8.9092 0.1008 10 8.5348
7.9162 0.2015 20 7.5592
7.3907 0.3023 30 6.9080
6.8587 0.4030 40 6.1464
5.7817 0.5038 50 5.2883
5.0792 0.6045 60 3.9477
4.1259 0.7053 70 2.7538
3.0821 0.8060 80 1.7983
2.2714 0.9068 90 1.4814
1.7994 1.0076 100 1.4092
1.4936 1.1083 110 1.3189
1.6535 1.2091 120 1.2445
1.3122 1.3098 130 1.2139
1.0667 1.4106 140 1.1800
1.274 1.5113 150 1.1507
1.1739 1.6121 160 1.1279
1.1871 1.7128 170 1.1094
1.2037 1.8136 180 1.0973
1.0839 1.9144 190 1.0832
1.0738 2.0151 200 1.0752
1.0955 2.1159 210 1.0695
1.1285 2.2166 220 1.0629
0.9973 2.3174 230 1.0574
1.0522 2.4181 240 1.0557
1.0803 2.5189 250 1.0458
1.0707 2.6196 260 1.0425
1.1868 2.7204 270 1.0384
1.0117 2.8212 280 1.0374
0.9206 2.9219 290 1.0347
1.0099 3.0227 300 1.0306
1.0459 3.1234 310 1.0307
1.0721 3.2242 320 1.0313
1.015 3.3249 330 1.0278
1.0358 3.4257 340 1.0237
0.9608 3.5264 350 1.0206
1.0416 3.6272 360 1.0202
0.9304 3.7280 370 1.0201
1.0447 3.8287 380 1.0187
1.0007 3.9295 390 1.0180
1.1681 4.0302 400 1.0168
1.0258 4.1310 410 1.0163
1.1054 4.2317 420 1.0153
0.907 4.3325 430 1.0154
0.935 4.4332 440 1.0151
0.9904 4.5340 450 1.0145
0.9735 4.6348 460 1.0142
0.9633 4.7355 470 1.0138
1.2809 4.8363 480 1.0136
1.0361 4.9370 490 1.0135

Framework versions

  • Transformers 4.41.2
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1