Hugging Face's logo

scottsuk0306
/
easylm-sft-gemma-2-9b

Safetensors
gemma2
easylm
trl
sft
Generated from Trainer
Model card Files Community
easylm-sft-gemma-2-9b / README.md
scottsuk0306's picture
scottsuk0306
End of training
b858522 verified
preview code
|
raw
history blame contribute delete
7.93 kB
metadata
license: gemma
base_model: google/gemma-2-9b
tags:
  - easylm
  - trl
  - sft
  - generated_from_trainer
  - trl
  - sft
  - generated_from_trainer
datasets:
  - alpaca_farm
model-index:
  - name: easylm-sft-gemma-2-9b
    results: []

easylm-sft-gemma-2-9b

This model is a fine-tuned version of google/gemma-2-9b on the alpaca_farm dataset. It achieves the following results on the evaluation set:

  • Loss: 0.7115

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 3e-06
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 8
  • total_train_batch_size: 16
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • num_epochs: 2

Training results

Training Loss Epoch Step Validation Loss
0.6921 0.016 10 0.6721
0.6141 0.032 20 0.6662
0.6665 0.048 30 0.6603
0.6116 0.064 40 0.6611
0.6102 0.08 50 0.6580
0.6886 0.096 60 0.6593
0.6415 0.112 70 0.6596
0.6214 0.128 80 0.6595
0.6816 0.144 90 0.6584
0.6481 0.16 100 0.6597
0.6022 0.176 110 0.6590
0.6703 0.192 120 0.6607
0.6742 0.208 130 0.6615
0.6369 0.224 140 0.6615
0.7142 0.24 150 0.6602
0.6707 0.256 160 0.6611
0.6629 0.272 170 0.6609
0.6299 0.288 180 0.6610
0.6351 0.304 190 0.6607
0.5885 0.32 200 0.6610
0.6613 0.336 210 0.6619
0.6151 0.352 220 0.6602
0.6342 0.368 230 0.6609
0.6376 0.384 240 0.6601
0.679 0.4 250 0.6601
0.6911 0.416 260 0.6593
0.6717 0.432 270 0.6592
0.6758 0.448 280 0.6603
0.6243 0.464 290 0.6603
0.643 0.48 300 0.6586
0.603 0.496 310 0.6573
0.6336 0.512 320 0.6568
0.6198 0.528 330 0.6569
0.6989 0.544 340 0.6578
0.6353 0.56 350 0.6570
0.6746 0.576 360 0.6568
0.6883 0.592 370 0.6571
0.6772 0.608 380 0.6566
0.6563 0.624 390 0.6564
0.6077 0.64 400 0.6554
0.6291 0.656 410 0.6552
0.6073 0.672 420 0.6547
0.6598 0.688 430 0.6551
0.593 0.704 440 0.6547
0.6352 0.72 450 0.6547
0.6216 0.736 460 0.6540
0.6937 0.752 470 0.6535
0.669 0.768 480 0.6530
0.6052 0.784 490 0.6525
0.6218 0.8 500 0.6525
0.6341 0.816 510 0.6526
0.6681 0.832 520 0.6522
0.6203 0.848 530 0.6516
0.6682 0.864 540 0.6506
0.6212 0.88 550 0.6501
0.6887 0.896 560 0.6502
0.64 0.912 570 0.6504
0.6176 0.928 580 0.6500
0.6285 0.944 590 0.6500
0.6661 0.96 600 0.6489
0.6537 0.976 610 0.6488
0.657 0.992 620 0.6482
0.4004 1.008 630 0.6503
0.4014 1.024 640 0.7170
0.4179 1.04 650 0.6923
0.3998 1.056 660 0.6921
0.3705 1.072 670 0.7054
0.3513 1.088 680 0.7036
0.3815 1.104 690 0.7025
0.3684 1.12 700 0.7049
0.3914 1.1360 710 0.7069
0.4082 1.152 720 0.7018
0.3494 1.168 730 0.7042
0.3715 1.184 740 0.7071
0.3675 1.2 750 0.7085
0.3319 1.216 760 0.7112
0.3823 1.232 770 0.7141
0.3571 1.248 780 0.7113
0.3503 1.264 790 0.7127
0.3742 1.28 800 0.7159
0.4087 1.296 810 0.7139
0.3781 1.312 820 0.7073
0.3475 1.328 830 0.7129
0.3724 1.3440 840 0.7113
0.3612 1.3600 850 0.7130
0.3254 1.376 860 0.7139
0.3626 1.392 870 0.7145
0.351 1.408 880 0.7147
0.3357 1.424 890 0.7105
0.371 1.44 900 0.7079
0.3566 1.456 910 0.7070
0.3762 1.472 920 0.7118
0.3755 1.488 930 0.7126
0.3595 1.504 940 0.7107
0.3828 1.52 950 0.7118
0.3793 1.536 960 0.7173
0.3446 1.552 970 0.7150
0.3707 1.568 980 0.7135
0.3604 1.584 990 0.7141
0.3441 1.6 1000 0.7137
0.3705 1.616 1010 0.7154
0.3857 1.6320 1020 0.7189
0.3952 1.6480 1030 0.7148
0.3815 1.6640 1040 0.7116
0.3507 1.6800 1050 0.7108
0.3662 1.696 1060 0.7124
0.3581 1.712 1070 0.7136
0.3867 1.728 1080 0.7132
0.3707 1.744 1090 0.7127
0.4078 1.76 1100 0.7122
0.3713 1.776 1110 0.7111
0.3525 1.792 1120 0.7110
0.3873 1.808 1130 0.7115
0.4008 1.8240 1140 0.7119
0.3889 1.8400 1150 0.7119
0.3591 1.8560 1160 0.7116
0.3843 1.8720 1170 0.7116
0.3713 1.888 1180 0.7115
0.3659 1.904 1190 0.7115
0.3588 1.92 1200 0.7115
0.3556 1.936 1210 0.7115
0.3278 1.952 1220 0.7116
0.3642 1.968 1230 0.7115
0.3718 1.984 1240 0.7115
0.3611 2.0 1250 0.7115

Framework versions

  • Transformers 4.43.3
  • Pytorch 2.3.0+cu121
  • Datasets 2.20.0
  • Tokenizers 0.19.1