gemma-2-9b-it-simpo
This model is a fine-tuned version of google/gemma-2-9b-it on the princeton-nlp/gemma2-ultrafeedback-armorm dataset. It achieves the following results on the evaluation set:
- Loss: 4.6824
- Rewards/chosen: -10.1512
- Rewards/rejected: -10.9667
- Rewards/accuracies: 0.5757
- Rewards/margins: 0.8155
- Logps/rejected: -1.0967
- Logps/chosen: -1.0151
- Logits/rejected: 1.1280
- Logits/chosen: 0.5690
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 8e-07
- train_batch_size: 1
- eval_batch_size: 2
- seed: 42
- gradient_accumulation_steps: 16
- total_train_batch_size: 16
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_ratio: 0.1
- num_epochs: 1
Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 7.6723 | 0.1074 | 400 | 7.5615 | -24.9077 | -25.5289 | 0.5263 | 0.6212 | -2.5529 | -2.4908 | 0.0773 | -0.4772 |
| 5.976 | 0.2149 | 800 | 5.6663 | -15.3560 | -15.8776 | 0.5304 | 0.5216 | -1.5878 | -1.5356 | 0.6816 | 0.1302 |
| 4.6292 | 0.3223 | 1200 | 4.9687 | -12.0618 | -12.7176 | 0.5520 | 0.6558 | -1.2718 | -1.2062 | 0.9567 | 0.4027 |
| 4.3654 | 0.4298 | 1600 | 4.8123 | -11.0979 | -11.8232 | 0.5561 | 0.7253 | -1.1823 | -1.1098 | 1.0648 | 0.5095 |
| 5.1152 | 0.5372 | 2000 | 4.7458 | -10.5834 | -11.3535 | 0.5675 | 0.7701 | -1.1354 | -1.0583 | 1.0999 | 0.5428 |
| 4.743 | 0.6446 | 2400 | 4.7077 | -10.3242 | -11.1208 | 0.5721 | 0.7966 | -1.1121 | -1.0324 | 1.1194 | 0.5613 |
| 4.9378 | 0.7521 | 2800 | 4.6912 | -10.1823 | -10.9893 | 0.5767 | 0.8071 | -1.0989 | -1.0182 | 1.1290 | 0.5703 |
| 4.2982 | 0.8595 | 3200 | 4.6826 | -10.1663 | -10.9832 | 0.5747 | 0.8168 | -1.0983 | -1.0166 | 1.1268 | 0.5678 |
| 5.4513 | 0.9669 | 3600 | 4.6824 | -10.1512 | -10.9667 | 0.5757 | 0.8155 | -1.0967 | -1.0151 | 1.1280 | 0.5690 |
Framework versions
- PEFT 0.7.1
- Transformers 4.44.2
- Pytorch 2.2.2+cu121
- Datasets 2.18.0
- Tokenizers 0.19.1
- Downloads last month
- 5
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support