chat_400_STEPS_05beta_1e7rate_CDPOSFT
This model is a fine-tuned version of tsavage68/chat_600STEPS_1e8rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 0.6919
- Rewards/chosen: -0.0067
- Rewards/rejected: -0.0110
- Rewards/accuracies: 0.4308
- Rewards/margins: 0.0043
- Logps/rejected: -18.8240
- Logps/chosen: -16.7680
- Logits/rejected: -0.5983
- Logits/chosen: -0.5982
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-07
- train_batch_size: 4
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 400
Training results
Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
---|---|---|---|---|---|---|---|---|---|---|---|
0.6929 | 0.0977 | 50 | 0.6947 | -0.0000 | 0.0016 | 0.4066 | -0.0016 | -18.7989 | -16.7547 | -0.5985 | -0.5983 |
0.694 | 0.1953 | 100 | 0.6903 | 0.0030 | -0.0047 | 0.4527 | 0.0076 | -18.8113 | -16.7487 | -0.5976 | -0.5975 |
0.6941 | 0.2930 | 150 | 0.6914 | -0.0038 | -0.0090 | 0.4330 | 0.0052 | -18.8200 | -16.7622 | -0.5986 | -0.5985 |
0.6934 | 0.3906 | 200 | 0.6941 | -0.0035 | -0.0032 | 0.4044 | -0.0003 | -18.8084 | -16.7616 | -0.5978 | -0.5977 |
0.6963 | 0.4883 | 250 | 0.6932 | -0.0058 | -0.0078 | 0.3890 | 0.0019 | -18.8175 | -16.7663 | -0.5978 | -0.5977 |
0.6898 | 0.5859 | 300 | 0.6900 | -0.0039 | -0.0122 | 0.4330 | 0.0084 | -18.8264 | -16.7624 | -0.5980 | -0.5979 |
0.6856 | 0.6836 | 350 | 0.6923 | -0.0075 | -0.0109 | 0.4571 | 0.0034 | -18.8237 | -16.7695 | -0.5975 | -0.5974 |
0.6978 | 0.7812 | 400 | 0.6919 | -0.0067 | -0.0110 | 0.4308 | 0.0043 | -18.8240 | -16.7680 | -0.5983 | -0.5982 |
Framework versions
- Transformers 4.40.1
- Pytorch 2.0.0+cu117
- Datasets 2.19.1
- Tokenizers 0.19.1
- Downloads last month
- 1