zephyr-7b-dpo-full / README.md
RikkiXu's picture
Model save
7dd2eeb verified
|
raw
history blame
2.38 kB
metadata
tags:
  - trl
  - dpo
  - generated_from_trainer
model-index:
  - name: zephyr-7b-dpo-full
    results: []

zephyr-7b-dpo-full

This model was trained from scratch on the None dataset. It achieves the following results on the evaluation set:

  • Loss: 1.9828
  • Rewards/chosen: -3.4223
  • Rewards/rejected: -2.1126
  • Rewards/accuracies: 0.3555
  • Rewards/margins: -1.3097
  • Logps/rejected: -521.2875
  • Logps/chosen: -405.8879
  • Logits/rejected: -4.9364
  • Logits/chosen: -4.7068

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-08
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 8
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 128
  • total_eval_batch_size: 64
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Rewards/chosen Rewards/rejected Rewards/accuracies Rewards/margins Logps/rejected Logps/chosen Logits/rejected Logits/chosen
0.3652 0.29 100 1.7488 -2.2159 -1.0097 0.3516 -1.2062 -519.0817 -403.4752 -4.9249 -4.6962
0.312 0.57 200 1.9596 -3.1949 -1.8164 0.3398 -1.3786 -520.6950 -405.4332 -4.9391 -4.7096
0.2993 0.86 300 1.9828 -3.4223 -2.1126 0.3555 -1.3097 -521.2875 -405.8879 -4.9364 -4.7068

Framework versions

  • Transformers 4.38.2
  • Pytorch 2.1.2+cu118
  • Datasets 2.16.1
  • Tokenizers 0.15.2