Hugging Face's logo

hZzy
/
qwen2.5-0.5b-expo-EXDPO2

Safetensors
qwen2
alignment-handbook
ndcg
trl
expo
Generated from Trainer
Model card Files Community
qwen2.5-0.5b-expo-EXDPO2 / README.md
hZzy's picture
hZzy
End of training
408d0d7 verified
preview code
|
raw
history blame contribute delete
3.94 kB
metadata
license: apache-2.0
base_model: hZzy/qwen2.5-0.5b-sft-news-IFT
tags:
  - alignment-handbook
  - ndcg
  - trl
  - expo
  - generated_from_trainer
  - trl
  - expo
  - generated_from_trainer
datasets:
  - hZzy/train_pairwise
model-index:
  - name: qwen2.5-0.5b-expo-EXDPO2
    results: []

Visualize in Weights & Biases

qwen2.5-0.5b-expo-EXDPO2

This model is a fine-tuned version of hZzy/qwen2.5-0.5b-sft-news-IFT on the hZzy/train_pairwise dataset. It achieves the following results on the evaluation set:

  • Loss: 1.0827
  • Logps: -99.3608
  • Logits: -1.3686
  • Objective: 1.0902
  • Dpo Loss: 0.6830
  • Regularize: 0.4072
  • Ranking Simple: 0.5221
  • Ranking Idealized: 0.5925
  • Ranking Idealized Expo: 0.5166

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 1e-07
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 4
  • gradient_accumulation_steps: 12
  • total_train_batch_size: 192
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 2

Training results

Training Loss Epoch Step Validation Loss Logps Logits Objective Dpo Loss Regularize Ranking Simple Ranking Idealized Ranking Idealized Expo
1.0911 0.1889 50 1.1043 -97.7941 -1.3092 1.1042 0.6905 0.4136 0.5166 0.5925 0.5166
1.0629 0.3778 100 1.0938 -98.1420 -1.3221 1.0948 0.6871 0.4077 0.5180 0.5925 0.5166
1.0597 0.5668 150 1.0886 -97.5760 -1.3388 1.0899 0.6843 0.4057 0.5193 0.5925 0.5166
1.0201 0.7557 200 1.0878 -98.5525 -1.3519 1.0928 0.6845 0.4083 0.5193 0.5925 0.5166
1.0173 0.9446 250 1.0863 -99.3374 -1.3586 1.0919 0.6845 0.4074 0.5193 0.5925 0.5166
0.9755 1.1335 300 1.0829 -99.1144 -1.3663 1.0893 0.6826 0.4067 0.5214 0.5925 0.5166
0.9708 1.3224 350 1.0829 -99.2669 -1.3642 1.0901 0.6833 0.4068 0.5221 0.5925 0.5166
0.968 1.5113 400 1.0829 -99.2520 -1.3682 1.0901 0.6832 0.4069 0.5214 0.5925 0.5166
0.9495 1.7003 450 1.0826 -99.3216 -1.3688 1.0900 0.6830 0.4070 0.5214 0.5925 0.5166
0.9463 1.8892 500 1.0827 -99.3605 -1.3686 1.0902 0.6830 0.4072 0.5221 0.5925 0.5166

Framework versions

  • Transformers 4.42.0
  • Pytorch 2.3.0+cu121
  • Datasets 2.19.1
  • Tokenizers 0.19.1