mazzaqq/Llama-dpo_large_snake

high_diff_eig

This model is a fine-tuned version of meta-llama/Llama-2-7b-chat-hf on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2215
• Rewards/chosen: -1.1270
• Rewards/rejected: -5.4377
• Rewards/accuracies: 0.9013
• Rewards/margins: 4.3107
• Logps/rejected: -113.3559
• Logps/chosen: -86.1622
• Logits/rejected: -1.0685
• Logits/chosen: -1.0587

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 4
• eval_batch_size: 2
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.2458	0.2	1126	0.2914	-0.1921	-3.0426	0.8618	2.8505	-89.4053	-76.8133	-0.9562	-0.9372
0.096	0.4	2252	0.2385	-1.1154	-5.1354	0.9035	4.0200	-110.3335	-86.0469	-1.0511	-1.0413
0.0547	0.6	3378	0.2271	-1.0829	-5.3656	0.9057	4.2827	-112.6357	-85.7219	-1.0669	-1.0570
0.1134	0.8	4504	0.2215	-1.1270	-5.4377	0.9013	4.3107	-113.3559	-86.1622	-1.0685	-1.0587

Framework versions

• PEFT 0.9.0
• Transformers 4.38.0
• Pytorch 2.1.2
• Datasets 2.15.0
• Tokenizers 0.15.2