simpo-lora-2

This model is a fine-tuned version of hatakeyama-llm-team/with_halcination_little_codes_ck5200 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5415
• Rewards/chosen: -5.7237
• Rewards/rejected: -12.8559
• Rewards/accuracies: 0.7368
• Rewards/margins: 7.1321
• Logps/rejected: -5.1424
• Logps/chosen: -2.2895
• Logits/rejected: -0.5526
• Logits/chosen: -0.5626

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 64
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.7791	0.1600	100	0.9134	-4.7310	-6.5490	0.7037	1.8179	-2.6196	-1.8924	-0.1767	0.0399
0.4822	0.3200	200	0.5821	-5.4149	-11.9630	0.7349	6.5482	-4.7852	-2.1659	0.0142	0.0556
0.5591	0.4800	300	0.5573	-5.5227	-12.0727	0.7417	6.5500	-4.8291	-2.2091	-0.6007	-0.5349
0.6293	0.6400	400	0.5447	-5.8900	-14.0923	0.7378	8.2023	-5.6369	-2.3560	-0.1891	-0.2711
0.5645	0.8000	500	0.5426	-5.7599	-13.0285	0.7368	7.2686	-5.2114	-2.3040	-0.5393	-0.5567
0.4763	0.9600	600	0.5415	-5.7237	-12.8559	0.7368	7.1321	-5.1424	-2.2895	-0.5526	-0.5626

Framework versions

• PEFT 0.11.1
• Transformers 4.41.1
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1