tsavage68/Summary4500_M2_350steps_1e8rate_05beta_CSFTDPO

Hyponatremia_M2_350steps_1e8rate_05beta_CSFTDPO

This model is a fine-tuned version of tsavage68/Summary4500_M2_200steps_1e7rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6748
• Rewards/chosen: -0.0068
• Rewards/rejected: -0.0657
• Rewards/accuracies: 0.5680
• Rewards/margins: 0.0590
• Logps/rejected: -152.8613
• Logps/chosen: -93.7533
• Logits/rejected: -2.3514
• Logits/chosen: -2.3039

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-08
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 100
• training_steps: 350

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.7203	0.0112	50	0.6987	-0.0033	-0.0121	0.5140	0.0089	-152.7540	-93.7463	-2.3525	-2.3051
0.7141	0.0224	100	0.7007	-0.0065	-0.0124	0.5120	0.0059	-152.7545	-93.7527	-2.3518	-2.3044
0.6852	0.0336	150	0.6771	-0.0123	-0.0648	0.5840	0.0525	-152.8595	-93.7644	-2.3522	-2.3047
0.6459	0.0448	200	0.6783	-0.0073	-0.0594	0.5780	0.0520	-152.8485	-93.7544	-2.3503	-2.3029
0.7282	0.0559	250	0.6723	-0.0149	-0.0813	0.5920	0.0664	-152.8924	-93.7697	-2.3516	-2.3041
0.6859	0.0671	300	0.6770	-0.0108	-0.0658	0.5540	0.0550	-152.8613	-93.7613	-2.3514	-2.3039
0.7327	0.0783	350	0.6748	-0.0068	-0.0657	0.5680	0.0590	-152.8613	-93.7533	-2.3514	-2.3039

Framework versions

• Transformers 4.42.4
• Pytorch 2.0.0+cu117
• Datasets 2.20.0
• Tokenizers 0.19.1