spin-margin1

This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0052
• Rewards/real: -2.4343
• Rewards/generated: -23.3733
• Rewards/accuracies: 1.0
• Rewards/margins: 20.9390
• Logps/generated: -328.0982
• Logps/real: -152.5396
• Logits/generated: -2.4931
• Logits/real: -2.1379

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/real	Rewards/generated	Rewards/accuracies	Rewards/margins	Logps/generated	Logps/real	Logits/generated	Logits/real
0.006	0.14	100	0.0103	0.8828	-8.9433	1.0	9.8261	-183.7986	-119.3683	-2.7878	-2.5728
0.0049	0.28	200	0.0059	-0.3283	-12.8574	1.0	12.5290	-222.9388	-131.4797	-2.6820	-2.3354
0.0008	0.41	300	0.0046	-0.3088	-17.1447	1.0	16.8359	-265.8120	-131.2846	-2.6102	-2.2508
0.0003	0.55	400	0.0040	-1.1157	-16.5400	1.0	15.4243	-259.7652	-139.3531	-2.7045	-2.2690
0.0013	0.69	500	0.0044	-0.9383	-19.5328	1.0	18.5945	-289.6934	-137.5797	-2.5767	-2.1514
0.0007	0.83	600	0.0047	-2.3358	-22.0378	1.0	19.7019	-314.7427	-151.5544	-2.5354	-2.2155
0.0012	0.96	700	0.0052	-2.4343	-23.3733	1.0	20.9390	-328.0982	-152.5396	-2.4931	-2.1379

Framework versions

• Transformers 4.37.0
• Pytorch 2.1.2+cu121
• Datasets 2.14.6
• Tokenizers 0.15.2