base-sft-safe-spin-v

This model is a fine-tuned version of alignment-handbook/zephyr-7b-sft-full on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.0738
• Rewards/real: -3.0711
• Rewards/generated: -13.2471
• Rewards/accuracies: 0.9713
• Rewards/margins: 10.1760
• Logps/generated: -228.7879
• Logps/real: -165.3767
• Logits/generated: -2.4198
• Logits/real: -2.4231

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-07
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 4
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/real	Rewards/generated	Rewards/accuracies	Rewards/margins	Logps/generated	Logps/real	Logits/generated	Logits/real
0.3742	0.06	100	0.2244	-0.3695	-6.6880	0.9658	6.3185	-163.1966	-138.3603	-2.7435	-2.7148
0.2528	0.12	200	0.1319	-1.2400	-17.8536	0.9697	16.6136	-274.8525	-147.0654	-2.4573	-2.4671
0.2066	0.17	300	0.1172	-1.6714	-19.7358	0.9618	18.0643	-293.6746	-151.3799	-2.4257	-2.3622
0.2207	0.23	400	0.1094	-1.9426	-20.6733	0.9729	18.7307	-303.0500	-154.0918	-2.4889	-2.4525
0.4379	0.29	500	0.1152	-1.0002	-8.3421	0.9666	7.3419	-179.7377	-144.6674	-2.3870	-2.3441
0.1517	0.35	600	0.0984	-1.6577	-12.9237	0.9745	11.2660	-225.5533	-151.2425	-2.2691	-2.2742
0.1708	0.41	700	0.0866	-1.9495	-14.1941	0.9745	12.2446	-238.2574	-154.1605	-2.2343	-2.2124
0.1135	0.47	800	0.0810	-3.0171	-16.4497	0.9785	13.4327	-260.8139	-164.8361	-2.1789	-2.1987
0.1364	0.52	900	0.0848	-2.5549	-14.8091	0.9729	12.2542	-244.4078	-160.2151	-2.3295	-2.3368
0.1142	0.58	1000	0.0902	-2.6698	-10.6438	0.9713	7.9740	-202.7553	-161.3638	-2.4644	-2.4787
0.1332	0.64	1100	0.0771	-2.7436	-11.8738	0.9785	9.1302	-215.0552	-162.1016	-2.4417	-2.4630
0.1007	0.7	1200	0.0758	-3.4115	-14.1899	0.9745	10.7784	-238.2156	-168.7807	-2.3948	-2.4255
0.1306	0.76	1300	0.0765	-2.4042	-11.1062	0.9753	8.7019	-207.3786	-158.7081	-2.5270	-2.5375
0.1084	0.81	1400	0.0760	-2.7805	-12.4025	0.9745	9.6220	-220.3422	-162.4709	-2.4762	-2.4848
0.1494	0.87	1500	0.0740	-3.0055	-13.0014	0.9713	9.9959	-226.3309	-164.7203	-2.4656	-2.4751
0.1099	0.93	1600	0.0774	-3.4971	-13.6736	0.9729	10.1765	-233.0532	-169.6366	-2.4253	-2.4320
0.0906	0.99	1700	0.0738	-3.0711	-13.2471	0.9713	10.1760	-228.7879	-165.3767	-2.4198	-2.4231

Framework versions

• Transformers 4.37.0
• Pytorch 2.1.2+cu121
• Datasets 2.14.6
• Tokenizers 0.15.2