IE_L3_1000steps_1e8rate_01beta_cSFTDPO

This model is a fine-tuned version of tsavage68/IE_L3_1000steps_1e6rate_SFT on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6897
• Rewards/chosen: -0.0098
• Rewards/rejected: -0.0175
• Rewards/accuracies: 0.4200
• Rewards/margins: 0.0078
• Logps/rejected: -75.8027
• Logps/chosen: -82.8953
• Logits/rejected: -0.7964
• Logits/chosen: -0.7394

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-08
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 100
• training_steps: 1000

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
0.6965	0.4	50	0.6929	-0.0030	-0.0041	0.3700	0.0011	-75.6681	-82.8275	-0.7963	-0.7392
0.6948	0.8	100	0.6908	-0.0022	-0.0074	0.4250	0.0052	-75.7008	-82.8198	-0.7961	-0.7393
0.6921	1.2	150	0.6946	-0.0077	-0.0055	0.375	-0.0022	-75.6824	-82.8750	-0.7972	-0.7399
0.6892	1.6	200	0.6941	-0.0042	-0.0030	0.3950	-0.0012	-75.6573	-82.8394	-0.7973	-0.7404
0.6937	2.0	250	0.6911	-0.0037	-0.0083	0.4000	0.0046	-75.7098	-82.8345	-0.7973	-0.7405
0.6933	2.4	300	0.6899	-0.0039	-0.0110	0.4300	0.0071	-75.7376	-82.8367	-0.7965	-0.7395
0.6915	2.8	350	0.6870	-0.0023	-0.0151	0.4700	0.0128	-75.7783	-82.8204	-0.7964	-0.7393
0.6933	3.2	400	0.6894	-0.0069	-0.0151	0.4100	0.0082	-75.7783	-82.8666	-0.7958	-0.7387
0.6981	3.6	450	0.6882	0.0006	-0.0100	0.4350	0.0106	-75.7275	-82.7918	-0.7968	-0.7398
0.6904	4.0	500	0.6896	-0.0001	-0.0078	0.4050	0.0077	-75.7054	-82.7989	-0.7958	-0.7391
0.6964	4.4	550	0.6867	-0.0021	-0.0157	0.4400	0.0136	-75.7838	-82.8187	-0.7965	-0.7396
0.6939	4.8	600	0.6902	0.0015	-0.0050	0.4000	0.0065	-75.6771	-82.7829	-0.7968	-0.7398
0.6963	5.2	650	0.6892	-0.0069	-0.0155	0.4200	0.0085	-75.7818	-82.8672	-0.7964	-0.7394
0.6951	5.6	700	0.6873	-0.0025	-0.0149	0.4650	0.0124	-75.7766	-82.8228	-0.7963	-0.7389
0.6855	6.0	750	0.6876	-0.0066	-0.0183	0.4550	0.0118	-75.8105	-82.8633	-0.7965	-0.7394
0.6873	6.4	800	0.6877	-0.0072	-0.0189	0.4550	0.0117	-75.8165	-82.8698	-0.7964	-0.7394
0.6848	6.8	850	0.6898	-0.0098	-0.0173	0.4100	0.0075	-75.8003	-82.8958	-0.7964	-0.7394
0.6983	7.2	900	0.6897	-0.0098	-0.0175	0.4200	0.0078	-75.8027	-82.8953	-0.7964	-0.7394
0.6859	7.6	950	0.6897	-0.0098	-0.0175	0.4200	0.0078	-75.8027	-82.8953	-0.7964	-0.7394
0.6888	8.0	1000	0.6897	-0.0098	-0.0175	0.4200	0.0078	-75.8027	-82.8953	-0.7964	-0.7394

Framework versions

• Transformers 4.44.2
• Pytorch 2.0.0+cu117
• Datasets 3.0.0
• Tokenizers 0.19.1