PairRM-V2-phi3-3-mini-unified-feedback

This model is a fine-tuned version of microsoft/Phi-3-mini-128k-instruct on the all dataset. It achieves the following results on the evaluation set:

• Loss: 0.2755

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 16
• total_train_batch_size: 128
• total_eval_batch_size: 8
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.05
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
0.3099	0.3245	500	0.3066
0.3073	0.6490	1000	0.2901
0.263	0.9736	1500	0.2846
0.2822	1.2981	2000	0.2831
0.2693	1.6226	2500	0.2787
0.2741	1.9471	3000	0.2778
0.2869	2.2716	3500	0.2762
0.2339	2.5961	4000	0.2756
0.2879	2.9207	4500	0.2755

Framework versions

• PEFT 0.11.1
• Transformers 4.43.1
• Pytorch 2.3.0+cu121
• Datasets 2.20.0
• Tokenizers 0.19.1