dpo5

This model is a fine-tuned version of deepseek-ai/deepseek-coder-1.3b-base on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: nan
• Rewards/chosen: nan
• Rewards/rejected: nan
• Rewards/accuracies: 0.0494
• Rewards/margins: nan
• Logps/rejected: nan
• Logps/chosen: nan
• Logits/rejected: nan
• Logits/chosen: nan

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.1
• train_batch_size: 4
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 8
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 200
• num_epochs: 4
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rewards/chosen	Rewards/rejected	Rewards/accuracies	Rewards/margins	Logps/rejected	Logps/chosen	Logits/rejected	Logits/chosen
4774.8922	0.5776	100	nan	nan	nan	0.0494	nan	nan	nan	nan	nan
0.0	1.1552	200	nan	nan	nan	0.0494	nan	nan	nan	nan	nan
0.0	1.7329	300	nan	nan	nan	0.0494	nan	nan	nan	nan	nan
0.0	2.3105	400	nan	nan	nan	0.0494	nan	nan	nan	nan	nan
0.0	2.8881	500	nan	nan	nan	0.0494	nan	nan	nan	nan	nan
19.2236	3.4657	600	nan	nan	nan	0.0494	nan	nan	nan	nan	nan

Framework versions

• Transformers 4.45.0
• Pytorch 2.5.1+cu124
• Datasets 2.19.2
• Tokenizers 0.20.3