results_medium

This model is a fine-tuned version of microsoft/Phi-3-medium-128k-instruct on the None dataset. It achieves the following results on the evaluation set:

• Loss: 1.1562

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 1
• eval_batch_size: 1
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 4
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
1.9442	0.2131	100	1.9833
1.8439	0.4262	200	1.8431
1.5387	0.6393	300	1.5840
1.4157	0.8524	400	1.3748
1.2628	1.0655	500	1.2339
1.2704	1.2786	600	1.1894
1.1634	1.4917	700	1.1763
1.2318	1.7048	800	1.1708
1.0665	1.9180	900	1.1683
1.1715	2.1311	1000	1.1634
1.1366	2.3442	1100	1.1617
1.1239	2.5573	1200	1.1584
1.1442	2.7704	1300	1.1568
1.1336	2.9835	1400	1.1562

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1