CH-Phi3m4k

This model is a fine-tuned version of microsoft/Phi-3-mini-4k-instruct on an casehold/casehold dataset. It achieves the following results on the evaluation set:

• Loss: 1.5304

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 1
• seed: 342
• gradient_accumulation_steps: 4
• total_train_batch_size: 32
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Validation F1
2.0053	0.1422	200	1.8927	0.722
1.6481	0.2844	400	1.6065	0.732
1.5653	0.4267	600	1.5864	0.746
1.5516	0.5689	800	1.5762	0.758
1.537	0.7111	1000	1.5689	0.768
1.5318	0.8533	1200	1.5637	0.772
1.5249	0.9956	1400	1.5595	0.774
1.5208	1.1378	1600	1.5550	0.768
1.5234	1.28	1800	1.5511	0.778
1.5099	1.4222	2000	1.5480	0.774
1.5144	1.5644	2200	1.5448	0.774
1.5139	1.7067	2400	1.5424	0.776
1.5013	1.8489	2600	1.5395	0.778
1.502	1.9911	2800	1.5377	0.770
1.4979	2.1333	3000	1.5358	0.768
1.491	2.2756	3200	1.5338	0.768
1.5025	2.4178	3400	1.5318	0.772
1.4933	2.56	3600	1.5304	0.772

Framework versions

• PEFT 0.11.1
• Transformers 4.41.2
• Pytorch 2.3.0+cu121
• Datasets 2.19.1
• Tokenizers 0.19.1