Llama3.1-8B-QA_CoT-MEDICAL-Instruct-r64

This model is a fine-tuned version of meta-llama/Llama-3.1-8B on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5788

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3.6e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Use OptimizerNames.PAGED_ADAMW_8BIT with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 1
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss
0.8399	0.0064	200	0.9350
0.9024	0.0128	400	0.8735
0.7805	0.0192	600	0.8382
0.905	0.0256	800	0.8193
0.762	0.032	1000	0.8070
0.5496	0.0384	1200	0.7964
0.1024	0.0448	1400	0.7896
0.5931	0.0512	1600	0.7859
0.6966	0.0576	1800	0.7791
0.7713	0.064	2000	0.7753
0.7055	0.0704	2200	0.7715
0.9559	0.0768	2400	0.7681
0.5054	0.0832	2600	0.7646
0.7603	0.0896	2800	0.7622
0.9815	0.096	3000	0.7593
0.6948	0.1024	3200	0.7569
1.1539	0.1088	3400	0.7535
0.9092	0.1152	3600	0.7504
0.4105	0.1216	3800	0.7478
0.8514	0.128	4000	0.7464
0.6902	0.1344	4200	0.7431
0.5141	0.1408	4400	0.7408
0.5374	0.1472	4600	0.7393
0.9075	0.1536	4800	0.7382
0.4641	0.16	5000	0.7352
0.9326	0.1664	5200	0.7327
0.9203	0.1728	5400	0.7315
0.9802	0.1792	5600	0.7294
0.8411	0.1856	5800	0.7262
0.8436	0.192	6000	0.7234
0.9267	0.1984	6200	0.7220
0.8866	0.2048	6400	0.7203
0.4686	0.2112	6600	0.7188
1.1163	0.2176	6800	0.7171
0.8547	0.224	7000	0.7152
0.6537	0.2304	7200	0.7132
0.562	0.2368	7400	0.7117
0.0866	0.2432	7600	0.7111
1.259	0.2496	7800	0.7083
0.5031	0.256	8000	0.7078
0.6377	0.2624	8200	0.7054
0.664	0.2688	8400	0.7039
0.7615	0.2752	8600	0.7023
0.9197	0.2816	8800	0.7009
0.9291	0.288	9000	0.6991
0.6651	0.2944	9200	0.6972
0.9469	0.3008	9400	0.6965
0.9489	0.3072	9600	0.6940
0.4859	0.3136	9800	0.6931
0.8835	0.32	10000	0.6911
0.5059	0.3264	10200	0.6899
0.9658	0.3328	10400	0.6886
0.4364	0.3392	10600	0.6866
0.8759	0.3456	10800	0.6856
0.6947	0.352	11000	0.6834
0.9081	0.3584	11200	0.6818
0.6082	0.3648	11400	0.6805
0.7717	0.3712	11600	0.6789
0.8324	0.3776	11800	0.6777
0.6286	0.384	12000	0.6762
0.8363	0.3904	12200	0.6741
0.5502	0.3968	12400	0.6734
0.4254	0.4032	12600	0.6720
0.8676	0.4096	12800	0.6708
0.8529	0.416	13000	0.6693
0.8278	0.4224	13200	0.6679
0.561	0.4288	13400	0.6665
0.8632	0.4352	13600	0.6646
0.5054	0.4416	13800	0.6633
0.7296	0.448	14000	0.6614
0.4644	0.4544	14200	0.6610
0.851	0.4608	14400	0.6586
0.8565	0.4672	14600	0.6577
0.5205	0.4736	14800	0.6569
0.8272	0.48	15000	0.6550
0.9689	0.4864	15200	0.6533
0.7433	0.4928	15400	0.6525
0.4175	0.4992	15600	0.6515
0.8158	0.5056	15800	0.6500
0.0571	0.512	16000	0.6486
0.3766	0.5184	16200	0.6474
0.546	0.5248	16400	0.6457
0.7811	0.5312	16600	0.6439
0.6592	0.5376	16800	0.6427
0.7379	0.544	17000	0.6413
0.8019	0.5504	17200	0.6403
0.6117	0.5568	17400	0.6386
0.9152	0.5632	17600	0.6373
0.8364	0.5696	17800	0.6359
0.6807	0.576	18000	0.6346
0.3434	0.5824	18200	0.6337
0.6248	0.5888	18400	0.6331
0.4177	0.5952	18600	0.6320
0.9378	0.6016	18800	0.6305
0.5745	0.608	19000	0.6288
0.5786	0.6144	19200	0.6268
0.6503	0.6208	19400	0.6260
0.3875	0.6272	19600	0.6250
0.3826	0.6336	19800	0.6244
0.5301	0.64	20000	0.6231
0.5212	0.6464	20200	0.6228
0.2531	0.6528	20400	0.6210
0.514	0.6592	20600	0.6198
0.7908	0.6656	20800	0.6188
0.7131	0.672	21000	0.6174
0.8612	0.6784	21200	0.6166
0.5492	0.6848	21400	0.6153
0.2713	0.6912	21600	0.6137
0.3661	0.6976	21800	0.6122
0.6265	0.704	22000	0.6114
0.7062	0.7104	22200	0.6104
0.2984	0.7168	22400	0.6094
0.6959	0.7232	22600	0.6079
0.8605	0.7296	22800	0.6066
0.5706	0.736	23000	0.6059
0.5996	0.7424	23200	0.6044
0.6127	0.7488	23400	0.6039
0.3297	0.7552	23600	0.6034
0.8156	0.7616	23800	0.6024
0.5316	0.768	24000	0.6011
0.6661	0.7744	24200	0.6005
0.8805	0.7808	24400	0.5997
0.676	0.7872	24600	0.5981
0.25	0.7936	24800	0.5972
0.3684	0.8	25000	0.5959
0.5218	0.8064	25200	0.5951
0.6316	0.8128	25400	0.5945
0.2007	0.8192	25600	0.5936
0.272	0.8256	25800	0.5929
0.4253	0.832	26000	0.5920
0.4194	0.8384	26200	0.5908
0.7739	0.8448	26400	0.5905
0.4304	0.8512	26600	0.5895
0.2536	0.8576	26800	0.5887
0.794	0.864	27000	0.5880
0.3571	0.8704	27200	0.5872
0.7102	0.8768	27400	0.5866
0.3415	0.8832	27600	0.5858
0.5687	0.8896	27800	0.5853
0.6717	0.896	28000	0.5846
0.6736	0.9024	28200	0.5841
0.4328	0.9088	28400	0.5837
0.5114	0.9152	28600	0.5830
0.6912	0.9216	28800	0.5825
0.6255	0.928	29000	0.5820
0.4711	0.9344	29200	0.5815
0.6466	0.9408	29400	0.5810
0.5242	0.9472	29600	0.5806
0.5089	0.9536	29800	0.5802
0.366	0.96	30000	0.5798
0.6565	0.9664	30200	0.5796
0.7487	0.9728	30400	0.5794
0.8204	0.9792	30600	0.5792
0.8001	0.9856	30800	0.5790
0.582	0.992	31000	0.5789
0.4883	0.9984	31200	0.5788

Framework versions

• PEFT 0.12.0
• Transformers 4.47.0
• Pytorch 2.5.1+cu124
• Datasets 3.0.0
• Tokenizers 0.21.0