mmlu_math_noaugse1_llama_lora

This model is a fine-tuned version of Daewon0808/prm800k_llama_fulltune on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.2988
• Prm accuracy: 0.8571
• Prm precision: 0.8739
• Prm recall: 0.9720
• Prm specificty: 0.2105
• Prm npv: 0.5714
• Prm f1: 0.9204
• Prm f1 neg: 0.3077
• Prm f1 auc: 0.5912
• Prm f1 auc (fixed): 0.8807

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 2
• eval_batch_size: 4
• seed: 908932403
• distributed_type: multi-GPU
• num_devices: 8
• gradient_accumulation_steps: 2
• total_train_batch_size: 32
• total_eval_batch_size: 32
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: cosine
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Prm accuracy	Prm precision	Prm recall	Prm specificty	Prm npv	Prm f1	Prm f1 neg	Prm f1 auc	Prm f1 auc (fixed)
No log	0	0	0.3535	0.8333	0.8772	0.9346	0.2632	0.4167	0.9050	0.3226	0.5989	0.8195
0.2794	0.0246	5	0.3514	0.8333	0.8707	0.9439	0.2105	0.4	0.9058	0.2759	0.5772	0.8155
0.2101	0.0493	10	0.4065	0.8571	0.856	1.0	0.0526	1.0	0.9224	0.1	0.5263	0.8377
0.2629	0.0739	15	0.4323	0.8571	0.856	1.0	0.0526	1.0	0.9224	0.1	0.5263	0.8598
0.2389	0.0985	20	0.3076	0.8492	0.8729	0.9626	0.2105	0.5	0.9156	0.2963	0.5866	0.8613
0.1863	0.1232	25	0.3250	0.8651	0.875	0.9813	0.2105	0.6667	0.9251	0.32	0.5959	0.8610
0.2801	0.1478	30	0.3046	0.8492	0.8729	0.9626	0.2105	0.5	0.9156	0.2963	0.5866	0.8694
0.2207	0.1724	35	0.2880	0.8571	0.8803	0.9626	0.2632	0.5556	0.9196	0.3571	0.6129	0.8731
0.25	0.1970	40	0.2856	0.8651	0.8879	0.9626	0.3158	0.6	0.9238	0.4138	0.6392	0.8709
0.2113	0.2217	45	0.2917	0.8571	0.8803	0.9626	0.2632	0.5556	0.9196	0.3571	0.6129	0.8719
0.1327	0.2463	50	0.2954	0.8651	0.8879	0.9626	0.3158	0.6	0.9238	0.4138	0.6392	0.8662
0.1599	0.2709	55	0.3042	0.8730	0.8889	0.9720	0.3158	0.6667	0.9286	0.4286	0.6439	0.8637
0.1855	0.2956	60	0.3409	0.8413	0.8655	0.9626	0.1579	0.4286	0.9115	0.2308	0.5603	0.8711
0.2119	0.3202	65	0.3051	0.8651	0.8879	0.9626	0.3158	0.6	0.9238	0.4138	0.6392	0.8660
0.156	0.3448	70	0.3285	0.8413	0.8655	0.9626	0.1579	0.4286	0.9115	0.2308	0.5603	0.8721
0.3401	0.3695	75	0.3260	0.8571	0.8678	0.9813	0.1579	0.6	0.9211	0.25	0.5696	0.8674
0.1796	0.3941	80	0.3070	0.8413	0.8655	0.9626	0.1579	0.4286	0.9115	0.2308	0.5603	0.8667
0.235	0.4187	85	0.3169	0.8492	0.8607	0.9813	0.1053	0.5	0.9170	0.1739	0.5433	0.8728
0.1436	0.4433	90	0.2999	0.8492	0.8729	0.9626	0.2105	0.5	0.9156	0.2963	0.5866	0.8746
0.178	0.4680	95	0.2941	0.8492	0.8729	0.9626	0.2105	0.5	0.9156	0.2963	0.5866	0.8736
0.1733	0.4926	100	0.3088	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8775
0.125	0.5172	105	0.3066	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8802
0.154	0.5419	110	0.2838	0.8730	0.8957	0.9626	0.3684	0.6364	0.9279	0.4667	0.6655	0.8795
0.1651	0.5665	115	0.2887	0.8651	0.8879	0.9626	0.3158	0.6	0.9238	0.4138	0.6392	0.8832
0.1942	0.5911	120	0.3150	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8898
0.2232	0.6158	125	0.3050	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8829
0.203	0.6404	130	0.2900	0.8730	0.8889	0.9720	0.3158	0.6667	0.9286	0.4286	0.6439	0.8773
0.2011	0.6650	135	0.2883	0.8730	0.8889	0.9720	0.3158	0.6667	0.9286	0.4286	0.6439	0.8819
0.1789	0.6897	140	0.2964	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8810
0.17	0.7143	145	0.3107	0.8730	0.8760	0.9907	0.2105	0.8	0.9298	0.3333	0.6006	0.8842
0.1512	0.7389	150	0.3141	0.8730	0.8760	0.9907	0.2105	0.8	0.9298	0.3333	0.6006	0.8842
0.1368	0.7635	155	0.3115	0.8651	0.875	0.9813	0.2105	0.6667	0.9251	0.32	0.5959	0.8854
0.1492	0.7882	160	0.3066	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8844
0.1946	0.8128	165	0.2986	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8819
0.1832	0.8374	170	0.2962	0.8651	0.8814	0.9720	0.2632	0.625	0.9244	0.3704	0.6176	0.8815
0.168	0.8621	175	0.2970	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8827
0.1285	0.8867	180	0.2977	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8810
0.1991	0.9113	185	0.2979	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8810
0.1585	0.9360	190	0.2975	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8815
0.2369	0.9606	195	0.2977	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8815
0.1665	0.9852	200	0.2988	0.8571	0.8739	0.9720	0.2105	0.5714	0.9204	0.3077	0.5912	0.8807

Framework versions

• PEFT 0.12.0
• Transformers 4.46.0
• Pytorch 2.4.0+cu118
• Datasets 3.0.0
• Tokenizers 0.20.1