mmlu_math_noaugse1_llama_lora

This model is a fine-tuned version of Daewon0808/prm800k_llama_fulltune on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 0.2988
  • Prm accuracy: 0.8571
  • Prm precision: 0.8739
  • Prm recall: 0.9720
  • Prm specificty: 0.2105
  • Prm npv: 0.5714
  • Prm f1: 0.9204
  • Prm f1 neg: 0.3077
  • Prm f1 auc: 0.5912
  • Prm f1 auc (fixed): 0.8807

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0001
  • train_batch_size: 2
  • eval_batch_size: 4
  • seed: 908932403
  • distributed_type: multi-GPU
  • num_devices: 8
  • gradient_accumulation_steps: 2
  • total_train_batch_size: 32
  • total_eval_batch_size: 32
  • optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
  • lr_scheduler_type: cosine
  • lr_scheduler_warmup_ratio: 0.1
  • num_epochs: 1

Training results

Training Loss Epoch Step Validation Loss Prm accuracy Prm precision Prm recall Prm specificty Prm npv Prm f1 Prm f1 neg Prm f1 auc Prm f1 auc (fixed)
No log 0 0 0.3535 0.8333 0.8772 0.9346 0.2632 0.4167 0.9050 0.3226 0.5989 0.8195
0.2794 0.0246 5 0.3514 0.8333 0.8707 0.9439 0.2105 0.4 0.9058 0.2759 0.5772 0.8155
0.2101 0.0493 10 0.4065 0.8571 0.856 1.0 0.0526 1.0 0.9224 0.1 0.5263 0.8377
0.2629 0.0739 15 0.4323 0.8571 0.856 1.0 0.0526 1.0 0.9224 0.1 0.5263 0.8598
0.2389 0.0985 20 0.3076 0.8492 0.8729 0.9626 0.2105 0.5 0.9156 0.2963 0.5866 0.8613
0.1863 0.1232 25 0.3250 0.8651 0.875 0.9813 0.2105 0.6667 0.9251 0.32 0.5959 0.8610
0.2801 0.1478 30 0.3046 0.8492 0.8729 0.9626 0.2105 0.5 0.9156 0.2963 0.5866 0.8694
0.2207 0.1724 35 0.2880 0.8571 0.8803 0.9626 0.2632 0.5556 0.9196 0.3571 0.6129 0.8731
0.25 0.1970 40 0.2856 0.8651 0.8879 0.9626 0.3158 0.6 0.9238 0.4138 0.6392 0.8709
0.2113 0.2217 45 0.2917 0.8571 0.8803 0.9626 0.2632 0.5556 0.9196 0.3571 0.6129 0.8719
0.1327 0.2463 50 0.2954 0.8651 0.8879 0.9626 0.3158 0.6 0.9238 0.4138 0.6392 0.8662
0.1599 0.2709 55 0.3042 0.8730 0.8889 0.9720 0.3158 0.6667 0.9286 0.4286 0.6439 0.8637
0.1855 0.2956 60 0.3409 0.8413 0.8655 0.9626 0.1579 0.4286 0.9115 0.2308 0.5603 0.8711
0.2119 0.3202 65 0.3051 0.8651 0.8879 0.9626 0.3158 0.6 0.9238 0.4138 0.6392 0.8660
0.156 0.3448 70 0.3285 0.8413 0.8655 0.9626 0.1579 0.4286 0.9115 0.2308 0.5603 0.8721
0.3401 0.3695 75 0.3260 0.8571 0.8678 0.9813 0.1579 0.6 0.9211 0.25 0.5696 0.8674
0.1796 0.3941 80 0.3070 0.8413 0.8655 0.9626 0.1579 0.4286 0.9115 0.2308 0.5603 0.8667
0.235 0.4187 85 0.3169 0.8492 0.8607 0.9813 0.1053 0.5 0.9170 0.1739 0.5433 0.8728
0.1436 0.4433 90 0.2999 0.8492 0.8729 0.9626 0.2105 0.5 0.9156 0.2963 0.5866 0.8746
0.178 0.4680 95 0.2941 0.8492 0.8729 0.9626 0.2105 0.5 0.9156 0.2963 0.5866 0.8736
0.1733 0.4926 100 0.3088 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8775
0.125 0.5172 105 0.3066 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8802
0.154 0.5419 110 0.2838 0.8730 0.8957 0.9626 0.3684 0.6364 0.9279 0.4667 0.6655 0.8795
0.1651 0.5665 115 0.2887 0.8651 0.8879 0.9626 0.3158 0.6 0.9238 0.4138 0.6392 0.8832
0.1942 0.5911 120 0.3150 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8898
0.2232 0.6158 125 0.3050 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8829
0.203 0.6404 130 0.2900 0.8730 0.8889 0.9720 0.3158 0.6667 0.9286 0.4286 0.6439 0.8773
0.2011 0.6650 135 0.2883 0.8730 0.8889 0.9720 0.3158 0.6667 0.9286 0.4286 0.6439 0.8819
0.1789 0.6897 140 0.2964 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8810
0.17 0.7143 145 0.3107 0.8730 0.8760 0.9907 0.2105 0.8 0.9298 0.3333 0.6006 0.8842
0.1512 0.7389 150 0.3141 0.8730 0.8760 0.9907 0.2105 0.8 0.9298 0.3333 0.6006 0.8842
0.1368 0.7635 155 0.3115 0.8651 0.875 0.9813 0.2105 0.6667 0.9251 0.32 0.5959 0.8854
0.1492 0.7882 160 0.3066 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8844
0.1946 0.8128 165 0.2986 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8819
0.1832 0.8374 170 0.2962 0.8651 0.8814 0.9720 0.2632 0.625 0.9244 0.3704 0.6176 0.8815
0.168 0.8621 175 0.2970 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8827
0.1285 0.8867 180 0.2977 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8810
0.1991 0.9113 185 0.2979 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8810
0.1585 0.9360 190 0.2975 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8815
0.2369 0.9606 195 0.2977 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8815
0.1665 0.9852 200 0.2988 0.8571 0.8739 0.9720 0.2105 0.5714 0.9204 0.3077 0.5912 0.8807

Framework versions

  • PEFT 0.12.0
  • Transformers 4.46.0
  • Pytorch 2.4.0+cu118
  • Datasets 3.0.0
  • Tokenizers 0.20.1
Downloads last month
11
Inference Providers NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API: The model has no pipeline_tag.

Model tree for Daewon0808/mmlu_math_noaugse1_llama_lora