Update README.md
Browse files
README.md
CHANGED
|
@@ -23,6 +23,4 @@ Prompt template:
|
|
| 23 |
GGML quants available [here](https://huggingface.co/TheBloke/Griffin-3B-GGML).</br>
|
| 24 |
GPTQ quants available [here](https://huggingface.co/TheBloke/Griffin-3B-GPTQ).
|
| 25 |
|
| 26 |
-
Note: Don't expect this model to be good, I was just starting out to finetune. So don't roast me please!
|
| 27 |
-
|
| 28 |
-
Benchmarks: <table><tr><th>hellaswag</th><td><table><tr><th>acc</th><td>0.5415255925114519</td></tr><tr><th>acc_stderr</th><td>0.004972543127767884</td></tr><tr><th>acc_norm</th><td>0.7211710814578769</td></tr><tr><th>acc_norm_stderr</th><td>0.004475067344626758</td></tr></table></td></tr><tr><th>hendrycksTest-abstract_algebra</th><td><table><tr><th>acc</th><td>0.22</td></tr><tr><th>acc_stderr</th><td>0.04163331998932269</td></tr><tr><th>acc_norm</th><td>0.22</td></tr><tr><th>acc_norm_stderr</th><td>0.04163331998932269</td></tr></table></td></tr><tr><th>hendrycksTest-anatomy</th><td><table><tr><th>acc</th><td>0.21481481481481482</td></tr><tr><th>acc_stderr</th><td>0.03547854198560824</td></tr><tr><th>acc_norm</th><td>0.21481481481481482</td></tr><tr><th>acc_norm_stderr</th><td>0.03547854198560824</td></tr></table></td></tr><tr><th>hendrycksTest-astronomy</th><td><table><tr><th>acc</th><td>0.2894736842105263</td></tr><tr><th>acc_stderr</th><td>0.03690677986137283</td></tr><tr><th>acc_norm</th><td>0.2894736842105263</td></tr><tr><th>acc_norm_stderr</th><td>0.03690677986137283</td></tr></table></td></tr><tr><th>hendrycksTest-business_ethics</th><td><table><tr><th>acc</th><td>0.37</td></tr><tr><th>acc_stderr</th><td>0.04852365870939099</td></tr><tr><th>acc_norm</th><td>0.37</td></tr><tr><th>acc_norm_stderr</th><td>0.04852365870939099</td></tr></table></td></tr><tr><th>hendrycksTest-clinical_knowledge</th><td><table><tr><th>acc</th><td>0.2641509433962264</td></tr><tr><th>acc_stderr</th><td>0.02713429162874172</td></tr><tr><th>acc_norm</th><td>0.2641509433962264</td></tr><tr><th>acc_norm_stderr</th><td>0.02713429162874172</td></tr></table></td></tr><tr><th>hendrycksTest-college_biology</th><td><table><tr><th>acc</th><td>0.2361111111111111</td></tr><tr><th>acc_stderr</th><td>0.03551446610810826</td></tr><tr><th>acc_norm</th><td>0.2361111111111111</td></tr><tr><th>acc_norm_stderr</th><td>0.03551446610810826</td></tr></table></td></tr><tr><th>hendrycksTest-college_chemistry</th><td><table><tr><th>acc</th><td>0.24</td></tr><tr><th>acc_stderr</th><td>0.04292346959909284</td></tr><tr><th>acc_norm</th><td>0.24</td></tr><tr><th>acc_norm_stderr</th><td>0.04292346959909284</td></tr></table></td></tr><tr><th>hendrycksTest-college_computer_science</th><td><table><tr><th>acc</th><td>0.22</td></tr><tr><th>acc_stderr</th><td>0.04163331998932269</td></tr><tr><th>acc_norm</th><td>0.22</td></tr><tr><th>acc_norm_stderr</th><td>0.04163331998932269</td></tr></table></td></tr><tr><th>hendrycksTest-college_mathematics</th><td><table><tr><th>acc</th><td>0.3</td></tr><tr><th>acc_stderr</th><td>0.046056618647183814</td></tr><tr><th>acc_norm</th><td>0.3</td></tr><tr><th>acc_norm_stderr</th><td>0.046056618647183814</td></tr></table></td></tr><tr><th>hendrycksTest-college_medicine</th><td><table><tr><th>acc</th><td>0.21965317919075145</td></tr><tr><th>acc_stderr</th><td>0.031568093627031744</td></tr><tr><th>acc_norm</th><td>0.21965317919075145</td></tr><tr><th>acc_norm_stderr</th><td>0.031568093627031744</td></tr></table></td></tr><tr><th>hendrycksTest-college_physics</th><td><table><tr><th>acc</th><td>0.19607843137254902</td></tr><tr><th>acc_stderr</th><td>0.03950581861179961</td></tr><tr><th>acc_norm</th><td>0.19607843137254902</td></tr><tr><th>acc_norm_stderr</th><td>0.03950581861179961</td></tr></table></td></tr><tr><th>hendrycksTest-computer_security</th><td><table><tr><th>acc</th><td>0.33</td></tr><tr><th>acc_stderr</th><td>0.04725815626252605</td></tr><tr><th>acc_norm</th><td>0.33</td></tr><tr><th>acc_norm_stderr</th><td>0.04725815626252605</td></tr></table></td></tr><tr><th>hendrycksTest-conceptual_physics</th><td><table><tr><th>acc</th><td>0.3148936170212766</td></tr><tr><th>acc_stderr</th><td>0.03036358219723817</td></tr><tr><th>acc_norm</th><td>0.3148936170212766</td></tr><tr><th>acc_norm_stderr</th><td>0.03036358219723817</td></tr></table></td></tr><tr><th>hendrycksTest-econometrics</th><td><table><tr><th>acc</th><td>0.22807017543859648</td></tr><tr><th>acc_stderr</th><td>0.03947152782669416</td></tr><tr><th>acc_norm</th><td>0.22807017543859648</td></tr><tr><th>acc_norm_stderr</th><td>0.03947152782669416</td></tr></table></td></tr><tr><th>hendrycksTest-electrical_engineering</th><td><table><tr><th>acc</th><td>0.2</td></tr><tr><th>acc_stderr</th><td>0.0333333333333333</td></tr><tr><th>acc_norm</th><td>0.2</td></tr><tr><th>acc_norm_stderr</th><td>0.0333333333333333</td></tr></table></td></tr><tr><th>hendrycksTest-elementary_mathematics</th><td><table><tr><th>acc</th><td>0.29365079365079366</td></tr><tr><th>acc_stderr</th><td>0.023456037383982022</td></tr><tr><th>acc_norm</th><td>0.29365079365079366</td></tr><tr><th>acc_norm_stderr</th><td>0.023456037383982022</td></tr></table></td></tr><tr><th>hendrycksTest-formal_logic</th><td><table><tr><th>acc</th><td>0.1984126984126984</td></tr><tr><th>acc_stderr</th><td>0.03567016675276862</td></tr><tr><th>acc_norm</th><td>0.1984126984126984</td></tr><tr><th>acc_norm_stderr</th><td>0.03567016675276862</td></tr></table></td></tr><tr><th>hendrycksTest-global_facts</th><td><table><tr><th>acc</th><td>0.32</td></tr><tr><th>acc_stderr</th><td>0.046882617226215034</td></tr><tr><th>acc_norm</th><td>0.32</td></tr><tr><th>acc_norm_stderr</th><td>0.046882617226215034</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_biology</th><td><table><tr><th>acc</th><td>0.23225806451612904</td></tr><tr><th>acc_stderr</th><td>0.02402225613030824</td></tr><tr><th>acc_norm</th><td>0.23225806451612904</td></tr><tr><th>acc_norm_stderr</th><td>0.02402225613030824</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_chemistry</th><td><table><tr><th>acc</th><td>0.23645320197044334</td></tr><tr><th>acc_stderr</th><td>0.029896114291733552</td></tr><tr><th>acc_norm</th><td>0.23645320197044334</td></tr><tr><th>acc_norm_stderr</th><td>0.029896114291733552</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_computer_science</th><td><table><tr><th>acc</th><td>0.26</td></tr><tr><th>acc_stderr</th><td>0.044084400227680794</td></tr><tr><th>acc_norm</th><td>0.26</td></tr><tr><th>acc_norm_stderr</th><td>0.044084400227680794</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_european_history</th><td><table><tr><th>acc</th><td>0.22424242424242424</td></tr><tr><th>acc_stderr</th><td>0.032568666616811015</td></tr><tr><th>acc_norm</th><td>0.22424242424242424</td></tr><tr><th>acc_norm_stderr</th><td>0.032568666616811015</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_geography</th><td><table><tr><th>acc</th><td>0.20707070707070707</td></tr><tr><th>acc_stderr</th><td>0.02886977846026705</td></tr><tr><th>acc_norm</th><td>0.20707070707070707</td></tr><tr><th>acc_norm_stderr</th><td>0.02886977846026705</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_government_and_politics</th><td><table><tr><th>acc</th><td>0.22797927461139897</td></tr><tr><th>acc_stderr</th><td>0.030276909945178274</td></tr><tr><th>acc_norm</th><td>0.22797927461139897</td></tr><tr><th>acc_norm_stderr</th><td>0.030276909945178274</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_macroeconomics</th><td><table><tr><th>acc</th><td>0.2358974358974359</td></tr><tr><th>acc_stderr</th><td>0.021525965407408726</td></tr><tr><th>acc_norm</th><td>0.2358974358974359</td></tr><tr><th>acc_norm_stderr</th><td>0.021525965407408726</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_mathematics</th><td><table><tr><th>acc</th><td>0.25925925925925924</td></tr><tr><th>acc_stderr</th><td>0.026719240783712163</td></tr><tr><th>acc_norm</th><td>0.25925925925925924</td></tr><tr><th>acc_norm_stderr</th><td>0.026719240783712163</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_microeconomics</th><td><table><tr><th>acc</th><td>0.2815126050420168</td></tr><tr><th>acc_stderr</th><td>0.029213549414372167</td></tr><tr><th>acc_norm</th><td>0.2815126050420168</td></tr><tr><th>acc_norm_stderr</th><td>0.029213549414372167</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_physics</th><td><table><tr><th>acc</th><td>0.2980132450331126</td></tr><tr><th>acc_stderr</th><td>0.037345356767871984</td></tr><tr><th>acc_norm</th><td>0.2980132450331126</td></tr><tr><th>acc_norm_stderr</th><td>0.037345356767871984</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_psychology</th><td><table><tr><th>acc</th><td>0.23669724770642203</td></tr><tr><th>acc_stderr</th><td>0.01822407811729907</td></tr><tr><th>acc_norm</th><td>0.23669724770642203</td></tr><tr><th>acc_norm_stderr</th><td>0.01822407811729907</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_statistics</th><td><table><tr><th>acc</th><td>0.1574074074074074</td></tr><tr><th>acc_stderr</th><td>0.02483717351824239</td></tr><tr><th>acc_norm</th><td>0.1574074074074074</td></tr><tr><th>acc_norm_stderr</th><td>0.02483717351824239</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_us_history</th><td><table><tr><th>acc</th><td>0.2647058823529412</td></tr><tr><th>acc_stderr</th><td>0.03096451792692341</td></tr><tr><th>acc_norm</th><td>0.2647058823529412</td></tr><tr><th>acc_norm_stderr</th><td>0.03096451792692341</td></tr></table></td></tr><tr><th>hendrycksTest-high_school_world_history</th><td><table><tr><th>acc</th><td>0.28270042194092826</td></tr><tr><th>acc_stderr</th><td>0.02931281415395592</td></tr><tr><th>acc_norm</th><td>0.28270042194092826</td></tr><tr><th>acc_norm_stderr</th><td>0.02931281415395592</td></tr></table></td></tr><tr><th>hendrycksTest-human_aging</th><td><table><tr><th>acc</th><td>0.39461883408071746</td></tr><tr><th>acc_stderr</th><td>0.03280400504755291</td></tr><tr><th>acc_norm</th><td>0.39461883408071746</td></tr><tr><th>acc_norm_stderr</th><td>0.03280400504755291</td></tr></table></td></tr><tr><th>hendrycksTest-human_sexuality</th><td><table><tr><th>acc</th><td>0.22900763358778625</td></tr><tr><th>acc_stderr</th><td>0.036853466317118506</td></tr><tr><th>acc_norm</th><td>0.22900763358778625</td></tr><tr><th>acc_norm_stderr</th><td>0.036853466317118506</td></tr></table></td></tr><tr><th>hendrycksTest-international_law</th><td><table><tr><th>acc</th><td>0.30578512396694213</td></tr><tr><th>acc_stderr</th><td>0.04205953933884124</td></tr><tr><th>acc_norm</th><td>0.30578512396694213</td></tr><tr><th>acc_norm_stderr</th><td>0.04205953933884124</td></tr></table></td></tr><tr><th>hendrycksTest-jurisprudence</th><td><table><tr><th>acc</th><td>0.32407407407407407</td></tr><tr><th>acc_stderr</th><td>0.04524596007030048</td></tr><tr><th>acc_norm</th><td>0.32407407407407407</td></tr><tr><th>acc_norm_stderr</th><td>0.04524596007030048</td></tr></table></td></tr><tr><th>hendrycksTest-logical_fallacies</th><td><table><tr><th>acc</th><td>0.22699386503067484</td></tr><tr><th>acc_stderr</th><td>0.03291099578615769</td></tr><tr><th>acc_norm</th><td>0.22699386503067484</td></tr><tr><th>acc_norm_stderr</th><td>0.03291099578615769</td></tr></table></td></tr><tr><th>hendrycksTest-machine_learning</th><td><table><tr><th>acc</th><td>0.2857142857142857</td></tr><tr><th>acc_stderr</th><td>0.04287858751340456</td></tr><tr><th>acc_norm</th><td>0.2857142857142857</td></tr><tr><th>acc_norm_stderr</th><td>0.04287858751340456</td></tr></table></td></tr><tr><th>hendrycksTest-management</th><td><table><tr><th>acc</th><td>0.27184466019417475</td></tr><tr><th>acc_stderr</th><td>0.044052680241409216</td></tr><tr><th>acc_norm</th><td>0.27184466019417475</td></tr><tr><th>acc_norm_stderr</th><td>0.044052680241409216</td></tr></table></td></tr><tr><th>hendrycksTest-marketing</th><td><table><tr><th>acc</th><td>0.27350427350427353</td></tr><tr><th>acc_stderr</th><td>0.02920254015343117</td></tr><tr><th>acc_norm</th><td>0.27350427350427353</td></tr><tr><th>acc_norm_stderr</th><td>0.02920254015343117</td></tr></table></td></tr><tr><th>hendrycksTest-medical_genetics</th><td><table><tr><th>acc</th><td>0.23</td></tr><tr><th>acc_stderr</th><td>0.04229525846816505</td></tr><tr><th>acc_norm</th><td>0.23</td></tr><tr><th>acc_norm_stderr</th><td>0.04229525846816505</td></tr></table></td></tr><tr><th>hendrycksTest-miscellaneous</th><td><table><tr><th>acc</th><td>0.2937420178799489</td></tr><tr><th>acc_stderr</th><td>0.016287759388491672</td></tr><tr><th>acc_norm</th><td>0.2937420178799489</td></tr><tr><th>acc_norm_stderr</th><td>0.016287759388491672</td></tr></table></td></tr><tr><th>hendrycksTest-moral_disputes</th><td><table><tr><th>acc</th><td>0.2774566473988439</td></tr><tr><th>acc_stderr</th><td>0.024105712607754307</td></tr><tr><th>acc_norm</th><td>0.2774566473988439</td></tr><tr><th>acc_norm_stderr</th><td>0.024105712607754307</td></tr></table></td></tr><tr><th>hendrycksTest-moral_scenarios</th><td><table><tr><th>acc</th><td>0.24916201117318434</td></tr><tr><th>acc_stderr</th><td>0.014465893829859933</td></tr><tr><th>acc_norm</th><td>0.24916201117318434</td></tr><tr><th>acc_norm_stderr</th><td>0.014465893829859933</td></tr></table></td></tr><tr><th>hendrycksTest-nutrition</th><td><table><tr><th>acc</th><td>0.24836601307189543</td></tr><tr><th>acc_stderr</th><td>0.024739981355113596</td></tr><tr><th>acc_norm</th><td>0.24836601307189543</td></tr><tr><th>acc_norm_stderr</th><td>0.024739981355113596</td></tr></table></td></tr><tr><th>hendrycksTest-philosophy</th><td><table><tr><th>acc</th><td>0.2733118971061093</td></tr><tr><th>acc_stderr</th><td>0.02531176597542612</td></tr><tr><th>acc_norm</th><td>0.2733118971061093</td></tr><tr><th>acc_norm_stderr</th><td>0.02531176597542612</td></tr></table></td></tr><tr><th>hendrycksTest-prehistory</th><td><table><tr><th>acc</th><td>0.2623456790123457</td></tr><tr><th>acc_stderr</th><td>0.024477222856135107</td></tr><tr><th>acc_norm</th><td>0.2623456790123457</td></tr><tr><th>acc_norm_stderr</th><td>0.024477222856135107</td></tr></table></td></tr><tr><th>hendrycksTest-professional_accounting</th><td><table><tr><th>acc</th><td>0.2801418439716312</td></tr><tr><th>acc_stderr</th><td>0.026789172351140228</td></tr><tr><th>acc_norm</th><td>0.2801418439716312</td></tr><tr><th>acc_norm_stderr</th><td>0.026789172351140228</td></tr></table></td></tr><tr><th>hendrycksTest-professional_law</th><td><table><tr><th>acc</th><td>0.24185136897001303</td></tr><tr><th>acc_stderr</th><td>0.010936550813827065</td></tr><tr><th>acc_norm</th><td>0.24185136897001303</td></tr><tr><th>acc_norm_stderr</th><td>0.010936550813827065</td></tr></table></td></tr><tr><th>hendrycksTest-professional_medicine</th><td><table><tr><th>acc</th><td>0.20220588235294118</td></tr><tr><th>acc_stderr</th><td>0.02439819298665492</td></tr><tr><th>acc_norm</th><td>0.20220588235294118</td></tr><tr><th>acc_norm_stderr</th><td>0.02439819298665492</td></tr></table></td></tr><tr><th>hendrycksTest-professional_psychology</th><td><table><tr><th>acc</th><td>0.25980392156862747</td></tr><tr><th>acc_stderr</th><td>0.017740899509177795</td></tr><tr><th>acc_norm</th><td>0.25980392156862747</td></tr><tr><th>acc_norm_stderr</th><td>0.017740899509177795</td></tr></table></td></tr><tr><th>hendrycksTest-public_relations</th><td><table><tr><th>acc</th><td>0.34545454545454546</td></tr><tr><th>acc_stderr</th><td>0.04554619617541054</td></tr><tr><th>acc_norm</th><td>0.34545454545454546</td></tr><tr><th>acc_norm_stderr</th><td>0.04554619617541054</td></tr></table></td></tr><tr><th>hendrycksTest-security_studies</th><td><table><tr><th>acc</th><td>0.23673469387755103</td></tr><tr><th>acc_stderr</th><td>0.02721283588407316</td></tr><tr><th>acc_norm</th><td>0.23673469387755103</td></tr><tr><th>acc_norm_stderr</th><td>0.02721283588407316</td></tr></table></td></tr><tr><th>hendrycksTest-sociology</th><td><table><tr><th>acc</th><td>0.23880597014925373</td></tr><tr><th>acc_stderr</th><td>0.030147775935409224</td></tr><tr><th>acc_norm</th><td>0.23880597014925373</td></tr><tr><th>acc_norm_stderr</th><td>0.030147775935409224</td></tr></table></td></tr><tr><th>hendrycksTest-us_foreign_policy</th><td><table><tr><th>acc</th><td>0.29</td></tr><tr><th>acc_stderr</th><td>0.04560480215720684</td></tr><tr><th>acc_norm</th><td>0.29</td></tr><tr><th>acc_norm_stderr</th><td>0.04560480215720684</td></tr></table></td></tr><tr><th>hendrycksTest-virology</th><td><table><tr><th>acc</th><td>0.3132530120481928</td></tr><tr><th>acc_stderr</th><td>0.036108050180310235</td></tr><tr><th>acc_norm</th><td>0.3132530120481928</td></tr><tr><th>acc_norm_stderr</th><td>0.036108050180310235</td></tr></table></td></tr><tr><th>hendrycksTest-world_religions</th><td><table><tr><th>acc</th><td>0.34502923976608185</td></tr><tr><th>acc_stderr</th><td>0.036459813773888065</td></tr><tr><th>acc_norm</th><td>0.34502923976608185</td></tr><tr><th>acc_norm_stderr</th><td>0.036459813773888065</td></tr></table></td></tr><tr><th>truthfulqa_mc</th><td><table><tr><th>mc1</th><td>0.23990208078335373</td></tr><tr><th>mc1_stderr</th><td>0.014948812679062133</td></tr><tr><th>mc2</th><td>0.38208865449224466</td></tr><tr><th>mc2_stderr</th><td>0.013866257677161405</td></tr></table></td></tr><tr><th>arc_challenge</th><td><table><tr><th>acc</th><td>0.3856655290102389</td></tr><tr><th>acc_stderr</th><td>0.014224250973257177</td></tr><tr><th>acc_norm</th><td>0.41723549488054607</td></tr><tr><th>acc_norm_stderr</th><td>0.014409825518403079</td></tr></table></td></tr></table>
|
|
|
|
| 23 |
GGML quants available [here](https://huggingface.co/TheBloke/Griffin-3B-GGML).</br>
|
| 24 |
GPTQ quants available [here](https://huggingface.co/TheBloke/Griffin-3B-GPTQ).
|
| 25 |
|
| 26 |
+
Note: Don't expect this model to be good, I was just starting out to finetune. So don't roast me please!
|
|
|
|
|
|