Update README.md
Browse files
README.md
CHANGED
@@ -31,4 +31,69 @@ Installation instruction:
|
|
31 |
Visit: https://github.com/OpenNMT/OpenNMT-py
|
32 |
make sure you install flash-attn and autoawq
|
33 |
|
34 |
-
Enjoy
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
31 |
Visit: https://github.com/OpenNMT/OpenNMT-py
|
32 |
make sure you install flash-attn and autoawq
|
33 |
|
34 |
+
Enjoy
|
35 |
+
|
36 |
+
detailed MMLU scoring:
|
37 |
+
```
|
38 |
+
ACC-abstract_algebra: 0.3600
|
39 |
+
ACC-anatomy: 0.6444
|
40 |
+
ACC-astronomy: 0.7303
|
41 |
+
ACC-business_ethics: 0.6400
|
42 |
+
ACC-clinical_knowledge: 0.7283
|
43 |
+
ACC-college_biology: 0.8056
|
44 |
+
ACC-college_chemistry: 0.5300
|
45 |
+
ACC-college_computer_science: 0.5900
|
46 |
+
ACC-college_mathematics: 0.3700
|
47 |
+
ACC-college_medicine: 0.6936
|
48 |
+
ACC-college_physics: 0.4510
|
49 |
+
ACC-computer_security: 0.7900
|
50 |
+
ACC-conceptual_physics: 0.6468
|
51 |
+
ACC-econometrics: 0.5614
|
52 |
+
ACC-electrical_engineering: 0.6414
|
53 |
+
ACC-elementary_mathematics: 0.4630
|
54 |
+
ACC-formal_logic: 0.4524
|
55 |
+
ACC-global_facts: 0.4600
|
56 |
+
ACC-high_school_biology: 0.8000
|
57 |
+
ACC-high_school_chemistry: 0.5320
|
58 |
+
ACC-high_school_computer_science: 0.7400
|
59 |
+
ACC-high_school_european_history: 0.8121
|
60 |
+
ACC-high_school_geography: 0.8081
|
61 |
+
ACC-high_school_government_and_politics: 0.9275
|
62 |
+
ACC-high_school_macroeconomics: 0.6923
|
63 |
+
ACC-high_school_mathematics: 0.3667
|
64 |
+
ACC-high_school_microeconomics: 0.7731
|
65 |
+
ACC-high_school_physics: 0.4636
|
66 |
+
ACC-high_school_psychology: 0.8569
|
67 |
+
ACC-high_school_statistics: 0.5278
|
68 |
+
ACC-high_school_us_history: 0.8431
|
69 |
+
ACC-high_school_world_history: 0.8650
|
70 |
+
ACC-human_aging: 0.7175
|
71 |
+
ACC-human_sexuality: 0.7710
|
72 |
+
ACC-international_law: 0.8347
|
73 |
+
ACC-jurisprudence: 0.7778
|
74 |
+
ACC-logical_fallacies: 0.7791
|
75 |
+
ACC-machine_learning: 0.5357
|
76 |
+
ACC-management: 0.7767
|
77 |
+
ACC-marketing: 0.9145
|
78 |
+
ACC-medical_genetics: 0.7100
|
79 |
+
ACC-miscellaneous: 0.8404
|
80 |
+
ACC-moral_disputes: 0.7775
|
81 |
+
ACC-moral_scenarios: 0.4112
|
82 |
+
ACC-nutrition: 0.7876
|
83 |
+
ACC-philosophy: 0.7492
|
84 |
+
ACC-prehistory: 0.7963
|
85 |
+
ACC-professional_accounting: 0.5177
|
86 |
+
ACC-professional_law: 0.5111
|
87 |
+
ACC-professional_medicine: 0.7390
|
88 |
+
ACC-professional_psychology: 0.7304
|
89 |
+
ACC-public_relations: 0.6727
|
90 |
+
ACC-security_studies: 0.7061
|
91 |
+
ACC-sociology: 0.8706
|
92 |
+
ACC-us_foreign_policy: 0.9100
|
93 |
+
ACC-virology: 0.5060
|
94 |
+
ACC-world_religions: 0.8538
|
95 |
+
ACC-all: 0.6707
|
96 |
+
[2023-12-22 16:35:03,999 INFO] total run time 7156.16
|
97 |
+
|
98 |
+
```
|
99 |
+
|