Commit
•
67ac45f
1
Parent(s):
e2ba6c7
commit files to HF hub
Browse files
README.md
CHANGED
@@ -135,6 +135,27 @@ GPTNeoXConfig {
|
|
135 |
|
136 |
```
|
137 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
138 |
## Disclaimer
|
139 |
|
140 |
Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.
|
|
|
135 |
|
136 |
```
|
137 |
|
138 |
+
## Model Validation
|
139 |
+
|
140 |
+
Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) out of the box:
|
141 |
+
|
142 |
+
|
143 |
+
| Task |Version| Metric |Value | |Stderr|
|
144 |
+
|-------------|------:|--------|-----:|---|-----:|
|
145 |
+
|arc_easy | 0|acc |0.6932|± |0.0095|
|
146 |
+
| | |acc_norm|0.6225|± |0.0099|
|
147 |
+
|openbookqa | 0|acc |0.2900|± |0.0203|
|
148 |
+
| | |acc_norm|0.3740|± |0.0217|
|
149 |
+
|winogrande | 0|acc |0.6369|± |0.0135|
|
150 |
+
|hellaswag | 0|acc |0.5140|± |0.0050|
|
151 |
+
| | |acc_norm|0.6803|± |0.0047|
|
152 |
+
|piqa | 0|acc |0.7682|± |0.0098|
|
153 |
+
| | |acc_norm|0.7661|± |0.0099|
|
154 |
+
|boolq | 1|acc |0.6685|± |0.0082|
|
155 |
+
|arc_challenge| 0|acc |0.3157|± |0.0136|
|
156 |
+
| | |acc_norm|0.3507|± |0.0139|
|
157 |
+
|
158 |
+
|
159 |
## Disclaimer
|
160 |
|
161 |
Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.
|