Commit
•
67ac45f
1
Parent(s):
e2ba6c7
commit files to HF hub
Browse files
README.md
CHANGED
|
@@ -135,6 +135,27 @@ GPTNeoXConfig {
|
|
| 135 |
|
| 136 |
```
|
| 137 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 138 |
## Disclaimer
|
| 139 |
|
| 140 |
Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.
|
|
|
|
| 135 |
|
| 136 |
```
|
| 137 |
|
| 138 |
+
## Model Validation
|
| 139 |
+
|
| 140 |
+
Model validation results using [EleutherAI lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) out of the box:
|
| 141 |
+
|
| 142 |
+
|
| 143 |
+
| Task |Version| Metric |Value | |Stderr|
|
| 144 |
+
|-------------|------:|--------|-----:|---|-----:|
|
| 145 |
+
|arc_easy | 0|acc |0.6932|± |0.0095|
|
| 146 |
+
| | |acc_norm|0.6225|± |0.0099|
|
| 147 |
+
|openbookqa | 0|acc |0.2900|± |0.0203|
|
| 148 |
+
| | |acc_norm|0.3740|± |0.0217|
|
| 149 |
+
|winogrande | 0|acc |0.6369|± |0.0135|
|
| 150 |
+
|hellaswag | 0|acc |0.5140|± |0.0050|
|
| 151 |
+
| | |acc_norm|0.6803|± |0.0047|
|
| 152 |
+
|piqa | 0|acc |0.7682|± |0.0098|
|
| 153 |
+
| | |acc_norm|0.7661|± |0.0099|
|
| 154 |
+
|boolq | 1|acc |0.6685|± |0.0082|
|
| 155 |
+
|arc_challenge| 0|acc |0.3157|± |0.0136|
|
| 156 |
+
| | |acc_norm|0.3507|± |0.0139|
|
| 157 |
+
|
| 158 |
+
|
| 159 |
## Disclaimer
|
| 160 |
|
| 161 |
Please read this disclaimer carefully before using the large language model provided in this repository. Your use of the model signifies your agreement to the following terms and conditions.
|