Create README.md
Browse files
README.md
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
```hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 32
|
| 2 |
+
| Task |Version| Metric |Value | |Stderr|
|
| 3 |
+
|-------------|------:|--------|-----:|---|-----:|
|
| 4 |
+
|arc_challenge| 0|acc |0.2355|_ |0.0124|
|
| 5 |
+
| | |acc_norm|0.2671|_ |0.0129|
|
| 6 |
+
|arc_easy | 0|acc |0.4444|_ |0.0102|
|
| 7 |
+
| | |acc_norm|0.4276|_ |0.0102|
|
| 8 |
+
|boolq | 1|acc |0.5358|_ |0.0087|
|
| 9 |
+
|hellaswag | 0|acc |0.3784|_ |0.0048|
|
| 10 |
+
| | |acc_norm|0.5034|_ |0.0050|
|
| 11 |
+
|openbookqa | 0|acc |0.1580|_ |0.0163|
|
| 12 |
+
| | |acc_norm|0.2840|_ |0.0202|
|
| 13 |
+
|piqa | 0|acc |0.6518|_ |0.0111|
|
| 14 |
+
| | |acc_norm|0.6464|_ |0.0112|
|
| 15 |
+
|winogrande | 0|acc |0.5422|_ |0.0140|
|
| 16 |
+
```
|