winglian commited on
Commit
cc30631
1 Parent(s): c7e344a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -1
README.md CHANGED
@@ -1,3 +1,4 @@
 
1
  ```
2
  hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 32
3
  | Task |Version| Metric |Value | |Stderr|
@@ -14,4 +15,25 @@ hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_
14
  |piqa | 0|acc |0.6518|_ |0.0111|
15
  | | |acc_norm|0.6464|_ |0.0112|
16
  |winogrande | 0|acc |0.5422|_ |0.0140|
17
- ```
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ### Single Pass
2
  ```
3
  hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 32
4
  | Task |Version| Metric |Value | |Stderr|
 
15
  |piqa | 0|acc |0.6518|_ |0.0111|
16
  | | |acc_norm|0.6464|_ |0.0112|
17
  |winogrande | 0|acc |0.5422|_ |0.0140|
18
+ ```
19
+
20
+
21
+ ### 16x Passees
22
+ ```
23
+ hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 32
24
+ | Task |Version| Metric |Value | |Stderr|
25
+ |-------------|------:|--------|-----:|---|-----:|
26
+ |arc_challenge| 0|acc |0.2287|_ |0.0123|
27
+ | | |acc_norm|0.2654|_ |0.0129|
28
+ |arc_easy | 0|acc |0.4785|_ |0.0103|
29
+ | | |acc_norm|0.4390|_ |0.0102|
30
+ |boolq | 1|acc |0.6076|_ |0.0085|
31
+ |hellaswag | 0|acc |0.3789|_ |0.0048|
32
+ | | |acc_norm|0.4877|_ |0.0050|
33
+ |openbookqa | 0|acc |0.1760|_ |0.0170|
34
+ | | |acc_norm|0.2800|_ |0.0201|
35
+ |piqa | 0|acc |0.6572|_ |0.0111|
36
+ | | |acc_norm|0.6589|_ |0.0111|
37
+ |winogrande | 0|acc |0.5541|_ |0.0140|
38
+ ```
39
+