winglian commited on
Commit
32fd5c7
1 Parent(s): 3158bdb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -20,20 +20,20 @@ hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_
20
 
21
  ### 16x Passees
22
  ```
23
- hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 32
24
  | Task |Version| Metric |Value | |Stderr|
25
  |-------------|------:|--------|-----:|---|-----:|
26
- |arc_challenge| 0|acc |0.2287|_ |0.0123|
27
- | | |acc_norm|0.2654|_ |0.0129|
28
- |arc_easy | 0|acc |0.4785|_ |0.0103|
29
- | | |acc_norm|0.4390|_ |0.0102|
30
- |boolq | 1|acc |0.6076|_ |0.0085|
31
- |hellaswag | 0|acc |0.3789|_ |0.0048|
32
- | | |acc_norm|0.4877|_ |0.0050|
33
- |openbookqa | 0|acc |0.1760|_ |0.0170|
34
  | | |acc_norm|0.2800|_ |0.0201|
35
- |piqa | 0|acc |0.6572|_ |0.0111|
36
- | | |acc_norm|0.6589|_ |0.0111|
37
- |winogrande | 0|acc |0.5541|_ |0.0140|
38
  ```
39
 
 
20
 
21
  ### 16x Passees
22
  ```
23
+ hf-causal-experimental (pretrained=openaccess-ai-collective/mighty-llama-1b,use_accelerate=True,dtype=bfloat16,trust_remote_code=True), limit: None, provide_description: False, num_fewshot: 0, batch_size: 64
24
  | Task |Version| Metric |Value | |Stderr|
25
  |-------------|------:|--------|-----:|---|-----:|
26
+ |arc_challenge| 0|acc |0.2466|_ |0.0126|
27
+ | | |acc_norm|0.2824|_ |0.0132|
28
+ |arc_easy | 0|acc |0.3649|_ |0.0099|
29
+ | | |acc_norm|0.3582|_ |0.0098|
30
+ |boolq | 1|acc |0.6214|_ |0.0085|
31
+ |hellaswag | 0|acc |0.3085|_ |0.0046|
32
+ | | |acc_norm|0.3614|_ |0.0048|
33
+ |openbookqa | 0|acc |0.1900|_ |0.0176|
34
  | | |acc_norm|0.2800|_ |0.0201|
35
+ |piqa | 0|acc |0.5702|_ |0.0116|
36
+ | | |acc_norm|0.5729|_ |0.0115|
37
+ |winogrande | 0|acc |0.5399|_ |0.0140|
38
  ```
39