loubnabnl HF staff lvwerra HF staff commited on
Commit
1c2dae0
1 Parent(s): 56438be

Update README.md (#2)

Browse files

- Update README.md (b2cd8079742a780faddfad707b3ebaa05feb39de)
- add humaneval (9a65c8fd8cbdac3ef66884458098127f54f52408)


Co-authored-by: Leandro von Werra <lvwerra@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +51 -0
README.md CHANGED
@@ -11,6 +11,57 @@ license: bigcode-openrail-m
11
  library_name: transformers
12
  tags:
13
  - code
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ---
15
 
16
  # StarCoder2
 
11
  library_name: transformers
12
  tags:
13
  - code
14
+ model-index:
15
+ - name: starcoder2-15b
16
+ results:
17
+ - task:
18
+ type: text-generation
19
+ dataset:
20
+ name: CruxEval-I
21
+ type: cruxeval-i
22
+ metrics:
23
+ - type: pass@1
24
+ value: 48.1
25
+ - task:
26
+ type: text-generation
27
+ dataset:
28
+ name: DS-1000
29
+ type: ds-1000
30
+ metrics:
31
+ - type: pass@1
32
+ value: 33.8
33
+ - task:
34
+ type: text-generation
35
+ dataset:
36
+ name: GSM8K (PAL)
37
+ type: gsm8k-pal
38
+ metrics:
39
+ - type: accuracy
40
+ value: 65.1
41
+ - task:
42
+ type: text-generation
43
+ dataset:
44
+ name: HumanEval+
45
+ type: humanevalplus
46
+ metrics:
47
+ - type: pass@1
48
+ value: 37.8
49
+ - task:
50
+ type: text-generation
51
+ dataset:
52
+ name: HumanEval
53
+ type: humaneval
54
+ metrics:
55
+ - type: pass@1
56
+ value: 46.3
57
+ - task:
58
+ type: text-generation
59
+ dataset:
60
+ name: RepoBench-v1.1
61
+ type: repobench-v1.1
62
+ metrics:
63
+ - type: edit-smiliarity
64
+ value: 74.08
65
  ---
66
 
67
  # StarCoder2