loubnabnl HF staff lvwerra HF staff commited on
Commit
4e1a8cf
1 Parent(s): 421d645

Update README.md (#1)

Browse files

- Update README.md (045118a53b7c5f9b375bf15aac797e3eb4c84feb)
- add humaneval (adbc2c4071e9942baa55c27738854ca2f5fbe851)


Co-authored-by: Leandro von Werra <lvwerra@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +51 -0
README.md CHANGED
@@ -11,6 +11,57 @@ license: bigcode-openrail-m
11
  library_name: transformers
12
  tags:
13
  - code
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
14
  ---
15
 
16
  # StarCoder2
 
11
  library_name: transformers
12
  tags:
13
  - code
14
+ model-index:
15
+ - name: starcoder2-3b
16
+ results:
17
+ - task:
18
+ type: text-generation
19
+ dataset:
20
+ name: CruxEval-I
21
+ type: cruxeval-i
22
+ metrics:
23
+ - type: pass@1
24
+ value: 32.7
25
+ - task:
26
+ type: text-generation
27
+ dataset:
28
+ name: DS-1000
29
+ type: ds-1000
30
+ metrics:
31
+ - type: pass@1
32
+ value: 25.0
33
+ - task:
34
+ type: text-generation
35
+ dataset:
36
+ name: GSM8K (PAL)
37
+ type: gsm8k-pal
38
+ metrics:
39
+ - type: accuracy
40
+ value: 27.7
41
+ - task:
42
+ type: text-generation
43
+ dataset:
44
+ name: HumanEval+
45
+ type: humanevalplus
46
+ metrics:
47
+ - type: pass@1
48
+ value: 27.4
49
+ - task:
50
+ type: text-generation
51
+ dataset:
52
+ name: HumanEval
53
+ type: humaneval
54
+ metrics:
55
+ - type: pass@1
56
+ value: 31.7
57
+ - task:
58
+ type: text-generation
59
+ dataset:
60
+ name: RepoBench-v1.1
61
+ type: repobench-v1.1
62
+ metrics:
63
+ - type: edit-smiliarity
64
+ value: 71.19
65
  ---
66
 
67
  # StarCoder2