ruixie commited on
Commit
0d0570c
1 Parent(s): b394858

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +13 -22
README.md CHANGED
@@ -87,29 +87,20 @@ Code Shell uses GPT-2 as its foundational architecture and incorporates technolo
87
  We selected the two most popular code evaluation datasets currently available (HumanEval and MBPP) to assess the model. Compared to the two most advanced 7b LLM for code, CodeLllama and Starcoder, Codeshell achieved the best results. The specific evaluation results are as follows.
88
 
89
  ### Pass@1
90
- | Task | codeshell | codellama-7B | starcoderbase-7B |
91
  | ------- | --------- | --------- | --------- |
92
- | humaneval | 33.48 | 29.44 | 27.80 |
93
- | mbpp | 39.08 | 37.60 | 34.16 |
94
- | multiple-java | 29.56 | 29.24 | 24.30 |
95
- | multiple-js | 33.60 | 31.30 | 27.02 |
96
- | multiple-r | 20.99 | 18.57 | 14.29 |
97
- | multiple-rkt | 12.48 | 12.55 | 10.43 |
98
- | multiple-cpp | 28.20 | 27.33 | 23.04 |
99
- | multiple-cs | 22.34 | 20.38 | 18.99 |
100
- | multiple-d | 8.59 | 11.60 | 8.08 |
101
- | multiple-go | 71.69 | 75.91 | 73.83 |
102
- | multiple-jl | 20.63 | 25.28 | 22.96 |
103
- | multiple-lua | 22.92 | 30.50 | 22.92 |
104
- | multiple-php | 30.43 | 25.96 | 22.11 |
105
- | multiple-pl | 15.65 | 17.45 | 16.40 |
106
- | multiple-py | 33.54 | 29.25 | 28.82 |
107
- | multiple-rb | 25.71 | 30.06 | 18.51 |
108
- | multiple-rs | 26.86 | 25.90 | 22.82 |
109
- | multiple-swift | 25.00 | 25.32 | 15.70 |
110
- | multiple-ts | 33.90 | 32.64 | 27.48 |
111
- | multiple-sh | 8.42 | 9.75 | 7.09 |
112
- | multiple-scala | 22.56 | 24.50 | 19.12 |
113
 
114
  # License
115
 
 
87
  We selected the two most popular code evaluation datasets currently available (HumanEval and MBPP) to assess the model. Compared to the two most advanced 7b LLM for code, CodeLllama and Starcoder, Codeshell achieved the best results. The specific evaluation results are as follows.
88
 
89
  ### Pass@1
90
+ | 任务 | codeshell | codellama | starcoder7b |
91
  | ------- | --------- | --------- | --------- |
92
+ | humaneval | **34.32** | 29.44 | 27.80 |
93
+ | multiple-js | **33.17** | 31.30 | 27.02 |
94
+ | multiple-java | **30.43** | 29.24 | 24.30 |
95
+ | multiple-cpp | **28.21** | 27.33 | 23.04 |
96
+ | multiple-swift | 24.30 | **25.32** | 15.70 |
97
+ | multiple-php | **30.87** | 25.96 | 22.11 |
98
+ | multiple-d | 8.85 | **11.60** | 8.08 |
99
+ | multiple-jl | 22.08 | **25.28** | 22.96 |
100
+ | multiple-lua | 22.39 | **30.50** | 22.92 |
101
+ | multiple-r | **20.52** | 18.57 | 14.29 |
102
+ | multiple-rkt | **17.20** | 12.55 | 10.43 |
103
+ | multiple-rs | 24.55 | **25.90** | 22.82 |
 
 
 
 
 
 
 
 
 
104
 
105
  # License
106