Update README.md
Browse files
README.md
CHANGED
|
@@ -45,14 +45,6 @@ This is the repository for the base 13B version finetuned based on [CodeLlama-13
|
|
| 45 |
| phi-2 | [opencsg/Opencsg-phi-2-v0.1](https://huggingface.co/opencsg/opencsg-phi-2-v0.1) |
|
| 46 |
|
| 47 |
|
| 48 |
-
| 7B | [opencsg/Opencsg-CodeLlama-7b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-7b-v0.1) |
|
| 49 |
-
| 13B | [opencsg/Opencsg-CodeLlama-13b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-13b-v0.1) |
|
| 50 |
-
| 34B | [opencsg/Opencsg-CodeLlama-34b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-34b-v0.1) |
|
| 51 |
-
| 7B | [opencsg/Opencsg-CodeLlama-7b-v0.2](https://huggingface.co/opencsg/opencsg-CodeLlama-7b-v0.2) |
|
| 52 |
-
| 13B | [opencsg/Opencsg-CodeLlama-13b-v0.2](https://huggingface.co/opencsg/opencsg-CodeLlama-13b-v0.2) |
|
| 53 |
-
| 34B | [opencsg/Opencsg-CodeLlama-34b-v0.2](https://huggingface.co/opencsg/opencsg-CodeLlama-34b-v0.2) |
|
| 54 |
-
|
| 55 |
-
|
| 56 |
|
| 57 |
## Model Eval
|
| 58 |
|
|
@@ -74,21 +66,6 @@ To simplify the comparison, we chosed the Pass@1 metric for the Python language,
|
|
| 74 |
| **opencsg-phi-2-v0.1** |**54.3**|
|
| 75 |
|
| 76 |
|
| 77 |
-
| CodeLlama-7b-hf | 30.5%|
|
| 78 |
-
| **opencsg-CodeLlama-7b-v0.1** | **43.9%** |
|
| 79 |
-
| **opencsg-CodeLlama-7b-v0.2** | **50.0%** |
|
| 80 |
-
| CodeLlama-13b-hf | 36.0%|
|
| 81 |
-
| **opencsg-CodeLlama-13b-v0.1** | **51.2%** |
|
| 82 |
-
| **opencsg-CodeLlama-13b-v0.2** | **55.5%** |
|
| 83 |
-
| CodeLlama-34b-hf | 48.2%|
|
| 84 |
-
| **opencsg-CodeLlama-34b-v0.1**| **56.1%** |
|
| 85 |
-
| **opencsg-CodeLlama-34b-v0.2**| **64.0%** |
|
| 86 |
-
| CodeLlama-70b-hf| 53.0% |
|
| 87 |
-
| CodeLlama-70b-Instruct-hf| **67.8%** |
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
| 91 |
-
|
| 92 |
|
| 93 |
|
| 94 |
|
|
@@ -171,12 +148,7 @@ opencsg-phi-2-v0.1是是一系列基于phi-2的通过全参数微调方法进行
|
|
| 171 |
|
| 172 |
| 模型大小 | 基座模型 |
|
| 173 |
| --- | ----------------------------------------------------------------------------- |
|
| 174 |
-
|
|
| 175 |
-
| 13B | [opencsg/Opencsg-CodeLlama-13b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-13b-v0.1) |
|
| 176 |
-
| 34B | [opencsg/Opencsg-CodeLlama-34b-v0.1](https://huggingface.co/opencsg/opencsg-CodeLlama-34b-v0.1) |
|
| 177 |
-
| 7B | [opencsg/Opencsg-CodeLlama-7b-v0.2](https://huggingface.co/opencsg/opencsg-CodeLlama-7b-v0.2) |
|
| 178 |
-
| 13B | [opencsg/Opencsg-CodeLlama-13b-v0.2](https://huggingface.co/opencsg/opencsg-CodeLlama-13b-v0.2) |
|
| 179 |
-
| 34B | [opencsg/Opencsg-CodeLlama-34b-v0.2](https://huggingface.co/opencsg/opencsg-CodeLlama-34b-v0.2) |
|
| 180 |
|
| 181 |
|
| 182 |
## 模型评估
|
|
@@ -199,17 +171,7 @@ HumanEval 是评估模型在代码生成方面性能的最常见的基准，尤
|
|
| 199 |
| **opencsg-phi-2-v0.1** |**54.3**|
|
| 200 |
|
| 201 |
|
| 202 |
-
|
| 203 |
-
| **opencsg-CodeLlama-7b-v0.1** | **43.9%** |
|
| 204 |
-
| **opencsg-CodeLlama-7b-v0.2** | **50.0%** |
|
| 205 |
-
| CodeLlama-13b-hf | 36.0%|
|
| 206 |
-
| **opencsg-CodeLlama-13b-v0.1** | **51.2%** |
|
| 207 |
-
| **opencsg-CodeLlama-13b-v0.2** | **55.5%** |
|
| 208 |
-
| CodeLlama-34b-hf | 48.2%|
|
| 209 |
-
| **opencsg-CodeLlama-34b-v0.1**| **56.1%** |
|
| 210 |
-
| **opencsg-CodeLlama-34b-v0.2**| **64.0%** |
|
| 211 |
-
| CodeLlama-70b-hf| 53.0% |
|
| 212 |
-
| CodeLlama-70b-Instruct-hf| **67.8%** |
|
| 213 |
|
| 214 |
**TODO**
|
| 215 |
- 未来我们将提供更多微调模型的在各基准上的分数。
|
|
|
|
| 45 |
| phi-2 | [opencsg/Opencsg-phi-2-v0.1](https://huggingface.co/opencsg/opencsg-phi-2-v0.1) |
|
| 46 |
|
| 47 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
## Model Eval
|
| 50 |
|
|
|
|
| 66 |
| **opencsg-phi-2-v0.1** |**54.3**|
|
| 67 |
|
| 68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
|
| 70 |
|
| 71 |
|
|
|
|
| 148 |
|
| 149 |
| 模型大小 | 基座模型 |
|
| 150 |
| --- | ----------------------------------------------------------------------------- |
|
| 151 |
+
| phi-2 | [opencsg/Opencsg-phi-2-v0.1](https://huggingface.co/opencsg/opencsg-phi-2-v0.1) |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 152 |
|
| 153 |
|
| 154 |
## 模型评估
|
|
|
|
| 171 |
| **opencsg-phi-2-v0.1** |**54.3**|
|
| 172 |
|
| 173 |
|
| 174 |
+
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 175 |
|
| 176 |
**TODO**
|
| 177 |
- 未来我们将提供更多微调模型的在各基准上的分数。
|