Update README.md
Browse files
README.md
CHANGED
|
@@ -25,6 +25,9 @@ model-index:
|
|
| 25 |
- name: pass@10
|
| 26 |
type: code_eval_outputs
|
| 27 |
value: 15
|
|
|
|
|
|
|
|
|
|
| 28 |
---
|
| 29 |
|
| 30 |
# CodeGen-350M-multi-xlcost-v2
|
|
@@ -135,6 +138,7 @@ We evaluated the model on the first 400 samples of XLCOST's [XLCost-single-promp
|
|
| 135 |
|--------|-----|-----|-----|-----|-----|
|
| 136 |
|pass@1 |3.325% |3.70% | 0.4% | 0.35% | 0.48% |
|
| 137 |
|pass@10 |15%| 14.5% | 3.5% | 3 % | 3.75% |
|
|
|
|
| 138 |
|
| 139 |
The [pass@k metric](https://huggingface.co/metrics/code_eval) tells the probability that at least one out of k generations passes the tests.
|
| 140 |
|
| 25 |
- name: pass@10
|
| 26 |
type: code_eval_outputs
|
| 27 |
value: 15
|
| 28 |
+
- name: codebleu
|
| 29 |
+
type: codebleu
|
| 30 |
+
value: 20.18191
|
| 31 |
---
|
| 32 |
|
| 33 |
# CodeGen-350M-multi-xlcost-v2
|
| 138 |
|--------|-----|-----|-----|-----|-----|
|
| 139 |
|pass@1 |3.325% |3.70% | 0.4% | 0.35% | 0.48% |
|
| 140 |
|pass@10 |15%| 14.5% | 3.5% | 3 % | 3.75% |
|
| 141 |
+
|CodeBLEU |20.18%| None | 15.15% | 19.42 % | 20.27% |
|
| 142 |
|
| 143 |
The [pass@k metric](https://huggingface.co/metrics/code_eval) tells the probability that at least one out of k generations passes the tests.
|
| 144 |
|