Mehdi Challakh
commited on
updated with gpt4-o evaluation entries
Browse files
README.md
CHANGED
|
@@ -101,15 +101,15 @@ print(answer)
|
|
| 101 |
|
| 102 |
## Performance
|
| 103 |
|
| 104 |
-
| Metric | stable-code-instruct-3b | stable-cypher-instruct-3b |
|
| 105 |
-
| ------------ | ----------------------- | ------------------------- |
|
| 106 |
-
| BLEU-4 | 19.07 | 88.63 |
|
| 107 |
-
| ROUGE-1 | 39.49 | 95.09 |
|
| 108 |
-
| ROUGE-2 | 24.82 | 90.71 |
|
| 109 |
-
| ROUGE-L | 29.63 | 91.51 |
|
| 110 |
-
| Jaro-Winkler | 52.21
|
| 111 |
-
| Jaccard | 25.55
|
| 112 |
-
| Pass@1 | 0.0
|
| 113 |
|
| 114 |
### Example
|
| 115 |
|
|
|
|
| 101 |
|
| 102 |
## Performance
|
| 103 |
|
| 104 |
+
| Metric | stable-code-instruct-3b | gpt4-o | stable-cypher-instruct-3b |
|
| 105 |
+
| ------------ | ----------------------- | ------ | ------------------------- |
|
| 106 |
+
| BLEU-4 | 19.07 | 32.35 | 88.63 |
|
| 107 |
+
| ROUGE-1 | 39.49 | 69.17 | 95.09 |
|
| 108 |
+
| ROUGE-2 | 24.82 | 46.97 | 90.71 |
|
| 109 |
+
| ROUGE-L | 29.63 | 65.24 | 91.51 |
|
| 110 |
+
| Jaro-Winkler | 52.21 | 86.38 | 95.69 |
|
| 111 |
+
| Jaccard | 25.55 | 72.80 | 90.78 |
|
| 112 |
+
| Pass@1 | 0.00 | 0.00 | 51.80 |
|
| 113 |
|
| 114 |
### Example
|
| 115 |
|