Commit
•
4e81136
1
Parent(s):
5d14a68
Corrected bolding in the HellaSwag table and adjusted the labels slightly
Browse files
README.md
CHANGED
|
@@ -90,25 +90,25 @@ python main.py \
|
|
| 90 |
|
| 91 |
### COPA (F1)
|
| 92 |
|
| 93 |
-
| Model
|
| 94 |
-
|
| 95 |
-
| [facebook/xglm-7.5B](https://huggingface.co/facebook/xglm-7.5B)
|
| 96 |
-
| [skt/ko-gpt-trinity-1.2B-v0.5](https://huggingface.co/skt/ko-gpt-trinity-1.2B-v0.5) †
|
| 97 |
-
| [kakaobrain/kogpt](https://huggingface.co/kakaobrain/kogpt) *
|
| 98 |
-
| [EleutherAI/polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) (ours)
|
| 99 |
-
| [EleutherAI/polyglot-ko-3.8b](https://huggingface.co/EleutherAI/polyglot-ko-3.8b) (
|
| 100 |
|
| 101 |
<img src="https://user-images.githubusercontent.com/19511788/192492576-cdd80c5c-7c90-43e3-8a4b-7a8486878f23.png" width="800px">
|
| 102 |
|
| 103 |
### HellaSwag (F1)
|
| 104 |
|
| 105 |
-
| Model
|
| 106 |
-
|
| 107 |
-
| [facebook/xglm-7.5B](https://huggingface.co/facebook/xglm-7.5B)
|
| 108 |
-
| [skt/ko-gpt-trinity-1.2B-v0.5](https://huggingface.co/skt/ko-gpt-trinity-1.2B-v0.5) †
|
| 109 |
-
| [kakaobrain/kogpt](https://huggingface.co/kakaobrain/kogpt) *
|
| 110 |
-
| [EleutherAI/polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) (ours)
|
| 111 |
-
| [EleutherAI/polyglot-ko-3.8b](https://huggingface.co/EleutherAI/polyglot-ko-3.8b) (
|
| 112 |
|
| 113 |
<img src="https://user-images.githubusercontent.com/19511788/192492585-a976ee38-2967-446a-b577-94f219228f4d.png" width="800px">
|
| 114 |
|
|
|
|
| 90 |
|
| 91 |
### COPA (F1)
|
| 92 |
|
| 93 |
+
| Model | params | 0-shot | 5-shot | 10-shot | 50-shot |
|
| 94 |
+
|------------------------------------------------------------------------------------------------|--------|--------|--------|---------|---------|
|
| 95 |
+
| [facebook/xglm-7.5B](https://huggingface.co/facebook/xglm-7.5B) | 7.5B | 0.6723 | 0.6731 | 0.6769 | 0.7119 |
|
| 96 |
+
| [skt/ko-gpt-trinity-1.2B-v0.5](https://huggingface.co/skt/ko-gpt-trinity-1.2B-v0.5) † | 1.2B | 0.6696 | 0.6477 | 0.6419 | 0.6514 |
|
| 97 |
+
| [kakaobrain/kogpt](https://huggingface.co/kakaobrain/kogpt) * | 6.0B | 0.7345 | 0.7287 | 0.7277 | 0.7479 |
|
| 98 |
+
| [EleutherAI/polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) (ours) | 1.3B | 0.7196 | 0.7193 | 0.7204 | 0.7206 |
|
| 99 |
+
| [EleutherAI/polyglot-ko-3.8b](https://huggingface.co/EleutherAI/polyglot-ko-3.8b) (this model) | 3.8B | **0.7595** | **0.7608** | **0.7638** | **0.7788** |
|
| 100 |
|
| 101 |
<img src="https://user-images.githubusercontent.com/19511788/192492576-cdd80c5c-7c90-43e3-8a4b-7a8486878f23.png" width="800px">
|
| 102 |
|
| 103 |
### HellaSwag (F1)
|
| 104 |
|
| 105 |
+
| Model | params | 0-shot | 5-shot | 10-shot | 50-shot |
|
| 106 |
+
|------------------------------------------------------------------------------------------------|--------|--------|--------|---------|---------|
|
| 107 |
+
| [facebook/xglm-7.5B](https://huggingface.co/facebook/xglm-7.5B) | 7.5B | 0.4261 | 0.437 | 0.4409 | 0.4517 |
|
| 108 |
+
| [skt/ko-gpt-trinity-1.2B-v0.5](https://huggingface.co/skt/ko-gpt-trinity-1.2B-v0.5) † | 1.2B | 0.4036 | 0.4 | 0.4011 | 0.4214 |
|
| 109 |
+
| [kakaobrain/kogpt](https://huggingface.co/kakaobrain/kogpt) * | 6.0B | **0.4599** | 0.456 | 0.4616 | 0.4754 |
|
| 110 |
+
| [EleutherAI/polyglot-ko-1.3b](https://huggingface.co/EleutherAI/polyglot-ko-1.3b) (ours) | 1.3B | 0.4013 | 0.3984 | 0.417 | 0.4416 |
|
| 111 |
+
| [EleutherAI/polyglot-ko-3.8b](https://huggingface.co/EleutherAI/polyglot-ko-3.8b) (this model) | 3.8B | 0.4438 | **0.4786** | **0.4737** | **0.4822** |
|
| 112 |
|
| 113 |
<img src="https://user-images.githubusercontent.com/19511788/192492585-a976ee38-2967-446a-b577-94f219228f4d.png" width="800px">
|
| 114 |
|