Spaces:
Runtime error
Runtime error
Ludwig Stumpp
commited on
Commit
•
5e1e4f6
1
Parent(s):
360209c
Align human eval format
Browse files
README.md
CHANGED
|
@@ -16,37 +16,37 @@ https://llm-leaderboard.streamlit.app/
|
|
| 16 |
| [cerebras-gpt-13b](https://huggingface.co/cerebras/Cerebras-GPT-13B) | yes | | | [0.635](https://www.mosaicml.com/blog/mpt-7b) | [0.258](https://www.mosaicml.com/blog/mpt-7b) | | [0.146](https://www.mosaicml.com/blog/mpt-7b) |
|
| 17 |
| [chatglm-6b](https://chatglm.cn/blog) | yes | [985](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 18 |
| [chinchilla-70b](https://arxiv.org/abs/2203.15556v1) | no | | | [0.774](https://arxiv.org/abs/2203.15556v1) | | [0.675](https://arxiv.org/abs/2203.15556v1) | |
|
| 19 |
-
| [code-cushman-001](https://arxiv.org/abs/2107.03374) | no | | [
|
| 20 |
-
| [code-davinci-002](https://arxiv.org/abs/2207.10397v2) | yes | | [
|
| 21 |
-
| [codegen-16B-mono](https://huggingface.co/Salesforce/codegen-16B-mono) | yes | | [
|
| 22 |
-
| [codegen-16B-multi](https://huggingface.co/Salesforce/codegen-16B-multi) | yes | | [
|
| 23 |
-
| [codegx-13b](http://keg.cs.tsinghua.edu.cn/codegeex/) | no | | [
|
| 24 |
-
| [codex-12b](https://arxiv.org/abs/2107.03374v2) | no | | [
|
| 25 |
| [dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) | yes | [944](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 26 |
| [eleuther-pythia-7b](https://huggingface.co/EleutherAI/pythia-6.9b) | yes | | | [0.667](https://www.mosaicml.com/blog/mpt-7b) | [0.265](https://www.mosaicml.com/blog/mpt-7b) | | [0.198](https://www.mosaicml.com/blog/mpt-7b) |
|
| 27 |
| [eleuther-pythia-12b](https://huggingface.co/EleutherAI/pythia-12b) | yes | | | [0.704](https://www.mosaicml.com/blog/mpt-7b) | [0.253](https://www.mosaicml.com/blog/mpt-7b) | | [0.233](https://www.mosaicml.com/blog/mpt-7b) |
|
| 28 |
| [fastchat-t5-3b](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0) | yes | [951](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 29 |
| [gal-120b](https://arxiv.org/abs/2211.09085v1) | no | | | | [0.526](https://paperswithcode.com/paper/galactica-a-large-language-model-for-science-1) | | |
|
| 30 |
| [gpt-3-175b](https://arxiv.org/abs/2005.14165) | no | | | | | [0.439](https://arxiv.org/abs/2005.14165) | |
|
| 31 |
-
| [gpt-3.5-175b](https://arxiv.org/abs/2303.08774v3) | yes | | [
|
| 32 |
-
| [gpt-4](https://arxiv.org/abs/2303.08774v3) | yes | | [
|
| 33 |
| [gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) | yes | | | [0.719](https://www.mosaicml.com/blog/mpt-7b) | [0.269](https://www.mosaicml.com/blog/mpt-7b) | [0.336](https://arxiv.org/abs/2204.06745v1) | [0.347](https://www.mosaicml.com/blog/mpt-7b) |
|
| 34 |
| [gpt-j-6b](https://huggingface.co/EleutherAI/gpt-j-6b) | yes | | | [0.683](https://www.mosaicml.com/blog/mpt-7b) | [0.261](https://www.mosaicml.com/blog/mpt-7b) | | [0.234](https://www.mosaicml.com/blog/mpt-7b) |
|
| 35 |
| [koala-13b](https://bair.berkeley.edu/blog/2023/04/03/koala/) | no | [1082](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 36 |
-
| [llama-7b](https://arxiv.org/abs/2302.13971) | no | | [
|
| 37 |
-
| [llama-13b](https://arxiv.org/abs/2302.13971) | no | [932](https://lmsys.org/blog/2023-05-03-arena/) | [
|
| 38 |
-
| [llama-33b](https://arxiv.org/abs/2302.13971) | no | | [
|
| 39 |
-
| [llama-65b](https://arxiv.org/abs/2302.13971) | no | | [
|
| 40 |
| [mpt-7b](https://huggingface.co/mosaicml/mpt-7b) | yes | | | [0.702](https://www.mosaicml.com/blog/mpt-7b) | [0.296](https://www.mosaicml.com/blog/mpt-7b) | | [0.343](https://www.mosaicml.com/blog/mpt-7b) |
|
| 41 |
| [oasst-pythia-12b](https://huggingface.co/OpenAssistant/pythia-12b-pre-v8-12.5k-steps) | yes | [1065](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 42 |
| [opt-7b](https://huggingface.co/facebook/opt-6.7b) | no | | | [0.677](https://www.mosaicml.com/blog/mpt-7b) | [0.251](https://www.mosaicml.com/blog/mpt-7b) | | [0.227](https://www.mosaicml.com/blog/mpt-7b) |
|
| 43 |
| [opt-13b](https://huggingface.co/facebook/opt-13b) | no | | | [0.692](https://www.mosaicml.com/blog/mpt-7b) | [0.257](https://www.mosaicml.com/blog/mpt-7b) | | [0.282](https://www.mosaicml.com/blog/mpt-7b) |
|
| 44 |
-
| [palm-540b](https://arxiv.org/abs/2204.02311v5) | no | | [
|
| 45 |
| [stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | yes | | | [0.533](https://www.mosaicml.com/blog/mpt-7b) | [0.251](https://www.mosaicml.com/blog/mpt-7b) | | [0.049](https://www.mosaicml.com/blog/mpt-7b) |
|
| 46 |
| [stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | no | [858](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 47 |
-
| [starcoder-base-16b](https://huggingface.co/bigcode/starcoderbase) | yes | | [
|
| 48 |
-
| [starcoder-16b](https://huggingface.co/bigcode/starcoder) | yes | | [
|
| 49 |
-
| [starcoder-16b (prompted)](https://huggingface.co/bigcode/starcoder) | yes | | [
|
| 50 |
| [vicuna-13b](https://huggingface.co/lmsys/vicuna-13b-delta-v0) | no | [1169](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 51 |
|
| 52 |
## Benchmarks
|
|
|
|
| 16 |
| [cerebras-gpt-13b](https://huggingface.co/cerebras/Cerebras-GPT-13B) | yes | | | [0.635](https://www.mosaicml.com/blog/mpt-7b) | [0.258](https://www.mosaicml.com/blog/mpt-7b) | | [0.146](https://www.mosaicml.com/blog/mpt-7b) |
|
| 17 |
| [chatglm-6b](https://chatglm.cn/blog) | yes | [985](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 18 |
| [chinchilla-70b](https://arxiv.org/abs/2203.15556v1) | no | | | [0.774](https://arxiv.org/abs/2203.15556v1) | | [0.675](https://arxiv.org/abs/2203.15556v1) | |
|
| 19 |
+
| [code-cushman-001](https://arxiv.org/abs/2107.03374) | no | | [0.335](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 20 |
+
| [code-davinci-002](https://arxiv.org/abs/2207.10397v2) | yes | | [0.658](https://arxiv.org/abs/2207.10397v2) | | | | |
|
| 21 |
+
| [codegen-16B-mono](https://huggingface.co/Salesforce/codegen-16B-mono) | yes | | [0.293](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 22 |
+
| [codegen-16B-multi](https://huggingface.co/Salesforce/codegen-16B-multi) | yes | | [0.183](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 23 |
+
| [codegx-13b](http://keg.cs.tsinghua.edu.cn/codegeex/) | no | | [0.229](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 24 |
+
| [codex-12b](https://arxiv.org/abs/2107.03374v2) | no | | [0.288](https://arxiv.org/abs/2107.03374v2) | | | [0.685](https://arxiv.org/abs/2301.12652v2) | |
|
| 25 |
| [dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) | yes | [944](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 26 |
| [eleuther-pythia-7b](https://huggingface.co/EleutherAI/pythia-6.9b) | yes | | | [0.667](https://www.mosaicml.com/blog/mpt-7b) | [0.265](https://www.mosaicml.com/blog/mpt-7b) | | [0.198](https://www.mosaicml.com/blog/mpt-7b) |
|
| 27 |
| [eleuther-pythia-12b](https://huggingface.co/EleutherAI/pythia-12b) | yes | | | [0.704](https://www.mosaicml.com/blog/mpt-7b) | [0.253](https://www.mosaicml.com/blog/mpt-7b) | | [0.233](https://www.mosaicml.com/blog/mpt-7b) |
|
| 28 |
| [fastchat-t5-3b](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0) | yes | [951](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 29 |
| [gal-120b](https://arxiv.org/abs/2211.09085v1) | no | | | | [0.526](https://paperswithcode.com/paper/galactica-a-large-language-model-for-science-1) | | |
|
| 30 |
| [gpt-3-175b](https://arxiv.org/abs/2005.14165) | no | | | | | [0.439](https://arxiv.org/abs/2005.14165) | |
|
| 31 |
+
| [gpt-3.5-175b](https://arxiv.org/abs/2303.08774v3) | yes | | [0.481](https://arxiv.org/abs/2303.08774v3) | [0.762](https://arxiv.org/abs/2303.08774v3) | | [0.700](https://arxiv.org/abs/2303.08774v3) | |
|
| 32 |
+
| [gpt-4](https://arxiv.org/abs/2303.08774v3) | yes | | [0.670](https://arxiv.org/abs/2303.08774v3) | | | [0.864](https://arxiv.org/abs/2303.08774v3) | |
|
| 33 |
| [gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) | yes | | | [0.719](https://www.mosaicml.com/blog/mpt-7b) | [0.269](https://www.mosaicml.com/blog/mpt-7b) | [0.336](https://arxiv.org/abs/2204.06745v1) | [0.347](https://www.mosaicml.com/blog/mpt-7b) |
|
| 34 |
| [gpt-j-6b](https://huggingface.co/EleutherAI/gpt-j-6b) | yes | | | [0.683](https://www.mosaicml.com/blog/mpt-7b) | [0.261](https://www.mosaicml.com/blog/mpt-7b) | | [0.234](https://www.mosaicml.com/blog/mpt-7b) |
|
| 35 |
| [koala-13b](https://bair.berkeley.edu/blog/2023/04/03/koala/) | no | [1082](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 36 |
+
| [llama-7b](https://arxiv.org/abs/2302.13971) | no | | [0.105](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | [0.738](https://www.mosaicml.com/blog/mpt-7b) | [0.302](https://www.mosaicml.com/blog/mpt-7b) | | [0.443](https://www.mosaicml.com/blog/mpt-7b) |
|
| 37 |
+
| [llama-13b](https://arxiv.org/abs/2302.13971) | no | [932](https://lmsys.org/blog/2023-05-03-arena/) | [0.158](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 38 |
+
| [llama-33b](https://arxiv.org/abs/2302.13971) | no | | [0.217](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 39 |
+
| [llama-65b](https://arxiv.org/abs/2302.13971) | no | | [0.237](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | [0.634](https://arxiv.org/abs/2302.13971v1) | |
|
| 40 |
| [mpt-7b](https://huggingface.co/mosaicml/mpt-7b) | yes | | | [0.702](https://www.mosaicml.com/blog/mpt-7b) | [0.296](https://www.mosaicml.com/blog/mpt-7b) | | [0.343](https://www.mosaicml.com/blog/mpt-7b) |
|
| 41 |
| [oasst-pythia-12b](https://huggingface.co/OpenAssistant/pythia-12b-pre-v8-12.5k-steps) | yes | [1065](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 42 |
| [opt-7b](https://huggingface.co/facebook/opt-6.7b) | no | | | [0.677](https://www.mosaicml.com/blog/mpt-7b) | [0.251](https://www.mosaicml.com/blog/mpt-7b) | | [0.227](https://www.mosaicml.com/blog/mpt-7b) |
|
| 43 |
| [opt-13b](https://huggingface.co/facebook/opt-13b) | no | | | [0.692](https://www.mosaicml.com/blog/mpt-7b) | [0.257](https://www.mosaicml.com/blog/mpt-7b) | | [0.282](https://www.mosaicml.com/blog/mpt-7b) |
|
| 44 |
+
| [palm-540b](https://arxiv.org/abs/2204.02311v5) | no | | [0.262](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | [0.779](https://arxiv.org/abs/2204.02311v5) | | [0.693](https://arxiv.org/abs/2204.02311v5) | |
|
| 45 |
| [stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | yes | | | [0.533](https://www.mosaicml.com/blog/mpt-7b) | [0.251](https://www.mosaicml.com/blog/mpt-7b) | | [0.049](https://www.mosaicml.com/blog/mpt-7b) |
|
| 46 |
| [stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | no | [858](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 47 |
+
| [starcoder-base-16b](https://huggingface.co/bigcode/starcoderbase) | yes | | [0.304](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 48 |
+
| [starcoder-16b](https://huggingface.co/bigcode/starcoder) | yes | | [0.336](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 49 |
+
| [starcoder-16b (prompted)](https://huggingface.co/bigcode/starcoder) | yes | | [0.408](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
| 50 |
| [vicuna-13b](https://huggingface.co/lmsys/vicuna-13b-delta-v0) | no | [1169](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
| 51 |
|
| 52 |
## Benchmarks
|