Spaces:
Running
Running
Ludwig Stumpp
commited on
Commit
•
9c17477
1
Parent(s):
21aaac9
Add MMLU few shot
Browse files
README.md
CHANGED
@@ -8,44 +8,45 @@ https://llm-leaderboard.streamlit.app/
|
|
8 |
|
9 |
## Leaderboard
|
10 |
|
11 |
-
| Model Name | Commercial Use? | Chatbot Arena Elo | HumanEval-Python (pass@1) | LAMBADA (zero-shot) | MMLU (zero-shot)
|
12 |
-
| -------------------------------------------------------------------------------------- | --------------- | ------------------------------------------------ | ------------------------------------------------------------------------------ | --------------------------------------------- |
|
13 |
-
| [alpaca-13b](https://crfm.stanford.edu/2023/03/13/alpaca.html) | no | [1008](https://lmsys.org/blog/2023-05-03-arena/) | | |
|
14 |
-
| [cerebras-gpt-7b](https://huggingface.co/cerebras/Cerebras-GPT-6.7B) | yes | | | [0.636](https://www.mosaicml.com/blog/mpt-7b) | [0.259](https://www.mosaicml.com/blog/mpt-7b)
|
15 |
-
| [cerebras-gpt-13b](https://huggingface.co/cerebras/Cerebras-GPT-13B) | yes | | | [0.635](https://www.mosaicml.com/blog/mpt-7b) | [0.258](https://www.mosaicml.com/blog/mpt-7b)
|
16 |
-
| [chatglm-6b](https://chatglm.cn/blog) | yes | [985](https://lmsys.org/blog/2023-05-03-arena/) | | |
|
17 |
-
| [chinchilla-70b](https://arxiv.org/abs/2203.15556v1) | no | | | [0.774](https://arxiv.org/abs/2203.15556v1) |
|
18 |
-
| [code-cushman-001](https://arxiv.org/abs/2107.03374) | no | | [33.5](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | |
|
19 |
-
| [code-davinci-002](https://arxiv.org/abs/2207.10397v2) | yes | | [65.8](https://arxiv.org/abs/2207.10397v2) | |
|
20 |
-
| [codegen-16B-mono](https://huggingface.co/Salesforce/codegen-16B-mono) | yes | | [29.3](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | |
|
21 |
-
| [codegen-16B-multi](https://huggingface.co/Salesforce/codegen-16B-multi) | yes | | [18.3](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | |
|
22 |
-
| [codegx-13b](http://keg.cs.tsinghua.edu.cn/codegeex/) | no | | [22.9](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | |
|
23 |
-
| [codex-12b](https://arxiv.org/abs/2107.03374v2) | no | | [28.
|
24 |
-
| [dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) | yes | [944](https://lmsys.org/blog/2023-05-03-arena/) | | |
|
25 |
-
| [eleuther-pythia-7b](https://huggingface.co/EleutherAI/pythia-6.9b) | yes | | | [0.667](https://www.mosaicml.com/blog/mpt-7b) | [0.265](https://www.mosaicml.com/blog/mpt-7b)
|
26 |
-
| [eleuther-pythia-12b](https://huggingface.co/EleutherAI/pythia-12b) | yes | | | [0.704](https://www.mosaicml.com/blog/mpt-7b) | [0.253](https://www.mosaicml.com/blog/mpt-7b)
|
27 |
-
| [fastchat-t5-3b](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0) | yes | [951](https://lmsys.org/blog/2023-05-03-arena/) | | |
|
28 |
-
| [gal-120b](https://arxiv.org/abs/2211.09085v1) | no | | | | [
|
29 |
-
| [gpt-3
|
30 |
-
| [gpt-
|
31 |
-
| [gpt-
|
32 |
-
| [gpt-
|
33 |
-
| [
|
34 |
-
| [
|
35 |
-
| [llama-
|
36 |
-
| [llama-
|
37 |
-
| [llama-
|
38 |
-
| [
|
39 |
-
| [
|
40 |
-
| [
|
41 |
-
| [opt-
|
42 |
-
| [
|
43 |
-
| [
|
44 |
-
| [stablelm-
|
45 |
-
| [
|
46 |
-
| [starcoder-16b](https://huggingface.co/bigcode/
|
47 |
-
| [starcoder-16b
|
48 |
-
| [
|
|
|
49 |
|
50 |
## Benchmarks
|
51 |
|
|
|
8 |
|
9 |
## Leaderboard
|
10 |
|
11 |
+
| Model Name | Commercial Use? | Chatbot Arena Elo | HumanEval-Python (pass@1) | LAMBADA (zero-shot) | MMLU (zero-shot) | MMLU (few-shot) | TriviaQA (zero-shot) |
|
12 |
+
| -------------------------------------------------------------------------------------- | --------------- | ------------------------------------------------ | ------------------------------------------------------------------------------ | --------------------------------------------- | ---------------------------------------------------------------------------------------- | ------------------------------------------- | --------------------------------------------- |
|
13 |
+
| [alpaca-13b](https://crfm.stanford.edu/2023/03/13/alpaca.html) | no | [1008](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
14 |
+
| [cerebras-gpt-7b](https://huggingface.co/cerebras/Cerebras-GPT-6.7B) | yes | | | [0.636](https://www.mosaicml.com/blog/mpt-7b) | [0.259](https://www.mosaicml.com/blog/mpt-7b) | | [0.141](https://www.mosaicml.com/blog/mpt-7b) |
|
15 |
+
| [cerebras-gpt-13b](https://huggingface.co/cerebras/Cerebras-GPT-13B) | yes | | | [0.635](https://www.mosaicml.com/blog/mpt-7b) | [0.258](https://www.mosaicml.com/blog/mpt-7b) | | [0.146](https://www.mosaicml.com/blog/mpt-7b) |
|
16 |
+
| [chatglm-6b](https://chatglm.cn/blog) | yes | [985](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
17 |
+
| [chinchilla-70b](https://arxiv.org/abs/2203.15556v1) | no | | | [0.774](https://arxiv.org/abs/2203.15556v1) | | [0.675](https://arxiv.org/abs/2203.15556v1) | |
|
18 |
+
| [code-cushman-001](https://arxiv.org/abs/2107.03374) | no | | [33.5](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
19 |
+
| [code-davinci-002](https://arxiv.org/abs/2207.10397v2) | yes | | [65.8](https://arxiv.org/abs/2207.10397v2) | | | | |
|
20 |
+
| [codegen-16B-mono](https://huggingface.co/Salesforce/codegen-16B-mono) | yes | | [29.3](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
21 |
+
| [codegen-16B-multi](https://huggingface.co/Salesforce/codegen-16B-multi) | yes | | [18.3](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
22 |
+
| [codegx-13b](http://keg.cs.tsinghua.edu.cn/codegeex/) | no | | [22.9](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
23 |
+
| [codex-12b](https://arxiv.org/abs/2107.03374v2) | no | | [28.8](https://arxiv.org/abs/2107.03374v2) | | | [0.685](https://arxiv.org/abs/2301.12652v2) | |
|
24 |
+
| [dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b) | yes | [944](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
25 |
+
| [eleuther-pythia-7b](https://huggingface.co/EleutherAI/pythia-6.9b) | yes | | | [0.667](https://www.mosaicml.com/blog/mpt-7b) | [0.265](https://www.mosaicml.com/blog/mpt-7b) | | [0.198](https://www.mosaicml.com/blog/mpt-7b) |
|
26 |
+
| [eleuther-pythia-12b](https://huggingface.co/EleutherAI/pythia-12b) | yes | | | [0.704](https://www.mosaicml.com/blog/mpt-7b) | [0.253](https://www.mosaicml.com/blog/mpt-7b) | | [0.233](https://www.mosaicml.com/blog/mpt-7b) |
|
27 |
+
| [fastchat-t5-3b](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0) | yes | [951](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
28 |
+
| [gal-120b](https://arxiv.org/abs/2211.09085v1) | no | | | | [0.526](https://paperswithcode.com/paper/galactica-a-large-language-model-for-science-1) | | |
|
29 |
+
| [gpt-3-175b](https://arxiv.org/abs/2005.14165) | no | | | | | [0.439](https://arxiv.org/abs/2005.14165) | |
|
30 |
+
| [gpt-3.5-175b](https://arxiv.org/abs/2303.08774v3) | yes | | [48.1](https://arxiv.org/abs/2303.08774v3) | [0.762](https://arxiv.org/abs/2303.08774v3) | | [0.700](https://arxiv.org/abs/2303.08774v3) | |
|
31 |
+
| [gpt-4](https://arxiv.org/abs/2303.08774v3) | yes | | [67.0](https://arxiv.org/abs/2303.08774v3) | | | [0.864](https://arxiv.org/abs/2303.08774v3) | |
|
32 |
+
| [gpt-neox-20b](https://huggingface.co/EleutherAI/gpt-neox-20b) | yes | | | [0.719](https://www.mosaicml.com/blog/mpt-7b) | [0.269](https://www.mosaicml.com/blog/mpt-7b) | [0.336](https://arxiv.org/abs/2204.06745v1) | [0.347](https://www.mosaicml.com/blog/mpt-7b) |
|
33 |
+
| [gpt-j-6b](https://huggingface.co/EleutherAI/gpt-j-6b) | yes | | | [0.683](https://www.mosaicml.com/blog/mpt-7b) | [0.261](https://www.mosaicml.com/blog/mpt-7b) | | [0.234](https://www.mosaicml.com/blog/mpt-7b) |
|
34 |
+
| [koala-13b](https://bair.berkeley.edu/blog/2023/04/03/koala/) | no | [1082](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
35 |
+
| [llama-7b](https://arxiv.org/abs/2302.13971) | no | | [10.5](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | [0.738](https://www.mosaicml.com/blog/mpt-7b) | [0.302](https://www.mosaicml.com/blog/mpt-7b) | | [0.443](https://www.mosaicml.com/blog/mpt-7b) |
|
36 |
+
| [llama-13b](https://arxiv.org/abs/2302.13971) | no | [932](https://lmsys.org/blog/2023-05-03-arena/) | [15.8](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
37 |
+
| [llama-33b](https://arxiv.org/abs/2302.13971) | no | | [21.7](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
38 |
+
| [llama-65b](https://arxiv.org/abs/2302.13971) | no | | [23.7](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | [0.634](https://arxiv.org/abs/2302.13971v1) | |
|
39 |
+
| [mpt-7b](https://huggingface.co/mosaicml/mpt-7b) | yes | | | [0.702](https://www.mosaicml.com/blog/mpt-7b) | [0.296](https://www.mosaicml.com/blog/mpt-7b) | | [0.343](https://www.mosaicml.com/blog/mpt-7b) |
|
40 |
+
| [oasst-pythia-12b](https://huggingface.co/OpenAssistant/pythia-12b-pre-v8-12.5k-steps) | yes | [1065](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
41 |
+
| [opt-7b](https://huggingface.co/facebook/opt-6.7b) | no | | | [0.677](https://www.mosaicml.com/blog/mpt-7b) | [0.251](https://www.mosaicml.com/blog/mpt-7b) | | [0.227](https://www.mosaicml.com/blog/mpt-7b) |
|
42 |
+
| [opt-13b](https://huggingface.co/facebook/opt-13b) | no | | | [0.692](https://www.mosaicml.com/blog/mpt-7b) | [0.257](https://www.mosaicml.com/blog/mpt-7b) | | [0.282](https://www.mosaicml.com/blog/mpt-7b) |
|
43 |
+
| [palm-540b](https://arxiv.org/abs/2204.02311v5) | no | | [26.2](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | [0.779](https://arxiv.org/abs/2204.02311v5) | | [0.693](https://arxiv.org/abs/2204.02311v5) | |
|
44 |
+
| [stablelm-base-alpha-7b](https://huggingface.co/stabilityai/stablelm-base-alpha-7b) | yes | | | [0.533](https://www.mosaicml.com/blog/mpt-7b) | [0.251](https://www.mosaicml.com/blog/mpt-7b) | | [0.049](https://www.mosaicml.com/blog/mpt-7b) |
|
45 |
+
| [stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b) | no | [858](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
46 |
+
| [starcoder-base-16b](https://huggingface.co/bigcode/starcoderbase) | yes | | [30.4](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
47 |
+
| [starcoder-16b](https://huggingface.co/bigcode/starcoder) | yes | | [33.6](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
48 |
+
| [starcoder-16b (prompted)](https://huggingface.co/bigcode/starcoder) | yes | | [40.8](https://drive.google.com/file/d/1cN-b9GnWtHzQRoE7M7gAEyivY0kl4BYs/view) | | | | |
|
49 |
+
| [vicuna-13b](https://huggingface.co/lmsys/vicuna-13b-delta-v0) | no | [1169](https://lmsys.org/blog/2023-05-03-arena/) | | | | | |
|
50 |
|
51 |
## Benchmarks
|
52 |
|