Hugging Face's logo

facebook
/
MobileLLM-125M

Text Generation
Transformers
PyTorch
mobilellm
custom_code
Model card Files Community
8
zechunliu commited on
Commit
0654ead
·
verified ·
1 Parent(s): 0919bd6

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -84,7 +84,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
84
 
85
  ## MobileLLM-125M
86
 
87
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
88
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
89
  | OPT-125M | 41.3 | 25.2 | 57.5 | 62.0 | 41.9 | 31.1 | 31.2 | 50.8 | 42.6 |
90
  | GPT-neo-125M | 40.7 | 24.8 | 61.3 | 62.5 | 41.9 | 29.7 | 31.6 | 50.7 | 42.9 |
@@ -94,7 +94,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
94
 
95
  ## MobileLLM-350M
96
 
97
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
98
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
99
  | OPT-350M | 41.9 | 25.7 | 54.0 | 64.8 | 42.6 | 36.2 | 33.3 | 52.4 | 43.9 |
100
  | Pythia-410M | 47.1 | 30.3 | 55.3 | 67.2 | 43.1 | 40.1 | 36.2 | 53.4 | 46.6 |
@@ -103,7 +103,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
103
 
104
  ## MobileLLM-600M
105
 
106
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
107
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
108
  | Qwen1.5-500M | 54.7 | 32.1 | 46.9 | 68.9 | 46.0 | 48.8 | 37.7 | 55.0 | 48.8 |
109
  | BLOOM-560M | 43.7 | 27.5 | 53.7 | 65.1 | 42.5 | 36.5 | 32.6 | 52.2 | 44.2 |
@@ -112,7 +112,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
112
 
113
  ## MobileLLM-1B
114
 
115
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
116
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
117
  | Pythia-1B | 49.9 | 30.4 | 58.7 | 69.2 | 43.3 | 47.4 | 38.6 | 52.2 | 48.7 |
118
  | MobiLlama-1B | 59.7 | 38.4 | 59.2 | 74.5 | 44.9 | 62.0 | 43.7 | 59.0 | 55.2 |
@@ -123,7 +123,7 @@ We evaluate the pretrained MobileLLM models on Zero-shot Common Sense Reasoning
123
 
124
  ## MobileLLM-1.5B
125
 
126
- | model | boolq | piqa | siqa | hellaswag | winogrande | arc_easy | arc_challenge | obqa | avg. |
127
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
128
  | GPT-neo-1.3B | 51.3 | 33.0 | 61.8 | 70.9 | 43.7 | 48.6 | 41.2 | 54.5 | 50.6 |
129
  | OPT-1.3B | 54.4 | 31.7 | 58.4 | 71.5 | 44.7 | 53.7 | 44.6 | 59.1 | 52.3 |
 
84
 
85
  ## MobileLLM-125M
86
 
87
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
88
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
89
  | OPT-125M | 41.3 | 25.2 | 57.5 | 62.0 | 41.9 | 31.1 | 31.2 | 50.8 | 42.6 |
90
  | GPT-neo-125M | 40.7 | 24.8 | 61.3 | 62.5 | 41.9 | 29.7 | 31.6 | 50.7 | 42.9 |
 
94
 
95
  ## MobileLLM-350M
96
 
97
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
98
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
99
  | OPT-350M | 41.9 | 25.7 | 54.0 | 64.8 | 42.6 | 36.2 | 33.3 | 52.4 | 43.9 |
100
  | Pythia-410M | 47.1 | 30.3 | 55.3 | 67.2 | 43.1 | 40.1 | 36.2 | 53.4 | 46.6 |
 
103
 
104
  ## MobileLLM-600M
105
 
106
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
107
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
108
  | Qwen1.5-500M | 54.7 | 32.1 | 46.9 | 68.9 | 46.0 | 48.8 | 37.7 | 55.0 | 48.8 |
109
  | BLOOM-560M | 43.7 | 27.5 | 53.7 | 65.1 | 42.5 | 36.5 | 32.6 | 52.2 | 44.2 |
 
112
 
113
  ## MobileLLM-1B
114
 
115
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
116
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
117
  | Pythia-1B | 49.9 | 30.4 | 58.7 | 69.2 | 43.3 | 47.4 | 38.6 | 52.2 | 48.7 |
118
  | MobiLlama-1B | 59.7 | 38.4 | 59.2 | 74.5 | 44.9 | 62.0 | 43.7 | 59.0 | 55.2 |
 
123
 
124
  ## MobileLLM-1.5B
125
 
126
+ | model | arc_easy | arc_challenge | boolq | piqa | siqa | hellaswag | obqa | winogrande | avg. |
127
  | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
128
  | GPT-neo-1.3B | 51.3 | 33.0 | 61.8 | 70.9 | 43.7 | 48.6 | 41.2 | 54.5 | 50.6 |
129
  | OPT-1.3B | 54.4 | 31.7 | 58.4 | 71.5 | 44.7 | 53.7 | 44.6 | 59.1 | 52.3 |