JingweiZuo
commited on
Commit
•
54ec485
1
Parent(s):
d2cd831
Update README.md
Browse files
README.md
CHANGED
@@ -1,10 +1,10 @@
|
|
1 |
-
---
|
2 |
-
datasets:
|
3 |
-
- tiiuae/falcon-refinedweb
|
4 |
-
- HuggingFaceFW/fineweb-edu
|
5 |
-
language:
|
6 |
-
- en
|
7 |
-
---
|
8 |
|
9 |
<img src="https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/falcon_mamba/thumbnail.png" alt="drawing" width="800"/>
|
10 |
|
@@ -210,10 +210,10 @@ We evaluate our model on all benchmarks of the leaderboard's version 2 using the
|
|
210 |
|
211 |
|
212 |
|
213 |
-
| `model name` |`ARC`|`HellaSwag` |`MMLU` |`Winogrande`|`TruthfulQA`|`GSM8K`|`Average` |
|
214 |
|:-----------------------------|:------:|:---------:|:-----:|:----------:|:----------:|:-----:|:----------------:|
|
215 |
| ***Pure SSM models*** | | | | | | | |
|
216 |
-
| `FalconMamba-7B`
|
217 |
| `TRI-ML/mamba-7b-rw` | 46.48 | 80.24 | 57.72 | 76.40 | - | 4.70 | - |
|
218 |
|***Hybrid SSM-attention models***| | | | | | | |
|
219 |
| `recurrentgemma-9b` |52.00 | 80.40 | 60.50 | 73.60 | 38.60 | 42.60 | 57.95 |
|
|
|
1 |
+
---
|
2 |
+
datasets:
|
3 |
+
- tiiuae/falcon-refinedweb
|
4 |
+
- HuggingFaceFW/fineweb-edu
|
5 |
+
language:
|
6 |
+
- en
|
7 |
+
---
|
8 |
|
9 |
<img src="https://huggingface.co/datasets/tiiuae/documentation-images/resolve/main/falcon_mamba/thumbnail.png" alt="drawing" width="800"/>
|
10 |
|
|
|
210 |
|
211 |
|
212 |
|
213 |
+
| `model name` |`ARC-c`|`HellaSwag` |`MMLU` |`Winogrande`|`TruthfulQA`|`GSM8K`|`Average` |
|
214 |
|:-----------------------------|:------:|:---------:|:-----:|:----------:|:----------:|:-----:|:----------------:|
|
215 |
| ***Pure SSM models*** | | | | | | | |
|
216 |
+
| `FalconMamba-7B` |62.03 | 80.82 | 62.11 | 73.64 | 53.42 | 52.54 | **64.09** |
|
217 |
| `TRI-ML/mamba-7b-rw` | 46.48 | 80.24 | 57.72 | 76.40 | - | 4.70 | - |
|
218 |
|***Hybrid SSM-attention models***| | | | | | | |
|
219 |
| `recurrentgemma-9b` |52.00 | 80.40 | 60.50 | 73.60 | 38.60 | 42.60 | 57.95 |
|