Text Generation
PyTorch
Safetensors
English
openlm
mamba
linear
Eval Results
sedrickkeh ivas-tri commited on
Commit
9b2853e
1 Parent(s): 189b8e7

Update README.md (#1)

Browse files

- Update README.md (6e42521084b3512816df7b6652b0138793faae5d)


Co-authored-by: Igor Vasiljevic <ivas-tri@users.noreply.huggingface.co>

Files changed (1) hide show
  1. README.md +9 -8
README.md CHANGED
@@ -148,14 +148,15 @@ Below we report the performance of Mamba 7B compared to other base models.
148
 
149
  <div class="evalTable">
150
 
151
- | | MMLU (5-shot) | HellaSwag | PIQA | Winogrande | ARC-E | ARC-C |
152
- | ----------------- | ---------------- | ------------- | -------- | -------------- | --------- | --------- |
153
- | Mamba-1.4B | 25.2 | 59.0 | 73.9 | 61.4 | 65.5 | 32.9 |
154
- | Mamba-2.8B | 26.3 | 66.2 | 75.8 | 63.4 | 69.7 | 36.3 |
155
- | Llama2-7B | 45.9 | 76.0 | 79.1 | 69.1 | 76.3 | 46.3 |
156
- | Gemma-7B | 62.9 | 80.7 | 81.9 | 73.7 | 81.1 | 53.2 |
157
- | Mistral-7B | 62.4 | 81.0 | 82.1 | 74.0 | 80.9 | 53.8 |
158
- | **Mamba-7B** | 33.3 | 77.9 | 81.0 | 71.8 | 77.5 | 46.7 |
 
159
 
160
  </div>
161
 
 
148
 
149
  <div class="evalTable">
150
 
151
+
152
+ | | HellaSwag | PIQA | Winogrande | ARC-E | ARC-C | MMLU (5-shot) |
153
+ | ----------------- | ------------- | -------- | -------------- | --------- | --------- | ---------------- |
154
+ | Mamba-1.4B | 59.0 | 73.9 | 61.4 | 65.5 | 32.9 | 25.2 |
155
+ | Mamba-2.8B | 71.0 | 78.1 | 65.9 | 68.2 | 41.7 | 26.2 |
156
+ | Llama2-7B | 76.0 | 79.1 | 69.1 | 76.3 | 46.3 | 45.9 |
157
+ | Gemma-7B | 80.7 | 81.9 | 73.7 | 81.1 | 53.2 | 62.9 |
158
+ | Mistral-7B | 81.0 | 82.1 | 74.0 | 80.9 | 53.8 | 62.4 |
159
+ | **Mamba-7B** | 77.9 | 81.0 | 71.8 | 77.5 | 46.7 | 33.3 |
160
 
161
  </div>
162