tiiuae
/

falcon-mamba-7b

Text Generation

Inference Endpoints

Model card Files Files and versions Community

Ilyas Chahed commited on Jul 24

Commit

adf712d

•

1 Parent(s): 5337fd7

Update README.md

Files changed (1) hide show

README.md +3 -2

README.md CHANGED Viewed

@@ -201,8 +201,9 @@ We evaluate our model on all benchmarks of the leaderboard's version 2 using the
 | `meta-llama/Meta-Llama-3-8B` | 14.55  | 24.50 | 3.25      | 7.38  | 6.24  | 24.55    | 13.41          | 60.24 | 82.23     | 66.70 | 78.45      | 42.93      | 45.19 | 62.62          |
 | `tiiuae/falcon2-11B`         | 32.61  | 21.94 | 2.34      | 2.8   | 7.53  | 15.44    | 13.78          | 59.73 | 82.91     | 58.37 | 78.30      | 52.56      | 53.83 | **64.28**      |
 | `mistralai/Mistral-7B-v0.1`  | 23.86  | 22.02 | 2.49      | 5.59  | 10.68 | 22.36    | 14.50          | 59.98 | 83.31     | 64.16 | 78.37      | 42.15      | 37.83 | 60.97          |
-| `Zyphra/Zamba-7B-v1`         | -      | -     | -         | -     | -     | -        | -              | 46.48 | 80.24     | 57.72 | 76.4       | -          | -     | -              |
-| Ours                         | 32.16  | 21.07 | 4.08      | 10.18 | 6.97  | 13.43    | **14.65**      | 61.69 | 80.63     | 61.05 | 74.03      | 53.60      | 51.86 | 63.81          |
 ## Throughput

 | `meta-llama/Meta-Llama-3-8B` | 14.55  | 24.50 | 3.25      | 7.38  | 6.24  | 24.55    | 13.41          | 60.24 | 82.23     | 66.70 | 78.45      | 42.93      | 45.19 | 62.62          |
 | `tiiuae/falcon2-11B`         | 32.61  | 21.94 | 2.34      | 2.8   | 7.53  | 15.44    | 13.78          | 59.73 | 82.91     | 58.37 | 78.30      | 52.56      | 53.83 | **64.28**      |
 | `mistralai/Mistral-7B-v0.1`  | 23.86  | 22.02 | 2.49      | 5.59  | 10.68 | 22.36    | 14.50          | 59.98 | 83.31     | 64.16 | 78.37      | 42.15      | 37.83 | 60.97          |
+| `Zyphra/Zamba-7B-v1`         | -      | -     | 3.32      | -     | -     | -        | -              | 46.48 | 80.24     | 57.72 | 76.4       | -          | -     | -              |
+| `TRI-ML/mamba-7b-rw`         | 22.46  | 6.71  | 0.45      | 1.12  | 5.51  | 1.69     | 6.25           | 46.48 | 80.24     | 57.72 | 76.4       | -          | -     | -              |
+| Ours                         | 33.36  | 19.88 | 3.63      | 8.05  | 10.85 | 14.47    | **15.04**      | 62.03 | 80.82     | 62.11 | 73.64      | 53.42      | 52.54 | 64.09          |
 ## Throughput