mlabonne commited on
Commit
0e9343a
1 Parent(s): c7018ef

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -16,6 +16,16 @@ This model is a Mixture of Experts (MoE) made with [mergekit](https://github.com
16
 
17
  ## 🏆 Evaluation
18
 
 
 
 
 
 
 
 
 
 
 
19
  | Model |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
20
  |--------------------------------------------------------------------|------:|------:|---------:|-------:|------:|
21
  |[**Beyonder-4x7B-v2**](https://huggingface.co/shadowml/Beyonder-4x7B-v2)| **45.29**| **75.95**| <u>**60.86**</u>| **46.4**| **57.13**|
 
16
 
17
  ## 🏆 Evaluation
18
 
19
+ Beyonder-4x7B-v2 is competitive with Mixtral-8x7B-Instruct-v0.1 on the Open LLM Leaderboard, while only having 4 experts instead of 8.
20
+
21
+ ![](https://i.imgur.com/5raBff0.png)
22
+
23
+ It also displays a significant improvement over the individual experts.
24
+
25
+ ![](https://i.imgur.com/7Idwkb0.png)
26
+
27
+ It also performs very well compared to other models on Nous benchmark suite. It's almost as good as the best Yi-34B fine-tune, which is a much bigger model: 24.2B parameters + only two experts are selected during inference (so ~12B) vs. 34B param.
28
+
29
  | Model |AGIEval|GPT4All|TruthfulQA|Bigbench|Average|
30
  |--------------------------------------------------------------------|------:|------:|---------:|-------:|------:|
31
  |[**Beyonder-4x7B-v2**](https://huggingface.co/shadowml/Beyonder-4x7B-v2)| **45.29**| **75.95**| <u>**60.86**</u>| **46.4**| **57.13**|