Update README.md
Browse files
README.md
CHANGED
@@ -19,6 +19,8 @@ Medchator-2x7b is a Mixure of Experts (MoE) made with the following models:
|
|
19 |
|
20 |
# Open LLM Leaderboard
|
21 |
|
|
|
|
|
22 |
| Model Name | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
|
23 |
| ------------------ | -------- | --------- | -------- | ---------- | ---------- | -------- |
|
24 |
| Orca-2-7b | **78.4** | 76.1 | 53.7 | **52.4** | 74.2 | **47.2** |
|
@@ -28,6 +30,21 @@ Medchator-2x7b is a Mixure of Experts (MoE) made with the following models:
|
|
28 |
| Medorca-2x7b | 54.1 | 76.04 | 54.1 | 48.04 | 74.51 | 20.64 |
|
29 |
| Medchator-2x7b | **57.59**| **78.14** | **56.13**| **48.77** | **75.3** | **32.83**|
|
30 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
31 |
## 🧩 Configuration
|
32 |
|
33 |
```yaml
|
|
|
19 |
|
20 |
# Open LLM Leaderboard
|
21 |
|
22 |
+
![image/png](https://cdn-uploads.huggingface.co/production/uploads/63486df1f8f01fcc4b23e97d/ZSMRhGuLrE-K1WNlfbDAG.png)
|
23 |
+
|
24 |
| Model Name | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
|
25 |
| ------------------ | -------- | --------- | -------- | ---------- | ---------- | -------- |
|
26 |
| Orca-2-7b | **78.4** | 76.1 | 53.7 | **52.4** | 74.2 | **47.2** |
|
|
|
30 |
| Medorca-2x7b | 54.1 | 76.04 | 54.1 | 48.04 | 74.51 | 20.64 |
|
31 |
| Medchator-2x7b | **57.59**| **78.14** | **56.13**| **48.77** | **75.3** | **32.83**|
|
32 |
|
33 |
+
## Medical Performance
|
34 |
+
|
35 |
+
Clinical Camel demonstrates competitive performance on medical benchmarks.
|
36 |
+
|
37 |
+
**Table: Five-Shot Performance of GPT3.5, llama-2-7b and Llama-2-70b on Various Medical Datasets**
|
38 |
+
|
39 |
+
| Dataset | Medchator-2x7b | GPT3.5 | Llama-2 7b | Llama-2 70b |
|
40 |
+
|----------------------------|----------------|--------|------------|-------------|
|
41 |
+
| MMLU Anatomy | 56.3 | 60.7 | 48.9 | 62.9 |
|
42 |
+
| MMLU Clinical Knowledge | 63.0 | 68.7 | 46.0 | 71.7 |
|
43 |
+
| MMLU College Biology | 63.8 | 72.9 | 47.2 | 84.7 |
|
44 |
+
| MMLU College Medicine | 50.9 | 63.6 | 42.8 | 64.2 |
|
45 |
+
| MMLU Medical Genetics | 67.0 | 68.0 | 55.0 | 74.0 |
|
46 |
+
| MMLU Professional Medicine | 55.1 | 69.8 | 53.6 | 75.0 |
|
47 |
+
|
48 |
## 🧩 Configuration
|
49 |
|
50 |
```yaml
|