Update README.md
Browse files
README.md
CHANGED
|
@@ -3,7 +3,6 @@ license: apache-2.0
|
|
| 3 |
tags:
|
| 4 |
- merge
|
| 5 |
- mergekit
|
| 6 |
-
- wanglab/ClinicalCamel-70B
|
| 7 |
- epfl-llm/meditron-70b
|
| 8 |
- allenai/tulu-2-dpo-70b
|
| 9 |
---
|
|
@@ -17,11 +16,32 @@ Medmerge-tulu-70b is a merge of the following models:
|
|
| 17 |
|
| 18 |
# Open LLM Leaderboard
|
| 19 |
|
|
|
|
|
|
|
| 20 |
| Model Name | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
|
| 21 |
| -------------------- | -------- | --------- | ------ | ---------- | ---------- | -------- |
|
| 22 |
| tulu-2-dpo-70b | 72.1 | 88.99 | 69.84 | 65.78 | 83.27 | 62.62 |
|
| 23 |
| Medmerge-tulu-70b | 67.81 | 87.46 | 70.1 | 47.89 | 83.43 | 56.56 |
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
## 🧩 Configuration
|
| 26 |
|
| 27 |
```yaml
|
|
|
|
| 3 |
tags:
|
| 4 |
- merge
|
| 5 |
- mergekit
|
|
|
|
| 6 |
- epfl-llm/meditron-70b
|
| 7 |
- allenai/tulu-2-dpo-70b
|
| 8 |
---
|
|
|
|
| 16 |
|
| 17 |
# Open LLM Leaderboard
|
| 18 |
|
| 19 |
+
|
| 20 |
+
|
| 21 |
| Model Name | ARC | HellaSwag | MMLU | TruthfulQA | Winogrande | GSM8K |
|
| 22 |
| -------------------- | -------- | --------- | ------ | ---------- | ---------- | -------- |
|
| 23 |
| tulu-2-dpo-70b | 72.1 | 88.99 | 69.84 | 65.78 | 83.27 | 62.62 |
|
| 24 |
| Medmerge-tulu-70b | 67.81 | 87.46 | 70.1 | 47.89 | 83.43 | 56.56 |
|
| 25 |
|
| 26 |
+
## Performance
|
| 27 |
+
|
| 28 |
+
Clinical Camel demonstrates competitive performance on medical benchmarks.
|
| 29 |
+
|
| 30 |
+
**Table: Five-Shot Performance of Clinical Camel-70B (C70), GPT3.5, GPT4, and Med-PaLM 2 on Various Medical Datasets**
|
| 31 |
+
|
| 32 |
+
| Dataset | Medmerge-tulu-70b | ClinicalCamel-70B | GPT3.5 | GPT4 | Med-PaLM 2 |
|
| 33 |
+
|-----------------------------|-------------------|-------------------|--------|-------|--------------|
|
| 34 |
+
| MMLU Anatomy | 66.6 | 65.2 | 60.7 | 80.0 | 77.8 |
|
| 35 |
+
| MMLU Clinical Knowledge | 72.0 | 72.8 | 68.7 | 86.4 | 88.3 |
|
| 36 |
+
| MMLU College Biology | 84.7 | 81.2 | 72.9 | 93.8 | 94.4 |
|
| 37 |
+
| MMLU College Medicine | 64.2 | 68.2 | 63.6 | 76.3 | 80.9 |
|
| 38 |
+
| MMLU Medical Genetics | 76.0 | 69.0 | 68.0 | 92.0 | 90.0 |
|
| 39 |
+
| MMLU Professional Medicine | 75.7 | 75.0 | 69.8 | 93.8 | 95.2 |
|
| 40 |
+
| MedMCQA | | 54.2 | 51.0 | 72.4 | 71.3 |
|
| 41 |
+
| MedQA (USMLE) | | 60.7 | 53.6 | 81.4 | 79.7 |
|
| 42 |
+
| PubMedQA | | 77.9 | 60.2 | 74.4 | 79.2 |
|
| 43 |
+
| USMLE Sample Exam | | 64.3 | 58.5 | 86.6 | - |
|
| 44 |
+
|
| 45 |
## 🧩 Configuration
|
| 46 |
|
| 47 |
```yaml
|