File size: 1,665 Bytes
301f600 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 |
---
base_model:
- aaditya/Llama3-OpenBioLLM-70B
library_name: transformers
tags:
- mergekit
- merge
license: llama3
language:
- en
---
# Llama-3-Galen-70B-v1
<img src="https://cdn-uploads.huggingface.co/production/uploads/60c8619d95d852a24572b025/R73wGdZE3GWeF9QZPvruG.jpeg" width="600" />
This is a merge of pre-trained language models created using [mergekit](https://github.com/cg123/mergekit).
## Merge Details
### Merge Method
This model was merged using the [DARE](https://arxiv.org/abs/2311.03099) [TIES](https://arxiv.org/abs/2306.01708) merge method using [aaditya/Llama3-OpenBioLLM-70B](https://huggingface.co/aaditya/Llama3-OpenBioLLM-70B) as a base.
### Evaluation
| Tasks |Version|Filter|n-shot| Metric |Value | |Stderr|
|---------------------|-------|------|-----:|--------|-----:|---|-----:|
|pubmedqa | 1|none | 0|acc |0.7820|± |0.0185|
|professional_medicine| 0|none | 0|acc |0.9375|± |0.0147|
|medical_genetics | 0|none | 0|acc |0.9300|± |0.0256|
|college_medicine | 0|none | 0|acc |0.8555|± |0.0268|
|college_biology | 0|none | 0|acc |0.9375|± |0.0202|
|clinical_knowledge | 0|none | 0|acc |0.9283|± |0.0159|
|anatomy | 0|none | 0|acc |0.8444|± |0.0313|
|medqa_4options |Yaml |none | 0|acc |0.7777|± |0.0117|
| | |none | 0|acc_norm|0.7777|± |0.0117|
|medmcqa |Yaml |none | 0|acc |0.7423|± |0.0068|
| | |none | 0|acc_norm|0.7423|± |0.0068|
**Average:** 0.8594
|