abhinand's picture
Upload folder using huggingface_hub
301f600 verified
metadata
base_model:
  - aaditya/Llama3-OpenBioLLM-70B
library_name: transformers
tags:
  - mergekit
  - merge
license: llama3
language:
  - en

Llama-3-Galen-70B-v1

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the DARE TIES merge method using aaditya/Llama3-OpenBioLLM-70B as a base.

Evaluation

Tasks Version Filter n-shot Metric Value Stderr
pubmedqa 1 none 0 acc 0.7820 ± 0.0185
professional_medicine 0 none 0 acc 0.9375 ± 0.0147
medical_genetics 0 none 0 acc 0.9300 ± 0.0256
college_medicine 0 none 0 acc 0.8555 ± 0.0268
college_biology 0 none 0 acc 0.9375 ± 0.0202
clinical_knowledge 0 none 0 acc 0.9283 ± 0.0159
anatomy 0 none 0 acc 0.8444 ± 0.0313
medqa_4options Yaml none 0 acc 0.7777 ± 0.0117
none 0 acc_norm 0.7777 ± 0.0117
medmcqa Yaml none 0 acc 0.7423 ± 0.0068
none 0 acc_norm 0.7423 ± 0.0068

Average: 0.8594