Edit model card

You need to agree to share your contact information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

You agree not to use the model for healthcare decision-making or commercial use

Log in or Sign Up to review the conditions and access this model content.

Clinical Camel

Model Description

Clinical Camel is an open large language model (LLM), fine-tuned on the LLaMA-2 70B architecture using QLoRA. It is tailored for the medical and clinical research, capable of processing and generating relevant content.

Review our pre-print for more details: Clinical Camel - Pre-print

Performance

Clinical Camel demonstrates competitive performance on medical benchmarks.

Table: Five-Shot Performance of Clinical Camel-70B (C70), GPT3.5, GPT4, and Med-PaLM 2 on Various Medical Datasets

Dataset ClinicalCamel-70B GPT3.5 GPT4 Med-PaLM 2
MMLU Anatomy 65.2 60.7 80.0 77.8
MMLU Clinical Knowledge 72.8 68.7 86.4 88.3
MMLU College Biology 81.2 72.9 93.8 94.4
MMLU College Medicine 68.2 63.6 76.3 80.9
MMLU Medical Genetics 69.0 68.0 92.0 90.0
MMLU Professional Medicine 75.0 69.8 93.8 95.2
MedMCQA 54.2 51.0 72.4 71.3
MedQA (USMLE) 60.7 53.6 81.4 79.7
PubMedQA 77.9 60.2 74.4 79.2
USMLE Sample Exam 64.3 58.5 86.6 -

Evaluation Datasets:

The performance of Clinical Camel was benchmarked across several datasets, including:

Evaluation Reproduction:

To reproduce the evaluations with lm-evaluation-harness see the 'TaskFiles' folder

Downloads last month
123

Spaces using wanglab/ClinicalCamel-70B 3