Spaces:
Restarting
on
CPU Upgrade
Restarting
on
CPU Upgrade
College biology isn't medicine
#4
by
maximegmd
- opened
No description provided.
maximegmd
changed pull request title from
College biology isn't medecine
to College biology isn't medicine
Hi @maximegmd , we also consider biology as part of the evaluation as it "evaluates" biomedical knowledge. We would like to evaluate the biomedical knowledge of the LLMs, not only medicine/clinical. Furthermore, MultiMedQA suggested by the Med-PaLM paper (Singhal et al., 2023) also includes MMLU college biology.
Let us know if you disagree, we would love to hear your take!
Hello,
I do not believe it evaluates biomedical knowledge. I went through the questions and the questions are mostly irrelevant (questions about animal behavior, geology, plants...).
While multimedqa did include it, they are in my opinion mistaken and it shows that involving medical doctors in these decisions is necessary.