M ed REQAL : Examining Medical Knowledge Recall of Large Language Models via Question Answering (2024)

Abstract: In recent years, Large Language Models (LLMs) have demonstrated an impressive ability to encode knowledge during pre-training on large text corpora. They can leverage this knowledge for downstream tasks like question answering (QA), even in complex areas involving health topics. Considering their high potential for facilitating clinical work in the future, understanding the quality of encoded medical knowledge and its recall in LLMs is an important step forward. In this study, we examine the capability of LLMs to exhibit medical knowledge recall by constructing a novel dataset derived from systematic reviews – studies synthesizing evidence-based answers for specific medical questions. Through experiments on the new MedREQAL dataset, comprising question-answer pairs extracted from rigorous systematic reviews, we assess six LLMs, such as GPT and Mixtral, analyzing their classification and generation performance. Our experimental insights into LLM performance on the novel biomedical QA dataset reveal the still challenging nature of this task.

## introduction the field of natural language processing (nlp) has been transformed with the advent of pre-trained large language models (llms). during their process of pre-training to predict the next token on massive amounts of text data, these models learn and construct an internalized representation of world knowledge @xcite . a popular domain of application of llms is healthcare, where they have the potential to democratize medical knowledge and facilitate access to healthcare, but also introduce risks of misinformation and lack of transparency @xcite . recent work has hinted at the fact that llms encode clinical knowledge rather well @xcite . in the medical world, systematic reviews are overview studies that synthesize the best available studies on a clearly defined medical research question. in them, the studies on a topic are analyzed, critically appraised, and their interpretations summarized by experts into a refined evidence-based conclusion @xcite . considering they synthesize the best available knowledge for a medical question into a concise answer, we see systematic reviews as a very well-suited proxy for testing the level of knowledge recall in llms. when prompted in a zero-shot setting, instruction-tuned llms also tend to generate a conclusion addressing the question based on their recalled knowledge from diverse sources. to examine how well the llms perform medical knowledge recall, we constructed a question-answering (qa) dataset originating from systematic reviews and evaluated the performance of six llms, probing their medical knowledge recall. based on experiments conducted on the newly constructed medreqal dataset, we analyze both the classification and generation performance, discussing insights about individual capabilities and limitations. our contributions include: (1) a novel dataset of biomedical question-answer pairs originating from rigorous systematic reviews, (2) experiments testing the zero-shot medical knowledge recall of six llms, and (3) deeper analysis of the dataset and experimental results. to ensure reproducibility, we provide the dataset and code in a public repository. foot_0 2 related work medicine is a common domain of application for nlp tasks @xcite . it is marked by the highly complex biomedical language and terminology. previous studies have highlighted the potential of llms in medical knowledge recall and exposed key gaps indicating the importance of further method development for creating safe and effective llms for health-related applications @xcite . biomedical question answering (bqa) can be split into four main categories: scientific, clinical, consumer, and examination @xcite . our dataset best fits into the scientific category, where questions and answers originate directly from biomedical research publications, and the most well-known datasets are pubmedqa @xcite and bioasq @xcite . while the usual qa setting works with provided documents, recent efforts have started advancing the open-domain qa setting, where evidence first has to be discovered in order to answer the question @xcite @xcite . also related is the task of scientific factchecking, focusing on assessing the veracity of claims based on relevant scientific evidence @xcite . the most similar dataset to medreqal is healthfc @xcite , which also uses systematic reviews as its evidence source for answers but focuses on popular health inquiries and retells the reviews in lay language. systematic reviews are most commonly associated in nlp with the task of their automatic construction @xcite , which is a highly complex task eliciting advanced biomedical knowledge and refined reasoning skills. to the best of our knowledge, we introduce the first bqa dataset directly generated from systematic reviews, all adhering rigorously to a predefined research protocol. the focus on important healthcare interventions, wide domain coverage, standardized structure, and rigorous process followed by medical experts to construct these reviews are all factors that made them a highly suitable source for the construction of a novel biomedical question-answering dataset. ## experimental setup we conduct our experiments using a variety of large language models. the models were instructed to answer the question (thus producing a long answer) and also predict a final label (one of the s/r/nei). for evaluation, predicted labels were collected and evaluated using classification metrics accuracy and macro-averaged f1 score, while the long answers were compared to the ground truth authors' conclusions with nlg evaluation metric rouge @xcite . since this metric focuses on lexical overlaps, we also use the bertscore metric, which captures semantic similarity @xcite . we chose three general-purpose llms and three biomedical llms. as a commercial state-of-theart llm, we include gpt-4 (turbo-0125) (ope-nai, 2023) in our comparison since it has demonstrated remarkable zero-shot performance on various nlp tasks. consequently, it is often used as a benchmark for comparing llms. we also include two recent open-source models that achieved impressive performance, namely the mistral-7b (instruct-v2) (jiang et al., 2023); and mixtral (jiang et al., 2024), based on a sparse mixture-ofexperts architecture @xcite . we found three domain-specific biomedical models that fit our use case: (1) pmc-llama 13b @xcite , which is an extension of llama @xcite additionally pre-trained on biomedical research articles from the database pubmed @xcite and later finetuned on various datasets; (2) medalpaca 7b @xcite , a biomedical extension of alpaca (instruction-tuned version of llama, @xcite fine-tuned on medical texts, encompassing resources such as medical flashcards, wikis, and dialogue datasets; and (3) chatdoctor (7b) @xcite , another extension of llama finetuned on patient-physician conversation datasets. since the models were instruction-tuned with different templates, we used for each of them the base prompt reported by authors in technical reports. the prompt additionally instructed the models to predict one of the three classes and write an explanation for its answer (which we deem the "long answer"). all prompts can be found in all the models were tested only in a zero-shot setting with just the question provided. providing the full text of the abstract would make the experiments focus on machine reading comprehension, which is not the goal of our study. we are interested in medical knowledge recall, which is best evaluated with a zero-shot setup. initially, we also experimented with few-shot learning, but it only slightly helped in learning the formatting of answers and did not affect f1 performance since answers are still dependent on internal knowledge. ## results and discussion the performance metrics and results are summarized in the difference in final classification results is apparent in to predict this label frequently. on the other hand, gpt-4 was very prone to answering questions affirmatively and saying they are supported by the latest medical research. conversely, mixtral was more reluctant to give definitive affirmative or negative verdicts, but correctly predicted in most cases that there is not enough reliable evidence to answer the question. this is demonstrated in an example in among the three biomedical llms, chatdoctor had by far the worst performance and resorted to answering almost every question affirmatively (we observed this even when accounting for different values of hyperparameters temperature or top p). interestingly enough, it was also the model with the highest bertscore value of its generated text, showing that its fine-tuning on patient-doctor conversations most likely led to more authentic phrasing and word choices when delivering medical information. on the opposite side of the spectrum, pmc-llama was the model that had by far the most refuted predictions (which were basically nonexistent for gpt-4 and mixtral). question do interventions to improve appropriate polypharmacy result in clinically significant improvement? conclusion it is unclear whether interventions to improve appropriate polypharmacy resulted in clinically significant improvement. (cole et al., 2023) [label: not enough info] ## conclusion we constructed a new biomedical qa dataset for testing the medical knowledge recall of llms. the dataset originates from systematic reviews, synthesized evidence-based studies on research questions, a well-suited proxy for knowledge recall probing. we tested the performance of three general llms and three biomedical llms, showing that the scale of the model or its domain-tuning is not always tied to better performance. deeper insights show that models have a moderately high level of recall and awareness of systematic reviews as quality evidence but still struggle with decisively concluding when there is not enough evidence to answer the question, leaving space for future improvements.