Natural Language Processing in Support of Evidence-based Medicine: A Scoping Review (2025)

Abstract: Evidence-based medicine (EBM) is at the forefront of modern healthcare, emphasizing the use of the best available scientific evidence to guide clinical decisions. Due to the sheer volume and rapid growth of medical literature and the high cost of curation, there is a critical need to investigate Natural Language Processing (NLP) methods to identify, appraise, synthesize, summarize, and disseminate evidence in EBM. This survey presents an in-depth review of 129 research studies on leveraging NLP for EBM, illustrating its pivotal role in enhancing clinical decision-making processes. The paper systematically explores how NLP supports the five fundamental steps of EBM—Ask, Acquire, Appraise, Apply, and Assess. The review not only identifies current limitations within the field but also proposes directions for future research, emphasizing the potential for NLP to revolutionize EBM by refining evidence extraction, evidence synthesis, appraisal, summarization, enhancing data comprehensibility, and facilitating a more efficient clinical workflow.

## introduction evidence-based medicine (ebm) is at the forefront of modern healthcare, emphasizing the use of the best available scientific evidence to guide clinical decisions @xcite . by integrating clinical expertise, patient values, and the most up-todate research data, ebm facilitates healthcare decisions by patients and the general public, clinicians, guideline developers, administrators, and policymakers @xcite @xcite . the foundation of ebm heavily relies on comprehensive research data from detailed textual sources such as clinical trial publications, cohort studies, and case reports @xcite . navigating this evidence hierarchy necessitates the use of advanced natural language processing (nlp) techniques, which are crucial for streamlining literature searches and extracting pico (patient/population, intervention, comparison, outcomes) elements @xcite . from the early utilization of statistical machine learning @xcite and recurrent neural networks @xcite , there has been a significant shift towards more advanced technologies such as transformer-based frameworks and large language models (llms). these modern approaches employ self-supervised pretraining and instruct-tuning @xcite to capture domain-specific knowledge @xcite , enhancing the accuracy and scalability of medical information processing @xcite . particularly, the recent advancements in llms have further propelled nlp capabilities within ebm, excelling in more complex tasks such as appraising and synthesizing evidence @xcite , differentiating and ranking evidence @xcite , generating humanlike responses, answering complex clinical questions @xcite , and identifying relevant clinical trials @xcite . despite these significant advancements, a comprehensive review summarizing nlp development and applications in ebm is still in demand. this paper seeks to fill the gap by offering a thorough review of essential nlp tasks in ebm, with a focus on evidence generation, such as evidence retrieval, extraction, synthesis, and summarization, as well as evidence adoption and evidence-based research, such as question-answering, clinical trial design and identification, and other cutting-edge studies across various clinical specialties. furthermore, we outline key benchmarks to facilitate the development of future nlp models. finally, we explore several potential avenues for future research. to better support both clinicians and researchers in making more informed clinical decisions and producing more comprehensive review literature, we have made these resources publicly available. 1 2 scope and literature selection our scoping review adheres to the preferred reporting items for systematic reviews and meta-analyses (prisma foot_1 ) guidelines, as illustrated in figure 1 . ## nlp techniques for ebm the entire ebm process consists of five steps, commonly referred to as the '5a's: ask, acquire, appraise, apply, and assess @xcite . nlp can be leveraged at each step to enhance the process ( ## ask -searching & selecting studies ebm can help researchers and clinicians draft a successful systematic review. after the scope and questions have been determined, the first step is to search for studies to include in the reviews and ensure they remain up to date. this step is typically achieved using nlp-based information retrieval techniques, which extract relevant information from large text corpora based on user queries. early heuristic methods involved structured, keyword-based queries to retrieve articles from repositories like medline or pubmed. these methods, while foundational, are limited by the high cost of expert annotation, maintenance, and domain sensitivity @xcite . despite these limitations, recent methods often rely on predefined rule-based strategies, e.g., sr[pt] and cqrs (navarro-ruan and haynes, 2022), to filter and compare the retrieved results for systematic reviews. in addition, while statistical machine learning and context-aware models @xcite have been widely adopted, they often lack scalability and struggle with less representative text embeddings. recent advancements are leaning towards transformer-based deep learning frameworks @xcite due to their scalability and the ability to integrate medical ontologies, improving domain-specific text representation through self-supervised pretraining. for example, @xcite ) used biobert's @xcite embeddings and attention mechanisms to improve query representation and biomedical literature retrieval in clinical practice. furthermore, the integration of generative ai models has advanced literature retrieval despite challenges like hallucination. for example, @xcite compared microsoft bing ai and chatgpt in accelerating the systematic literature search for a clinical review on peyronie disease treatment, finding both can speed up the search process. ## acquire -collecting data ebm is designed to identify all studies relevant to their research questions and synthesize data regarding the study design, risk of bias, and results. therefore, the findings of ebm heavily depend on decisions about which data from these studies are presented and analyzed. the data collected should be accurate, complete, and accessible for future review, updates, and data-sharing purposes. here we describe nlp approaches used to extract data directly from journal articles and other studies' reports. ## appraise, synthesize, and summarize evidence this task screens the included studies for risk of bias and appraises them for quality to ensure that healthcare decisions are informed by the most reliable and relevant evidence. once the appraisal is complete, the next step is synthesizing evidence by combining findings from multiple studies, often using meta-analyses. finally, these synthesized insights are summarized into concise, actionable conclusions. ## apply and assess: adoption, refinement, and research transitioning from the evidence generation and synthesis, the next critical step is its adoption and refinement, facilitated by an 'evidence-based research' approach. adoption and refinement are crucial to consistently reassessing and enhancing clinical evidence, particularly when existing evidence gaps lead to unmet needs of clinicians and patients. evidence-based research further ensures that these gaps inform future clinical studies. here, we summarize several applications identified from our literature review that align with this topic. ## ebm benchmark dataset here, we summarize the benchmarks used in nlp and ebm (supplementary the existing datasets are often built upon general texts rather than medical-specific content. for example, cnn-dailymail @xcite is used for evidence summarization, but it is not medicalrelated. we also noticed that the primary data sources for these benchmarks are scholarly articles from pubmed and clinical trials. ## challenges and future directions ebm is an important, rewarding, and dynamic field that organizes current data to improve healthcare decision-making. by integrating the best available evidence with a healthcare professional's experience and the patient's values, ebm aims to optimize health outcomes. our focus here is on retrieving, extracting, appraising, synthesizing, and summarizing evidence from biomedical literature such as clinical trials, cohort studies, and case reports. however, conducting these analyses can be both demanding and time-consuming. in this study, we explore key nlp techniques that can streamline and facilitate this process. our review indicates that nlp-based systems or pipelines have achieved impressive results in ebm, such as extracting entities like pico, enhancing the information retrieval engines, automating the evidence synthesis, assessing evidence quality, ranking the evidence with the highest confidence, summarizing the information, and answering questions. at the same time, as in any other evolving area, there remain challenges ahead. for example, generative models in ebm tasks have demonstrated impressive fluency and scalability, yet their tendency to hallucinate facts, lack source attribution, and sensitivity to prompt phrasing remain significant limitations for clinical use. a core challenge is the validation and trustworthiness of generated outputs, especially in high-stakes domains like medicine. mechanisms such as rag offer potential mitigations but require further development and evaluation. from another perspective, particularly in handling diseases with limited literature or annotated data @xcite . few-shot learning holds significant potential, as it enables models to generalize effectively from a small number of examples, reducing the dependency on large, annotated datasets. this data-efficient approach is crucial for ebm tasks in under-researched areas, such as rare diseases, where annotated resources are scarce. fewshot learning can help these models adapt quickly to specific clinical needs, allowing for more accurate information extraction, question answering, and evidence synthesis, even with minimal training data. additionally, there is a pressing need for more benchmark datasets, especially for evidence synthesis and appraisal and question answering. current resources often rely on general corpora rather than those specifically oriented toward medical content, limiting the development of specialized nlp applications. researchers can consider and build more meaningful datasets. moreover, nlp-based tools have not yet been widely applied across all medical specialties, such as urology and hepatology, indicating room for expansion in these areas. another future direction for nlp in ebm involves incorporating real-world data from various sources, such as mobile devices, social media, and genomics. these data sources capture rich and diverse information beyond traditional clinical records, offering valuable insights into patient behaviors, lifestyle, environmental factors, and genetic predispositions. for example, data from mobile health apps and wearable devices can provide real-time health metrics. at the same time, social media posts may reveal patient self-reported outcomes or experiences that are often missed in clinical settings. integrating genomic data adds another layer, enabling family history and personalized genomic code into disease risk and treatment response. furthermore, the "black box" nature of many nlp models limits their interpretability and accountability. biases within training data can restrict nlp's effectiveness and fairness across diverse patient demographics. additionally, the high computational demands and the need for domain expertise in both nlp and healthcare are resource-intensive. to fully realize the potential of nlp for ebm in real-world clinical workflows often involve interdisciplinary scenarios that span multiple conditions, comorbidities, and patient subpopulations. to address these complexities, nlp systems for ebm must evolve toward more holistic, adaptable frameworks capable of reasoning across diverse clinical questions and integrating heterogeneous data sources. addressing these limitations is important for enabling efficiency and ultimately contributing to a safer, more equitable healthcare landscape. ## conclusion our comprehensive review of over 600 papers resulted in the selection of 129 studies that focus on critical aspects of nlp within ebm. we first provide an overview of ebm, followed by a survey of nlp methods and techniques that address each step of the ebm process. we also explore use cases that demonstrate the application of ebm in various scenarios. additionally, we review popular datasets and benchmarks. finally, we present open challenges and future directions for research in this field. as nlp technologies evolve, they offer promising prospects for harnessing vast amounts of unstructured data, thus supporting clinical and research applications.