Automatic Extraction of Disease Risk Factors from Medical Publications (2024)

Abstract: We present a novel approach to automating the identification of risk factors for diseases from medical literature, leveraging pre-trained models in the bio-medical domain, while tuning them for the specific task. Faced with the challenges of the diverse and unstructured nature of medical articles, our study introduces a multi-step system to first identify relevant articles, then classify them based on the presence of risk factor discussions and, finally, extract specific risk factor information for a disease through a question-answering model. Our contributions include the development of a comprehensive pipeline for the automated extraction of risk factors and the compilation of several datasets, which can serve as valuable resources for further research in this area. These datasets encompass a wide range of diseases, as well as their associated risk factors, meticulously identified and validated through a fine-grained evaluation scheme. We conducted both automatic and thorough manual evaluation, demonstrating encouraging results. We also highlight the importance of improving models and expanding dataset comprehensiveness to keep pace with the rapidly evolving field of medical research.

## introduction automatic identification of risk factors for diseases plays a pivotal role in preventive medicine, enabling healthcare professionals to formulate effective prevention strategies and improve patient outcomes. traditionally, this process has relied heavily on manual review of extensive medical literature, a time-consuming and labor-intensive task, hindering knowledge accessibility and effective usage. as a concrete example, recently, compelling evidence has emerged linking lipoprotein a (lp(a)) -a particle operating similarly to the more familiar ldl molecule -to the pathogenesis of atherosclerosis and subsequent coronary artery disease, commonly referred to as myocardial infarction (mi). despite the established role of lp(a) as a risk factor @xcite , many primary care clinicians remain inadequately informed, occasionally lacking knowledge regarding its testing procedures. moreover, in a conversation with a board-certified professor of interventional cardiology, he disclosed receiving frequent inquiries from other clinicians questioning the necessity of referrals for lp(a) testing. this highlights the pressing need for an automated tool capable of screening vast amounts of scientific literature and identifying prominent risk factors for various diseases. despite significant advances in the field of natural language processing, automatic extraction of disease risk factors from scientific medical literature remains a challenging endeavor. contrary to the analysis of electronic health records @xcite @xcite , here the primary challenge lies in the diverse and unstructured nature of medical publications, where risk factors are described in various contexts and formats. what is more, the continuous discovery of new risk factors necessitates a dynamic approach that can adapt to the evolving body of medical knowledge. this study introduces a novel approach to automating the identification of disease risk factors from medical literature. utilizing pre-trained large language models, based on biobert @xcite , we developed a multi-step system, that first identifies relevant medical articles, classifies them based on the presence of risk factor discussions, and then extracts specific risk factor information through a question answering (qa) model. our approach to extraction of disease risk factors is illustrated in figure 1: (1) medical abstracts are retrieved from pubmed, (2) a specifically fine-tuned binary classifier is used to identify abstracts with risk factors information, and (3) textual spans containing risk factors are extracted via a question answering model, fine-tuned on manually annotated qa items. the contribution of this work, therefore, twofold: first, we present a comprehensive pipeline for automated extraction of risk factors. second, we compile and make available several datasets that can serve as valuable resources for future research in this field. these datasets include a carefully annotated, large and diverse set of over 1,700 risk factors associated with 15 diseases, as well as set of over 160,000 automatically extracted risk factors, foot_0 with almost 1,500 manually assessed for their quality, using a fine-grained annotation scheme. 2 we survey the related work in section 2 and detail on collection and annotation of our datasets in section 3. we next describe our approach to the task and report experimental results in section 4. human evaluation results are presented in section 5. discussion of the difficulty of the task and the limitations of this work are presented in section 6. we conclude this study in section 7. ## related work automatic identification of disease risk factors through the analysis of medical texts has garnered interest across various research domains, particularly in applying natural language processing and machine learning techniques to electronic health records (ehrs) and electronic medical records (emrs). here we review key contributions in this area, highlighting approaches that parallel and diverge from our focus on free-text medical articles. @xcite explore the utilization of deep learning models to extract heart disease risk factors from ehrs. the approach, grounded in analyzing structured data within ehrs, contrasts with our exploration of unstructured text in medical literature, underscoring the diversity in data sources for risk factor identification. @xcite attempt at mining clinical texts for risk factor identification using association rules. specifically, they handle data in xml format from the diabetes register, indicating a structured approach to data analysis. this work differs from ours in terms of both data source type (clinical narratives), as well as in our broader application to unstructured, free-text medical articles and the use of pre-trained large language models (llms) for the task of text understanding. a comprehensive work on identifying risk factors for heart disease (from clinical data) over time was done in a shared task organized by uthealth foot_2 @xcite . @xcite offer an overview of nlp applications in analyzing clinical notes for chronic disease management, highlighting the increasingly significant contribution of language models to healthcare applications. in the domain of precision medicine, @xcite focus on extracting semantic information and assessing sentiments in clinical notes. various works have employed data mining and machine learning (ml) techniques for identifying risk factors from patient data @xcite , or clinical outcome prediction @xcite @xcite . recently, the identification of risk factors for delirium prediction, a rare adverse reaction observed in covid-19 patients, was developed utilizing ml applied to nursing records @xcite . additional line of studies focuses on building language models specifically-tailored for medical literature related tasks @xcite @xcite . several significant contributions have been made in the field of biomedical relation extraction, which includes identifying factors that predispose individuals to diseases. the semrep @xcite tool extracts semantic predications from biomedical texts, including relationships such as "predisposes". the outputs of semrep have been used to create semmeddb @xcite , a large-scale repository of semantic predications from pubmed. building on these resources, bio-prep @xcite employs deep learning techniques for predicate classification. the biored @xcite dataset includes a "positive correlation" relation between diseases and other biomedical entities like genes and chemicals. conclusion while the majority of existing research focuses on analyzing structured electronic health records and electronic medical records to identify disease risk factors, our study pushes beyond these confines by examining free-text medical literature. processing unstructured medical text introduces distinct challenges, especially due to language complexity, variation, and the potential for nuanced double meanings, and even worse, due to the necessity to discern context accurately. consequently, it opens up expansive opportunities for subtle understandings of disease risk factors, facilitating both research and practical applications. ## dataset data collection process for this work can be viewed as a three-step process: (i) collection of the set of disease names spanning multiple disease families, (ii) manual annotation of scientific article abstracts containing explicit mention of risk factors of a subset of diseases -"abstracts seed", and (iii) manual annotation of risk factors description (span) in abstract texts found in (ii) -"risk factors seed". we detail on each step in this multi-phase procedure. ## methodology and experiments we further describe in detail our methodological approach, experimental setup and results. ## human evaluation we next manually evaluated a random sample of 1,485 extracted risk factors spanning 29 various diseases (constituting roughly 1% of the full set of extracted factors), based on their validity and relevance to the disease in question. ## discussion and limitations our study, while contributing valuable insights into the automation of risk factor identification from medical publications, is subject to several limitations that merit a thorough discussion. one of the primary limitations is the challenge of accurately distinguishing risk factors specifically associated with the disease in question (type 1) from valid risk factors that are not directly related to the disease under investigation (type 2). while our models demonstrated a high capacity for identifying potential risk factors, the precision in contextualizing these factors to specific diseases varied. this aspect highlights a critical area for future research, emphasizing the need for enhanced specificity in the models to improve their utility in targeted medical research and practice. moreover, the study's reliance on free-text medical articles introduces variability in the data quality and representation. the unstructured nature of these texts and the diversity in how risk factors are described pose significant challenges for both the binary classification and question-answering models. efforts to standardize data representation and improve model robustness against such variability are essential steps forward. the datasets used in this study, while extensive, are not exhaustive. the landscape of medical research is continuously evolving, with new findings emerging regularly. the datasets, therefore, rep-disease abstract excerpt (identified risk factor highlighted) marker chronic myeloid leukemia [...] results: previous diagnoses of dyspepsia, gastritis or peptic ulcers, as well as previous proton pump inhibitor (ppi) medication, were all associated with a significantly increased risk of cml (rrs, 1.5-2.0; @xmath0.0005-0.05). meanwhile, neither inflammatory bowel disease nor intake of nsaids were associated with cml, indicating that it is not gastrointestinal ulcer or inflammation per se that influences risk. [...] 1 cystic fibrosis background: cystic fibrosis, like other chronic diseases, is a risk factor for the development of elevated symptoms of depression and anxiety. [...] patient anxiety (or 2.33) and depression (or 4.09) were significantly associated with forced expiratory volume in one second (fev1) <40% and forced vital capacity (fvc) <80% (or 1.60 and 1.61, respectively). conclusions: cystic fibrosis increases the risk of developing anxiety and depression in female patients and in mothers. finally, the study's scope was constrained by the computational resources available. future work could explore more complex models or ensemble approaches that might offer improved accuracy but require more substantial computational power. despite these limitations, this study represents a significant step toward automating the identification of disease risk factors from medical literature. acknowledging and addressing these limitations in future research will be crucial for advancing the field and enhancing the practical applicability of these technologies in healthcare. ## conclusions and future work this study presented an approach to identifying and extracting disease risk factors from free-text medical articles using advanced natural language processing techniques, specifically leveraging the capabilities of the pre-trained biobert-based architecture. our methodology involved a multi-step process, including the retrieval of relevant articles, binary classification to filter articles discussing risk factors, and a question-answering model to extract specific risk factor information. we have demonstrated the potential of language technologies to significantly enhance the efficiency and effectiveness of risk factor identification in medical literature. our contributions to this field are twofold: the presentation of an automated pipeline for risk factor extraction and the creation of valuable datasets for future research. while our study marks an advancement in the automated extraction of risk factors from medical literature, there remain several avenues for future research and development. our future directions include introducing improvements to qa model's accuracy and specificity, integration of additional data sources, and evaluation of more advanced llms for the task of risk factors identification. furthermore, inspired by recent findings that automatic annotations generated by models like gpt-4 can achieve results comparable to human annotations, we plan to investigate the use of gpt-4 for the task of risk factors annotation, and compare its performance with human experts.