Accelerating the Discovery of Semantic Associations from Medical Literature: Mining Relations Between Diseases and Symptoms (2022)

Abstract: Medical literature is a vast and constantly expanding source of information about diseases, their diagnoses and treatments. One of the ways to extract insights from this type of data is through mining association rules between such entities. However, existing solutions do not take into account the semantics of sentences from which entity co-occurrences are extracted. We propose a scalable solution for the automated discovery of semantic associations between different entities such as diseases and their symptoms. Our approach employs the UMLS semantic network and a binary relation classification model trained with distant supervision to validate and help ranking the most likely entity associations pairs extracted with frequency-based association rule mining algorithms. We evaluate the proposed system on the task of extracting disease-symptom associations from a collection of over 14M PubMed abstracts and validate our results against a publicly available known list of disease-symptom pairs.

## introduction scientific literature is a valuable resource for accelerating scientific discovery in several fields, from computer science to physics, and healthcare (kumar and tipney, 2014). however, the overwhelming amount of articles that need to be inspected requires extensive computational approaches as well as modeling knowledge in an appropriate machine readable form. many efforts have been recently tackling the problem of transforming the unstructured knowledge from scientific papers to knowledge graphs that enable the extraction of actionable insights @xcite @xcite . due to the phenomenal growth of pubmed and medline publications @xcite , the medical domain would particularly benefit from the creation of comprehensive knowledge graphs. extracting relations between entities is particularly valuable in the medical and biomedical domains where scientists need to extract semantic relations between medical concepts, such as protein and protein, gene and protein, drug and drug, and drug and disease. these relations can be extracted from biomedical literature available from various sources and have already been made accessible in different databases such as biogrid @xcite or pdid @xcite . the extraction of these associations from biomedical literature, however, is often time consuming and computationally expensive. hence, these databases become quickly outdated if they are not updated at the same rate as new scientific discoveries are published. in fact, to the best of our knowledge, the usage of fully automated tools for the extraction of information from these sources is very limited. in this paper, we propose an efficient end-to-end pipeline for the extraction of semantic relations between medical concepts. we evaluate our approach on disease-symptom associations discovery, but it can also be applied to other relation types and use cases. while disease-symptom information is widely published in medical bibliography, mining such information from literature, electronic health records, or even from user generated content, can accelerate the detection of new symptoms, diseases or variants. early detection is particularly important in public health surveillance, both in detecting new pandemics (as was the case for covid-19), identifying new symptoms associated with known diseases (also seen in covid-19), or even for detecting the resurgence of disease outbreaks in certain countries -as was the case for the ebola outbreak of 2014 in west africa. being able to recognize relations between medical concepts also means that biomedical or clinical texts can be automatically processed at scale, resulting in tools to support decision-making, clinical trial screening, and pharmacovigilance @xcite . the proposed pipeline operates similarly to a search engine and can therefore return up-to-date information as its database of documents can be updated in near real time. our main contributions are: • a scalable and easily updatable elasticsearchbased solution to store and query annotated medical literature documents; • an efficient association rules discovery system based on (i) an association rule mining algorithm, (ii) umls semantic network information and (iii) a binary relation classification model trained with distant supervision; • a solution for human-in-the-loop verification and interpretation of the discovered diseasesymptoms associations. in addition, our solution uses open source libraries and models, and does not require expensive annotation efforts. ## related work relation extraction from biomedical text, clinical discharge notes and medical articles have been widely investigated. among the early systems, @xcite proposed a combination of text mining and association rules to determine the association of drugs and diseases. more recently, increasing numbers of machine learning-based approaches have been developed exploiting supervised learning and feature engineering. many of them have looked at identifying modifiers related to important clinical entities, such as medication features @xcite . an interesting relation extraction task was proposed in the 2010 i2b2/va challenge @xcite . task organizers released an annotated dataset with medical concepts, assertions and relations, and participants where asked to extract both concepts and assertion as well as specific relations between relevant clinical entities in text. all the top-ranked systems used machine learning-based methods with extensive feature engineering. for example, @xcite proposed a support vector machine (svm) based system with additional rules to capture linguistic patterns of relations, while de @xcite investigated machine learning using large-dimensional features derived from both textual information and other external sources. differently from those approaches, we focus on disease-symptom associations, which are useful for a wide variety of healthcare applications and our approach does not require costly manual annotations or extensive feature engineering. more recently, deep learning models have also been applied to solve the same task. @xcite introduce a concept association mining framework based on word embeddings learned through neural networks allowing diseases and related symptoms to be visualized in chronological order. @xcite have used a semi-supervised approach based on variational autoencoder for biomedical relation extraction where a multi-layer convolutional neural network (cnn) was used together with bidirectional long short-term memory networks (bi-lstms) to encode drug-drug, protein-protein and chemical-protein interactions. similar tasks were carried out by @xcite who introduced a multi task learning framework leveraging a structured self-attentive network together with adversarial learning, and their approach covers also the task of medical concept relation extraction. our approach explores a different solution, combining pattern mining with nlp algorithms and leveraging more than 14m pubmed abstract, to exploit the vastness of publications to its fullest. ## proposed system the proposed system for the discovery of semantic entity associations is divided into three parts: 1) a sentence annotation system, 2) an elasticsearch foot_0 index and 3) a querying and filtering component. the latter can be further divided into three units: (i) when deployed, a user can query our system providing a disease name or its umls unique identifier and receive a ranked list of symptoms that are likely associated to it, together with a few sentences from the indexed data that motivate the discovered disease-symptom associations. the system facilitates the addition of new documents -such as newly published paperswith the automated annotation of the new textual data and a call to the index api to insert the new records to the index without any updates to the results filtering pipeline. a diagram of the proposed system is depicted in figure 1 . ## evaluation we evaluate the proposed system on the task of discovering symptoms associated to different diseases from a collection of over 14m pubmed abstracts published between 2000 and 2022. we assess the performance of the proposed pipeline in terms of recall, f1 score, and precision@k (p@k), with @xmath0, 3, 5}, defined as the number of relevant items among the top k ranked by the system divided by k. as ground truth data, we consider a subset of the disease-symptoms knowledge database @xcite published by columbia university and available online. 3 this dataset contains a list of disease-symptom pairs extracted from textual discharge summaries of patients at new york presbyterian hospital. we only considered 54 out of 134 diseases from the ground truth data since for the remaining 80 no symptoms were ever associated -i.e. mentioned in the same sentence -with the respective disease in our subset of pubmed abstracts. after pruning, our ground truth data contained an average of 1.89 symptoms per disease (@xmath1, @xmath2, standard de-@xmath3.10). for this reason, we limit the number of candidate symptoms returned by our approach to a maximum of 10 per disease after ranking them, as explained before. to determine the parameters α and β used to compute the final ranking score of each disease-symptoms candidate, we perform a 2-fold cross validation optimizing the p@1 performance measure of the entire pipeline. ## conclusions, limitations and future work we describe an end-to-end pipeline for the accelerated discovery of medical entity associations from textual data. we evaluate the proposed approach on the task of extracting disease-symptom pairs from medical literature. the main advantages of the proposed system are (i) its capability to discover new entity associations from collections of millions of scientific abstracts, (ii) the ability to easily include new scientific data in its index and therefore to perform up-to-date predictions, (iii) the independence from human annotations, (iv) the interpretability of its predictions given supporting evidence, (v) its low computational complexity, and (vi) the reliance only on open source libraries and models. we believe the adoption of systems like this in the healthcare industry could help medical professionals and researchers in making better-informed decisions. such systems could also accelerate scientific discovery by giving researchers the ability to quickly verify potential entity associations claims against scientific literature, or discover new symptoms if used with additional data sources. our pipeline could also be employed to extend existing resources such as the umls ontology with new entity relations. the proposed approach allows researchers to verify the generated entity associations claims by providing statements from indexed scientific documents that motivate such claims. despite the encouraging results, the quality of the associations discovered is limited by the accuracy of the entity annotator (incorrect annotations observed in recognizing acronyms may lead to inaccurate entity associations). similarly, the available umls semantic types may not accurately describe the different categories and this can also introduce noise. in addition, the system uses only pubmed abstract, while the entire text could provide more information. as future work, we would like test the pipeline over a larger ground truth, explore relations between other concepts and explore the issue of veracity of the extracted claims in cases where the opinions in scientific literature change over time. we are also planning to evaluate possible improvements to our pipeline, in particular at the stage where the brc and arm scores are combined, exploring some machine learning-based approaches for learning to rank.