Rationalizing Medical Relation Prediction from Corpus-level Statistics (2020)

Abstract: Nowadays, the interpretability of machine learning models is becoming increasingly important, especially in the medical domain. Aiming to shed some light on how to rationalize medical relation prediction, we present a new interpretable framework inspired by existing theories on how human memory works, e.g., theories of recall and recognition. Given the corpus-level statistics, i.e., a global co-occurrence graph of a clinical text corpus, to predict the relations between two entities, we first recall rich contexts associated with the target entities, and then recognize relational interactions between these contexts to form model rationales, which will contribute to the final prediction. We conduct experiments on a real-world public clinical dataset and show that our framework can not only achieve competitive predictive performance against a comprehensive list of neural baseline models, but also present rationales to justify its prediction. We further collaborate with medical experts deeply to verify the usefulness of our model rationales for clinical decision making.

## introduction predicting relations between entities from a text corpus is a crucial task in order to extract structured knowledge, which can empower a broad range of downstream tasks, e.g., question answering @xcite , dialogue systems @xcite , reasoning @xcite , etc. there has been a large amount of existing work focusing on predicting relations based on raw texts (e.g., sentences, paragraphs) mentioning two entities @xcite @xcite @xcite @xcite . in this paper, we study a relatively new setting in which we predict relations between entities based on the global co-occurrence statistics aggregated from a text corpus, and focus on medical relations and clinical texts in electronic medical records @xcite . the corpus-level statistics present a holistic graph view of all entities in the corpus, which will greatly facilitate the relation inference, and can better preserve patient privacy than raw or even de-identified textual content and are becoming a popular substitute for the latter in the research community for studying emr data @xcite . to predict relations between entities based on a global co-occurrence graph, intuitively, one can first optimize the graph embedding or global word embedding @xcite @xcite , and then develop a relation classifier @xcite @xcite based on the embedding vectors of the two entities. however, such kind of neural frameworks often lack the desired interpretability, which is especially important for the medical domain. in general, despite their superior predictive performance in many nlp tasks, the opaque decision-making process of neural models has concerned their adoption in high stakes domains like medicine, finance, and judiciary @xcite . building models that provide reasonable explanations and have increased transparency can remarkably enhance user trust @xcite . in this paper, we aim to develop such a model for our medical relation prediction task. to start with, we draw inspiration from the existing theories on cognitive processes about how human memory works, e.g., two types of memory retrieval (recall and recognition) @xcite . basically, in the recall process, humans tend to retrieve contextual associations from long-term memory. for example, given the word "paris", one may think of "eiffel tower" or "france", which are strongly associated with "paris" @xcite @xcite . besides, there is a strong correlation between the association strength and the co-occurrence graph @xcite . in the recognition process, humans typically recognize if they have seen a certain piece of information before. figure 1 shows an example in the context of relation prediction. assume a model is to predict whether aspirin may treat headache or not (that "aspirin may treat headache" is a known fact, and we choose this relation triple for illustration purposes). it is desirable if the model could perform the aforementioned two types of memory processes and produce rationales to base its prediction upon: (1) recall. what entities are associated with aspirin? what entities are associated with headache? (2) recognition. do those associated entities hold certain relations, which can be leveraged as clues to predict the target relation? for instance, a model could first retrieve a relevant entity pain relief for the tail entity headache as they co-occur frequently, and then recognize there is a chance that aspirin can lead to pain relief (i.e., formulate model rationales or assumptions), based on which it could finally make a correct prediction (aspirin, may treat, headache). now we formalize such intuition to rationalize the relation prediction task. our framework consists of three stages, global association recall (cogstage-1), assumption formation and representation (cogstage-2), and prediction decision making (cogstage-3), shown in figure 2 . cogstage-1 ## background different from existing work using raw texts for relation extraction, we assume a global co-occurrence graph (i.e., corpus-level statistics) is given, which was pre-constructed based on a text corpus d, and denote it as an undirected graph @xmath0, e), where each vertex @xmath1. counts reflect how frequent two entities appear in the same context (e.g., co-occur in the same sentence, document, or a certain time frame). in this paper, we focus on clinical co-occurrence graph in which vertices are medical terms extracted from clinical notes. nevertheless, as we will see later, our framework is very general and can be applied to other relations with corpus-level statistics. our motivation for working under this setting lies in three folds: (1) such graph data is stripped of raw textual contexts and thus, has a better preserving of patient privacy @xcite , which makes itself easier to be constructed and shared under the hippa protected environments @xcite for medical institutes @xcite ; (2) compared with open-domain relation extraction, entities holding a medical relation oftentimes do not co-occur in a local context (e.g., a sentence or paragraph). for instance, we observe that in a widely used clinical co-occurrence graph @xcite , which is also employed for our experiments later, of all entity pairs holding the treatment relation according to umls (unified medical language system), only about 11.4% have a co-occurrence link (i.e., co-occur in clinical notes within a time frame like 1 day or 7 days); finally, our relation prediction task is formulated as: given the global statistics g and an entity pair, we predict whether they hold a relation r (e.g., may treat), and moreover provide a set of model rationales t composed of relation triples for the prediction. for the example in figure 1 , we aim to build a model that will not only accurately predict the may treat relation, but also provide meaningful rationales on how the prediction is made, which are crucial for gaining trust from clinicians. ## methodology following a high-level framework illustration in figure 2 , we show a more detailed overview in figure 3 and introduce each component as follows. ## experiments in this section, we first introduce our experimental setup, e.g, the corpus-level co-occurrence statistics and datasets used for our experiments, and then compare our model with a list of comprehensive competitive baselines in terms of predictive performance. moreover, we conduct expert evaluations as well as case studies to demonstrate the usefulness of our model rationales. ## related work relation extraction (re) typically focuses on predicting relations between two entities based on their text mentions, and has been well studied in both open domain @xcite @xcite @xcite and biomedical domain @xcite @xcite @xcite . among them, most state-of-the-art work develops various powerful neural models by leveraging human annotations, linguistic patterns, distance supervision, etc. more recently, an increasing amount of work has been proposed to improve model's transparency and interpretability. for example, @xcite visualizes self-attention weights learned from bert @xcite to explain relation prediction. however, such text-based interpretable models tend to provide explanations within a local context (e.g., words in a single sentence mentioning target entities), which may not capture a holistic view of all entities and their relations stored in a text corpus. we believe that such a holistic view is important for interpreting relations and can be provided to some degree by the global statistics from a text corpus. moreover, global statistics have been widely used in the clinical domain as they can better preserve patient privacy @xcite . on the other hand, in recent years, graph embedding techniques @xcite @xcite have been widely applied to learn node representations based on graph structure. representation learning based on global statistics from a text corpus (i.e., co-occurrence graph) has also been studied @xcite . after employing such methods to learn entity embeddings, a number of relation classifiers @xcite @xcite @xcite can be adopted for relation prediction. we compare our method with such frameworks to show its competitive predictive accuracy. however, such frameworks tend to be difficult to interpret as they provide little or no explanations on how decisions are made. in this paper, we focus more on model interpretability than predictive accuracy, and draw inspirations from existing cognitive theories of recall and recognition to develop a new framework, which is our core contribution. another line of research related to interpreting relation prediction is path-based knowledge graph (kg) reasoning @xcite @xcite @xcite . in particular, existing paths mined from millions of relational links in knowledge graphs can be used to provide justifications for relation predictions. for example, to explain microsoft and usa may hold the relation countryofheadquarters, by traversing a kg, one can extract the path microsoft ## conclusion in this paper, we propose an interpretable framework to rationalize medical relation prediction based on corpus-level statistics. our framework is inspired by existing cognitive theories on human memory recall and recognition, and can be easily understood by users as well as provide reasonable explanations to justify its prediction. essentially, it leverages corpus-level statistics to recall associative contexts and recognizes their relational connections as model rationales. compared with a comprehensive list of baseline models, our model obtains competitive predictive performances. moreover, we demonstrate its interpretability via expert evaluation and case studies.