Improving Medical NLI Using Context-Aware Domain Knowledge (2020)

Abstract: Domain knowledge is important to understand both the lexical and relational associations of words in natural language text, especially for domain-specific tasks like Natural Language Inference (NLI) in the medical domain, where due to the lack of a large annotated dataset such knowledge cannot be implicitly learned during training. However, because of the linguistic idiosyncrasies of clinical texts (e.g., shorthand jargon), solely relying on domain knowledge from an external knowledge base (e.g., UMLS) can lead to wrong inference predictions as it disregards contextual information and, hence, does not return the most relevant mapping. To remedy this, we devise a knowledge adaptive approach for medical NLI that encodes the premise/hypothesis texts by leveraging supplementary external knowledge, alongside the UMLS, based on the word contexts. By incorporating refined domain knowledge at both the lexical and relational levels through a multi-source attention mechanism, it is able to align the token-level interactions between the premise and hypothesis more effectively. Comprehensive experiments and case study on the recently released MedNLI dataset are conducted to validate the effectiveness of the proposed approach.

## introduction natural language inference is a fundamentally important but challenging task in natural language processing (nlp) as it requires understanding and reasoning over natural language texts @xcite . as a result, a good performing this work is licensed under a creative commons attribution 4.0 international license. license details: http: //creativecommons.org/licenses/by/4.0/ . nli system is considered indispensable for downstream nlp applications such as question answering and automatic text summarization @xcite . given a pair of sentences, a premise and a hypothesis ℎ, the goal of nli is to determine whether the semantic relationship between and ℎ is among , and . the ability to understand natural language text innately requires to deal with background knowledge foot_0 @xcite . a robust nli model usually needs to reason over two types of background knowledge -lexical and relational @xcite . the former pertains to understanding the concepts expressed by the words in the text, while the latter learns the semantic relations between the different concepts. when performing nli on open domain data, it is assumed that the background knowledge will be implicitly learned from the training corpora. remednli. the words in red, "dmii" and "dm" in and "diabetes" in ℎ are semantically similar at the lexical level. the umls relation "co-occurs" of the highlighted words in green in to "diabetes" in ℎ manifests the inferential signal "comorbidities". for specialized domains (e.g., medical), however, large nli datasets are extremely scarce and the required implicit knowledge beyond the text surface cannot be extracted from limited data. for example, the recently released mednli dataset @xcite , albeit being the single publicly available nli dataset in the clinical domain, contains only around 13,000 expert annotated sentence pairs 2 . therefore, most current literature on medical nli @xcite have capitalized on the prior semantic knowledge that is encoded in external resources (e.g., umls). nevertheless, a limitation of the existing suite of medical ontologies such as umls is that they retrieve mapping (i.e., lexical/relational) irrespective of the textual context, which could mislead the inference model. this is worsened by the distinct linguistic idiosyncrasies present in clinical texts, wherein phrases are compressed with shorthand jargon (i.e., abbreviations) for physicians' convenience. specifically, this instigates three challenges: 1) some semantically important words do not map to any matching concept in the umls 3 , 2) a wrong concept mapping is returned that does not reflect the word's actual meaning and 3) the noise introduced by wrong concept mapping could get carried forward when retrieving the relational mapping. to address the aforementioned issues, we devise an approach for the nli problem in the medical domain that is equipped with an adaptive encoding scheme to integrate context-relevant domain knowledge into the text representation more effectively. in particular, a mixture model is employed to adaptively leverage supplementary external resources, that provide contextual evidence to disambiguate the concept sense for each word, and thus facilitate in learning more semantically refined text embeddings, as well as, account for the missing words. furthermore, in order to infuse important inferential clues between the 2 compared to open domain nli datasets 3 we call such words "missing words" in this paper premise-hypothesis tokens, context-relevant relational embeddings elicited from knowledge graph is encoded through multi-source attention mechanism. we dub the proposed framework as multi-source knowledge adaptive inference network (muskan). ## approach overview we treat the task of natural language inference (nli) as a supervised classification task and state it as follows: given a premise @xmath0, ..., here, and ℎ are the -th and -th concept, and and ℎ are the -th and -th relation triple of the premise and hypothesis respectively. note that a word / ℎ in the premise/hypothesis could also be an abbreviation, which here we collectively call word. as aforementioned, although incorporating domain knowledge from the umls helps to understand medical semantics in the text that go beyond basic linguistic understanding, it could also aggravate the inference process due to the missing words and the inaccurate mappings. to this end, we supplement the umls with other external resources in order to soft-align the context-relevant domain information to each word in the text. figure 2 illustrates a high-level overview of the architecture of our proposed model, muskan. it follows the encode-match-classify framework of general text-based nli models @xcite . in the encoder layer, an adaptive encoding scheme encodes each word in the premise/hypothesis sentence by integrating refined concept embeddings into the text representation, where the refinement is done by leveraging contextual evidence from the supplementary external resources. then in the matching layer, the adaptive lexical encodings are enhanced with refined relational information codified in knowledge graphs using multi-source attention, that facilitates in semantically aligning and aggregating the interactions between the premise-hypothesis words. finally, the classification layer composes the pair of sentences to a fixed length vector and predicts their relation. more details of each component will be presented in the next sections. ## conclusion and future work this work discloses the effectiveness of contextaware domain knowledge in medical nli and proposes a systematic approach to infuse such knowledge using an adaptive encoding scheme. by employing a multi-source attention mechanism that is able to model both the lexical and relational semantics, it is able to mitigate the noise introduced by the abbreviation-like jargon prevalent in medical text. through both qualitative and quantitative analysis, our proposed framework advances the limited work so far done on medical nli. there are several possible directions that could premise: in the ed, initial vs revealed t 98.9, hr 73, bp 121/90, rr 15, o2 sat 98% on ra.