Extractive Medical Entity Disambiguation with Memory Mechanism and Memorized Entity Information (2024)

Abstract: Medical entity disambiguation (MED) aims to ground medical mentions in text with ontological entities in knowledge bases (KBs). A notable challenge of MED is the long medical text usually contains multiple entities’ mentions with intricate correlations. However, limited by computation overhead, many existing methods consider only a single candidate entity mention during the disambiguation process. As such, they focus only on local MED optimal while ignoring the sole-mention disambiguation possibly boosted by richer context from other mentions’ disambiguating processes – missing global optimal on entity combination in the text. Motivated by this, we propose a new approach called Extractive Medical Entity Disambiguation with Memory Mechanism and Memorized Entity Information (M3E). Specifically, we reformulate MED as a text extraction task, which simultaneously accepts the context of medical mentions, all possible candidate entities, and entity definitions, and it is then trained to extract the text span corresponding to the correct entity. Upon our new formulation, 1) to alleviate the computation overhead from the enriched context, we devise a memory mechanism module that performs memory caching, retrieval, fusion and cross-network residual; and 2) to utilize the disambiguation clues from other mentions, we design an auxiliary disambiguation module that employs a gating mechanism to assist the disambiguation of remaining mentions. Extensive experiments on two benchmark datasets demonstrate the superiority of M3E over the state-of-the-art MED methods on all metrics.

## introduction associating medical mentions in a given biomedical text with their corresponding correct entities * corresponding author 1 the source code and datasets of this paper can be obtained from https://github.com/stubborn-z/mmme . in recent years, a series of disambiguation meth-ods have been proposed in the literature of deep representation learning. they can be roughly categorized into matching and generative med, improving benchmark performance in a row. first, med was defined as a matching problem between medical mentions and candidate entities. as such, some works @xcite utilized dual encoders to separately encode medical mentions and candidate entities, adopting attention mechanisms to enhance the interaction between them. however, these methods only consider a single candidate entity for a medical mention and do not fully utilize semantic knowledge in kbs. to mitigate this, another research line presents generative med methods @xcite with sacrifice of efficiency, which utilize generative pretrained language models, inputting medical mentions with context and outputting corresponding entities through pre-training and fine-tuning. nonetheless, these generative med methods still suffer from a lack of context information as they only consider the context of the current medical mention, while ignoring richer context from other mentions' disambiguating processes. for example, as shown in figure 1 (a), when encountering a medical mention in a sentence, humans simultaneously consider the context of the mention, all possible candidates and the semantic knowledge of entities, focusing on global information to comprehensively understand the whole context and determine the correct entity. what's worse, these methods usually disambiguate each medical mention individually without considering the semantic clues when disambiguating other entities. as such, they focus only on local med optimization, overlooking the potential benefits of richer context from the disambiguation processes of other mentions. this results in missing the global optimization of entity combinations within the text. continue the example above: as depicted in figure 1(b) , humans utilize the memorized entity information as disambiguation clues to further infer the remaining medical mentions. therefore, the disambiguation pattern of existing med methods is inconsistent with human cognitive behaviors. inspired by the extractive entity disambiguation in the general domain @xcite and the human cognitive behaviors in figure 1 , we propose a new paradigm that reformulates med as a medical text extraction task. unlike previous medical entity disambiguation methods, we introduce a memory mechanism module to alleviate the compu-tational burden of our newly formulated med task, enabling the model to simultaneously consider the context of medical mentions, all possible candidate entities and the semantic knowledge of candidates during the disambiguation process. additionally, we propose an auxiliary disambiguation module that leverages the semantic disambiguation clues from memorized entities, i.e., the richer knowledge from the disambiguation process of other mentions to assist in disambiguating the remaining medical mentions. our approach achieves globally optimal disambiguation by focusing on global med optimization our main contributions are three-fold: generative med. generative med methods usually utilize generative language models pretrained on various medical corpora as fundamental architectures, and finetune them to infer correct entities of medical mentions. for example, biobart @xcite ) is a biomedical autoregressive generative language model pre-trained on pubmed abstracts, which is further fine-tuned on the medical entity disambiguation task, yielding promising results. genbioel @xcite first proposes kb-guided pre-training to inject synonyms and definition knowledge into the generative language model and then proposes synonym-aware fine-tuning to select correct entities for mentions. ## conclusion in this paper, we propose a novel med framework called extractive medical entity disambiguation with memory mechanism and memorized entity information (m 3 e). our new paradigm reformulates med as a text extraction task, which simultaneously accepts the context of mentions, all possible candidate entities and entity definitions. to alleviate the computational burden caused by the increased input length, we devise a memory mechanism module, which performs memory caching, retrieval, fusion, and cross-network residual connections to enhance the model's efficiency by capturing and utilizing global context information more effectively. additionally, we implement an auxiliary disambiguation module, which leverages memorized entities' semantic clues in the disambiguation process of other mentions, aiding in the disambiguation of the remaining mentions. therefore, our method achieves globally optimal disambiguation by emphasizing global med optimization. extensive experimental results on three public benchmark datasets demonstrate that m 3 e consistently outperforms the representative and state-of-the-art med baselines. notably, our model achieves an f1 score approximately 1 point higher than the suboptimal baseline on all datasets, showing great potential for further exploration.