Clustering-based Inference for Biomedical Entity Linking (2021)

Abstract: Due to large number of entities in biomedical knowledge bases, only a small fraction of entities have corresponding labelled training data. This necessitates entity linking models which are able to link mentions of unseen entities using learned representations of entities. Previous approaches link each mention independently, ignoring the relationships within and across documents between the entity mentions. These relations can be very useful for linking mentions in biomedical text where linking decisions are often difficult due mentions having a generic or a highly specialized form. In this paper, we introduce a model in which linking decisions can be made not merely by linking to a knowledge base entity but also by grouping multiple mentions together via clustering and jointly making linking predictions. In experiments on the largest publicly available biomedical dataset, we improve the best independent prediction for entity linking by 3.0 points of accuracy, and our clustering-based inference model further improves entity linking by 2.3 points.

## introduction ambiguity is inherent in the way entities are mentioned in natural language text. grounding such ambiguous mentions to their corresponding entities, the task of entity linking, is critical to many applications: automated knowledge base construction and completion @xcite , information retrieval @xcite , smart assistants @xcite , question answering @xcite ), text mining @xcite . consider the excerpt of text from a biomedical research paper in figure 1 , the three highlighted mentions (expression, facial expressions, and facially expressive) all link to the same entity, namely c0517243 -facial expresson in the leading biomedical kb, unified medical language system (umls) @xcite . the mention expression is highly ambiguous and easily confused with the more prevalent en- [c0017262] name: gene expression type: biologic function qualifiers: expression gene, gene expression, expression, expressed, gene expression figure 1 : biomedical entity linking. all three highlighted mentions refer to the same entity. the mention expression is clearly related to the other two highlighted mentions which are much less ambiguous. if considered independently expression is more closely related to an incorrect entity. in this paper, we propose an entity linking model in which entity mentions are either (1) linked directly to an entity in the knowledge-base or (2) join a cluster of other mentions and link as a cluster to an entity in the knowledge-base. some mentions may be difficult to link directly to their referent ground truth entity, but may have very clear coreference relationships to other mentions. so long as one mention among the group of mentions clustered together links to the correct entity the entire cluster can be correctly classified. this provides for a joint, tranductive-like, inference procedure for linking. we describe both the inference procedure as well as training objective for optimizing the model's inference procedure, based on recent work on supervised clustering @xcite . it is important to note that our approach does not aim to do joint coreference and linking, but rather makes joint linking predictions by clustering together mentions that are difficult to link directly to the knowledge-base. for instance, in figure 1 , the mention expression may be difficult to link to the ground truth facial expression entity in the knowledge-base because the mention can refer to a large number of entities. however, the local syntactic and semantic information of the paragraph give strong signals that expression is coreferent with facial expression, which is easily linked to the correct entity. we perform experiments on two biomedical entity linking datsets: medmentions @xcite , the largest publicly available dataset as well as the benchmark bc5cdr @xcite . we find that our approach improves over our strongest baseline by 2.3 points of accuracy on medmentions and 0.8 points of accuracy on bc5cdr over the baseline method @xcite . we further analyze the performance of our approach and observe that (1) our method better handles ambiguous mention surface forms (as in the example shown in figure 1 ) and (2) our method can correctly link mentions even when the candidate generation step fails to provide the correct entity as a candidate. ## background we denote the set of all mentions across all documents as plainly m. the task of entity linking is to classify each mention m i as referent to a single entity e i from a kb of entities. we use e(m i ) to refer to the ground truth entity of mention m i and êi to refer to the predicted entity. knowledge-bases. we assume that we are given a knowledge-base corresponding to a closed world of entities. these kbs are typically massive: english wikipedia contains just over 6m entities foot_0 and the 2020 release of the umls contains 4.28m entities 2 . we describe in sections 5.1 & 5.2 the details of the kbs used in each of the experiments. ## model in this section, we describe our clustering-based approach for jointly making entity linking predictions for a set of mentions. our proposed inference method builds a graph where the nodes are the union of all of the mentions and entities and the edges have weights denoting the affinities between the endpoints. to make linking decisions, we cluster the nodes of the graph such that each cluster contains exactly one entity, following which each mention is assigned to the entity in its cluster. ## training in this section, we explain the training procedure for the affinity models ϕ(•, •) and ψ(•, •) used by the clustering inference procedure. we train the mention-mention and mention-entity models independently in a way that allows the affinities to be comparable when performing inference. let d i be the document containing m i 7: we use triplet max-margin based training objectives to train both models. the most important aspect of our procedure is how we pick negatives during training. for the mention-entity model, we restrict our negatives to be from the candidate set. for the mention-mention model, we restrict our negatives to come from mentions within the same document. from these sets of possible negatives we choose the top-k most offending ones according the instantaneous state of the model -i.e. the negatives with highest predicted affinities according to the model at that point during training. the following sections detail the training procedures for both models. ## experiments we evaluate on biomedical entity linking using the medmentions @xcite and bc5cdr @xcite datasets. we compare to state-ofthe-art methods. we then analyze the performance of our method in more detail and provide qualitative examples demonstrating our approaches' ability to use mention-mention relationships to improve candidate generation and linking. ## related work entity linking is widely studied and often focused on linking mentions to wikipedia entities (also known as wikification) @xcite @xcite @xcite . entity linking is often done independently for each mention in the document @xcite or by modeling dependencies between predictions of entities in a document @xcite @xcite . in the biomedical domain, unified medical language system (umls) is often used as a knowledge-base for entities @xcite . while umls is a rich ontology of concepts and relationships between them, this domain is low resource compared to wikipedia with respect to number of labeled training data for each entity mention. this leads to a zero-shot setting in datasets such as medmentions @xcite where new entities are seen at test time. previous work has addressed this zero-shot setting using models of the type hierarchy @xcite . this previous work @xcite uses an unrealistic candidate generation setting where the true positive candidate is within the candidate set and/or entities are limited to those in the dataset rather than those in the knowledge-base. mention-mention relationships are also explored in @xcite which extends the pairwise crf model @xcite to use mention-level relationships in addition to entity relationships. these works use attention in a way to build the context representation of the mentions. however, as mentioned by @xcite is not well suited for zero-shot linking. coreference (both within and across documents) has also been explored by past work @xcite . this work uses an iterative procedure that performs hard clustering for the sake of aggregating the contexts of entity mentions. @xcite presents a crf-based model for joint ner, within-document coreference, and linking. they show that jointly modeling these three tasks improves performance over the independent baselines. this differs from our work since we do not require coreference decisions to be correct in order to make correct linking decisions. other work performs joint entity and event coreference @xcite without linking. ## conclusion in this work, we presented a novel clustering-based inference procedure which enables joint entity linking predictions. we evaluate the effectiveness of our approach on the two biomedical entity linking datasets, including the largest publicly available dataset. we show through analysis that our approach is better suited to link mentions with ambiguous surface forms and link mentions where the ground truth entity is not in the candidate set.