Fast and Effective Biomedical Entity Linking Using a Dual Encoder (2021)

Abstract: Biomedical entity linking is the task of identifying mentions of biomedical concepts in text documents and mapping them to canonical entities in a target thesaurus. Recent advancements in entity linking using BERT-based models follow a retrieve and rerank paradigm, where the candidate entities are first selected using a retriever model, and then the retrieved candidates are ranked by a reranker model. While this paradigm produces state-of-the-art results, they are slow both at training and test time as they can process only one mention at a time. To mitigate these issues, we propose a BERT-based dual encoder model that resolves multiple mentions in a document in one shot. We show that our proposed model is multiple times faster than existing BERT-based models while being competitive in accuracy for biomedical entity linking. Additionally, we modify our dual encoder model for end-to-end biomedical entity linking that performs both mention span detection and entity disambiguation and out-performs two recently proposed models.

## introduction entity linking is the task of identifying mentions of named entities (or other terms) in a text document and disambiguating them by mapping them to canonical entities (or concepts) listed in a reference knowledge graph @xcite . this is an essential step in information extraction, and therefore has been studied extensively both in domainspecific and domain-agnostic settings. recent stateof-the-art models @xcite attempt to learn better representations of mentions and candidates using the rich contextual information encoded in pre-trained language models such as bert @xcite . these models follow a retrieve and rerank paradigm, which consists of two separate steps: first, the can-didate entities are selected using a retrieval model. subsequently, the retrieved candidates are ranked by a reranker model. although this approach has yielded strong results, owing primarily to the powerful contextual representation learning ability of bert-based encoders, these models typically process a single mention at a time. processing one mention at a time incurs a substantial overhead both during training and test time, leading to a system that is slow and impractical. in this paper, we propose a collective entity linking method that processes an entire document only once, such that all entity mentions within it are linked to their respective target entities in the knowledge base in one pass. compared to the popular entity linking model blink @xcite , our model is up to 25x faster. blink deploys two separately trainable models for candidate retrieval and reranking. in contrast, our method learns a single model that can perform both the retrieval and reranking steps of entity linking. our model does not require candidate retrieval at inference time, as our dual encoder approach allows us to compare each mention to all entities in the target knowledge base, thus significantly reducing the overhead at inference time. we evaluate our method on two particularly challenging datasets from the biomedical domain. in recent times, there is an increased focus on information extraction from biomedical text such as biomedical academic publications, electronic health records, discharge summaries of patients, or clinical reports. extracting named concepts from biomedical text requires domain expertise. existing automatic extraction methods, including the methods and tools catering to the biomedical domain @xcite @xcite , often perform poorly due to the inherent challenges of biomedical text: ## model given a document @xmath0, . . . , x d t ] of t tokens with n mentions {m 1 , . . . , m n } and a set of m entities {e 1 , . . . , e m } in a target knowledge base or thesaurus e, the task of collective entity disambiguation consists in mapping each entity mention m k in the document to a target entity t @xmath1. each mention in the document d may span over one or multiple tokens, denoted by pairs (i, j) of start and end index positions such that m @xmath2, . . . , x d j ]. ## conclusion this paper introduces a biomedical entity linking approach using bert-based dual encoders to disambiguate multiple mentions of biomedical concepts in a document in a single shot. we show empirically that our method achieves higher accuracy and recall than other competitive baseline models in significantly less training and inference time. we also showed that our method is significantly better than two recently proposed biomedical entity linking models for the end-to-end entity disambiguation task when subjected to multi-task