Simple Hierarchical Multi-Task Neural End-To-End Entity Linking for Biomedical Text (2020)

Abstract: Recognising and linking entities is a crucial first step to many tasks in biomedical text analysis, such as relation extraction and target identification. Traditionally, biomedical entity linking methods rely heavily on heuristic rules and predefined, often domain-specific features. The features try to capture the properties of entities and complex multi-step architectures to detect, and subsequently link entity mentions. We propose a significant simplification to the biomedical entity linking setup that does not rely on any heuristic methods. The system performs all the steps of the entity linking task jointly in either single or two stages. We explore the use of hierarchical multi-task learning, using mention recognition and entity typing tasks as auxiliary tasks. We show that hierarchical multi-task models consistently outperform single-task models when trained tasks are homogeneous. We evaluate the performance of our models on the biomedical entity linking benchmarks using MedMentions and BC5CDR datasets. We achieve state-of-theart results on the challenging MedMentions dataset, and comparable results on BC5CDR.

## introduction & related work the task of identifying and linking mentions of entities to the corresponding knowledge base is a key component of biomedical natural language processing, strongly influencing the overall performance of such systems. the existing biomedical entity linking systems can usually be broken down into two stages: (1) mention recognition (mr) where the goal is to recognise the spans of entity mentions in text and (2) entity linking (el, also referred as entity normalisation or standardisation), which given a potential mention, tries to link it to an appropriate type and entity. often, the entity linking task includes the entity typing (et) and entity disambiguation (ed) as separate steps, with the former task aiming to identify the type of the mention, such as gene, protein or disease before passing it to the entity disambiguation stage, which effectively grounds the mention to an appropriate entity. widely studied in the general domain, entity linking is particularly challenging for the biomedical text. this is mostly due to the size of the ontology, (here referred to as the knowledge base), high syntactic and semantic overlap between types and entities, the complexity of terms, as well as the lack of availability of annotated text. due to these challenges, the majority of the existing methods rely on hand-crafted complex rules and architectures including semi-markov methods @xcite , approximate dictionary matching @xcite or use a set of external domain-specific tools with manually curated ontologies @xcite . these methods often include multiple steps, each of these steps carrying over the errors to the subsequent stages. nevertheless, these tasks are usually interdependent and have been proven to often benefit from a joint objective @xcite . recently, both in the general and biomedical domain, there has been a steady shift to neural methods to solve el @xcite , leveraging a range of methods including the use of entity embeddings @xcite , multi-task learning @xcite , and others @xcite . there have also been a plethora of mixed methods combining heuristic approaches such as approximate dictionary matching with language models @xcite . this work focuses on multi-task approaches to end-to-end entity linking, which has already been studied in the biomedical domain. these include ones leveraging pre-trained language models @xcite @xcite , model dependency @xcite and building out a cross-sharing model structure @xcite ). an interesting approach has been proposed by @xcite , where authors established a multi-task deep learning model that trained ner and el models in parallel, with each task leveraging feedback from the other. a model with a similar setup and architecture to the one here, casting the el problem as a simple per token classification problem has been outlined by @xcite . nevertheless, its application domain, architecture, and training regime strongly differ from the one proposed here. in this study, we investigate the use of a significantly simpler model, drawing on a set of recent developments in nlp, such as pre-trained language models, hierarchical and multi-task learning to outline a simple, yet effective approach for biomedical end-to-end entity linking. we evaluate our models on three tasks, mention recognition, entity typing, and entity linking, investigating different task setups and architectures on the medmentions and biocreative v cdr corpora. our contributions are as follows: ## conclusion & future work in this work, we have proposed a simple neural approach to end-to-end entity linking for biomedical text which makes no use of heuristic features. we have proven that the problem can benefit from the hierarchical multi-task learning when tasks are homogeneous. we report state-of-the-art results on el on the full medmentions dataset and comparable results on the mr and et tasks on bc5cdr @xcite . the work could easily be extended by, for example, using the output of the pt setup as features or by further developing the hierarchical multi-task framework of end-to-end entity linking problem. moreover, the additional parameters such as output class weights or loss scaling which has not been used here could be easily adapted to a particular problem.