# On the Use of Knowledge Transfer Techniques for Biomedical Named Entity Recognition +
###### Abstract  
Biomedical named entity recognition (BioNER) is a preliminary task for many other tasks, e.g., relation extraction and semantic search. Extracting the text of interest from biomedical documents becomes more demanding as the availability of online data is increasing. Deep learning models have been adopted for biomedical named entity recognition (BioNER) as deep learning has been found very successful in many other tasks. Nevertheless, the complex structure of biomedical text data is still a challenging aspect for deep learning models. Limited annotated biomedical text data make it more difficult to train deep learning models with millions of trainable parameters. The single-task model, which focuses on learning a specific task, has issues in learning complex feature representations from a limited quantity of annotated data. Moreover, manually constructing annotated data is a time-consuming job. It is, therefore, vital to exploit other efficient ways to train deep learning models on the available annotated data. This work enhances the performance of the BioNER task by taking advantage of various knowledge transfer techniques: multitask learning and transfer learning. This work presents two multitask models (MTMs), which learn shared features and task-specific features by implementing the shared and task-specific layers. In addition, the presented trained MTM is also fine-tuned for each specific dataset to tailor it from a general features representation to a specialized features representation. The presented empirical results and statistical analysis from this work illustrate that the proposed techniques enhance significantly the performance of the corresponding single-task model (STM). biomedical named entity recognition; deep learning; single-task model; ELMo; transfer learning; multitask learning +
Footnote †: Copyright © 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons. recognition (NER). Named entities refer to the proper nouns presented in the sentences. NER recognizes text of interest and labels them into predefined categories such as person, geographical location, organization, etc. NER is considered a sequence-labeling problem that determines the output tag of the input words presented in the sentence [1; 2; 3; 4]. IE has also become a critical activity in the biomedical domain, as biomedical text is also publishing at an increasing rate. Biomedical named entity recognition (BioNER) identifies the biomedical concepts and assigns them to predefined categories such as genes, chemicals, diseases, etc., as shown in Figure 1. In practice, performing the BioNER task is more challenging than a standard NER task as biomedical documents are different from standard text data (e.g., newspaper articles) in several ways. Although there are certain practices followed by researchers for writing biomedical concepts, still, no strict rules exist for the biomedical domain. With open and growing biomedical literature, it becomes more challenging to follow the same naming convention. Another issue concerns the classification of the entities. Different human annotators, even with the same background, can sometimes associate the same word with diverse medical concepts, e.g., \"_p53_\" corresponds to a protein in the GENIA corpus. In contrast, the HUGA nomenclature annotates it as a gene \"_TP53_\" [5]. It is also very common in biomedical texts to use different spelling variations for the same entity. For example, \"IL12\", \"IL 12\", or \"IL-12\" refer to the same entity but adopt different writing conventions [6]. Another challenging aspect for BioNER is learning synonyms present in the text, e.g., _PTEN_ and _MMAC1_ both represent the same gene entity but with different synonyms. Furthermore, long compound-word entities make the learning process for the BioNER model more complicated, as these entities are expressed using different types of characters.