# On the Use of Knowledge Transfer Techniques for Biomedical Named Entity Recognition +
### Multitask Learning  
It has been observed that the performance of a deep learning model highly correlates with the quantity of annotated data, i.e., the model performance improves with the quantity of data available. Unfortunately, various biomedical tasks lack enough annotated text data, and for this reason, in many cases, deep learning models cannot generalize well. Producing manually annotated data is an expensive and time-consuming job. One solution to such a barrier is to get the benefit of other associated methods that share common features. In a single-task learning approach, various tasks cannot get any benefits from each other as their features cannot be shared among them. The multitask learning (MTL) approach allows different tasks to share their knowledge among themselves using shared layers in their architecture, thus helping to improve the performance of another task [12]. MTL is an inductive learning process that learns to generalize by utilizing the knowledge of different tasks [13]. When the tasks are sufficiently related, they can provide an inductive bias that forces models to learn generally useful representations [14]. Two different methods are used in the MTL approach, i.e., hard parameter sharing and soft parameter sharing, which are shown in Figure 2. Hard parameter sharing is the most common method used in MTL, where a complete sharing (i.e., parameters) of hidden layers among different tasks is done. This article also focuses on the hard-parameter-sharing approach. In soft parameter sharing, separate models are created for different tasks. These models are then somehow enforced to loosely match the parameters of the shared layers, most commonly done by regularizing the parameters of the shared layers. The MTM can be seen as an implicit data augmentation technique as well. Jointly training various models helps them to transfer their knowledge implicitly using a shared layer(s) [15]. The MTL strategy, therefore, increases the size of the data available to the MTM. The MTL approach helps the MTM to learn those features that cannot be learned in the single-task learning approach for any specific task. In other words, it is difficult for an STM to learn complex features of task \(B\), but it is easier to learn when it is trained along with task \(A\). This is due to the fact that task \(A\) provides supervision to the MTM when it comes to learning complex features of task \(B\). Figure 2: Hard-parameter- vs. soft-parameter-shared MTMs: (**a**) hard-parameter-shared MTM; (**b**) soft-parameter-shared MTM. MTL optimizes the model during its training to produce a generalized version of the MTM. In single-task learning, a model is more prone to overfitting for a specific task, whereas MTL decreases the possibility of trapping into overfitting as the model has to learn the common representation for all tasks. Training more tasks brings more generalization for MTMs. In MTL, a model focuses on more relevant features, as some tasks give information about irrelevant and relevant features in high-dimensional and noisy data. Similarly, the noise presented in the dataset has less impact in an MTL approach, since the noise is averaged during the training. By using the MTL approach, we keep certain layers of the model shared among various tasks while retaining some layers task-specific, this helps a model to learn useful features for its current task. Training associated tasks simultaneously allow the model to optimize the values of its parameters. MTL has been widely applied in many domains, e.g., computer vision [16], speech recognition [10], and drug discovery [11]. [PERSON] and [PERSON] [17] used a CNN-based MTM and trained multiple NLP tasks jointly such as POS tagging, NER, chunking, etc. [18] showed that using the MTL approach, their model increased the performance for historical spelling normalization. [PERSON] and [PERSON] [19] used the MTL approach for different domains, i.e., Chinese word segmentation and named entity recognition. [PERSON] et al.