# On the Use of Knowledge Transfer Techniques for Biomedical Named Entity Recognition +
## 2 Proposed Methods  
### MTM-CNN Model  
In this section, a multitask model (MTM-CNN) is proposed that consists of a convolutional neural network (CNN) layer and BiLSTM layers as shown in Figure 3. The proposed MTM-CNN model varies from the model introduced by [34; 35] in different ways. [PERSON] et al. [34] proposed a multitask model (MTM) based on a CNN to perform BioNER. However, they only focused on the word-level features ignoring the character-level ones. Although the word-level features give much information about the entities, the character-level features help to extract common subword structures among the same entities. Moreover, using only the word-level features can lead to out-of-vocabulary problems when a specific word is not found in the pretrained word embedding. In addition, [PERSON] et al. [35] also performed BioNER using MTM with a single shared BiLSTM layer and found that the word-level and character-level features enhanced the performance of the MTM. Our proposed approach uses an orthographic-level representation of words, while models presented by [PERSON] et al. [34] and [PERSON] et al. [35] do not explicitly consider this feature. Various studies have utilized words' orthographic-level information for their models [36; 37; 38]. The words' orthographic-level information provides some explicit information to the model, which can help deep learning models to learn orthographic-level features implicitly. This can also help the conditional random field (CRF) whose outputs highly depend on handcrafted features [39]. In an MTM-CNN, the orthographic-level information is used, speculating that the MTM-CNN can extract additional hidden features about the current entity. In this section, the orthographic-level feature is referred to as _case-level features_ and both terms can be used interchangeably. The orthographic-level (case-level) representation in the MTM-CNN considers the structure-level information of a word. The case-level features considered in the experiments include the capitalization features of a word, e.g., whether all letters in the specific word are capitalized or lower case, or if the specific word starts with a capital or lower-case letter, whether the word contains digits or all alphabetic characters, etc. Differently from [PERSON] et al. [35], the proposed MTM-CNN utilizes a CNN (represented by the circled 1 in Figure 3) instead of BiLSTM, to extract character-level features. Note that the model proposed by [PERSON] et al. [34] does not take into account the character-level features. Character-level information encourages the model to pull out generic subword structures among the same entities. Additionally, only considering the word-level features can be prone to the out-of-vocabulary problem. Many of the state-of-the-art approaches use a CNN at the character level [40; 41] due to its unique feature extraction ability. A CNN perceives global-level features from local-level ones and therefore allows the CNN to pull out additional veiled features [34]. Third, the MTM-CNN implements stacked layers of BiLSTM units in contrast to [35]. The stacked BiLSTM units encourage each of them to perceive the hidden pattern of the data exhibited at various time stamps. This helps the BiLSTM network to gain knowledge of the features at a more abstract level. The first innermost BiLSTM layer (marked by the circled 2 in Figure 3) is shared among all the tasks while the second layer of BiLSTM (shown by the circled 3 in Figure 3) is task-specific. The MTM-CNN exploits CRF (represented by the circled 4 in the figure) at the output layer for final sequence labeling. CRF performs a tagging of the current token by considering neighboring tags at a sentence level [38]. [PERSON] et al. [42] performed experiments with both CRF and Softmax and concluded that CRF produced better results compared to Softmax.