# A Comparative Analysis of Active Learning for Biomedical Text Mining
## 2 Related Work
### Natural Language Processing (NLP)  
NLP is the intersection of computing science and linguistics that includes dissecting and understanding common human language from both speech and written texts. Over the years, NLP has been used in various applications such as email filtering [36], irony and sarcasm detection [37] document organisation [38], sentiment and opinion mining prediction [39, 40, 41], hate speech detection [42, 43, 44], question answering [45], content mining [46], biomedical text mining [47, 48], and many more [8, 49, 50].  
In biomedical named entity recognition (BioNER), [PERSON] et al. [51] initially created embeddings of words on unlabelled texts of biological topics using neural networks, going on to establish a multi-layer neural network to obtain cutting edge output. [PERSON] et al. [52] mixed sentence vectors and twin word embeddings and utilised the BiLSTM on biomedical texts to identify domain-relevant entities. To identify drug entities, [PERSON] et al. [53] developed their model, BiLSTM-CRF. CNN was utilised to get the representation of features on a character level. This was done with representations on a word level and used as data to be fed to the BiLSTM-CRF for BioNER. In biomedical literature, many words can cause information redundancy whilst neural network models are trained for feature capture, preventing critical information from being obtained. This may cause the crucial areas not to focus on the BioNER models, and loss of information could occur. It is a salient focus to make models of neural networks attentive to areas of importance. In machine translation, [PERSON] et al. [54] suggest the attention focusing mechanism. Taking the decoder model into account, the focus can be made on the initial text's essential bits as it is decoded, reducing information loss. An attention-based BiLSTM-CRF model is used by [PERSON] et al. [55] for BioNER on a document level. They optimise the tagging inconsistency problem by using, between various sentences, mechanisms that are attention-focused. The best results are obtained on CHEMDNER and CDR corpora using this approach.  
Several other works have investigated the benefit of contextual models in biomedical and clinical areas. Several researchers trained ELMo on biomedical corpora and presented BioELMo and found that BioELMo beats ELMo on BioNER tasks [56; 57]. Along with their work, a pre-trained BioELMo model was published, enabling further clinical research. [PERSON] et al. [58] released Scientific BERT (SciBERT), where BERT was trained on the scientific texts. In non-contextual embedding, BERT has been usually superior and better than ELMo. Similarly, innovative wireless connectivity techniques could be applied to the remote execution of these techniques [59; 60; 61; 62]  
[PERSON] et al. [63], trained the BERT on clinical notes corpora, using complex task-specific models to improve both traditional embedding and ELMo embedding i2b2 2010 and 2012 BioNER. Similarly, in another study, a new domain-specific language model, BioBERT [64], trained a BERT model on biomedical documents from PMC abstracts and articles from PubMed that resulted in improved BioNER results. [PERSON] et al. [65] introduced Biomedical Language Understanding Evaluation (BLUE), a collection of resources for evaluating and analysing natural biomedical language representation models.