Active Learning for Assisted Corpus Construction: A Case Study in Knowledge Discovery from Biomedical Text (2021)

Abstract: This paper presents an active learning approach that aims to reduce the human effort required during the annotation of natural language corpora composed of entities and semantic relations. Our approach assists human annotators by intelligently selecting the most informative sentences to annotate and then pre-annotating them with a few highly accurate entities and semantic relations. We define an uncertainty-based query strategy with a weighted density factor, using similarity metrics based on sentence embeddings. As a case study, we evaluate our approach via simulation in a biomedical corpus and estimate the potential reduction in total annotation time. Experimental results suggest that the query strategy reduces by between 35% and 40% the number of sentences that must be manually annotated to develop systems able to reach a target F1 score, while the pre-annotation strategy produces an additional 24% reduction in the total annotation time. Overall, our preliminary experiments suggest that as much as 60% of the annotation time could be saved while producing corpora that have the same usefulness for training machine learning algorithms. An open-source computational tool that implements the aforementioned strategies is presented and published online for the research community.

## introduction machine learning, and specifically supervised learning, is one of the most effective tools for automating complex cognitive tasks, such as recognising objects in images or understanding natural language text. one of the main bottlenecks of supervised learning is the need for high-quality datasets of labelled samples on which statistical models can be trained. these datasets are usually built by human experts in a lengthy and costly manual process. active learning @xcite is an alternative paradigm to conventional supervised learning that has been proposed to reduce the costs involved in manual annotation . the key idea underlying active learning is that a learning algorithm can perform better with less training examples if it is allowed to actively select which examples to learn from @xcite . in the supervised learning context, this paradigm changes the role of the human expert. in conventional supervised learning contexts, the human expert guides the learning process by providing a large dataset of labelled examples. however, in active learning the active role is shifted to the algorithm and the human expert becomes an oracle, participating in a labelling-training-query loop. in the active paradigm, a model is incrementally built by training on a partial collection of samples and then selecting one or more unlabelled samples to query the human oracle for labels and increase the training set. this approach introduces the new problem of how to best select the query samples so as to maximise the model's performance while minimising the effort of the human participant. the simplest active learning scenario consists of the classification of independent elements x i drawn from a pool of unlabelled samples. examples range from image classification @xcite to sentiment mining @xcite , in which the minimal level of sampling (e.g., an image or text document) corresponds to the minimal level of decision. i.e, a single label is assigned to each x i . more complex scenarios arise when the decision level is more fine-grained than the sampling level. in the domain of text mining, an interesting scenario is the task of entity and relation extraction from natural language text @xcite . in this scenario the sampling level is a sentence, but the minimal level of decision involves each token or pair of tokens in the sentence, and furthermore, these decisions are in general not independent within the same sentence. in this case, it is not trivial to estimate how informative an unlabelled sample will be, since each sample has several sources of uncertainty. this research focuses on the tasks of entity and relation extraction, proposing an active learning strategy to reduce the overall time of annotation for human experts, by actively selecting the most informative sentences to annotate. we also consider the problem of providing some instances of entities and relations pre-annotated to further reduce the annotation time for the human, while minimising the number of erroneous suggestions. in contrast with the usual formulation, we focus on the problem of obtaining the corpus per-se, and the task of training the underlying machine learning models is considered as means to an end rather than as the primary objective. the contributions of this research can be summarised as follows: the remaining sections of this paper are organised as follows. section 2 reviews the most relevant research related with active learning in general and specifically for entity and relation extraction. section 3 presents the formal definition for our active learning approach. section 4 describes a computational prototype tool that implements this strategy. section 5 evaluates our proposal in the context of a corpus of entities and relations in the biomedical domain. section 6 presents a discussion of the most relevant insights that our research provides. finally, section 7 presents the main conclusions of our research. ## related works this section reviews some of the most relevant research related with active learning in general, and specifically focused on entity detection and relation extraction. one of the most important design decisions in active learning is how to intelligently select the novel unlabelled samples in the most efficient way. the underlying assumption is that we want to train a model to the highest possible performance (measured in precision, f 1 , etc.) while minimising the human cost (measured in time, number of samples manually labelled, or any other suitable metric). this requirement is often framed as the selection of the most informative unlabelled samples, and formalised in terms of a query strategy @xcite . the most common query strategies for general-purpose active learning can be grouped into the following categories: expected model change (iii) and variance/error reduction (iv) strategies are heavily dependent on the specific learning model used. in contrast, uncertainty sampling (i) and query by committee (ii) are applicable in general with a high degree of model agnosticism. furthermore, relevant subsets of both strategies can be formalised under a single framework if we define the uncertainty as a measure of the entropy of the model's predicted output. in this framework, query-by-committee can be implemented via weighted voting, thereby assigning empirical probabilities to the possible outputs. weighted density is a complimentary strategy in which the most informative samples are weighted by how representative they are of the input space, for example, by measuring their similarity to the remaining samples @xcite . this approach attempts to counter-balance a noticeable tendency to select outliers as the most informative samples -a problem associated with other query strategies-since outliers are often the samples that create the highest amount of uncertainty, disagreement or hypothesis change. recent advances in natural language processing have produced an increased interest in active learning to alleviate the requirement for large annotated corpora @xcite . @xcite compare several strategies for active learning in sequence labelling scenarios, concluding that query strategies based on measures of sequence entropy combined with weighted sampling outperform other variants. @xcite propose a comprehensive benchmark to evaluate different active learning strategies for entity matching. in the task of named entity recognition, crf models have been used to select query samples @xcite . the task of relation extraction also benefits from active learning approaches, both in general-purpose settings @xcite and in domainspecific settings @xcite . however, despite the growing body of research, it is still a challenge to apply active learning in joint entity recognition and relation extraction, especially in scenarios with low resources @xcite . ## computational prototype the strategy presented in section 3 is implemented as a web application that can be integrated with the brat annotation tool @xcite . this application is independent of brat and relies only on the file system to iteratively select batches of sentences and apply suggestions. the web interface is simple to use, allowing the user to ask for a new batch, and decide whether to accept, modify or discard the annotation suggestions (see figure 2 ). this tool is compatible with any entity and relation annotation schema that can be represented in brat standof ann format @xcite . as explained in section 3, two different machine learning models m e and m r must be implemented to evaluate the informativeness metric i(•). these models must provide probability estimates for each label, and should be efficient enough to be trained in the same time it takes a human annotator to annotate a single batch, such that the new batch is always ready. for the previous reasons, we selected two simple machine learning models based on standard formulations for the problems of entity recognition and relation extraction respectively. for entity model m e , we select a conditional random field (crf) classification model with syntactic and semantic features extracted with the spacy library. the extracted features include coarse and fine-grained part-of-speech tagging, lemmatization, a standard ner labeling, as well as indicator variables for several syntactic patterns (e.g., numbers, dates, punctuation, emails, urls, etc.). by this means, the entity recognition problem is framed as a sequence tagging problem using the bilouv encoding and viterbi decoding. special hand-crafted rules are designed to account for multi-word entities with discontinuous word spans. the uncertainty of each entity h(e i k ) is estimated by the normalized marginal probabilities of the crf model on the token sequence, averaging the probabilities of the tokens that correspond to the same entity. despite its simplicity, this model achieves an f 1 score of 0.78 in the entity extraction subtask of the ehealth-kd challenge 2020, which is competitive with state-of-the-art techniques in past benchmarks @xcite . in the case of the relation model m r , this subtask is more complex and simple baselines perform significantly worse than state-of-the-art models. however, since complex models cannot be trained in the required time, we decided to maintain a simple baseline. the problem of extracting all relations in a sentence is modeled as a set of independent classification problems between all pairs of entities in the sentence. each pair is represented by the same characteristics used in the entity recognition subtask, applied to both entities under analysis, plus a bag-of-words encoding of the tokens that appear in the smallest dependency subtree that contains both entities. the uncertainty h(r i j ) of each pair is computed from the marginal probabilities provided by a logistic regression model trained on each pair representation. for the information density metric id(•) an implementation of doc2vec from the gensim library is used. ## experiments and results to validate the effectiveness of the active learning strategy proposed in this research we selected a recent manually annotated corpus of spanish sentences in the biomedical domain, i.e., the ehealth-kd 2020 corpus @xcite . this selection was motivated by the relative complexity of the annotation schema proposed in this corpus, which contains different entity and relation types, multi-word tokens, overlapping annotations and other characteristics that make it a challenging annotation process even for human experts @xcite . the corpus contains a total of 1300 sentences in spanish, manually annotated with a general-purpose entity-relation schema. of these, a set of 1000 sentences is used as the unlabeled pool u, and the remaining 300 are used for testing the final performance of all machine learning models trained in the experimentation. the corpus has been split following the authors' recommendations. figure 2 shows an illustrative example of the annotation schema applied to 3 exemplary english sentences, in the context of the prototype application developed in this research. we simulated the assisted annotation process to evaluate the effect to annotating the corpus using active learning strategies versus annotating the corpus in the original order without suggestions (baseline). as the process of annotating a corpus is expensive, it was simulated using the gold annotations in the training collection. the improvement can be estimated by comparing how many sentences need annotating to reach a specific performance of the machine learning algorithms (measured in terms of f 1 in the testing collection). to study the relative impact of the different components of our query strategy, we evaluated three different variants. they consisted of using the full query strategy proposed in section 3 with β = 1, as well as considering only entity uncertainty ĥ(e i ) and relation uncertainty ĥ(r i ) respectively. figure 3 shows how the f 1 metric improved with each batch of sentences, for the first 500 sentences. the target f 1 is the final score obtained by training the models m e and m r on the full 1000 sentences of the corpus. in general, the curves that correspond to the active learning strategy (i.e., assisted variants) approach the target f 1 significantly faster than the unassisted baseline. to illustrate the degree of time-reduction achieved, figure 4 shows the minimum number of sentences that must be annotated to reach different relative target f 1 scores. for example, after annotating the first 400 sentences it is possible to achieve a 95% of the ultimate f 1 score when using all the corpus. however, to reach the target score, the first 880 out of a 1000 sentences must be annotated if the corpus is annotated in the original order (baseline). by contrast, using our active learning strategy only between 530 to 580 sentences must be annotated to reach the same target f 1 , thereby saving between a 35% and a 40% of human annotation time. another interesting finding is to estimate the extent to which the suggested annotations further reduce the total annotation time. a human annotator using the tool will need to accept some of the suggested annotations, correct the ones that are wrong and annotate the ones that are missing. each of these actions has a different cost in time. for quantifying the improvement in overall time that the suggestions produce, we assigned a relative cost (in terms of abstract time units) to each of the following types of annotations: missing annotations: annotations that the model did not suggest and the human annotator must produce, have a cost of 1 time unit. spurious annotations: annotations that the model suggested and are wrong, which must be eliminated by the human annotator, have a cost of 2 time units. correct annotations: since the human annotator must at least recognise the annotation is correct, the cost is 0.25 time units. partial annotations: annotations that are partially correct either because the spans are partially covering or the label is wrong, have a cost of 0.5 time units. this cost structure assumes that the problem of correcting wrong annotations is more complex than simply producing the correct annotations, while acknowledging that even agreeing with correct annotations has a non-zero cost. for an active learning strategy to be helpful it must provide enough correct annotations to outweigh the cost of correcting the wrong annotations; hence, it should prioritise precision over recall. figure 5 shows the relative effect (in terms of reducing the overall annotation time) of enabling annotation suggestions for different combinations of the entity and relation thresholds u e and u r . it can be observed that on average, the entity suggestions produce a positive effect (green colour) across a wide range of thresholds, while the relation suggestions tend to produce a negative effect as more suggestions are allowed. this is a direct consequence of the m r relation model's performance, which achieves at most an f 1 score of 0.27, while the entity model m e achieves up to a 0.78 score. the optimal time reduction is achieved for an entity threshold u @xmath0.4 and a relation threshold u @xmath1, producing an estimated 24% reduction in the total annotation time. interestingly, these parameters result in an overall performance for the machine learning model of f 1 = 0.54, with a precision of 0.78 and a recall of 0.41. as expected, given the asymmetrical cost structure, it is preferable to prioritise precision rather than recall for the annotation suggestions. figure 4 : minimum number of sentences necessary to reach a specific performance relative to the target f 1 with each active learning strategy for sentence suggestion. the machine learning model used for entity recognition m e achieves a result comparable with the state-of-the-art in this corpus while being simple enough to be trained during annotation. not only does the model produce a significant reduction in the number of sentences that need to be annotated but it also is capable of pre-annotating entities that are often correct, even when factoring in the significantly higher cost of correcting the wrong suggestions. by contrast, the relation extraction model m r performed significantly worse than current state-of-the-art in this corpus. however, even if the pre-annotated relations suggested by this model are on average not beneficial, it is interesting to note that using only the uncertainty of relations ĥ(r i ) as a query strategy still produces a significant time reduction (see figure 4 , assisted (relations)). unfortunately, all good performing models for this problem are composed of complex deep learning architectures that cannot be trained sufficiently fast enough to be used during the annotation process in a commodity hardware. indeed, using simple models is necessary in active learning scenarios where algorithms must be trained interactively, but even without considering this issue, there are additional factors to consider related to model complexity. we argue that an interesting trade-off exists between the capacity of a model and its usefulness for active learning. very simple models (underfit) will have a high uncertainty in all samples, while very complex models (overfit) will overestimate their certainty. in both cases, the informativeness i(•) for all sentences will be very similar, and there is no sensible way to choose the most informative ones. this suggests that there may be an optimal middle ground where the model learns enough to provide useful suggestions while still maintaining a healthy level of uncertainty. the fact that even weak baselines (like the relation model m r ) are still a useful source of information when actively selecting unlabelled sentences is one surprising conclusion of our research. this seems to suggest that even in very complex scenarios where state-of-the-art models are impossible to train interactively, using weaker surrogate models can still provide a significant benefit for human-in-the-loop learning. regarding the generalisation of our approach, the fact that the corpus is in the spanish language is irrelevant for our experimental results since the machine learning models used are language-agnostic and no language-specific heuristics were applied. hence, these results should generalise to other languages and annotation schemas albeit with different baseline f 1 scores according to the complexity of the underlying learning problem. nevertheless, we are interest in evaluating our approach using languages other than english since the creation of linguistic resources is one of the main difficulties of nlp research, especially for other languages. an ongoing research priority is to validate this strategy on other corpora with different annotation schemas. in future work, we will explore how to explicitly control the complexity of a model during the active learning process by controlling the model's capacity. two strategies that can be analysed are the use of ensemble methods and deep learning architectures with early stopping. in both cases, the intuitive idea is to iteratively refine a machine learning model up to the point where a sufficiently good performance is achieved but before the model overfits on the small labelled set of sentences, such that uncertainty measures are still relevant. another interesting scenario in which to apply this approach is when many annotators exist for the same text. in this case, the models can learn contradictory hypotheses due to differences between annotators. interestingly, in the case of active learning, this is a positive phenomena, since inter-annotator disagreement is a meaningful proxy for annotation difficulty. active learning models trained on a pool of sentences with multiple, possibly contradictory annotations, will naturally tend to select sentences that are more likely to cause disagreement between annotators. in this context, it can be interesting to explore query-by-ensemble methods where each model in the ensemble is trained on a different annotator's pool to maximise model variance. ## conclusions in this article, we present an approach for reducing the time involved in manually annotating a corpus of natural language sentences that contains entities and relations. this approach uses active learning with uncertainty sampling and weighted density, and provides an estimated reduction of 60% of total annotation time in a simulated experiment with a real corpus. this improvement is derived from two independent factors: intelligently sorting which sentences to annotate and providing pre-annotated suggestions with a high-degree of certainty. the proposed strategies have been implemented into a computational tool that is applicable to a broad range of corpus annotation schemas and is available for the research community.