Re-weighting Tokens: A Simple and Effective Active Learning Strategy for Named Entity Recognition (2023)

Abstract: Active learning, a widely adopted technique for enhancing machine learning models in text and image classification tasks with limited annotation resources, has received relatively little attention in the domain of Named Entity Recognition (NER). The challenge of data imbalance in NER has hindered the effectiveness of active learning, as sequence labellers lack sufficient learning signals. To address these challenges, this paper presents a novel re-weighting-based active learning strategy that assigns dynamic smoothing weights to individual tokens. This adaptable strategy is compatible with various token-level acquisition functions and contributes to the development of robust active learners. Experimental results on multiple corpora demonstrate the substantial performance improvement achieved by incorporating our re-weighting strategy into existing acquisition functions, validating its practical efficacy. We will release our implementation upon the publication of this paper.

## introduction the effectiveness of deep neural networks has been well-established on diverse sequence tagging tasks, including named entity recognition (ner) @xcite . however, the evolving complexity of these models demands substantial labeled data, making annotation a resource-intensive endeavor. active learning (al) addresses this by astutely focusing on information-rich sequences, substantially reducing data that needs expert annotation. such a strategy proves invaluable in ner, as it brings down costs and accelerates learning by honing in on complex or dynamic entities. al's iterative process smartly picks subsets from an unlabeled pool, leveraging expert-annotated data to refine the model. modern al approaches primarily pivot on model uncertainty @xcite @xcite @xcite or data diversity (sener and savarese, * * corresponding author 2017; @xcite @xcite . the primary challenge in applying active learning to sequence tagging lies in addressing data imbalance at the entity level. traditional methods of active sequence tagging, such as those referenced by @xcite @xcite , typically generate scores for sentences by summing or averaging the tokens within them, thereby treating each token equally. @xcite attempted to address this by segmenting sentences for token-level selections, however this led to loss of context and semantic meaning, impairing human understanding. to effectively tackle this data imbalance issue, it is imperative to focus on the token level within the sentences. by implementing a strategy that redistributes sentence selection through assigning weights to the tags of the tokens, a more balanced and efficient handling of the data can be achieved. in supervised learning (sl) and semi-supervised learning (ssl), re-weighting is a widely researched method to tackle data imbalance. however, they have limitations to be directly applied to active learning. for example, @xcite ; @xcite assume the labeled data and the unlabeled data have the same label distribution, while we usually have no prior information about the unlabeled data in active learning; @xcite use the confusion matrix on the labeled data to approximate the label distribution of the unlabeled data. in active learning, the labeled data pool can be quite small, so the approximation can cause strong biases; @xcite suggest a gradientbased method to compute weights, while computing gradients significantly increases the computational cost for active learning. in this paper, we focus on the feasibility of adapting reweighting-based methods from sl, ssl to active sequence tagging, which has not been developed in existing works. our contributions can be summarized as follows: these traditional approaches, however, have their limitations. @xcite pointed out that summing over all tokens, as done in previous methods, can introduce bias towards longer sentences. to overcome this, they proposed maximum normalized log-probability (mnlp), a normalized version that mitigates this issue. in a more recent development, lowest token probability (ltp) @xcite selects the sentence containing the token with the lowest probability. additionally, bayesian active learning by disagreement (bald) @xcite uses monte-carlo dropout to approximate the bayesian posterior, which makes it feasible to approximate the mutual information @xcite between outputs and model parameters. it is worth noting that we exclude subsequencebased active learning methods @xcite @xcite ) from our comparison. in their setting, the query is a subsequence instead of the whole sentence. as they admitted, it may be not practical in the real world because subsequences can be meaningless. human experts usually need the context to label tokens. re-weighting methods for supervised learning and semi-supervised learning. many real-world datasets exhibit a "long-tailed" label distribution @xcite @xcite . imbalanced supervised learning has been widely researched. re-weighting the loss @xcite @xcite ) is one of the popular methods on this topic. in recent semi-supervised learning methods, re-weighting is gradually gaining attention. @xcite utilize the unlabeled data with pseudo labels according to an estimated class distribution, @xcite develop a framework to refine class distribution, @xcite use the effective number defined by @xcite to produce adaptive weights. ## experimental setups datasets: in this section, we evaluate the efficacy of re-weighted active learning on three widely used ner datasets, which are listed as follows, the statistical data of each dataset is provided in the appendix b. active learning settings: the initial size of the labeled data pool is 30, the number of active learning iterations is 10, the query size varies from {15, 30, 50}. in each active learning iteration, we selected a batch of unlabeled data, queried their labels, added them to the labeled data pool, and then re-trained the model. we evaluated the performance of algorithms by their mean f1-scores with 95% confidence interval for five trials on the test dataset. baselines: we considered five baselines: (1) randomly querying, (2) least confidence (lc) @xcite , (3) maximum normalized logprobability (mnlp) @xcite , (4) sequence entropy (se) @xcite , (5) bayesian active learning by disagreement (bald) @xcite . the details of the baselines are mentioned in the appendix a. hyperparameter setting: as our reweightingbased methods have a hyperparameter β to be tuned, we first conduct a grid search over {0.01, 0.1, 0.2, 0.5, 1} in the re-weighting lc case on each dataset, and fix the optimal value β = 0.1 for all experiments. please see appendix c for the performances of all hyperparameters. ## conclusion in this work, we pioneered the use of re-weighting in active learning for ner to tackle the label imbalance issue. our method uses the class sizes in the labeled data pool to generate a weight for each token. we empirically demonstrate that our reweighting-based method can consistently improve performance for each baseline, across a large range of ner datasets and query sizes.