Instance-adaptive training with noise-robust losses against noisy labels (2021)

Abstract: In order to alleviate the huge demand for annotated datasets for different tasks, many recent natural language processing datasets have adopted automated pipelines for fast-tracking usable data. However, model training with such datasets poses a challenge because popular optimization objectives are not robust to label noise induced in the annotation generation process. Several noise-robust losses have been proposed and evaluated on tasks in computer vision, but they generally use a single dataset-wise hyperparamter to control the strength of noise resistance. This work proposes novel instance-adaptive training frameworks to change single dataset-wise hyperparameters of noise resistance in such losses to be instance-wise. Such instance-wise noise resistance hyperparameters are predicted by special instance-level label quality predictors, which are trained along with the main classification models. Experiments on noisy and corrupted NLP datasets show that proposed instance-adaptive training frameworks help increase the noise-robustness provided by such losses, promoting the use of the frameworks and associated losses in NLP models trained with noisy data.

## introduction the wide availability of neural network models has allowed development of novel and complex natural language processing tasks, many of which are in low-resource settings. with new definitions of tasks comes challenges of constructing new datasets, which is still an expensive and timeintensive endeavor. many researchers have resorted to constructing datasets by using completely automated pipelines (e.g. @xcite @xcite @xcite . however, silver labels collected this way are still quite noisy compared to expert annotation. because such methods have been gaining popularity and practicality, it is impor-tant to explore ways to ensure good performance in spite of noisy labels in training data. the widely-used cross entropy (ce) loss as the optimization objective in classification tasks has been shown to overfit to label noise @xcite . several noise-robust losses have been designed for training models with noisy labels @xcite @xcite , which were a convenient way to address the noisy label issue and shown to be more robust than ce. experiments are usually conducted on computer vision datasets such as cifar @xcite and @xcite . these noise-robust losses usually have hyperparameters for determining the strength of the noiserobustness at the dataset level. however, individual training instances may have different amounts of noise, derived from biases within models used in the automated pipeline. moreover, noisy labels in natural language datasets potentially pose a greater challenge because instances of the same true label may not share similar surface features. therefore, this work focuses on the improvement of training with noisy labels using noise-robust losses in nlp. we propose two robust training frameworks where the noise-robustness hyperparameters are instancespecific. they are predicted by label quality predictors, which are trained either jointly or iteratively with main models in order to take advantage of any correlation between label quality and input features. such frameworks are tested with many noise-robust losses on several noisy and corrupted nlp datasets. results from experiments show that: ## noise-robust losses we first define a dataset d for single-label classification as a tuple of input features and corresponding labels {x i , y i } n @xmath0, where y @xmath1, ..., k} is the annotated label and y @xmath2, 1} k is a one-hot representation of the annotated label with k total possible classes for training instance i. given a classification model f with trainable parameters θ, the predicted conditional distribution of the classes from the model is d @xmath3. training the model f is then trying to find the set of parameters θ * which minimizes the empirical risk θ * = arg min θ n @xmath4, y i ), with l being a loss function which takes the model output and the annotated label, and returns a non-negative value. ce is the commonly used loss for classification, which is defined as the negative log-likelihood of the annotated class ce(y, d) = -y log f (x). theoretical results @xcite have shown that losses which satisfy k @xmath5, ȳk ) = c, ∀@xmath6, ∀f, with c being some constant and ȳk being a onehot representation of a label at k, are robust against symmetric and label-dependent noise with noise rate η < k-1 k , which is the probability that the annotated label y is not the true label ŷ. however, for losses which cannot satisfy this condition where the sum of loss values with respect to all classes is constant, they are more noise-robust if the above term is bounded instead of unbounded. examples of each condition include ce being unbounded, mean squared error being bounded and mean absolute error (mae) being constant. ## instance-adaptive noise-robust training frameworks when supervised models are used in pipelines to generate silver labels, the resulted machineannotated dataset reflects biases and inaccuracies learned by such models, which may be caused by spurious relations between instance-level features, such as words, phrases and syntactic constructions, and labels in datasets on which these models are trained. this in turn causes some instances more likely to receive noisy silver labels than others. ideally, each training instance should have its own noise-robustness β value, which is certainly hard to manually tune. we propose that each instance should be assigned a different β with its value calculated by a label quality function q : {x i , y i } → β i , β @xmath7, 1). foot_0 motivated by the intuition that label errors in automated pipelines are correlated with difficult input features or learned biases, we model the function q with a neural network, which is expected to capture the complex relationship between inputs and quality of silver labels. we propose two instance-adaptive frameworks for training classification models with noise-robust losses with instance-specific β: the first training framework jointly trains the main model and the data-quality predictor (section 3.1) and the second one takes additional supervision of label quality and iteratively trains the main module and the data predictor (section 3.2), shown in figure 1b and 1c . ## experiments for all experiments, a model selection procedure similar to a common use case is adopted: one β µ is first selected from 0.1 to 0.95 in increments of 0.05, and performance of the trained models on the noisy development dataset with 3 different random seeds and the chosen β is compared. performance of the model with the best development result is reported for the noisy datasets to simulate the common use case. means and variances of the three models with the best performing β µ are reported for the clean datasets for better understanding of model behavior. all other hyperparameters of the models, such as the learning rate and the batch size, are tuned with the ce loss and kept fixed. 3 the experiment conditions include six different noiserobust losses including wce, bsl s , bsl h , gce, sce and er-gce, along with ls and ce as baselines. a for sce is set to -4 following previous work @xcite . they also include three 3 detailed information on the models and training procedures can be found in the appendix. ## related work there have been many different approaches to address the noisy label problem. one such approach relies on knowledge of clean labels @xcite @xcite , while another tries to estimate the label-dependent @xcite or annotatordependent @xcite noise distributions, many with neural network layers @xcite @xcite . such methods have seen some application in natural language processing @xcite @xcite . different training strategies have also been proposed to increase the robustness @xcite , many of which require training of auxiliary networks to reweight samples @xcite @xcite . complementary labels @xcite are also used for negative learning for robustness @xcite . regularization techniques such as drop-out @xcite also show positive results in combating noisy labels. @xcite proposed adding auxiliary variables into normal loss functions for regularization, which act as instance-specific priors over the predicted distributions to ease the training difficulty when labels are noisy. conceptually similar to the joint training with instance-adaptive βs proposed in this work, the regularization method in @xcite ) may be complementary to noise-robust losses explored in this work, because noise-robust losses may enjoy further improvement when combined with trainable instance-specific priors. noise-robust losses are another way to counter label noise @xcite . the noise-robustness of some losses, such as mae, was shown theoretically in @xcite . new noise-robust losses have also been proposed where some of the losses have a passive component attached to ce for noise-robustness @xcite , which was used by @xcite for reading comprehension with noisy data. others behave like a mixture of mae and ce @xcite . other methods, such as normalization @xcite of losses and use of determinant-based mutual information @xcite as a finetuning loss have also shown to be robust to noise. ## conclusion this work focuses on combating noisy labels in nlp datasets by means of adaptive training with noise-robust losses. two novel instance-adaptive training frameworks are proposed and investigated along with several noise-robust losses including a new er-gce loss. experiments on different datasets show the effectiveness of the approach: the adaptive training frameworks help models achieve the best performance on noisy datasets, and the er-gce shows great noise-robustness among the previously proposed losses.