Measuring Robustness for NLP (2022)

Abstract: The quality of Natural Language Processing (NLP) models is typically measured by the accuracy or error rate of a predefined test set. Because the evaluation and optimization of these measures are narrowed down to a specific domain like news and cannot be generalized to other domains like Twitter, we often observe that a system reported with human parity results generates surprising errors in real-life use scenarios. We address this weakness with a new approach that uses an NLP quality measure based on robustness. Unlike previous work that has defined robustness using Minimax to bound worst cases, we measure robustness based on the consistency of cross-domain accuracy and introduce the coefficient of variation and (epsilon, gamma)-Robustness. Our measures demonstrate higher agreements with human evaluation than accuracy scores like BLEU on ranking Machine Translation (MT) systems. Our experiments of sentiment analysis and MT tasks show that incorporating our robustness measures into learning objectives significantly enhances the final NLP prediction accuracy over various domains, such as biomedical and social media.

## introduction evaluation criteria serve as learning objectives and model assessment standards and are crucial for nlp research. conventional evaluation methods compute the accuracy or errors compared to the reference on a predefined test set, such as bleu or ter. since the evaluation results highly depend on the test set, they may not apply to real-world test inputs that come from an unknown distribution. for example, one may query a legal document while the system is trained on the news. therefore, it is necessary to define a measure that can give an idea of how robust system performance will be on unseen test data so that we can predict whether our model is generalizable to new domains. there has been influential investigation into defining robustness instead of accuracy alone @xcite @xcite . robustness of a machine learning model can be described as the characteristic of how accurate the model is in making its predictions when tested on a new (but similar) dataset. for example, one definition of robustness claims "if a testing sample is similar to a training sample, then the testing error is close to the training error" @xcite . such testing samples, which are samples dissimilar from training samples are known as adversarial examples. many studies on robustness measures @xcite @xcite focus on the worst-case scenarios with adversarial inputs. however, because the worst-case appears very infrequently, in particular, if artificially simulated @xcite @xcite , a worst-case analysis has the inherent problem that if the worst case is far from the typical, then the quantification assigns a numerical value to cases that occur rarely, like outliers. for instance, if a system accuracy has a very small variance in a million test cases but fails dramatically in one, then minimax will label this system less robust than a system with a high accuracy variance. while focusing on the worst-case can be important for some areas of computer science and engineering like astronautics, but in our view, it is usually not desirable in the nlp context. in our view, robustness is a notion that should address typical behavior, not atypical and rare behavior. note that if the worst-case differs a lot from the typical, a statistical notion of robustness should make this apparent. additionally, a useful formalization should ignore atypical behavior of our system -because only focusing on a worst-case is not practical. in this paper, we introduce novel definitions of robustness for nlp systems. instead of defining robustness in terms of worst-case performance, we define it in terms of the typical behavior of the system and the consistency in their performance of the system. under this definition, the more inconsistent the model predictions are, the less robust the model is. specifically, we simulate unknown test domains using leaving-one-out cross-validation to measure the variance of model accuracy on a set of test domains. more precisely, we introduce the coefficient of variation to quantify the variants of accuracy that is non-linear. we further expand this notion to a probabilistic definition, called (ϵ, γ)-robustness, where the higher probability of consistent behavior, the more robust the system. we provide statistical guarantees on the performance of an nlp model through the following assertion: both variation of coefficient and (ϵ, γ)-robustness takes a new direction away from the worst-case result to a general consistency of the model quality. they are meta-evaluation methods that measure the consistency of user-defined quality metrics. thus, any standard evaluation metrics can be applied to our paradigm. our robustness measures are evaluated by comparing with human rankings on system outputs and by simulating unknown test domains using cross-validation. our experimental results show that our robustness measures have higher agreements with human rankings on machine translation (mt) system submissions than bleu scores. in summary, the main contributions of this work include: the rest of the paper is as follows: in section 2, we describe the three definitions of robustness. in section 2.4, we connect the coefficient of variation with (ϵ, γ)-robustness and show their relationship. section 3 describes the six algorithms to compute robustness metrics we used during our experimentation. section 4 describes our evaluation methods including cross-validation and manual comparison. section 5 describes the experiments that evaluate our robustness measures. in section 6, we discuss the previous literature on robustness in machine learning. section 7 concludes the paper. ## definition of robustness we define robustness as the consistency in the behavior of a machine-learned system. we think of it as the standard or typical behavior of the system. the more this behavior deviates from the typical, the less robust the system is defined to be. notice that this definition does not give a notion of whether the performance of the system is good or bad. a system with consistent terrible performance is still a robust system under this definition. for example, in terms of mt, this definition becomes the consistency in translation performance for a trained mt system. ## experiments data & tools we evaluate our robustness metrics in machine translation and sentiment classification. for machine translation, we use four different english-french nmt models trained on wmt'14, biomedical'18, islwt'17, and mtnt'18 datasets. the models are trained using 35m, 2m, 200k, and 40k sentences, respectively. the wmt'14 model is the pre-trained model provided by facebook research @xcite and the remaining were trained in house using convs2s toolkit @xcite till convergence. the development data used for the in house trained models include khresmoi for biomedical, test2014 and test2015 for iwslt, and mtnt'18 development for mtnt. the four different test data were also from the same domains, including wmt (newstest14), biomedical (edp2018), iwslt (test2017), and mtnt (mtnt2018). for sentiment classification, we use amazon product review dataset @xcite . specifically, we train four models using pre-processed balanced reviews from dvd, kitchen, electronics and books domains where each review is labeled with 0 or 1. we randomly split 15% of the sentences as validation set and the rest as training set. we evaluate the classification model on unprocessed reviews from 21 domains different from the training domain, and compute the mean, var and coefficient of variance of these 21 test accuracy scores. then we leave the apparel, baby, beauty, grocery and music domain out to compute the mean, variance, and the coefficient of variation on other 20 domains. different from machine translation that uses sentence level performance score (i.e. bleu) to compute robustness, we use domain/document level performance score (i.e. accuracy) to compute robustness metrics in sentiment classification. ## related work there have been substantial amount of work trying to improve the robustness of nlp models. among those, a majority of focus lies on the vulnerability of nlp models to input perturbations, such as generating adversarial examples. for instance, @xcite uses controllable attributes irrelevant to task labels to generate diverse adversarial texts. @xcite uses pre-trained masked language ## conclusion we introduce variance, coefficient of variation and (ϵ, γ)-robustness to measure the robustness of an nlp model's typical behavior. our robustness metrics outperform bleu in mt system performance rankings and highly agree with human robustness assessment. our work demonstrates a successful step towards general robustness evaluation and optimization goals.