Effects of Human Adversarial and Affable Samples on BERT Generalization (2023)

Abstract: BERT-based models have had strong performance on leaderboards, yet have been demonstrably worse in real-world settings requiring generalization. Limited quantities of training data is considered a key impediment to achieving generalizability in machine learning. In this paper, we examine the impact of training data quality, not quantity, on a model’s generalizability. We consider two characteristics of training data: the portion of human-adversarial (h-adversarial), i.e. sample pairs with seemingly minor differences but different ground-truth labels, and human-affable (h-affable) training samples, i.e. sample pairs with minor differences but the same ground-truth label. We find that for a fixed size of training samples, as a rule of thumb, having 10-30% h-adversarial instances improves the precision, and therefore F1, by up to 20 points in the tasks of text classification and relation extraction. Increasing h-adversarials beyond this range can result in performance plateaus or even degradation. In contrast, h-affables may not contribute to a model’s generalizability and may even degrade generalization performance.

## introduction deep learning models have dominated natural language processing (nlp) task leaderboards as seen in benchmarks such as glue @xcite , where the model performance is assessed using a held-out test set. however, the performance of these models may not be replicable outside of a given test set @xcite , or they may not capture any relevant understanding of how language works @xcite . benchmark test sets may not be sufficiently representative @xcite . these factors make it difficult to know when a model will work in practice and raise questions about model generalizability. as an example of a possible disconnect between nlp models and what might be considered to be training set alien language label random timbuktu scio ghy positive shyp meyo mlue neutral h-adversarial timbuktu scio ghy positive timbuktu meyo ghy neutral conventional wisdom in machine learning is: the more data, the better. limited training data is acknowledged as a key obstacle to achieving generalizability. in this paper, we study how human adversarial samples (h-adversarials) and human affable samples (h-affables) can impact generalisation performance. we define h-adversarials as samples that look very similar to each other, but have different ground-truth labels. we use the term "human adversarial" to differentiate it from adversarial machine learning which aims to fool the machines into making incorrect predictions by applying minor perturbations to a source sample, where the ground-truth label of the perturbed sample remains the same as its corresponding source @xcite . we define h-affable samples as training samples that are very similar and have the same ground-truth label. the intuition behind h-adversarials is to explicitly guide the model to learn discriminatory features through training data, thus reducing its reliance on spurious features (see data augmentation strategies for adding training samples by making minor perturbations that preserve (h-affables) or alter (h-adversarials) instance labels have been shown to improve model robustness @xcite . here, we explore the question, "how do various proportions of haffables and h-adversarials in training samples affect the model's generalization performance?" we examine three nlp classification tasks, including text classification, relation extraction, and keyword detection, and perform an empirical study on the impact of h-adversarial and h-affable samples. the experimental results confirm our hypothesis -with fixed training data size, a model's performance can exhibit a significant improvement with a moderate proportion of h-adversarial samples, while h-affable samples can contribute to lowering model performance. these results reveal how data quality, not data quantity alone, may affect model performance, thereby providing insights into data annotation and curation procedures. with limited data annotation budgets, creating high-quality training samples is critical to guide a model towards learning discriminative features. ultimately, this will lead to robust generalization performance. ## h-adversarials and h-affables given a source sample s assigned with the groundtruth label l, we define a h-adversarial sample sl ′ as a sample generated by applying δ changes to s such that l is changed to a different label l ′ , where δ is less than a pre-defined threshold ϵ: similarly, we define a h-affable sample sl as a sample generated by applying δ changes to s but the ground-truth label l remains the same: in this paper, we focus on the tasks of text classification and relation extraction, where a sample corresponds to a text. given a reference sample s and another sample s, we measure their input difference δ s,s using the word error rate based on levenshtein edit distance @xcite : given a training set for an nlp classification task, we define the h-adversarial rate r ll ′ hv for the class label l and l ′ as the portion of samples in class l ′ that have an h-adversarial sample in class l: here, s l is the set of training samples labelled with l, s l ′ is the set of samples labelled with l ′ , and ñl ′ is the number of samples labelled with l ′ that are within ϵ edit distance to at least one sample labelled with l, i.e., can be derived by applying less than ϵ changes to another sample with label l. the value of r ll ′ hv can range between [0, |s l ′ |], where 0 indicates that there are no h-adversarial samples for label @xmath0. when r ll ′ hv is greater than 1, it indicates that a single source sample s l can have multiple h-adversarial samples sl ′ . it's worthwhile to note that in such cases, these h-adversarials are all similar to s l in terms of input, and may haffables for each other. we define h-affable rate as r l hf for label l as follows: the h-adversarial and h-affable samples can be curated by domain experts, and also can be generated automatically in some cases if appropriate rules can be defined given the context of a task. taking a binary relation extraction task as an example, a given source text annotated with participating entities can be used to generate multiple samples by marking different entities as involved in the relationship, as shown in ## method we examine the impact of h-adversarials and haffables on three different nlp classification tasks, including text classification, relation extraction, and a customized keyword detection task (detailed in section 3.1.2). next, we introduce these tasks and datasets, focusing on the generation of hadversarials and h-affables in these tasks. ## conclusion data annotation and curation is the most human intensive, expensive and time consuming aspect of developing machine learning applications. curating large volumes of data without understanding how the data distribution contributes to improving a model can result in misdirected effort. in addition, the resulting models with poor generalization performance may be inadequate to automate or aid manual curation efforts. hence, understanding the characteristics of training samples is key to ensuring that the right variety of samples are collected to create robust machine learning applications. in our work, we find that additional similar examples sharing a label, h-affables, may not contribute to improving performance despite increasing training data quantity. similar examples with divergent labels, h-adversarials, add more value. our work contributes insights into the preparation of high-quality training data to support robust model generalization. specifically, we demonstrate how moderate quantities of h-adversarials in combination with more diverse samples can substantially improve robustness.