Characterizing the Impacts of Instances on Robustness (2023)

Abstract: Building robust deep neural networks (DNNs) against adversarial attacks is an important but challenging task. Previous defense approaches mainly focus on developing new model structures or training algorithms, but they do little to tap the potential of training instances, especially instances with robust patterns carring innate robustness. In this paper, we show that robust and non-robust instances in the training dataset, though are both important for test performance, have contrary impacts on robustness, which makes it possible to build a highly robust model by leveraging the training dataset in a more effective way. We propose a new method that can distinguish between robust instances from non-robust ones according to the model’s sensitivity to perturbations on individual instances during training. Surprisingly, we find that the model under standard training easily overfits the robust instances by relying on their simple patterns before the model completely learns their robust features. Finally, we propose a new mitigation algorithm to further release the potential of robust instances. Experimental results show that proper use of robust instances in the original dataset is a new line to achieve highly robust models.

## introduction deep neural networks (dnns) have made significant progress in a number of fields, such as computer vision @xcite and natural language processing @xcite , but they are susceptible to adversarial examples, which are crafted by adding small, human-imperceptible adversarial perturbations to normal examples @xcite . to improve the robustness of models, many techniques have been developed, such as robust architecture search @xcite , model pruning @xcite , adversarial training @xcite and regularizations @xcite . however, most of these defensive approaches focus on developing new model structures or training algorithms, ignoring the fact that training data has a decisive impact on the trained model. it is widely believed that the more abundant the labeled data, the higher the likelihood of learning diverse features, which in turn leads to well generalized models @xcite . however, in practice, adversarial robustness remains a challenge that cannot be solved simply by scaling up the dataset @xcite ). on the one hand, recent theoretical work argues that training a model invariant to adversarial perturbations requires a much larger dataset than that is required for standard generalization @xcite . on the other hand, the model tends to use any available signal to maximize accuracy, and thus, adversarial examples can arise as a result of manipulating highly predictive but fragile features in the data @xcite . the above evidences indicate that, adversarial vulnerability is not only associated with the training data size, but is also an inherent property of the data. most of existing defense methods treat all data equally, which requires us to have a closer look at the dataset whether all instances contribute equally to improving the robustness of the model. in this paper, we focus on exploring the relationship between training data and adversarial robustness, with the aim of figuring out the following questions: q1: which instances are important for adversarial robustness, and how do we find them? we delve into the training dynamics of each instance and find that instances have different robustness. even without the help of adversarial training, a portion of the data progressively becomes more robust to perturbations, and these instances are called robust instances. 1 when we train models on these data in isolation, they are more helpful in improving robustness than other subsets of data with the same size. motivated by this phenomenon, we propose a metric based on the adversarial loss of each instance across the training epochs to indicate the impact of training instances on robustness. as shown in figure 2 , this metric reveals three distinct regions in the dataset: a region with inherently robust instances, a region with non-robust instances, and a region with instances that fluctuate between robust and non-robust. based on the proposed metrics, a significant portion of robust instances can be selected from the training dataset to significantly improve the robustness of the model. q2: why is the benefit of robust instances held back when mixed with other training instances? how to make the best out of them to improve robustness? dnns exhibit memorization effects in that they first memorize easy and clean patterns, and then hard and noisy ones @xcite . the robust instances have simple and straightforward task patterns that better align with human perception. we find that the model under standard training easily overfits the robust instances by relying on their simple patterns before the model starts to learn their robust features, which limits the power of robust instances. to address this problem, we propose a new mitigation algorithm that impedes overconfident predictions by regularizations for robust instances to avoid overfitting. the proposed method effectively releases the potential of robust instances, while other instances contribute little to robustness improvement. in particular, our contributions are: our analysis and results are verified by extensive experiments. ## characterizing robust instances we find that the model have different sensitivities to perturbations of the instances during the training phase, and this property is strongly correlated with the robustness of the trained models. based on this, we propose an approach to identify these innately robust instances and demonstrate that they contribute more to robustness when trained in isolation. ## data selection using data maps the data map shows the different regions in the dataset. it is natural to wonder what role instances from different regions play in learning and adversarial robustness. we answer this question empirically by training the model solely on instances selected from each region, and then performing standard the training strategy is simple and straightforward -we train the model from scratch on the subsets of the training data selected by ranking instances based on the statistics described above. we hypothesize that robust and swing instances are more important for improving the robustness of the model because they have more robust features and more stable to perturbations as training proceeds. we compare the performance of the models trained on different data regions with other baselines. all considered subsets contain 50% of the training data (to control the effect of training data size on performance). baselines. the most natural baseline is using all of the data (100% train). our data selection baselines consider the subsets of 50% of the most robust (50% robust), 50% of the most non-robust (50% non-robust) and 50% of the most indecisive (50% swing), which is a trade-off between robust and non-robust instances. finally, we also compare our models trained on data subsets with a textual adversarial training method, freelb @xcite , which is a strong defense baseline in nlp (100% freelb). results. we report accuracy on the test set to evaluate generalization performance, and accuracy under attack using textfooler @xcite as the attacker to measure adversarial robustness. in figure 3 , we show the evolution of generalization performance and adversarial robustness when we change the size of the selected subsets of data. each point in the figure corresponds to retraining the model from scratch (with the same hyperparameters as the base model) on an increasingly larger subset of the training data. we observe that the generalization performance improves rapidly when we increase the size of non-robust and swing subsets. this means that the original training dataset is redundant to the model, and training the model on a small portion of the data also gives excellent generalization performance. comparatively, by in-creasing the number of robust and swing, 50% of the data is sufficient to achieve excellent robustness performance, while more data actually hurts robustness. on the one hand, we require sufficient data to improve the generalization, and on the other hand, excessive well-generalized data will harm the robustness. we cannot rely on data selection alone to get the best generalization and robustness at the same time, and in the next sections we will show how to leverage instances from different regions to achieve win-win results in performance. ## why do robust instances fail? in the previous section, we observed that robust instances are import for adversarial robustness. this leads us to wonder why robust instances fail in competition with non-robust instances, and is it inevitable when we train these data together? in this section, we provide further insight into the training procedure by investigating the interactions between robust and non-robust instances. ## experiments in this section, we provide experimental results using bert-base @xcite as a backbone model on the sst-2 @xcite , qqp @xcite and agnews @xcite dataset baseline accuracy robustness sst-2 standard training 92.1 6.1 + flooding 91.9 46.0 + label smoothing 92.5 41.4 qqp standard training 90.1 20.8 + flooding 90.9 39.4 + label smoothing 91.0 45.1 results. ## related work text attacks typically generate adversarial examples by manipulating characters @xcite , words @xcite @xcite @xcite , phrases @xcite of the original input, or even entire sentences @xcite , to to counter adversarial attacks, a number of defense methods have been developed, such as adversarial training @xcite @xcite , information compression @xcite , and model pruning @xcite . however, most of these defensive approaches focus on developing new model structures and training algorithms, ignoring the fact that training data has a decisive impact on the robustness of the model. in this paper, we propose a new defense method from a data perspective to improve the robustness of the model by better utilizing the robust instances in the original dataset. a body of work tends to view the existence of adversarial examples as an inevitable consequence of using high-dimensional inputs and the statistical fluctuations due to data size and data noise @xcite . however, @xcite claim that adversarial vulnerability is a direct result of sensitivity to wellgeneralizing features in the data. data-related studies in the field of robustness focus on improving the robustness of models using more unlabeled data @xcite and data augmentation @xcite . @xcite find that low-quality data may not be useful or even detrimental to adversarial robustness. to the best of our knowledge, no work has attempted to characterize the impact of each instance in the training dataset on robustness. ## conclusion in this paper, we address the challenge of understanding the impact of training instances on robustness, particularly to improve the robustness of the model. we study the adversarial losses of each instance during training and show how these losses can be used as a metric to identify robust instances. our empirical results suggest that the proposed metric is a very promising measure for characterizing the contribution of training instances to robustness, and can be used to prune out non-robust instances to construct a dataset that is inherently robust. furthermore, we show that standard training can easily overfit robust instances by relying on their simple patterns before the model learns the robust features. the robustness of the model can be significantly improved by mitigating the overfitting of the model to robust instances during the standard training. further investigations in this direction may lead to new technologies for adversarial defense.