L earn2 W eight: Parameter Adaptation against Similar-domain Adversarial Attacks (2022)

Abstract: Recent work in black-box adversarial attacks for NLP systems has attracted attention. Prior black-box attacks assume that attackers can observe output labels from target models based on selected inputs. In this work, inspired by adversarial transferability, we propose a new type of black-box NLP adversarial attack that an attacker can choose a similar domain and transfer the adversarial examples to the target domain and cause poor performance in target model. Based on domain adaptation theory, we then propose a defensive strategy, called Learn2Weight, which trains to predict the weight adjustments for target model in order to defense the attack of similar-domain adversarial examples. Using Amazon multi-domain sentiment classification dataset, we empirically show that Learn2Weight model is effective against the attack compared to standard black-box defense methods such as adversarial training and defense distillation. This work contributes to the growing literature on machine learning safety.

## introduction as machine learning models are applied to more and more real-world tasks, addressing machine learning safety is becoming an increasingly pressing issue. deep learning algorithms have been shown to be vulnerable to adversarial examples @xcite @xcite . in particular, prior black-box adversarial attacks assume that the adversary is not aware of the target model architecture, parameters or training data, but is capable of querying the target model with supplied inputs and obtaining the output predictions. the phenomenon that adversarial examples generated from one model may also be adversarial to another model is known as adversarial transferability @xcite . motivated by adversarial transferability, we conjecture another black-box attack pipeline where the adversary does not even need to have access to the target model nor query labels from crafted inputs. instead, as long as the adversary knows the task of the target, they can choose a similar domain to build a substitute model, and then attack the target model with adversarial examples that are generated from the attack domain. the similar-domain adversarial attack may be more practical than prior blackbox attacks as label querying from the target model is not needed. this attack can be illustrated with the following example (figure 1b ) in medical insurance fraud @xcite . insurance companies may use hypothetical opioid risk models to classify the likelihood (high/low) of a patient to abuse the opioids to be prescribed, based on the patient's medical history as text input. physicians can run the original patient history through the attack pipeline to generate an adversarial patient history, where the original is more likely to be rejected ("high" risk) and the adversarial is more likely to be accepted ("low" risk). perturbations in patient history could be, for example, a slight perturbation from "alcohol abuse" to "alcohol dependence", and it may successfully fool the insurance company's model. based on domain adaption theory @xcite , we conjecture that domain-variant features cause the success of the similar-domain attack. the adversarial examples with domain-variant features are likely to reside in the low-density regions (far away from decision boundary) of the empirical distribution of the target training data which could fool the target model @xcite . literature indicates that worsened generalizability is a tradeoff faced by existing defenses such as adversarial training @xcite and domain generalization techniques @xcite . in trying to increase robustness against adversarial inputs, a model faces a tradeoff of weakened accuracy towards clean inputs. given that an adversarial training loss function is composed of a loss against attack domain target domain attack data attack model correctly classified sentences misclassified sentences labels labels adversarial attack algorithm 1 2 3 adversarial samples labels 4 target data target model 0 5 predicted labels for adversarial samples 6 (a) generalized architecture of similarity-based attacks. original patient history opioid risk model (insurance company) label: high painkiller patient history data (attack domain) surrogate model (physician) correctly-classified sentences, filtered for similarity of symptoms / to original history fast gradient sign method perturbed patient history opioid risk model (insurance company) label: low (b) flow of how an adversary physician can leverage similarity attack to fool opioid risk models. to defend against this similar-domain adversarial attack, we propose a meta learning approach, learn2weight, so that the target model's decision boundary can adapt to the examples from lowdensity regions. experiments confirm the effectiveness of our approach against the similar-domain attack over other baseline defense methods. moreover, our approach is able to improve robustness accuracy without losing the target model's standard generalization accuracy. our contribution can be summarized as follows † : zhang et al. (2020) provides a survey of adversarial attacks in nlp. existing research proposes different attack methods for generating adversarial text examples @xcite @xcite . the crafted adversarial text examples have been shown to fool state-of-the-art nlp systems, e.g. bert @xcite . a large body of adversarial attack research focuses on black-box attack where the adversary builds a substitute model by querying the target model with supplied inputs and obtaining the output predictions. the key idea behind such black-box attack is that adversarial examples generated from one model may also be misclassified by another model, which is known as adversarial transferability @xcite . while prior work examines the transferability between different models trained over the same dataset, or the transferability between the same or different models trained over disjoint subsets of a dataset, our work examines the adversarial transferability between different domains, which we call a similar-domain adversarial attack. ## defending against similar-domain adversarial attack in order to defend against a similarity based adversarial attack, it is critical to block adversarial transferability. adversarial training is the most intuitive yet effective defense strategy for adversarial attack @xcite . however, this may not be effective for two reasons. first, there is no formal guidance for generating similar-domain adversarial examples because the defender has no idea what the attack data domain is. second, simply feeding the target model with adversarial examples may even hurt the generalization of the target model @xcite @xcite , which is also confirmed in our experiments. ## conclusion in this newly-proposed, empirically-effective similar-domain adversarial attack, an adversary can choose a similar domain to the target task, build a substitute model and produce adversarial examples to fool the target model. we also propose a defense strategy, learn2weight, that learns to adapt the target model's weight using crafted adversarial examples. compared with other adversarial defense strategies, learn2weight can improve the target model robustness against the similar-domain attack. our method demonstrates properties of a good adversarial defense, such as adopting a defense architecture that adapts to situations/inputs rather than compromising standard error versus robustness error, to leverage class-independent properties in domain-variant text, and factoring in domain similarity in adversarial robustness.