White-Box Multi-Objective Adversarial Attack on Dialogue Generation (2023)

Abstract: Pre-trained transformers are popular in state-of-the-art dialogue generation (DG) systems. Such language models are, however, vulnerable to various adversarial samples as studied in traditional tasks such as text classification, which inspires our curiosity about their robustness in DG systems. One main challenge of attacking DG models is that perturbations on the current sentence can hardly degrade the response accuracy because the unchanged chat histories are also considered for decision-making. Instead of merely pursuing pitfalls of performance metrics such as BLEU, ROUGE, we observe that crafting adversarial samples to force longer generation outputs benefits attack effectiveness—the generated responses are typically irrelevant, lengthy, and repetitive. To this end, we propose a white-box multi-objective attack method called DGSlow. Specifically, DGSlow balances two objectives—generation accuracy and length, via a gradient-based multi-objective optimizer and applies an adaptive searching mechanism to iteratively craft adversarial samples with only a few modifications. Comprehensive experiments on four benchmark datasets demonstrate that DGSlow could significantly degrade state-of-the-art DG models with a higher success rate than traditional accuracy-based methods. Besides, our crafted sentences also exhibit strong transferability in attacking other models.

## introduction pre-trained transformers have achieved remarkable success in dialogue generation (dg) @xcite @xcite , e.g., the ubiquitous chat agents and voice-embedded chat-bots. however, such powerful models are fragile when encountering adversarial samples crafted by small and imperceptible perturbations @xcite . recent studies have revealed the vulnerability of deep learning in traditional tasks such as text classification @xcite @xcite and neural machine translation @xcite . nonetheless, investigating the robustness of dg systems has not received much attention. crafting dg adversarial samples is notably more challenging due to the conversational paradigm, where we can only modify the current utterance while the models make decisions also based on previous chat history @xcite . this renders small perturbations even more negligible for degrading the output quality. an intuitive adaptation of existing accuracy-based attacks, especially black-box methods @xcite @xcite that merely pursue pitfalls for performance metrics, cannot effectively tackle such issues. alternatively, we observed that adversarial perturbations forcing longer outputs are more effective against dg models, as longer generated responses are generally more semanticirrelevant to the references. besides, such an objective is non-trivial because current large language models can handle and generate substantially long outputs. this implies the two attacking objectivesgeneration accuracy and length, can somehow be correlated and jointly approximated. to this end, we propose a novel attack method targeting the two objectives called dgslow, which produces semantic-preserving adversarial samples and achieves a higher attack success rate on dg models. specifically, we define two objectiveoriented losses corresponding to the response accuracy and length. instead of integrating both objectives and applying human-based parameter tuning, which is inefficient and resource-consuming, we propose a gradient-based multi-objective optimizer to estimate an optimal pareto-stationary solution @xcite . the derived gradients serve as indicators of the significance of each word in a dg instance. then we iteratively substi-tute those keywords using masked language modeling (mlm) @xcite and validate the correctness of crafted samples. the intuition is to maintain semantics and grammatical correctness with minimum word replacements @xcite . finally, we define a unique fitness function that considers both objectives for selecting promising crafted samples. unlike existing techniques that apply either greedy or random search, we design an adaptive search algorithm where the selection criteria are dynamically based on the current iteration and candidates' quality. our intuition is to avoid the search strapped in a local minimum and further improve efficiency. we conduct comprehensive attacking experiments on three pre-trained transformers over four dg benchmark datasets to evaluate the effectiveness of our method. evaluation results demonstrate that dgslow overall outperforms all baseline methods in terms of higher attack success rate, better semantic preservance, and longer as well as more irrelevant generation outputs. we further investigate the transferability of dgslow on different models to illustrate its practicality and usability in real-world applications. our main contributions are as follows: ## dialogue adversarial generation suppose a chat bot aims to model conversations between two persons. we follow the settings @xcite where each person has a persona (e.g. the same process applies for b to keep the conversation going. in the following, we first define the optimization goal of dg adversarial samples and then introduce our multi-objective optimization followed by a searchbased adversarial attack framework. ## conclusions in this paper, we propose dgslow-a white-box multi-objective adversarial attack that can effectively degrade the performance of dg models. specifically, dgslow targets to craft adversarial samples that can induce long and irrelevant outputs. to fulfill the two objectives, it first defines two objective-oriented losses and applies a gradientbased multi-objective optimizer to locate key words for higher attack success rate. then, dgslow perturbs words with semantic-preserving substitutions and selects promising candidates to iteratively approximate an optima solution. experimental results show that dgslow achieves state-of-the-art results regarding the attack success rate, the quality of adversarial samples, and the dg performance degradation. we also show that adversarial samples generated by dgslow on a model can effectively attack other models, proving the practicability of our attack in real-world scenarios.