Finding a Needle in the Adversarial Haystack: A Targeted Paraphrasing Approach For Uncovering Edge Cases with Minimal Distribution Distortion (2024)

Abstract: Adversarial attacks against Language models (LMs) are a significant concern. In particular, adversarial samples exploit the model’s sensitivity to small input changes. While these changes appear insignificant on the semantics of the input sample, they result in significant decay in model performance. In this paper, we propose Targeted Paraphrasing via RL (TPRL), an approach to automatically learn a policy to generate challenging samples that improve the model’s performance. TPRL leverages FLAN-T5, a language model, as a generator and employs a self-learned policy using a proximal policy optimization to generate the adversarial examples automatically. TPRL’s reward is based on the confusion induced in the classifier, preserving the original text meaning through a Mutual Implication score. We demonstrate & evaluate TPRL’s effectiveness in discovering natural adversarial attacks and improving model performance through extensive experiments on four diverse NLP classification tasks via Automatic & Human evaluation. TPRL outperforms strong baselines, exhibits generalizability across classifiers and datasets, and combines the strengths of language modeling and reinforcement learning to generate diverse and influential adversarial examples.

## introduction lms have made impressive advancements in classification, question-answering, and machine translation. however, they are susceptible to adversarial attacks, which exploit their vulnerability to small input changes @xcite @xcite @xcite . these attacks introduce variations not encountered during training. two approaches to address these vulnerabilities are data augmentation-based techniques @xcite @xcite and adversarial training-based approaches @xcite ; xu et @xcite . expanding training data using predesigned samples generated by data augmentation methods can assist classifier training. however, generated samples may lack an adversarial nature @xcite , leading to confusion and inaccurate classification. adversarial training-based approaches @xcite @xcite @xcite address this limitation by generating challenging examples. this improves the model's ability to handle difficult subsets of data, enhancing robustness and performance. generating diverse and semantically meaningful adversarial examples is challenging due to limited operations like word addition, deletion, or substitution. this lack of diversity in word-level generation often results in generated sentences with identical vectors to the original, offering little insight into the model's behavior. recent studies @xcite show that character manipulation or word swap methods can produce irrelevant and incoherent samples, altering the original text's meaning. these methods are unsuitable for real-world applications and can harm the model. attacks lacking semantic significance expose the model's blind spots, are easily detectable and removable, and do not represent real-world examples encountered during deployment. to address this, researchers have explored sentence-level generation methods. @xcite show promising results in generating diverse examples but may result in irrelevant and obtaining labeled data is challenging. another approach employs a paraphrasing technique using lms. however, this method lacks targeted fine-tuning to break the classifier and has limited available styles. in this paper, we introduce a novel approach called tprl that leverages flan-t5 for targeted paraphrasing, enhancing classifier performance through reinforcement learning. we employ a two-step process: first, we train a diverse paraphrasing model using flan-t5 and then fine-tune it with rl to preserve text meaning while generating targeted adversarial examples. we evaluate tprl on four classification tasks, both automatically and with human evaluation, demonstrating its ability to improve classifier robustness and effectiveness. our findings can be summarized as follows: ## background this section briefly introduces and formalizes textual adversarial attacks for text classification and employs rl in language models for generating adversarial attacks and other tasks. ## collecting labeled paraphrasing pairs this section will show the selected datasets for training the paraphrase model. afterward, we will outline a systematic procedure for filtering those datasets to maximize the paraphrase pairs' diversity, similarity, and relevance. ## experiments we assess the effectiveness of targeted paraphrasing via rl adversarial attacks (tprl) on four dis-tinct classification tasks: sentiment analysis, news topic classification, hate speech detection, and offensive speech detection. to this end, we carefully select relevant datasets, outline our implementation details, establish baseline methods, and specify evaluation metrics. ## ablation & analysis studies in this section, we conduct ablations and analyze tprl's key components' impact on baseline improvements. trpl diversity enhances classifier performance: we explore diversity's impact on performance, comparing tprl with the baseline methods. similarity scores increased with reduced diversity post-adversarial training, while the mi metric sustains diversity. human evaluations support mi's alignment. also, we use gpt-3.5 to confirm tprl's diversity; details in appendix g. the importance of targeted component: our tprl implementation has a targeted component using rl. however, we evaluated using only the fine-tuned paraphraser without rl (untp), which showed inconsistent performance compared to tprl ( number of generated samples impact: we explore the link between increasing sample size and the resultant performance improvements for both tprl and a vanilla classifier. tprl's average sample generation per dataset: sst-2: 359, sst-5: 2993, hs: 135, off: 407, ag's news: 3825. we examined the correlation between sample count (normalized by total set size) and tprl's performance boost, finding no significant correlation (pearson correlation of 0.144, p ≪ 0.05). the insignificance stems from performance enhancements' variability due to classifier weaknesses. thus, we introduced a mutual implication filtering criterion of 0.5 for selecting high-quality samples, improving model performance without irrelevant sampling. original: i complain all the time about seeing the same ideas repeated in films over and over again, but the bourne identity proves that a fresh take is always possible. adversarial: ive always been complaining that movies repeat the same ideas but the identity of bourne proves that you dont need to reinvent the wheel original: something must be done to halt this terrible wave of immigrants entering britian adversarial: in order to stop a terrible wave of immigration into britian it is necessary to do something original: i live in oslo and race mixing is a huge problem adversarial: theres a huge problem of race mixing in oslo original: william shatner as a pompous professor is the sole bright spot adversarial: the only bright spot of the movie is william shatner in a pompous professor original: if you enjoy more thoughtful comedies with interesting conflicted characters; this one is for you adversarial: this is a good comedy for those who like thoughtful comedies with interesting characters original: any ways smearing a single dna steam with black dna is horrible adversarial: any way it seems disgusting to smear one steam with black dna original: we spend so much time on black people when the real problem is asians and hispanics adversarial: we spend too much time and effort worrying about black while the asians and hispanics were the ones ## conclusion in this paper, we propose tprl, an adversarial generation method designed to improve classification model robustness. our innovation involves reinforcement training to learn various attack strategies automatically. we confirm tprl's effectiveness in four classification tasks, consistently producing high-quality adversarial samples that represent edge cases with minimal distortion in the data distribution.