RAFT : Realistic Attacks to Fool Text Detectors (2024)

Abstract: Large language models (LLMs) have exhibited remarkable fluency across various tasks. However, their unethical applications, such as disseminating disinformation, have become a growing concern. Although recent works have proposed a number of LLM detection methods, their robustness and reliability remain unclear. In this paper, we present RAFT: a grammar error-free black-box attack against existing LLM detectors. In contrast to previous attacks for language models, our method exploits the transferability of LLM embeddings at the word-level while preserving the original text quality. We leverage an auxiliary embedding to greedily select candidate words to perturb against the target detector. Experiments reveal that our attack effectively compromises all detectors in the study across various domains by up to 99%, and are transferable across source models. Manual human evaluation studies show our attacks are realistic and indistinguishable from original human-written text. We also show that examples generated by RAFT can be used to train adversarially robust detectors. Our work shows that current LLM detectors are not adversarially robust, underscoring the urgent need for more resilient detection mechanisms.

## introduction large language models (llms) such as chatgpt @xcite , llama @xcite , and gpt-4 (openai, 2023) have exhibited transformative abilities to generate remarkably fluent and cogent long-form text in response to user queries. however, llms have been misused to disseminate disinformation, commit academic dishonesty, and launch targeted spear phishing campaigns against vulnerable populations @xcite . to mitigate harm from malicious use, the capability to distinguish machine-generated text and humanwritten text is paramount. ## related work substitution-based attacks against nlp targets: while existing gradient-based adversarial attacks are effective in the vision and speech domains @xcite , attacks in the text domain present unique challenges due to its discrete nature. attacks in the natural language domain are also constrained by language fluency, semantic consistency, and human prediction consistency. @xcite introduce a black-box word substitution-based attack that fulfills all these criteria by utilizing semantic similarity and pos matching to greedily replace words with synonyms until a successful attack. in this work, we use an llm instead to generate semantically consistent word replacement candidates and greedily select the pos-consistent word that most effectively attacks the target detector. llm detector attack frameworks: existing algorithms for detecting machine-generated text can be categorized into three categories: supervised classifiers @xcite @xcite , watermark detectors @xcite @xcite , and zero-shot statistical-based methods @xcite @xcite @xcite . as new detection methods continue to be developed, parallel efforts in red-teaming these detectors have also gained momentum. the primary techniques for attacking them include text paraphrasing and word replacement. @xcite present dipper, a paraphrase generation model that can be conditioned on surrounding context to effectively attack state-of-the-art detectors while controlling output diversity and maintaining semantic consistency. @xcite iterate on this method and present a recursive paraphrasing attack that breaks watermarking and retrieval-based detectors with slight degradation in text quality by using a lightweight t5-based paraphraser model. these attack frameworks are more vulnerable to attack since they are reliant on one large fine-tuned llm model and utilize smaller paraphrasing models that are relatively weaker than the source llm. @xcite introduce a word replacement method that utilizes an llm to randomly generate substitution candidates for multiple words, selecting the optimal replacement candidates using an iterative evolutionary search algorithm to minimize detection score. we improve upon this framework by introducing a replaceable proxy scoring model that uses an auxiliary llm embedding to rank which words in the machine-generated text should be replaced and greedily select llm-generated candidates that effectively subvert the target detector. ## (b) fast-detectgpt figure 4 : study on the impact of different mask percentages. we use opt-2.7b and roberta-large as proxy scoring models to attack log rank and fast-detectgpt detectors on the xsum dataset at 1%, 5%, 10%, 15%, and 20% masking rates while measuring detection performance and text quality in terms of auroc and perplexity. the auroc approaches 0 at around 10% with a moderate increase in perplexity. masking percentages beyond 15% degrade text quality across both detectors. ## conclusion we introduce raft, an adversarial attack framework for subverting machine-generated text detectors by leveraging auxiliary llm embeddings. our method effectively identifies optimal words to perturb using a proxy llm embedding and perturbs them such that the original text remains semantically consistent, grammar error-free, and reads fluently. experimental results and manual annotation exercises show that our method successfully compromises various llm detection methods while maintaining text quality and semantic consistency, highlighting the need for robust llm content detectors. we also demonstrate that the outputs from raft can be used to enhance the resilience of existing detectors through adversarial training.