Title: Adversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text
Venue: NeurIPS 2025
Accepted: yes
Keywords: AI Text Detection

Adversarial Attacks
Abstract: The increasing capabilities of Large Language Models (LLMs) have raised concerns about their misuse in AI-generated plagiarism and social engineering. While various AI-generated text detectors have been proposed to mitigate these risks, many remain vulnerable to simple evasion techniques such as paraphrasing. However, recent detectors have shown greater robustness against such basic attacks. In this work, we introduce \textbf{Adversarial Paraphrasing}, a training-free attack framework that universally humanizes any AI-generated text to evade detection more effectively. Our approach leverages an off-the-shelf instruction-following LLM to paraphrase AI-generated content under the guidance of an AI text detector, producing adversarial examples that are specifically optimized to bypass detection. Extensive experiments show that our attack is both broadly effective and highly transferable across several detection systems. For instance, compared to simple paraphrasing attack—which, ironically, increases the true positive at 1\% false positive (T@1\%F) by 8.57\% on RADAR and 15.03\% on Fast-DetectGPT—adversarial paraphrasing, guided by OpenAI-RoBERTa-Large, reduces T@1\%F by 64.49\% on RADAR and a striking 98.96\% on Fast-DetectGPT. Across a diverse set of detectors—including neural network-based, watermark-based, and zero-shot approaches—our attack achieves an average T@1\%F reduction of 87.88\% under the guidance of OpenAI-RoBERTa-Large. 
We also analyze the tradeoff between text quality and our attack success to find that our method can significantly reduce detection rates, with mostly a slight degradation in text quality.
Our novel adversarial setup highlights the need for more robust and resilient detection strategies in the light of increasingly sophisticated evasion techniques.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Adversarial Paraphrasing, a method that rewrites AI-generated text to appear more human-like by fooling AI text detectors. The authors use a decoding strategy that, at each step, selects the next token that results in the lowest score from a pre-trained AI detector. By doing so, the paraphrased output gradually avoids detection, despite being originally generated by a language model. The method is tested across multiple detectors and shows strong performance in reducing detectability without significantly harming fluency.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. The idea of using a detector to guide the paraphrasing is creative and different from simple paraphrasing approaches.\n2. The authors tested across a wide range of AI detectors and included analyses on detectability, fluency, and perplexity.\n3. Reduction in detection rates is impressive and clearly beats baseline methods.\n\n**Weaknesses**\n1. The method selects tokens that minimize the AI-detector score at each step, but this local strategy does not ensure the overall text is natural, coherent, or stylistically human. It may result in text that is merely **undetectable** rather than genuinely **human-like**.\n\n2. The detail about the detector scoring is missing. The paper doesn’t explain what happens when all the k=50 candidates get high detector scores. How is the tie broken?\n\n4. GPT-4o ratings compare against AI text, not human text baseline.\n5. There's no analysis of failure cases or discussion about when this method breaks down.\n7. Running detectors 50 times per token seems prohibitively expensive. There is no analysis of computational overhead for detector guidance generation.\n8. There are no ablation studies showing how results change with different values of p, k or other guidance strategies.\n\nQUESTIONS:\nMost of my concerns are already outlined in the Weaknesses section, but I include a few additional questions here for clarification:\n\n1. To what extent can we trust that always choosing the token with the lowest detector score actually leads to more human-like text, rather than just fooling the detector?\n1. Did the authors try selecting tokens that were not the lowest-scoring to see if they also avoided detection?\n2. I would like to see a discussion on failure cases — where the attack didn’t help or actually made detection worse. Could adversarial paraphrasing ever backfire?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces:\n> a universal, training-free framework for humanizing AI-generated text, a novel attack designed to effectively and efficiently rephrase AI-generated text into “human-like text” that evades a wide range of detectors\n\nIn more detail, the approach involves paraphrasing using an instruction-tuned model (Llama-3-8B) where the decoding procedure is re-weighted based on scores from a machine-text detector. The idea is to bias the paraphrase model to produce tokens that trick the AI text detectors. Experiments using a wide range of detectors suggest that the approach is effective in degrading the performance of detectors. Further analysis also looks at the quality of the paraphrases, suggesting that they remain fluent despite fooling detectors.\n\nSTRENGTHS AND WEAKNESSES:\n**Pros**\n* The proposed attack is simple and empirically successful in the goal of defeating AI text detection. To the extent that this drives research in more robust detection methods, this is a “pro.”\n* The paper is well-written with clear explanations of related work, experiments, and results.\n* The empirical evaluations are fairly comprehensive.\n\n**Cons**\n* The quality of the paraphrases may be a serious limitation of the proposed approach. For example, although the base LLM may be capable of producing fluent paraphrases, the detector-fooling rescoring has no such fluency considerations. By selecting the token “with the lowest detector score” as the generated token, it  seems possible to produce highly disfluent outputs that defeat the detector while producing text that would look weird to any human reader. \n* I’m not sure that the perplexity results in Table 3 address this concern, since the rescored tokens are taken from the “top” of the next-token distribution. However, likely tokens may not be semantically plausible. Thus, while fluent under a LM, the paraphrased text may lose important semantic information. I would have liked to see further metrics assessing this, such as semantic similar scores (e.g., SBERT).\n* The proposed prompting approach aims to modify the style of the text while keeping the content unchanged. There is a relatively large literature on style transfer (e.g., https://arxiv.org/abs/2406.15586). Prompting approaches are known to have limitations, being unable to materially change the style. This may limit the ability to modify the original style via prompting alone. \n* The terminology is a bit misleading. Defining “human-likeness” as fooling an AI-text detector is a bit of a stretch when these detectors may well be exploiting various shortcuts for detection that have nothing to do with closing the gap between machine and human writing. One potential way to validate the claim of “humanizing” the text would be to look at a measure of stylistic similarity between human and machine text, as in https://arxiv.org/abs/2401.06712. This confusing terminology is repeated throughout the paper, including the technical description of the rescoring approach (L181-182).\n* I also quibble with the “training-free” terminology, since the approach relies on a machine-text detector during decoding, which seem to be restricted to _supervised_ methods in the experiments (RoBERTa classifiers, MAGE, and RADAR).\n* No discussion of limitations.\n\nQUESTIONS:\n* Do you have any further evaluations assessing the quality of the paraphrases? e.g. SBERT similarity to the originals?\n\n* The approach has a number of hyper-parameters. How were hyper-parameters set?\n\nLIMITATIONS:\nThere is no meaningful discussion of limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Adversarial Paraphrasing, a training-free attack that rewrites AI-generated text using an LLM under the guidance of a detector’s feedback. Unlike simple paraphrasing, this method generates detector-aware adversarial examples that significantly reduce detection performance across a wide range of detectors, while remaining high text quality.\n\nSTRENGTHS AND WEAKNESSES:\n**Strength:**\n\n- An adversarial paraphraser, using a signal by a target detector during decoding, with strong attack effectiveness and transferability.\n- Comprehensive experiments on attack effectiveness, its transferability, and text quality of the proposed attacker under various detectors\n\n**Weaknesses:**\n\n- Lack of a naive and simple baseline to demonstrate the necessity of incorporating a detection score, especially during decoding as a proposed method.\n    - We can naturally think that a naive baseline would be firstly paraphrasing multiple times and then picking a paraphrased text with the lowest AI score by a target detector.\n    - Given that most detectors in this study operate on the entire text rather than token by token, it remains unclear why this needs to be incorporated at each decoding step. Furthermore, in the early stages of decoding, the generated text is relatively short, and the reliability of AI scores on such partial outputs is questionable, as detectors generally perform worse with limited context.\n- Questionable intuitions on the transferability of the proposed attacker.\n    - In Section 3, they demonstrated that a well-trained detector represents the human-like distribution, and the AI score can be used for guidance for the universal transferable attacker.\n    - However, considering a strong, well-trained detector does not always seem to cause higher transferability of attacks. In Table 2, MAGE appears to be the most effective detector among the non-watermarking methods. However, attackers optimized against MAGE do not necessarily cause the largest performance drop for the other detectors.\n\nQUESTIONS:\nWhile I acknowledge the strong attacking performance of the proposed detector, I’m wondering: given that methods like RADAR are designed to adapt sequentially to new attacks, would it be able to detect the proposed attacks if they were incorporated into its training data?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a framework designed to evade AI-generated text detectors by transforming machine-generated content into human-like text. The authors propose Adversarial Paraphrasing, a training-free attack that leverages an instruction-tuned LLM to paraphrase text while being guided by an AI text detector to optimize for evasion. Experiments demonstrate that adversarial paraphrasing significantly reduces detection rates compared to baseline methods like simple or recursive paraphrasing. The attack maintains high text quality, as evaluated through perplexity scores and automated GPT-4o ratings. It also provides empirical evidence of the attack’s universality.\n\nSTRENGTHS AND WEAKNESSES:\n1.\tOverall, the paper is well-written. My main concern lies in the claimed trade-off between text quality and detector evasion. First, the definition of “text quality” is somewhat vague. Is it about preserving semantic similarity to human-written text, linguistic fluency, or other dimensions? Fundamentally, text quality and detector evasion are not necessarily conflicting objectives. for example, a human writer can produce high-quality text that is also recognized as human-written by detectors. Therefore, I find the premise of this trade-off unclear in the current context and suggest that it be clarified more explicitly.\n\n2.\tIn Figure 5, the lower part is difficult to interpret. The authors conclude that “simple paraphrases outperform adversarial paraphrases less than half of the time,” but this is not immediately evident from the figure. There appear to be three categories: noadv wins, ties, and adv wins. Does \"noadv\" refer to “simple paraphrases”? If so, this should be clearly stated to aid interpretation.\n\n3.\tThe proposed method involves iterative, detector-guided token selection, which likely increases computational cost. The paper does not include a detailed efficiency analysis, for example, in terms of latency or resource requirements, especially for large-scale deployment scenarios.\n\n4.\tThe effectiveness of adversarial paraphrasing depends heavily on feedback from a specific detector. It would be useful to discuss how the method would generalize if future detectors adopt fundamentally different detection mechanisms, which may render the current approach ineffective.\n\nQUESTIONS:\nsee above \"Weaknesses\" part.\n\nLIMITATIONS:\nyes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}