Title: DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm
Venue: NeurIPS 2025
Accepted: yes
Keywords: AI-generated Text Detection
Abstract: The rapid advancement of large language models (LLMs) has blurred the line between AI-generated and human-written text. This progress brings societal risks such as misinformation, authorship ambiguity, and intellectual property concerns, highlighting the urgent need for reliable AI-generated text detection methods. However, recent advances in generative language modeling have resulted in significant overlap between the feature distributions of human-written and AI-generated text, blurring classification boundaries and making accurate detection increasingly challenging. To address the above challenges, we propose a DNA-inspired perspective, leveraging a repair-based process to directly and interpretably capture the intrinsic differences between human-written and AI-generated text. Building on this perspective, we introduce **DNA-DetectLLM**, a zero-shot detection method for distinguishing AI-generated and human-written text. The method constructs an ideal AI-generated sequence for each input, iteratively repairs non-optimal tokens, and quantifies the cumulative repair effort as an interpretable detection signal. Empirical evaluations demonstrate that our method achieves state-of-the-art detection performance and exhibits strong robustness against various adversarial attacks and input lengths. Specifically, DNA-DetectLLM achieves relative improvements of **5.55\%** in AUROC and **2.08\%** in F1 score across multiple public benchmark datasets. Code and data are available at https://github.com/Xiaoweizhu57/DNA-DetectLLM.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is inspired by the DNA repair process in biology. The paper proposes a new way to detect AI-generated text: iteratively “repairing” a text sequence and measuring how hard it is to make it match the ideal AI-generated version. Then, the paper proposes the difficulty of the repair process of fixing fraction quantification. We introduced one to capture the cumulative effort required to complete the conversion. Last, by comparing the repair score with the calibrated threshold, DNA-DetectLLM robustly distinguishes AI-generated text and human-written text, using the basic differences that deviate from the ideal generation.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper is well written and easy to follow.\n- The idea is novel and motivated by a strong analogy from biology.\n- The experiments are well-designed and show strong results.\n- The baselines used for comparison are mostly appropriate and fairly broad.\n\n**Weaknesses**\n\n1. The datasets used in the main experiments (XSum, Writing, ArXiv) are quite similar—they are all long-form writing tasks. A more challenging dataset, such as **PubMedQA**, which is used in Fast-DetectGPT and focuses on scientific question answering, would be helpful to test how well the method works in a different domain.\n\n2. It would be useful to include **GPTZero**, a popular commercial detection tool, as a baseline. This would help readers understand how the method compares to widely-used systems.\n\n3. The authors should include comparisons with more recent and stronger baselines:\n- **DNA-GPT** (ICLR 2024)\n- **Text Fluoroscopy** (EMNLP 2024)\n- **ImBD** (AAAI 2025)\n- **Biscope** (NeurIPS 2024)\n\n4. The paper mentions using outputs from GPT-4 Turbo, Gemini 2.0 Flash, and Claude 3.7 Sonnet. However, it does not provide details such as the **temperature** or **model versions** used to generate these outputs. This information is important for reproducibility. Also, since GPT-4 Turbo is now being phased out, the authors could consider using GPT-4o for future experiments.\n\nQUESTIONS:\nPlease See Weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-28"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe article proposes a novel method called DNA-DetectLLM, which adopts a perspective inspired by DNA repair to directly and interpretably analyze the intrinsic differences between human-written and AI-generated texts. This method constructs an ideal AI-generated sequence for each input, iteratively repairs suboptimal tokens, and quantifies the cumulative repair effort as an interpretable detection signal.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The proposed method is both interesting and well-motivated.\n- The evaluation of the method is robust, considering mainstream and challenging detection scenarios, with necessary ablation studies conducted.\n\n**Weaknesses**\n- The authors seem to have a misunderstanding of the current state of research on detection methods. For example, in line 30, it is stated that “training-free approaches leverage intrinsic statistical differences to distinguish human-written and AI-generated texts,” but in reality, these methods are not entirely “training-free.” They also require extracting feature scores on a training set, which are then used for detection during the testing phase. It would be beneficial to revise the corresponding statements with reference to [1].\n- Following the previous weakness, it appears that the authors did not separate the training set from the testing set, leading to unfair comparisons in Table 1 and the related ablation studies. For a fair comparison, training-based methods and training-free methods should be trained on the same training set and evaluated on the same testing set.\n\nOverall, this is a good paper, but there seems to be a significant issue with the experimental setup. If this issue can be resolved, I would be happy to recommend accepting this paper.\n\n**Other Suggestion**\n\nIn the related work section, the authors discuss the differences between their approach and previous studies. However, I believe there are still some similar works that have not been mentioned. Overall, the paper assumes that human-written text resembles a mutated chain, where token selection deviates from the optimal probability, resulting in measurable differences. It is worth noting that some works also adopt similar ideas, such as Revise-Detect. [2], which allows LLMs to rewrite the text themselves to fix tokens that deviate from the optimal probability, and GECScore [3], which introduces grammatical errors and LLM's correction preferences to measure the sequence differences between the two types of text. The key difference is that these works operate at the sequence level for \"repair\" and \"correction,\" whereas the proposed method in this paper performs iterative \"repair\" and \"correction\" at the token level. I speculate that the token-by-token repair process may provide more controllable and fine-grained measurements of the deviation patterns between the two types of text. Therefore, comparing this work with such methods would be beneficial in highlighting the advantages of the approach presented in the paper.\n\n[1] A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions. Computational Linguistics, 2025.\n\n[2] Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT. EMNLP, 2023.\n\n[3] Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore. COLING, 2025.\n\nQUESTIONS:\nI am curious about the experimental setup for the ablation studies. Is it in-distribution or out-of-distribution? For example, in terms of text length, was the model trained on texts of the same length and then tested on texts of the same length?\n\nLIMITATIONS:\nNone", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new method for the unsupervised detection of LLM-generated text. The proposed method is a modified version of the cross-perplexity method from Binoculars [1]; however, instead of computing the ratio of perplexity to cross-perplexity, it computes the \"conditional score\", i.e., the ratio of the probability of argmax tokens given the observed document with the cross-perplexity.\n\n[1] Hans, A., Schwarzschild, A., Cherepanova, V., Kazemi, H., Saha, A., Goldblum, M., ... & Goldstein, T. (2024, July). Spotting LLMs with binoculars: zero-shot detection of machine-generated text. In Proceedings of the 41st International Conference on Machine Learning (pp. 17519-17537).\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The paper is generally well-written and (mostly) easy to understand. It does a good job of differentiating between unsupervised and supervised baselines and reports competitive approaches in both categories\n\n2. The results are really quite impressive given the large difference between the scoring models (Falcon-7B, LLaMA-7B) and the target models (GPT-4-turbo, Gemini-2.0-flash, Claude-3.7-Sonnet). This is in pretty stark contrast to some existing methods like DetectGPT, which experience large performance degradations when the scoring and target models don't match\n\n3. The paper experiments on a wide range of benchmarks, models, and baseline approaches. It includes experiments on paraphrasing and editing attacks, analyzes performance as a function of the number of tokens, and evaluates different scoring methods. Very thorough!  \n\nWeaknesses:\n\n1. I found the \"DNA\" inspiration to be more confusing than helpful. As far as I can tell, the \"ideal sequence\" here is just a sequence of argmax tokens which are selected independently of one another, so the ideal sequence might actually be very low probability or ungrammatical under a language model\n\n2. The method in this paper seems to be pretty closely motivated by Binoculars. While I think this is fine, it could be more explicit about it and justify some of the decisions made in the Binoculars paper (e.g., why cross-perplexity) in order to be more easily read as a standalone paper. For example, my understanding is that the models experimented on are paired as (Falcon-7B-Instruct, Falcon-7B) and (Llama-2-7B, Llama-7B) because the cross-perplexity computation requires models to share a tokenizer, but this is not explained in the paper\n\nQUESTIONS:\nN/A\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe propose approach is a zero shot approach for LLM generated sequence detection. Given an input sequence (text utterance), ideal AI generated sequence is obtained using greedily selecting the most probable token at each position using probability distribution obtained using Falcon-7B-Instruct. Once both ideal AI generated sequence and original input sequence is available. The position where there is a change is compared and referred as mutations. The input sequence is aligned with ideal LLM generated sequence using an iterative token-level modifications referred as mutation repair mechanism. A score is accumulated throughout the repair. The accumulated score is compared against a calibrated threshold to label input sequence as AI generated or human written. Author run exhaustive experiments on multiple dataset and compare with several existing work to show competitive performance from proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\nMajor Strength: \n\n1. Approach is novel, there are existing work on DNA inspired zero shot algorithms for AI generated text detection e.g. R1 and R2. However, proposed approach is different w.r.t them. \n2. Performance from proposed approach is competitive and author compare with several recent methods also like Lastdee++. Author also run experiments to show method is more robust than existing method from attacks like insertion, deletion, substitution and paraphrase.\n3. Since method is based on accumulated score between ideal AI sequence and input sequence. Existing work show that detection performance can be sensitive to length of the sequence. Detection become more challenging for smaller length sequence. Author show proposed approach is robust to different sequence lengths\n4.  Author also show that proposed approach works independent of base LLM used to generate ideal AI generated sequence.\n5. Paper is well written and easy to understand.\n\nMinor Weakness: Using different base LLMs and reporting detection performance using them in Table 2 or Appendix can be helpful.\n\nR1: https://arxiv.org/html/2305.12519v2\nR2: https://openreview.net/forum?id=Xlayxj2fWp\n\nQUESTIONS:\nRefer above\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}