DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm
Authors: Xiaowei Zhu
Yubing Ren
Fang Fang
Qingfeng Tan
Shi Wang
Yanan Cao
Venue: NeurIPS 2025 spotlight ()
Keywords: AI-generated Text Detection
Primary area: deep_learning

Abstract: The rapid advancement of large language models (LLMs) has blurred the line between AI-generated and human-written text. This progress brings societal risks such as misinformation, authorship ambiguity, and intellectual property concerns, highlighting the urgent need for reliable AI-generated text detection methods. However, recent advances in generative language modeling have resulted in significant overlap between the feature distributions of human-written and AI-generated text, blurring classification boundaries and making accurate detection increasingly challenging. To address the above challenges, we propose a DNA-inspired perspective, leveraging a repair-based process to directly and interpretably capture the intrinsic differences between human-written and AI-generated text. Building on this perspective, we introduce **DNA-DetectLLM**, a zero-shot detection method for distinguishing AI-generated and human-written text. The method constructs an ideal AI-generated sequence for each input, iteratively repairs non-optimal tokens, and quantifies the cumulative repair effort as an interpretable detection signal. Empirical evaluations demonstrate that our method achieves state-of-the-art detection performance and exhibits strong robustness against various adversarial attacks and input lengths. Specifically, DNA-DetectLLM achieves relative improvements of **5.55\%** in AUROC and **2.08\%** in F1 score across multiple public benchmark datasets. Code and data are available at https://github.com/Xiaoweizhu57/DNA-DetectLLM.

DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm
Xiaowei Zhu! Yubing R Fang
Qingfeng Tan Shi Wang Yanan Cao
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China 2School of Cyher Security, University of Chinese Academy of Sciences, Beijing, China University International College, Macau University of Science and Technology, Macau Cyberspace Institute of Advanced Technology. Guangzhou University, Guangzhou, China Institute of Computing Science, Chinese Academy of Sciences, Beijing, China {zhuxiaowei, renyubing] ac. cn
Abstract
The rapid advancement of large language models (LLMs) has blurred the line between Al-generated and human-written text. This progress brings societal risks such as misinformation, authorship ambiguity, and intellectual property concerns, highlighting the urgent need for reliable Al-generated text detection methods. However, recent advances in generative language modeling have re- sulted in significant overlap between the feature distributions of humann-ritten and Al-generated text, blurring classification boundaries and making accurate detection increasingly challenging. To address the above challenges, pose a DNA DNA-inspired perspective, leveraging repair- based process to directly and interpretably capture the intrinsic differences between numan-writted and Al-generated text. Building on this perspective, we introduce DNA-DetectLLM, zero-shot detection method for distinguishing Al-generated and numan-writted text. The method constructs an ideal Al-generated sequence for each input, it- eratively repairs non-optimal tokens, and quantifies the cumulative repair effort as an interpretable detection signal. Empirical evaluations demonstrate that our method achieves -tate-of-the-art detection performance and exhibits strong ro- bustness against various adversarial attacks and input lengths. Specifically, DNA- DetectLLM achieves relative improvements 5.55% in AUROC and 2.08% F1 score across multiple public benchmark datasets. Code and data are available at htttps://github. com Himm/Xiaai http///xaaiiii//////// /DNA -etet---eeeLLLL
pro-
Introduction
The rapid advancement of large language models (LLMs) has created increasingly human-like textual content, substantially narrowing the distinguishable gap between Al-generated and human- written text. While these improvements have catalyzed significant technological breakthroughs, they simultancously pose critical societal challenges, including misinformation dissemination, authorship ambiguity, and threats to intellectual property rights 2 I [T] Consequently, there is urgent and growing need for effective and reliable methods to accurately detect Al-generated text.
While significant research efforts have been dedicated to Al-generated text detection, existing method- ologies typically adopt either training-baat based or training-fres methods. Training based methods depend upon large volumes of annotated data, limiting their scalability and
28
16
12
39 [1]]
"Coresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Traditional Paradigns
DNA-DetectLLN (Nutation-Repair Paradign)
tomplate strand
overlapping regions
X4
Repair
Poor Performance
Score
Mutation-Repair Paradign
Mutate
Feuer over lapping regions
replicated
Robust Performance
Figure 1: Illustration of the Mutation- Repair Paradigm. The input sequence can be analogized to replicating strand, while the ideal Al-generated sequence corresponds to the template strand.
generalization to new domains. In contrast, training-free approaches 22
14 36
leverage intrinsic
statistical differences to distinguish human-written and Al-generated texts. Both paradigms funda- mentally operate by attempting to identify distinct, separable boundaries within the feature space. However, recent advancements in generative language modeling have produced outputs increasingly indistinguishable from human- authnn content, causing these classification boundaries to become progressively blurred. Empirical studies [7] have highlighted substantial overlap regions in the feature distributions of -witten and Al Al-generated texts, significantly undermining detection accuracy in practical scenarios. Therefore, one capable of more precisely and intrinsically capturing differences between the generative processes of Al and human writing is urgently needed.
In molecular biology, DNA's double-helix structure ensures stable transmission genetic information, yet mutations during replication introduce variations that can lead to individual differences or even diseases such as cancer. In a similar vein, an idcal Al-generated text sequence can be seen as a "template strand", representing the most probable token choices at each position. Sllman-writter texts, by contrast, resemble mutated strands, where token selections deviate from the optimal probabilities, creating measurable differences. Inspired by this biological mechanism, we propose new perspective for Al Al-generated text detection: by analogizing to DNA base-repair processes, we iteratively "correct" non-optimal tokens in text and measure the difficulty of restoring the ideal Al-generated form. This repair- based approach captures the intrinsic divergence between -e-geeerteed and human- written texts in direct and interpretable manner.
Building on this intuition, we propose DNA-Detectl L.M a novel method for zero-shot detection of AI- generated texts. For each input sequence, we first construct its corresponding ideal Al sequence that is, the sequence formed by greedily selecting the most probable token at each position under a reference language model. We then perform a token-by-token repair process the input sequence, progressively modifying tokens toward their optimal choices until the sequence fully aligns with the ideal AI sequence. To quantify the difficulty of this repair process, introduce repair score that captures the cumulative effort required to complete the transformation. Finally, by comparing the repair score against calibrated threshold, texts from human-written ones, leveraging the fundamental differences in their deviation patterns from ideal generation.
robustly distinguishes Al Al-generaeed
DNA-DetectL.LM, consistently achieves state-of-the-art performance across multiple datasets and LLMs. In particular, it obtains relative improvements of 5.55% % in AUROC and 2.08% in Fl score on three public benchmark datasets. Additionally, the method exhibits notable robustness against various adversarial attacks and across different input lengths. Efficiency experiments further indicate rapid detection capability, processing each sample in under 0.8s.
Our contributions are summarized as follows:
Inspired by the mutation and repair mechanisms of nucleotide bases in DNA replication, we introduce the mutation-repain paradigm into Al-generated text detection.

---

We propose DMA-DetectLLM, novel zero-shot method for detecting Al Al-generated text that incrementally repairs mutated tokens within the imput sequence until it perfectly aligns with the ideal Al-generated sequence, subsequently quantifying the repair difficulty metric for text detection.
Extensive evaluations validate that DNA-Detectl offers a reliable, efficient, and broadly generalizable solution for Al-generated text detection, with consistent gains across various detection settings.
2
Related Works
Detecting Al-generated text is essential for enhancing public trust and preventing misuse, driving growing interest from both academia and industry. Beyond watermarking techniques (20), which embed identifiable markers during generation, current post hoc detection methods are broadly categorized into training-based and training-free methods.
Training-based Methods. Such approaches typically involve training classification models to
distinguish between Al-generated and Homan--riiteen texts. Specifically, early efforts by OpenAI [28] employed RoBERTa-based models for training text classifiers. Subsequently, RADAR [16 introduced adversarial learning enhance the robustness against paraphrased texts. DeTeCtive 12
utilized multi-level contrastive learning to map texts generated by different LLMs into corresponding
feature spaces, classifying them based on similarity metrics. DPIC 39 extracted deep textual features
by reconstructing prompts and regenerating texts. Biscope [TT] proposed employing bidirectional
cross-entropy loss to extract statistical features for binary classifier training. R-Detect (29 employs a nonparametric kernel relative test to detect Al-generated text, thereby reducing the false positive rate compared to two-sample tests. However, existing research [5] 32) indicates that training-basee methods consistently overfit to in-distribution features, resulting in poor generalization to out-of- distribution (OOD) texts. Consequently, researchers have increasingly focused on developing more universally applicable training-free methods.
Training-free Methods. These training-free methods emphasize exploiting probabilistic charac- teristies of texts, constructing statistical scores based on specifie hypotheses, and making decisions according to the comparison of scores against thresholds. For example, LogRank [9] Likelihood [15], and Entropy [17] calculate the average probability ranking, likelihood probabilities, and entropy values to measure the uncertainty of Al-generated texts. DetectGPT [22] pioneered a paradigm that uses perturbations generate numerous contrast samples evaluate the overall distribution. Although methods such as DetectLL (30) and DNA-GPT (37) have further developed this paradigm, their efficiency limitations prevent real-time or large-scall detection implementations. Fast- DetectGPT [4] has since updated sampling techniques to compute conditional probability curvature, significantly improving detection efficiency and broadening potential applications. Binoculars [[4] achieved state-of-the-are classification performance by calculating perpleeeryy from dual-model perspectives. Lastde++ (36) proposed focusing on local textual features by calculating Diversity Entropy to optimize classification performance.
3
DNA-DetectLLM
3.1
Preliminary
This study primarily involves two statistical metries: log-perplexity, which quantifies the average token-level negative log-likelihood under a single model, and cross-perplexity. which captures the average per-token cross-entropy between the probability distributions of two models:
log PPLM, (8)
log PMr (x:/(re)),
T
(I)
log X-PPLM,
Σ
PM, (2a((tes) log (arl(ai/ee)))
3

---

Al-generated or Imput
Obtainning Ideal sequence
Mutation
Repair
Repair Mechanism
Based Detection
President Trump met with North Korean leader Kim Jong-un in June 2018.
Referomea
Repair murated tnlenn
Reyaiir soure
Inpot Soquemed
O generae ht
Muzatd tolen Iceal
F Sappe
Thraloed Comperooa
Impat sequemoz
Iteratielly repair
Lets get result.
busy ss
Let's shead detect it.
Idea sequence
Hamsnt-uriten Yest
Do
The ropair is complete
Thresheld
Algonerated
Figure 2: Overview of
where is the input sequence of length L, æ; denotes the i-th token, and Par (ar(2,(<<)) is the conditional probability of r; given its preceding tokens under reference model M1 observer model M2. Furthermore, their ratio (s)) has been empirically demonstrated to serve as an effective score for distinguishing Al-generated text [14). To quantify the effect of local token-level modifications these metrics, this work introduces the conditional log-perplexity and conditional score:
a = log PPLM, (5]s)
log PPLM, (8]s)
log PM, (i.)
(2)
L
log X- (
where 5 denotes the sequence obtained by modifying tokens in the input sequence
3.2 Overview of DNA-DetectLLM
The entire workflow of DNA-DetectLLM can summarized key steps, shown in Figure 2
Step 1:
Obtaining the ideal Al-generated Sequence. We construct the ideal Al-generated sequence given input by greedily selecting the most probable token at each position.
Step 2: Mutation Repair Mechanism. We perform iterative token-level modifications on the input sequence until it fully aligns with the ideal Al-generated sequence.
Step 3:
Repair Score-Based Detection. We introduce a repair score to quantify the difficulty of the repair, which is compared against calibrated threshold to determine the detection result.
3.3 Obtaining the ideal Al-generated Sequence
We propose the concept of an ideal Al-generated sequence &, analogous to the error-free template strand in DNA replication, where each token is selected by maximizing the conditional probability its position:
$ = { 82
fz). where 2; = arg max PM, (2/z2s), REV
(3)
with V denoting the vocabulary and deg (xi,..., Ti-1 representing the preceding i- tokens of the input sequence
3.4 Mutation Repair Mechanism
We treat non-max probability tokens as mutated tokens and max- mux-probabillty tokens as ideal tokens. Analogous to the mutation-repair paradigm in DNA, we propose a mutation repair mechanism aiming to uncover fundamental differences in deviation patterns between Al-generated and human-written texts. Under this mechanism, mutated tokens in the input sequence are iteratively repaired with their ideal tokens step by step, until the input fully aligns with the ideal form:
3i E {2), hL + 2; arg max PM, (Z)z)))
if + 8.,
(4)
BEV

---

Hamin Som
Machino Soora
Homon
Machino Rig)
Hamar Midline
Mochira Midinn
Mbr Randon Repat Rasiom Rejei:
High-bllon
Low-toggg
Secuentia Ropuir
Figure 3: Variation of repair scores Across Different Repair Strategies.
3.5 Repair Score-Based Detection
We introduce the repair score R(s) to quantify the difficulty of the repair process, defined as the average conditional score accumulated throughout the repair trajectory:
1 T+1
Σ'= log PPLM, (8x|s)
R(s)
(scls)
(T + 1o X-PPL.M,,Ms
where 8; is the sequence after repair steps, and T the total number of mutated tokens to be corrected.
Human-written texts typically exhibit more substantial mutations, resulting in greater repair difficulty. In contrast, Al-generated texts are generally easier to repair. Accordingly, the detection result for the input sequence determined as:
Hriten Text, R(s)) > R(s) S T.
D(s)
(6)
Al-generated Text,
3.6 Sensitivity to Repair Order and Score Simplification
Figure 3 shows that different repair orders yield varying repair scores the same input sequence, due to the unequal impact of each mutated token on the conditional score. Mutated tokens can be broadly categorized into high- and probbabiiylity types. Repairing low-probe cow-robbability tokens typically causes larger shifts in the conditional score, while high-probability tokens lead to smaller changes. To systematically analyze the influence of repair order, we identify four types of principal repair strategies as follows:
High-to-low:
Repairing high-probability tokens first, followed by low-probability ones, results in convex "repair curve" with a higher repair score.
Low-to-high: Repairing low-probability tokens before high-probability ones yields con- cave "repair curve" with a lower repair score.
Sequential Repair: Tokens repaired in their original order of appearance in the input sequence, regardless of their probability values.
Random Repair: Tokens are repaired in randomly chosen order. Averaging the repair scores across multiple random repairs leads to a more stable estimate.
These findings highlight the sensitivity of the repair score to the chosen repair order. Performing multiple random repairs effectively mitigates biases, resulting in repair score close to the midpoint between the initial and final scores. Consequently, we further derive that the average repair score converges the number of random repairs N approaches infinity. The derivation is as follows:
Let(5,, Let(51,02...-. ,2....,577 5r) be a set of non-negative real numbers satisfying Σi, 6, (s)) (8)s). Morcover, 8, = (s)- - (s) quantifies the impact repairing the current token on the score. Since the influence of each token repair on the conditional log-perplexity is independent and fixed, the set (d), 82...., consists fixed values for given input sequence, whose order varies depending on the repair strategy. It further follows:
-
8, for 1.2,...,T. T, where (()) = ( (s).
5

---

For specific permutation Φ E [1, N. the corresponding repair score R
is:
T ((8)
T
(s) T+1
6
=(()
T+1 (T-
(8)
Since each token is equally likely to repaired at random, the expected value of 6 appearing in the i-th position across all permutations is (a(s)) - (8]s)). We further derive the expected value the repair score follows:
o(s) - -8888)
T(T + 1)
[[[[+)] = o(s) T+1
E
800
(T- (T-i+1)= (8) T+1
T
2
Therefore, as the number of random permutations converges as follows:
approaches infinity, the average repair score
N Σ (s) ==
N
I
lim N-000
= lim N
Σ n=1
Σ
(1) (sels)
= (0((s) + o(]s)). (10)
N
N
T+1
We thus simplify the repair score to R(s) while avoiding intermediate score computations during repair.
} (a(() + (i]ss)) improving detection performance
4 Experiments
4.1 Experimental Setup
Datasets. To evaluate performance across diverse domains, we collect 4,800 human- texts from three representative tasks: news article writing (XSum (23), story generation (WritingPrompts 8), and academic writing (Arxiv (24) ). For each text, we construct task-specific prompts (see Appendix C and generate corresponding Al outputs using three advanced LLMs: GPT-4 Turbo, Gemini-2.0 Flash, and Claude-3.7 Sonnet. We further sample 2,000 balanced examples from each of
three high-quality detection benchmarks-MR (33], DetectRL (35] and RealDet and comprehensive evaluation across real-world scenarios.
ensure fair
Metries. We adopt the area under the receiver operating characteristic curve (AUROC 18] and F1 score evaluate detection performance, where higher values indicate better separability between human-written and AI-generated texts.
Baselines. We compare DNA-DetectLLA, with existing training-based and training-free methods. For training-based methods, we include OpenAl-D (28), Biscope [1] and R-Detect 229.. For training- free methods, we consider classic zero-shot detectors including Likelihood [13]. LogRank (9, and Entropy [17). along with several recent SOTA approaches such as DetectGPT [22] Fast-DetecrGPT
[4. Binoculars [14]. and Lastde++ [36].
More baseline comparisons are provided Appendix B
Implementation details. In real-world detection scenarios, the source and distribution of textual data are often unknown, constituting an out-of-distribution (OOD) detection problem. To ensure faimess for training-based methods, we exclusively train on the HC3 dataset [00, which is entirely disjoint from the test sets. For training-fre free methods, the choice of LLM used for scoring can introduce significant performance variation [3]. To eliminate this factor, standardize the reference (or scoring) model across all methods by employing Balcon-7B-Instrrnee [25] to compute token generation probabilities. Moreover, Fast-DetectGPT, Binoculars, Lastde++, and DNA-DetectLLM utilize Falcon-7B (25) as the observer (or sampling) model, while DetectGPT uses T5-3B (26). During testing, the maximum input token length is capped 1024. More details are in Appendix D
4.2 Main Results
Table compares the detection performance of DNA-DetectI.LLM against other baselines across different writing tasks and various generation models. DNA-DetccILLM consistently achieves

---

Table AUROC (%) of detectors on -iiitann-ittnnn vs, Al-generated text across datasets and LLMs.
XSum Gemini-2.0 Flash
WritingPrompt
Arsiv
Detectors
GPT-4 Turbo
Chando337 Sonnet
GPT-4
Gemini-2.0
Claude-3.7 Sonnet
GPT-4 Turbo
Gemini-2.0 Flash
Claude-3.7 Somnet
Avg.
Turbo
Flash
Truining- bassd Methods
OpenAl-D
60.51
68.9
62.96
50.94
59.47
57.28
49.63
51.40
68.57
58.85
Biscope
75.08
95.63
94.09
80.08
98.71
98.05
82.53
99.74
96.61
91.17
R-Detect
63.56
45.63
51.13
73.58
71.07
75.74
56.47
57.24
53.55
60.89
Training-free Methods
Entropy
72.26
54.85
74.90
87.85
90.36
91.17
45.60
79.96
80.42
75.26
Likelibood
70.03
69.50
70.39
80.82
95.52
85.85
57.87
93.60
86.24
78.87
LogRank
69.61
69.26
69.81
78.90
94.53
84.13
58.17
94.15
85.80
78.26
DetectGPT
61.50
68.09
61.36
72.80
89.33
78.
56.10
92.18
90.27
74.42
Tust DetectGPT
98.33
92.54
94,30
97.79
98.58
94.14
92.12
99.80
98.21
96,20
Binoculars
98.06
95.56
96.83
97.73
99.53
97.29
93.69
99.87
97.99
97.39
Laside++
97.25
90.87
92.77
95.12
98.19
92.36
90.43
99.54
97.57
94,90
DNA-DetecLLLM
99.31
96,65
98.45
98,86
99.72
98.51
95.00
99.88
98,35
98,30
DNA-DetectLLMM with Other Repair Orders
Low-to-high
98,94
94.98
97.31
29
99.41
97.77
92.48
99.75
97.91
97.43
High-to-low
99.14
97.22
98.63
98.37
94.67
99.77
98.16
Sequential Ropair
98.90
97.82
98.37
96.80
99.77
98.55
95.78
99.91
98.21
98.23
Table Detection performance (AUROC and Fl score) on public benchmark datasets.
M4
DetectRL Multi-LLM
DetectRL Multi-Dumaie
RealDet
Avg.
AUROC
F.
AUROC
Fi
AUROC
Fi
AUROC
Fi
AUROC
F.
OpenAI-D
77.51
71.18
78.15
71.90
74.60
70.03
84.75
77.47
78.75
72.65
Biscope
79.74
73.08
79.97
73.20
76.52
71.64
92.88
86.90
82.28
76.21
R-Detect
61.91
67.14
67.40
66.56
79.19
73.38
65.93
67.72
68.61
68.70
Entropy
83.72
79.10
64.30
71.92
47.82
69.24
75.42
74.72
67.82
73.75
Likelibood
85.77
78.38
66.82
66.71
48.96
66.69
85.35
79.75
71.73
72.88
LogRank
87.50
80.70
67.30
66.71
50.55
66.69
86.28
80.69
72.91
73.70
DetectGPT
73.13
70.11
49.57
66.67
34.67
66.67
78.69
73.80
59.02
69.31
DeleeGet
89.77
84.12
82.26
75.93
74.98
68.91
93.25
90.00
85.07
79.74
Binoculars
90.00
87.40
83.21
82.87
77.45
80.20
93.64
90.51
86.08
85.25
Lastdo++
91.43
84.97
75.36
69.24
67.30
93.90
89.41
82.00
77.57
DNA-Detect.LM
91.74
87.72
88.97
84.85
88.23
84.94
94.48
90.58
90.86
87.02
state-of-the-art performance under all settings, with an average AUROC of 98.30%, representing relative improvement of 0.93%. Specifically, it yields relative gains of 1.36 % 0.87%. and 0.58% on the XSum, WritingPromptts and Arxiv datasets, respectively, demonstrating strong cross- domain generalization. This strong generalization dynamically capture generation discrepancies between domain-specinn text and its ideal Al-generated counterpart through the mutation-reye -rrpair mechanism, enabling robust identification of versus Al-generated text across diverse domains.
be attributed to DNA- DNN--DetctLLMM' ability to
Table the real-world detection performance of all methods on three high-quality public benchmarks. DNA-DetectLLay demonstrates superior reliability, with average AUROC and Fl score improvements of 5.55 % and 2.08 %. respectively. Notably, it achieves significant AUROC gains on challenging DetectRL settings- 6.92% % on Multi-LLM and 13.92% % on Multi-Domain. This improvement can be altributed to the inherent difficulty of DetectRL, where both positive and negative samples may include mixtures texts with the same label due to the dataset's construction. Such complexity hinders traditional training- free methods that rely fixed statistical scores. In contrast, DNA- MrA-DetectLLM accurately computes repair scores for intricately constructed input texts by aligning them with their respective ideal Al-generated sequences. This flexible repair-based scoring allows for more accurate detection under distributional overlap and ambiguous cases, underscoring the practical utility of DNA-DetectLLA in complex detection scenarios.
4.3 Robustness
4.3.1
Robustness against Various Attacks
Figure 4 illlstrates the AUROC curves of DNA-DetectL.LM and other baselines against various attacks (see Appendix F for details). We conducted experiments texts generated by GPT-4 Turbo, Gemini-2.0 Flash, and Claude-3.7 Sonnet, incorporating two distinct attacks: token-level edits and paraphrasing, Editing attacks involved random insertion, deletion, substitution of tokens at rates of
7

---

OpenAIiD Biscoge
Detect Eutropy
Likelhhood LogRank
DetectGPT
Binoculars Lastde++
DNA DeteDelecaL..M
Fost- Vart-DolecttiP1
Insertion
Deletion
Substtttttonn
Paraphrase
Iiee Puste Rats
Toue Pustiive Slae
Tree Portes ve Haste
GPT Turton
Palse Fosuive
Postive
Palse Postiive Rate
Faloe Posttive
Comiia 2. Flash Trua Postive Kuze
True Posiivve Bate
Iree Fesennn More
Tree Visstine Mate
34
CA
CA
Falas
Chanie 3.5 Somert Your pustin Katt
Plssinn Russ
Coopinn More
Trae Postiire Hatte
Tmm
Flise Positive Rate
Faise Postive Rate
False Positive Rate
Fdite Posutvve Rate
Figure 4: AUROC curves of DNA-DetcetLLM and baselines under paraphrasing and editing attacks.
Gemini
Claude Sonnet
95-
AUnOc
-00
85
DesecggTT
80
DeteciSPT
Binoculars
Binooubrs
350
200
Length
Length
Length
Figure 5: Detection performance (AUROC) on input texts truncated to the target number of tokens.
1%. The paraphrasing attacks employed DIPPER 21 to rephrase Al-generated texts. To maintain clean labels, attacks were exclusively applied to Al-generated texts.
The results demonstrate that DNA-DetectL exhibits strong robustness against variety of adver- sarial attacks. For instance, on GPT-4 Turbo-generated text, our method achieves relative AUROC improvements of 6.65% % 3.17%. 6.62 %, and 0.81% under insertion, deletion, substitution, and paraphrasing attacks, respectively. Notably, the improvement is particularly pronounced under low false positive rate (FPR) conditions. We attribute this robustness to the observation that although token-level edits are limited in scope, they can substantially alter the generation probability dis- tribution of the input sequence while having minimal impact on its ideal Al-generated sequence. As result, DNA-DetectLLMM is still able to compute accurate repair scores for reliable detection. Moreover, even under paraphrasing attacks using Dipper, the method effectively captures intrinsic deviations from the ideal sequence and maintains high AUROC of 97. 97.23%. These findings highlight DNA-DetectL..MN LL's capacity to detect adversarially manipulated Al-generated text, even when such attacks are designed to evade detection.
4.3.2 Robustness on Different Lengths
Prior research indicates that token length significantly affects detection performance, with shorter texts proving challenging to detect. We investigate the impact by truncating the input

---

Denway Legfoni
Datuet
5.30
12.
1.72
ATROC (5)
0.75
0.78
Thene (4)
0.59
C3
0.30 0.31
0.30
0.06
6.05
DetectRi. MoRH-LL
DetertRL Multi-Domain
Detection Methods
Figure 6: Comparison of DMMs per-
Figure 7: Comparison of time costs for processing a single sample for each method.
formance when using different LLM pairs.
texts to various target tokens. Figure presents the detection performance across varying lengths for five methods: L,,,,,, Binoculars, Fast-DetecIGPT, Lastde++, and Biscope. Results show that MMN-DetecLLLM consistently outperforms all baselines across varying lengths. On GPT- Turbo-generated text, achieves an average AUROC improvement of 2.46 %. While all methods benefit from longer inputs, LM exhibits greater advantage token length of 40, it surpasses the second-bes best method by 3.38 %. 1.80% and 3.27 %, respectively. These findings suggest that our method enables detection at shorter lengths by extracting more discriminative features from limited textual input.
shorter texts. At
4.4
Ablation Studies
We further evaluated the importance of repair order and different base LLMs through two types of ablation experiments. More detailed ablation studies are available in Appendix G
Repair Score-based Detection under Various Repair Orders. Table [ compares the detection
performance of DMA-DetectLLLM under various repair orders. When the repair order is changed to High-to-low, Low-to-high, or Sequential Repair, a slight performance drop is observed. Although these strategies still outperform other baselines, they require recalculating the conditional score after each mutated token repair, resulting in significantly increased computational cost. In contrast, the simplified repair score (R(s) = (s(s) + (š]s))) maintains strong performance while improving efficiency by an order of magnitude, highlighting its necessity in practical deployment.
DNA-DetecLLLM's Performance with Different M₁ and M2. Figure oeeallutess four different LLM combinations: "Falcon- -B--oonn---Isttrctt + Falcon- 7B". "Llama-3-ae 3-8------BBBu--- + Llama-34 --B",, "Mistral-7B - Mistral-7B", and Allama-2-7B LLama-7B" Results demonstrate that any of these combinations significantly outperform existing baselines, with an average performance improvement of 15.28%. Interestingly, the combination -Llama-2-7B: + Llama-7B" slightly exceeds the default combination
Ballcon-BB-Intrrune + Falcon-7B" used in our main experiments, achieving AUROC of 92.4% and 90.7%. These findings highlight the inherent effectiveness of LM suggesting its robust detection performance is not reliant on any specific LLM combination, with potential for further enhancement through better LLM pairings.
4.5 Efficiency Analysis
Efficiency is critical for Al-generated text detection, as slow detection speeds hinder large-scale or real-time monitoring in practical scenarios. Figure iillustrates the average processing time per sample for each method. To eliminate the confounding factor of text length, we randomly sampled 1,000 long texts from the RealDet dataset, truncated them to 300 tokens, and measured average detection cost with a batch size of 1. We observe that training-based methods such as Biscope and OpenAI-D were the fastest, requiring less than ss per text, but these methods entail significant training overhead. Among training-free methods, classical methods like Likelihood, Logrank, and Entropy are faster, with inference times around 0.3s, but their detection accuracy did not meet our

---

requirements. DNA-DetectLLM, Binoculars, and Fast- DeteetGGeGGPT processed each sample in 0.8s, with DNA-DetccLLLM achieving the better detection performance.
Conclusion
In this paper, introduce DNA-DetectLLMM, novel zero-shot Al-generated text detection method via a DNA-inspired mutation-repair paradigm. Extensive experiments demonstrate that DNA- DetectLLM consistently achieves SOTA detection performance while exhibiting strong robustness across diverse scenarios. We hope our work offers new insights and perspectives for Al-generated text detection and plan further explore the mutation-repair paradigm enhance detection performance.
Acknowledgments
This work is supported by the Postdoctoral Fellowship Program of CPSF under Grant Number GZC20251076, and the National Natural Science Foundation of China (No. U2336202...
References
[1] David Ifeoluwa Adelani, Hao Thi Mai, Fuming Fang, Huy Hoang Nguyen, Junichi Yamag- ishi, and Isao Echizen. Generating centiment-preserving fake online reviews using neural language models and their human- and machine- detection. In International Conference on Advanced Information Networking and Applications, 2019.
[2] Alim Al Ayub Ahmed, Ayman Aljabouh, Praveen Kumar Donepudi, and Myung Suh Choi. Detecting fake news using machine learning A systematic literature review, 2021.
[3] Guangsheng Bao, Yanbin Zhao, Juncai He, and Yue Zhang. Glimpse: Enabling white-box meth- ods to use proprietary models for zero-shot I -----geeraee text detection. In The Thirteenth International Conference on Learning Representations, 2025.
[4] Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang. and Yue Zhang. Fast-detectGPT: Efficient zero-shot detection of machine- muchine-generatee text via conditional probability curvature. In The Twelfth International Conference on Learning Representations, 2024.
[5] Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. On the possibilities of ai-generated text detection, 2023.
[6] Jiaqi Chen, Xiaoye Zhu, Tianyang Liu, Ying Chen, Chen Xinhui, Yiwen Yuan, Chak Tou Leong, Zuchao Li, Long Tang, Lei Zhang, Chenyu Yan, Guanghao Mei, Jie Zhang, and Lefei Zhang. Imitate before detect: Aligning machine stylistic preference for machine-revised text detection. Proceedings of the AAAI Conference on Artificial Intelligence, 39(22):23559-2533 235677, Apr. 2025.
[7] Liam Dugan, Alyssa Hwang, Filip Trhlik, Andrew Zhu, Josh Magnus Ludan, Hainiu Xu, Daphne Ippolito, and Chris Callison- Burch. RAID: A shared benchmark for robust evaluation of machine- generated text detectors. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12492, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
machine-generated
[8] Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889-898. Mel- bourne, Australia, July 2018. Association for Computational Linguistics.
[9] Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. GLTR: Statistical detection and visualization of generated text. Marta R. Costa-jussà and Enrique Alfonseca, editors, Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111-116, Florence, Italy, July 2019, Association for Computa- tional Linguistics.
10

---

[10] Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. How close chatgpt to human experts? comparison corpus, evaluation, and detection, 2023.
[11] Hanxi Guo, Siyuan Cheng, Xiaolong Jin, Zhuo Zhang, Kaiyuan Zhang, Guanhong Tao, Guangyu Shen, and Xiangyu Zhang. Biscope: Ai-generated text detection by checking memorization of preceding tokens. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 104065-104090. Curran Associates, Inc., 2024.
[12] Xun Guo, Shan Zhang, Yongxin He, Ting Zhang, Wanquan Feng, Haibin Huang, and Chongyang Ma. Detective: Detecting zi-generated text via multi-level contrastive learning. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 88320-88347. Curran Associates, Inc., 2024.
[13] Zhiwei Guo, Yu Shen, Ali Kashif Bashir, Muhammad Imran, Neeraj Kumar, Di Zhang, and Keping Yu. Robust spammer detection using collaborative neural network in internet-of -fftheggf applications. IEEE Internet of Things Journal, 8(12):9549-99) 9558, 2021.
[14] Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting LLMs with binoculars: Zero-shot detection of machine-generaceee text, 2024.
[15] Tatsunori B. Hashimoto, Hugh Zhang, and Percy Liang. Unifying human and statistical evaluation for natural language generation. In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long and Short Papers), pages 1689-1701, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
[16] Xiaomeng Hu, Pin-Yu Chen, and Tsung- Yi Ho. Radar: Robust ai-text detection via adversarial learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 15077-15095. Curran Associates, Inc., 2023.
[17] Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. Automatic detection of generated text is easiest when humans are fooled. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5808-1822, Online, July 2020. Association for Computational Linguistics.
[18] Alberto Jiménez-Valle Valverde. Insights into the area under the receiver operating characteristic curve (auc) as a discrimination measure in species distribution modelling. Global Ecology and Biogeography, 21 21(4):498-500 2012.
[19] Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A dataset for biomedical research question answering. In Kentaro Inui, Jing Jiang. Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-UCNLP), pages 2567-2577, Hong Kong, China, November 2019. Asso- ciation for Computational Linguistics.
[20] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. watermark for large language models. CoRR, abs/2301.10226, 2023.
[21] Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit lyyer. Para- phrasing evades detectors of ai-generated text, but retrieval an effective defense. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 27469-27500. Curran Associates, Inc., 2023.

---

[22] Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. DetectGPT: Zero-shot machine-gee geneenttd text detection using probability curvature. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, vol- ume 202 of Proceedings of Machine Learning Research, pages 24962.2 PMLR, 23-29 Jul 2023.
[23] Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun ichi Tsujii, editors, Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797-1807 Brussels, Belgium, October-Novembee 2018. Association for Computational Linguistics.
[24] Sayak Paul and Soumik Rakshit. arxiv paper abstracts. 2021.
Sttpp://www.kaqe com/
[25] Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The Refined dataset for Falcon LLM: outperforming curated corpora with web data, and web data only. arXiv preprint arXiv:2306.01 16, 2023.
[26] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67, 2020.
[27] Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected?, 2025.
(28) Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert- Herhert-Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. Release strategies and the social impacts of language models. arXiv preprint arXiv: 190030 2019.
[29] Yiliao Song, Zhenqiao Yuan, Shuhai Zhang, Zhen Fang. Jun Yu, and Feng Liu. Deep kernel relative test for machine-generated text detection. In The Thirteenth International Conference on Learning Representations, 2025.
[30] Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. Detectlim: Leveraging log rank information for zero-shot detection of machine-generatel text, 2023.
[31] Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, QINGHUA ZHANG, Ruifeng Li, Chao Xu, and Yunhe Wang. Multiscale positive-unlabeled detection of Al-generated texts. In The Twelfth International Conference on Learning Representations, 2024.
[32] Adaku Uchendu, Thai Lc. Kai Shu, and Dongwon Lee. Authorship attribution for neural text generation. In Bonnie Webber, Trevor Cohn, Yulan Hc, and Yang Liu, editors, Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 1384-8339 Online, November 2020. Association for Computational Linguistics.
[33] Yuxia Wang, Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, Thomas Arnold, Alham Fikri Aji, Nizar Habash, Iryna Gurevych, and Preslav Nakov. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generatee text detection. In Yvette Graham and Matthew Purver, editors, Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume I: Long Papers), pages 1369-1407, St. Julian's. 's Malta, March 2024. Association for Computational Linguistics.
[34] Junchao WU, Runzhe Zhan, Derek F. Wong, Shu Yang. Xuebo Liu, Lidia S. Chao, and Min Zhang. Who wrote this? the key to pero-shot LLM LLL--genrrteed text detection is GECScore. In Owen Rambow, Leo Wanner, Mariannaa Apidianaki, Hend AI-Khalifa, Barbara Di Eugenio, and Steven Schockaert, editors, Proceedings of the 31st International Conference on Computational Linguistics, pages 10275-10292, Abu Dhabi, UAE, January 2025. Association for Computa- tional Linguistics.
12

---

[35] Junchao Wu, Runzhe Zhan, Derek F. Wong, Shu Yang. Xinyi Yang, Yulin Yuan, and Lidia S. Chao. DetectRL: Benchmarking LLM-gennrated text detection in real-world scenarios. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
[36] Yihuai Xu, Yongwei Wang, Yifei Bi, Huangsen Cao, Zhouhan Lin, Yu Zhao, and Fei Wu. Training-free free Imm-generated text detection by mining token probability sequences. CoRR. abs/2410.06072, 2024.
[37] Xianjun Yang, Wei Cheng, Yue Wu, Linda Ruth Petzold, William Yang Wang, and Haifeng Chen. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. In ICLR, 2024.
[38] Xianjun Yang. Wei Cheng, Yue Wu, Linda Ruth Petzold, William Yang Wang, and Haifeng Chen. DNA-GPT: Divergent n n-gram analysis for training free detection of GPT ETT-generated text. In The Twelfth International Conference on Learning Representations, 2024.
[39] Xiao Yu, Yuang Qi, Kejiang Chen, Guoqiang Chen, Xi Yang, Pengyuan Zhu, Xiuwei Shang, Weiming Zhang, and Nenghai Yu. Dpic: Decoupling prompt and intrinsic characteristics for I1m generated text detection. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tom- czak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages
Curran Associates, Inc., 2024.
[40] Biru Zhu, Lifan Yuan, Ganqu Cui, Yangyi Chen, Chong Fu, Bingxiang He, Yangdong Deng, Zhiyuan Liu, Maosong Sun, and Ming Gu. Beat LLMs at their own game: Zero-shot LLM- generated text detection via querying ChatGPT. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7470-7483, Singapore, December 2023. Association for Computational Linguistics.
[41] Xiaowei Zhu, Yubing Ren, Yanan Cao, Xixun Lin, Fang Fang. and Yangxi Li. Reliably bound- ing false positives: A zero-shot machine-generatee text detection framework via multiscaled conformal prediction, 2025.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The methodological pipeline and contributions of our approach, including the dataset construction, are clearly outlined in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of this study in Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification: A complete derivation of the simplified repair score, including all assumptions, is provided in Section 3.6
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We fully disclose all information necessary to reproduce the main experimental results, including but not limited to the details provided in Sections and Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
15

---

The authors should provide scripts to reproduce all experimental results for the new reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, described in the supplemental material?
Question:
Answer: [Yes]
Justification: We have included the data and code the supplementary material, along with detailed instructions to facilitate reproduction of the main experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
wublic/guides/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ https/cccppbbcccguuudddoooo Coaee//ddeeSSbmmmmma ssi onPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
proposed method and baselines. only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specify all the training and test details in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: While we do not report error bars, we evaluate our method and all baselines using AUROC and F1 scores.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
16

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
Computational resources are described detail in Appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification: We have thoroughly read the NeurIPS Code of Ethics and ensured compliance in all aspects of our research.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the potential societal impacts of our work in Appendix B
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
17

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
Not involved in misusing.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We provide proper citations for all baseline methods and datasets used in this study.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
18

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: All code, datasets, and models are well-documented and openly available through access in the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
19

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
The LLMs used in this study are explicitly described in Sections and
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttppri///eurpss.... http///rpppscccoooeeeeeeeee///// ccc/Comfferrenccs/222//LLLL for what should or should be described.
20

---

A
Limitations
Due to memory constraints, we were unable scale up the batch size for evaluating method efficiency under ample computational resources. As result, this study does not fully explore performance differences real-world scenarios involving large-scale, real-time monitoring Al-generated texts. The reported efficiency results are based on relative comparisons under uniform small-batch setting.
B Broader Impacts
The proposed DNA-DetectLLMM contributes to the field of Al-generated text detection by improving accuracy and introducing a novel detection perspective to the research community. Its enhanced reliability may serve a reference in socially relevant scenarios where auxiliary judgment is required. However, we emphasize that detection inherently carries a risk of implication or accusation. While method demonstrates strong performance, we strongly oppose the use of its outputs direct evidence in punitive or disciplinary contexts. Regardless of its accuracy, such applications could lead to serious consequences and misuse.
C Prompt Design for Main Data
Table Examples of input prompts and corresponding outputs across different writing tasks, where the outputs are sampled from GPT---generrted texts.
Writing Task
Input Prompt
Output
News Article Writing
A police source told the BBC that infiltrator from the Taliban had allowed militants into the police station the regional capital Lashkar Gah last night. Please continue.
The militants, believed to be associated with the Taliban, reportedly raided the police station and engaged in prolonged gun battle with the officers...
Story Generation
Two kids entered the Rockmount Zoo. Please continue.
As they excitedly crossed the threshold, their eyes widened at the sight of the vibrantly colored parrots squawking from the treetops...
Academic Writing
Please write abstract based on the following title: "Pure Exploration and Regret Minimization Matching Bandits".
This paper delves into the field of pure exploration and regret minimization in the context of matching bandits problems- important machine learning...
Table 3 presents the general-purpose prompts we designed for different writing tasks. Using these prompts, we generated 4,800 Al-generated texts corresppndiig to numan-written texts across GPT-4 Turbo, Gemini-2.0 Flash, and Claude-3.7 Sonnet, which were used in our experiments. To ensure reproducibility, we explicitly report the generation parameters for each API call (note that Top-k is not manually configurable):
*
GPT-4 Turbo: gpt-4- tur-o- -aarrboo2222--0 2024-04-09, Temperature = 1.0, Top-p 1.0.
* Gemini 2.0 Flash:
Temperature 1.0, Top-p 0.95.
Claude 3.7 Sonnet:
claude -laude---77-smmeet022202222 Temperature 1.0, Top-p = 1.0.
D Main Experiments Supplement
All experiments are conducted on single NVIDIA A100 GPU with 80GB of memory. Unless otherwise specified, default settings are used for temperature, top-k, and other generation parameters. No additional hyperparameter tuning is involved in this study. For training-based methods (Biscope and R-Detect), models are trained on 4,000 balanced samples from the HC3 dataset, and the best- performing checkpoints selected based on validation performance separate 2.000-sample
21

---

validation set. In the main experiments, all reported FI scores are the maximum values obtained by selecting the optimal threshold based on the positive and negative scores of each method.
Table 4: Comparison of F1 score across public benchmark datasets.
Method
M4
DetectRL Multi-LLLM DetectRL, Multi-Domain
RealDet
Avg.
OpenAI-D
68.01
70.55
67.89
70.41
69.22
Biscope
71.75
72.00
68.91
81.23
73.97
R-Detect
67.14
66.56
66.26
67.55
66.88
Entropy
75.39
70.10
55.04
60.33
65.22
Likelihood
66.82
66.62
66.60
66.87
66.73
LogRank
66,80
66,69
66.60
66.82
66.73
DetectGPT
54.53
42.41
30.79
66.59
48.58
Fast- Getecttt
81.27
75.84
68.06
84.72
77.47
Binoculars
84.82
80.97
76,24
82.15
81.05
Lastde++
82.74
69.17
61.72
84.59
74.56
DNA-DetectLLM
85.15
84.49
83.94
84.72
84.58
To ensure fairer performance comparison, select fixed thresholds for all methods based on scores computed on a separate clean dataset (c.g.. DNA-DetcotLLMM: 0.6533, Binoculars: 0.9366, etc.). Subsequently,
recompute and report the F1 scores in Table 4 usingg these fixed thresholds. The clean dataset used for threshold selection consists over 3,000 samples generated by GPT-4, Gemini, and Claude based numan-writted texts sourced from XSum, WritingPrompt, and Arxiv.
E
Additional Performance Comparisons
Table 5: Comparison of AUROC (%) across benchmark datasets.
Method
XSum
WritingPrompt
Arxiv
PubMedQA
Avg.
Revise-Detect
39.73
65.54
95.31
66.86
GECScore
70.84
66.31
64.91
67.35
DNA-GPT
65,46
75.22
70.13
82.32
73.28
ImBD
88.07
93.06
91.06
92.59
91.20
GPTZero
99.01
98.54
94.42
88.48
95.11
DNA-DetecLLLM
99.31
98.86
95.00
97.08
97.56
As shown in Table we expanded our comparative experiments to include additional baselines Revise- Detect 440, GECScore [34. DNA-GPT [38] IMBD (6] (recent but non-state-off the-art methods), and GPTZero (a widely used commercial detector). We also incorporated results on the biomedical short-text dataset PubMedQA [19], which further demonstrate the strong and consistent performance of DNA-Detectl across diverse domains and detection settings.
Robustness Experiments
In this study, we do not consider adversarial attacks human-written texts, evasion in such cases generally inconsequential. Instead, we focus on adversarial scenarios involving Al-generated texts, introducing two common attack types: paraphrasing and token-level editing. For paraphrasing attacks, we employ DIPPER with hyperparameters set to a lexical diversity 60 and a syntactic diversity of 60. This level of paraphrasing is sufficient to potentially bypass SOTA detectors. For editing attacks, we tokenize the input using the GPT-2 tokenizer and apply random insertions, deletions, and substitutions to 1% of the tokens. The inserted or substituted tokens are sampled uniformly from the tokenizer's vocabulary.
Table 6 Table 7 and Table & report the detection performance of all methods under various adver- sarial attacks across different Al Al-generated texts. Notably, DNA-DetecIL consistently achieves strong performance across all adversarial scenarios, demonstrating robustness to both editing and paraphrasing attacks. contrast, training-free methods are significantly affected by token-level edits, while training-baa based methods are more vulnerable to paraphrasing- based attacks.
22

---

Table 6: AUROC (%) for GPT-4 Turbo against various attacks.
Method
Insertion
Deletion
Substitution
Paraphrase
OpenAI-D
51.54
52.95
51.62
89.50
Biscope
60.75
62.66
58.59
41.16
R-Detect
61.71
61.27
61.08
57.49
Entropy
57.52
63.38
55,44
36.04
Likelihood
51.54
60.69
49.89
49.33
LogRank
52.19
60.69
50.45
50.48
DetectGPT
43.78
52.36
39.67
44.60
Fast- Dett-eeectGPT
86.16
92.05
85.89
96.70
Binoculars
87.55
93.32
87.28
97.23
Lastde++
71.60
88.51
71.34
94.94
DNA- MMA-DetectLLM
93.37
96.28
93.06
98.02
Table 7: AUROC (%) for Gemini-2.0 Flash against various attacks.
Method
Insertion
Deletion
Substitution
Paraphrase
OpenAI-D
52.28
56.72
53.02
86.46
Biscope
95.48
95.79
95.15
52.61
R-Detect
56.40
56.37
56,84
52.97
Entropy
65.59
71.11
63.47
33.50
Likelihood
76.41
82.86
74.73
47.02
LogRank
76.79
82.93
74.98
47.65
DetectGPT
68.74
76.01
66.60
49.68
Fast- Dett
95.39
96.57
95.23
95.27
Binoculars
97.41
98.06
97.35
96.78
Lastde++
90.39
95.31
90.16
93.41
DNA- DNA-DetectL.IMM
97.72
98.16
97.69
97.80
Table AUROC (%) for Claude-3.7 Sonnet against various attacks.
Method
Insertion
Deletion
Substitution
Paraphrase
OpenAI-D
57.99
60.50
57.80
83.62
Biscope
93.99
93.47
93.65
44.34
R-Detect
58.44
58.24
58.06
53.02
Entropy
73.32
78.29
70.87
35.93
Likelihood
64.11
73.01
61.93
42.80
LogRank
64.26
72.79
61.99
44.21
DetectGPT
60.39
68.81
57.54
45.37
Fast- Rest-DetectGPT
86.38
92.74
85,34
92.56
Binoculars
89.81
95.34
88,88
95.33
Lastde++
73.47
89.85
72.54
90.43
ONA-DetectLLLN
93.77
96.89
93.25
96.81
Table 9: F1 score (%) under paraphrasing and editing attacks
Gomini-1.0
Chudle3.7
Aug.
Subsiitution
Paroptuase
Subsciution
Purpptras
Pispprisee
50.76
56.34
85.80
Detect
66.65
65.63
66.63
66.42
61.92
08.69
68.27
71.36
$7.20
63.39
Licelhood
65.65
66.54
66.54
66.69
06.55
06.02
LegRank
65.1
65.6
66.57
66.57
66.50
66.65
66.65
06.00
DeseiGPT
21,90
30.87
17.13
21.04
$4.86
65.84
$3.22
30.18
43.03
$1.86
38.79
22.13
37.73
Fist DesscGPT
77.10
$4.25
76.24
$9.38
$7.50
$9.04
$7.85
95.03
77.00
85.22
75.81
$4.50
$3.55
Bincculars
81.31
$7.16
$1.15
91.50
92.44
93108
92.28
91.65
83.15
88.71
82.35
$9.78
$7.50
Lasdeee
60185
90.07
60.39
87.35
$2.86
$7.16
$1.39
85.87
61.45
92.22
61.25
83.02
76.22
DNA-DedectLL.M
86.74
90.36
W6.01
93,09
94.12
94.94
93.91
93.63
87.28
91.58
8681
91.06
90.50
23

---

Table 9 reports the Fl-score performances for the primary robustness experiments. Notably, DNA- DetectLLM consistently exhibits superior robustness compared to other baselines in practical scenar- ios.
G
Ablation Study Supplement
Table 10: Ablation results across different repair strategies and datasets.
Setting
XSum
WP
Arxiv
M4
RealDet
Avg.
Time Cost (s)
Default
98.14
99.03
97.74
91.74
94.48
96.23
0.78
Low-to-High
97.08
98.49
96.71
92.42
94.71
95.88
14.11
High-to-Low
98.25
98.94
97.29
89.56
93.59
95.53
14.45
Sequential Repair
98.36
98.37
97.97
91.26
93.71
95.93
14.55
Table I0 compares the detection performance and inference time of DNN-DetectLLL under different repair orders. The results show that the default setting achieves superior performance compared to alternative strategies, while reducing computation time by nearly 20x, highlighting its practical efficiency.
24


Reviews:
[{"review_id": "AB7laYojLV", "reviewer": "Reviewer_gfB8", "summary": "The propose approach is a zero shot approach for LLM generated sequence detection. Given an input sequence (text utterance), ideal AI generated sequence is obtained using greedily selecting the most probable token at each position using probability distribution obtained using Falcon-7B-Instruct. Once both ideal AI generated sequence and original input sequence is available. The position where there is a change is compared and referred as mutations. The input sequence is aligned with ideal LLM generated sequence using an iterative token-level modifications referred as mutation repair mechanism. A score is accumulated throughout the repair. The accumulated score is compared against a calibrated threshold to label input sequence as AI generated or human written. Author run exhaustive experiments on multiple dataset and compare with several existing work to show competitive performance from proposed approach.", "questions": "Refer above", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Major Strength: \n\n1. Approach is novel, there are existing work on DNA inspired zero shot algorithms for AI generated text detection e.g. R1 and R2. However, proposed approach is different w.r.t them. \n2. Performance from proposed approach is competitive and author compare with several recent methods also like Lastdee++. Author also run experiments to show method is more robust than existing method from attacks like insertion, deletion, substitution and paraphrase.\n3. Since method is based on accumulated score between ideal AI sequence and input sequence. Existing work show that detection performance can be sensitive to length of the sequence. Detection become more challenging for smaller length sequence. Author show proposed approach is robust to different sequence lengths\n4.  Author also show that proposed approach works independent of base LLM used to generate ideal AI generated sequence.\n5. Paper is well written and easy to understand.\n\nMinor Weakness: Using different base LLMs and reporting detection performance using them in Table 2 or Appendix can be helpful.\n\nR1: https://arxiv.org/html/2305.12519v2\nR2: https://openreview.net/forum?id=Xlayxj2fWp", "was_revised": true, "final_justification": "I thank author for clearing minor concerns about LLM pairing I had and concerns from other reviewers. I look forward to reading more details about it in Appendix in the final version. I stay with my initial recommendation to accept.", "rebuttal": "Dear Reviewer,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our manuscript and providing valuable suggestions. We are truly delighted by your positive assessment of our work. In response to your feedback, please find our reply below:\n\n**Response to Weakness:** Figure 6 in our manuscript illustrates the performance of DNA-DetectLLM using various base LLMs on the DetectRL dataset. We have identified that further improvements can be achieved through optimal LLM pairing. To better highlight and clarify these experimental findings, we will include an additional comparative table in the Appendix in the final version.\n\nBest regards,\n\nAll authors"}, {"review_id": "ZPxEgg1BTs", "reviewer": "Reviewer_284H", "summary": "The paper proposes a new method for the unsupervised detection of LLM-generated text. The proposed method is a modified version of the cross-perplexity method from Binoculars [1]; however, instead of computing the ratio of perplexity to cross-perplexity, it computes the \"conditional score\", i.e., the ratio of the probability of argmax tokens given the observed document with the cross-perplexity.\n\n[1] Hans, A., Schwarzschild, A., Cherepanova, V., Kazemi, H., Saha, A., Goldblum, M., ... & Goldstein, T. (2024, July). Spotting LLMs with binoculars: zero-shot detection of machine-generated text. In Proceedings of the 41st International Conference on Machine Learning (pp. 17519-17537).", "questions": "N/A", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is generally well-written and (mostly) easy to understand. It does a good job of differentiating between unsupervised and supervised baselines and reports competitive approaches in both categories\n\n2. The results are really quite impressive given the large difference between the scoring models (Falcon-7B, LLaMA-7B) and the target models (GPT-4-turbo, Gemini-2.0-flash, Claude-3.7-Sonnet). This is in pretty stark contrast to some existing methods like DetectGPT, which experience large performance degradations when the scoring and target models don't match\n\n3. The paper experiments on a wide range of benchmarks, models, and baseline approaches. It includes experiments on paraphrasing and editing attacks, analyzes performance as a function of the number of tokens, and evaluates different scoring methods. Very thorough!  \n\nWeaknesses:\n\n1. I found the \"DNA\" inspiration to be more confusing than helpful. As far as I can tell, the \"ideal sequence\" here is just a sequence of argmax tokens which are selected independently of one another, so the ideal sequence might actually be very low probability or ungrammatical under a language model\n\n2. The method in this paper seems to be pretty closely motivated by Binoculars. While I think this is fine, it could be more explicit about it and justify some of the decisions made in the Binoculars paper (e.g., why cross-perplexity) in order to be more easily read as a standalone paper. For example, my understanding is that the models experimented on are paired as (Falcon-7B-Instruct, Falcon-7B) and (Llama-2-7B, Llama-7B) because the cross-perplexity computation requires models to share a tokenizer, but this is not explained in the paper", "was_revised": true, "final_justification": "Strong results, very thorough experiments, and good generalization between disparate scoring and target models.", "rebuttal": "Dear Reviewer,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our manuscript and providing valuable suggestions. In response to your feedback, we offer the following point-by-point replies:\n\n**R1 (to W1):** The ideal sequence is derived from the input text and corresponds one-to-one with it. When the ideal sequence exists independently as a text sequence, it may have a lower probability or deviate from grammatical rules due to independent selection. However, from a fine-grained perspective, each token in the ideal sequence represents the most likely token for the corresponding position in the input text, similar to how nucleotides on the template strand of DNA replication are healthy (non-mutated). In contrast, the tokens (those not of the maximum probability) in the original text may correspond to mutated nucleotides on the DNA replication strand. These inspired our thinking to introduce the mutation-repair paradigm, analogous to DNA, into the detection process.\n\n**R2 (to W2):** Expand on the Binoculars score to further clarify and strengthen the paper’s readability. The mutation score in DNA-DetectLLM improves upon the Binoculars score because the latter (the ratio of log perplexity to cross-perplexity) is highly effective at capturing AI-generated text with high perplexity. A key condition for calculating the cross-perplexity from dual model perspectives is that the tokenizers for the LLMs must be identical. We will provide a detailed explanation of the above points in Section 3.1 (Preliminary) of the final version of the manuscript.\n\nBest regards,\n\nAll authors"}, {"review_id": "er6ZdWNYX4", "reviewer": "Reviewer_LFW4", "summary": "The article proposes a novel method called DNA-DetectLLM, which adopts a perspective inspired by DNA repair to directly and interpretably analyze the intrinsic differences between human-written and AI-generated texts. This method constructs an ideal AI-generated sequence for each input, iteratively repairs suboptimal tokens, and quantifies the cumulative repair effort as an interpretable detection signal.", "questions": "I am curious about the experimental setup for the ablation studies. Is it in-distribution or out-of-distribution? For example, in terms of text length, was the model trained on texts of the same length and then tested on texts of the same length?", "limitations": "None", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "**Strengths**\n- The proposed method is both interesting and well-motivated.\n- The evaluation of the method is robust, considering mainstream and challenging detection scenarios, with necessary ablation studies conducted.\n\n**Weaknesses**\n- The authors seem to have a misunderstanding of the current state of research on detection methods. For example, in line 30, it is stated that “training-free approaches leverage intrinsic statistical differences to distinguish human-written and AI-generated texts,” but in reality, these methods are not entirely “training-free.” They also require extracting feature scores on a training set, which are then used for detection during the testing phase. It would be beneficial to revise the corresponding statements with reference to [1].\n- Following the previous weakness, it appears that the authors did not separate the training set from the testing set, leading to unfair comparisons in Table 1 and the related ablation studies. For a fair comparison, training-based methods and training-free methods should be trained on the same training set and evaluated on the same testing set.\n\nOverall, this is a good paper, but there seems to be a significant issue with the experimental setup. If this issue can be resolved, I would be happy to recommend accepting this paper.\n\n**Other Suggestion**\n\nIn the related work section, the authors discuss the differences between their approach and previous studies. However, I believe there are still some similar works that have not been mentioned. Overall, the paper assumes that human-written text resembles a mutated chain, where token selection deviates from the optimal probability, resulting in measurable differences. It is worth noting that some works also adopt similar ideas, such as Revise-Detect. [2], which allows LLMs to rewrite the text themselves to fix tokens that deviate from the optimal probability, and GECScore [3], which introduces grammatical errors and LLM's correction preferences to measure the sequence differences between the two types of text. The key difference is that these works operate at the sequence level for \"repair\" and \"correction,\" whereas the proposed method in this paper performs iterative \"repair\" and \"correction\" at the token level. I speculate that the token-by-token repair process may provide more controllable and fine-grained measurements of the deviation patterns between the two types of text. Therefore, comparing this work with such methods would be beneficial in highlighting the advantages of the approach presented in the paper.\n\n[1] A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions. Computational Linguistics, 2025.\n\n[2] Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT. EMNLP, 2023.\n\n[3] Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore. COLING, 2025.", "was_revised": true, "final_justification": "Given that my major concerns have been addressed and the paper now presents clearly justified, robust, and interesting results, I believe it is suitable for acceptance. I encourage the authors to further clarify the limitations of the supervised detector comparisons in the final version to avoid any misinterpretation. On balance, the paper makes a meaningful contribution to the research community, and I recommend acceptance.", "rebuttal": "Dear Reviewer,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our manuscript and providing valuable suggestions. In response to your feedback, we offer the following point-by-point replies:\n\n**R1 (to Weakness): Training-free Methods Require No Threshold Training under AUROC Evaluation.**\n\nWe understand your point regarding training-free methods not being entirely \"training-free.\" For example, methods like GECScore [1], aside from using predefined thresholds, can also optimize a distribution-specific \"optimal threshold\" based on training data. Similarly, other training-free methods also require identifying an “optimal threshold” based on feature scores from training data to enhance classification performance in practical applications. However, it is equally important to emphasize that when these methods calculate statistical scores based on relative differences, they indeed do not require additional training. For instance, Fast-DetectGPT [2] calculates log conditional probability differences across multiple samples using sampling and scoring models (LLMs in evaluation mode).\n\nIn our experiments, we primarily adopt AUROC, a threshold-independent evaluation metric. Therefore, none of the training-free methods in our experiments involves training on data within the same distribution to identify an \"optimal threshold.\" Instead, they calculate statistical scores for positive and negative samples and directly convert these scores into AUROC values, enabling a fair comparison across methods. Meanwhile, the training-based methods are trained on the general HC3 dataset [3] to simulate real-world out-of-distribution (OOD) scenarios (see Section 4.1), providing a relatively fair evaluation against the zero-shot methods that require no additional training. We hope this clarification addresses your concerns regarding our training and evaluation.\n\nP.S. The computation of F1 scores presented in Table 2 is explained in Appendix D of the manuscript.\n\n**R2 (to Suggestion):** Thank you for your suggestion and for highlighting additional related work. We will expand Section 2 to include a discussion of sequence-level revise or rewrite-based approaches, such as Revise-Detect [4] and GECScore, which assess textual differences through revision-based strategies. As shown in the supplementary Table 1 below, we evaluated the detection performance of Revise-Detect and GECScore on GPT-4-generated texts. Our results show that these methods display substantial performance variability across different data distributions and are considerably less reliable than DNA-DetectLLM. Interestingly, Revise-Detect performs well on the Arxiv dataset. We believe this may be due to the formal writing style of the dataset, where the LLM used for revision (GPT-4o) can more effectively capture the homology of GPT-4-generated texts. To more fully demonstrate the advantages of DNA-DetectLLM, we will include these experimental results in the Appendix of the final version.\n\n**Table 1. AUROC(%) on GPT-4-generated Texts.**\n| Method          | Xsum | WritingPrompt | Arxiv | Avg.   |\n|-----------------|------------------|-------------------|-------------------|--------|\n| ***Revise-Detect***   | 39.73            | 65.54             | 95.31             | 66.86  |\n| ***GECScore***       | 70.84            | 66.31             | 64.91             | 67.35  |\n| ***DNA-DetectLLM***   | 99.31            | 98.86             | 95.00             | **97.72**  |\n\n**R3 (to Question):** The training-free methods do not require any additional training and compute AUROC scores directly based on statistical scoring. In contrast, training-based methods such as Biscope are trained on the HC3 dataset and do not incorporate any explicit handling of input text length. In the robustness experiment on text length, we only truncate the input text during the testing phase, without modifying the training process.\n\n[1] Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore. COLING, 2025.\n\n[2] Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature. ICLR, 2024.\n\n[3] How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection. 2023.\n\n[4] Beat LLMs at Their Own Game: Zero-Shot LLM-Generated Text Detection via Querying ChatGPT. EMNLP, 2023.\n\nBest regards,\n\nAll authors"}, {"review_id": "D5eto3yPYz", "reviewer": "Reviewer_YKTY", "summary": "This paper is inspired by the DNA repair process in biology. The paper proposes a new way to detect AI-generated text: iteratively “repairing” a text sequence and measuring how hard it is to make it match the ideal AI-generated version. Then, the paper proposes the difficulty of the repair process of fixing fraction quantification. We introduced one to capture the cumulative effort required to complete the conversion. Last, by comparing the repair score with the calibrated threshold, DNA-DetectLLM robustly distinguishes AI-generated text and human-written text, using the basic differences that deviate from the ideal generation.", "questions": "Please See Weaknesses.", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The paper is well written and easy to follow.\n- The idea is novel and motivated by a strong analogy from biology.\n- The experiments are well-designed and show strong results.\n- The baselines used for comparison are mostly appropriate and fairly broad.\n\n**Weaknesses**\n\n1. The datasets used in the main experiments (XSum, Writing, ArXiv) are quite similar—they are all long-form writing tasks. A more challenging dataset, such as **PubMedQA**, which is used in Fast-DetectGPT and focuses on scientific question answering, would be helpful to test how well the method works in a different domain.\n\n2. It would be useful to include **GPTZero**, a popular commercial detection tool, as a baseline. This would help readers understand how the method compares to widely-used systems.\n\n3. The authors should include comparisons with more recent and stronger baselines:\n- **DNA-GPT** (ICLR 2024)\n- **Text Fluoroscopy** (EMNLP 2024)\n- **ImBD** (AAAI 2025)\n- **Biscope** (NeurIPS 2024)\n\n4. The paper mentions using outputs from GPT-4 Turbo, Gemini 2.0 Flash, and Claude 3.7 Sonnet. However, it does not provide details such as the **temperature** or **model versions** used to generate these outputs. This information is important for reproducibility. Also, since GPT-4 Turbo is now being phased out, the authors could consider using GPT-4o for future experiments.", "was_revised": true, "final_justification": "I have resolved my doubts. I gave this paper a score of 4 because it performs very well on the current benchmarks, and compared to the previous method, DNA-GPT, it has achieved a significant improvement in speed, which is important.", "rebuttal": "Dear Reviewer,\n\nWe sincerely appreciate the time and effort you have dedicated to reviewing our manuscript and providing valuable suggestions. In response to your feedback, we offer the following point-by-point replies:\n\n**R1 (to W1 & W3): On Dataset and Baseline Expansions**\n\nTo further evaluate generalization performance, we have included results on **PubMedQA**, a more challenging biomedical short-text dataset, in Table 1.\n\nOur experiments already incorporate several of the most recent and state-of-the-art baselines, such as **Biscope** (NeurIPS 2024), **R-Detect** (ICLR 2025), and **Lastde++** (ICLR 2025). In response to your suggestion, we have additionally included several relevant recent methods (e.g., DNA-GPT and ImBD) in Table 1 for a more comprehensive comparison. Note that **DNA-GPT** uses the same scoring model (Falcon-7B-Instruct) as the other methods, and **ImBD** is pretrained on the HC3 dataset.\nAs shown in Table 1, **DNA-DetectLLM** maintains strong detection performance on PubMedQA (AUROC = **97.08%**), clearly demonstrating its superior generalization ability across diverse distributions.\n\n**Table 1. AUROC(%) on GPT-4-generated Texts.**\n| Method           | XSum  | WritingPrompt | Arxiv | PubMedQA |\n|------------------|--------|----------------|--------|-----------|\n| ***Biscope***          | -      | -              | -      | 92.39     |\n| ***Fast-DetectGPT***   | -      | -              | -      | 96.31     |\n| ***Binoculars***       | -      | -              | -      | 96.48     |\n| ***Lastde++***         | -      | -              | -      | 87.41     |\n| ***DNA-GPT***          | 65.46  | 75.22          | 70.13  | 82.32     |\n| ***ImBD***             | 88.07  | 93.06          | 91.06  | 92.59     |\n| ***DNA-DetectLLM***    | **99.31**  | **98.86**          | **95.00**  | **97.08**     |\n\n**R2 (to W2): On the Use of GPTZero as a Baseline**\n\nGPTZero is a popular commercial detection tool that was included as a baseline in earlier works such as Fast-DetectGPT. However, many subsequent studies have shown that GPTZero performs significantly less reliably than strong zero-shot methods. Recent state-of-the-art approaches (e.g., **Biscope**, **Lastde++**) have also opted not to include GPTZero in comparisons, instead focusing on benchmarking against more advanced methods. Additionally, the inference cost of GPTZero is relatively high.\nGiven these considerations, we believe that including GPTZero as a baseline in our work is not essential.\n\n**R3 (to W4): On the Use of GPT-4 Turbo, Gemini 2.0 Flash, and Claude 3.7 Sonnet**\n\nThe model IDs for the text generators used are as follows:\n\n* **GPT-4 Turbo**: `gpt-4-turbo-2024-04-09`\n\n* **Gemini 2.0 Flash**: `gemini-2.0-flash-001`\n\n* **Claude 3.7 Sonnet**: `claude-3-7-sonnet@20250219`\n\nAll generations were performed using the default official parameter settings provided by the respective APIs. We will include further implementation details and code snippets in the Appendix of the final version to enhance the reproducibility of our results.\nWe once again thank you for your constructive suggestions and hope that our responses have adequately addressed your concerns.\n\nBest regards,\n\nAll authors"}]

Author rebuttal:

Meta-review: This paper introduces a clever new way to detect AI-generated text. Inspired by DNA repair, the method, called DNA-DetectLLM, measures the effort needed to fix a piece of text to make it look like an LLM-generated sequence. The core idea is that human writing, with its natural imperfections and stylistic choices, will require a lot more "repair" than text that was already generated by an LLM.

The reviewers appreciated the idea and the impressive results. In response to feedback from the reviewers, the authors added more baselines, including recent academic work like DNA-GPT and ImBD, and importantly, the commercial tool GPTZero. They also tested their method on the PubMedQA dataset, showing robustness in a completely different domain (short-form biomedical text). They also adopted a more rigorous fixed-threshold method for calculating F1 scores across all experiments, 

I think this is a strong submission and appreciate how this approach doesn't require any training and is interpretable as well.
Decision: Accept (spotlight)