Diversity Boosts AI-Generated Text Detection
Authors: Advik Raj Basani
Pin-Yu Chen
Venue: Accepted by TMLR ()

Abstract: Detecting AI-generated text is an increasing necessity to combat misuse of LLMs in education, business compliance, journalism, and social media, where synthetic fluency can mask misinformation or deception. While prior detectors often rely on token-level likelihoods or opaque black-box classifiers, these approaches struggle against high-quality generations and offer little interpretability. In this work, we propose DivEye, a novel detection framework that captures how unpredictability fluctuates across a text using surprisal-based features. Motivated by the observation that human-authored text exhibits richer variability in lexical and structural unpredictability than LLM outputs, DivEye captures this signal through a set of interpretable statistical features. Our method outperforms existing zero-shot detectors by up to $33.2$% and achieves competitive performance with fine-tuned baselines across multiple benchmarks. DivEye is robust to paraphrasing and adversarial attacks, generalizes well across domains and models, and improves the performance of existing detectors by up to $18.7$% when used as an auxiliary signal. Beyond detection, DivEye provides interpretable insights into why a text is flagged, pointing to rhythmic unpredictability as a powerful and underexplored signal for LLM detection.

Published in Transactions on Machine Learning Research (02/2026)
Diversity Boosts Al-Generated Text Detection
Advik Raj Basani
f20221155@gou wil.ooooaablllllaaiiii..
Birla Institute of Technology and Science, Goa
Pin- Yu Chen IBM Research
pin-gu.chen@ibma chen Wit.m.niiimmmooo
Reviewed OpenReview: https: https://openreviee
openreview. net/ per/oomm? id= WscAU9It11
Abstract
Detecting Al-generated text is an increasing necessity to combat misuse of LLMs in educa- tion, business compliance, journalism, and social modia, where synthetic fluency can mask misinformation or deception. While prior detectors often rely on token-level likelihoods or opaque black-box classifiers, these approaches struggle against high-quality generations and offer little interpretability. In this work, we propose DivEye, a novel detection framework that captures how unpredictability fluctuates across text using surprisal-basee features. Motivated by the observation that human-authored texi exhibits richer variability in lexical and structural unpredictability than LLM outputs, DivEye captures this signal through set of interpretable statistical features. Our method outperforms existing zero-shot detec- lors by up to 33.2% and achieves competitive performance with fine-luned baselines across multiple benchmarks. DivEye is robust paraphrasing and adversarial attacks, generalizes well across domains and models, and improves the performance of existing detectors by up to 18.7% when used insights into why text is flagged, pointing to rhythmic unpredictability as powerful and underexplored signal for LLM detection.
auxiliary signal. Beyond detection, DivEye provides interpretable
1 Introduction
Large Language Models (LLMs) have become deeply integrated into daily human workflows, powering appli- cations from personal assistants to academic writing (Alahdab 2024 Meyer et al. 2023 Lund et al., 2023 and content creation (Hu et al. 2024 Yuan el al., 2022). Their fluency and generalizaa
capabilities make them highly useful, but this same fluency enables growing number of concerning applications. Al-generated text can now be scamlessly inserted into essays, news articles, logal briefs, scientific abstracts, and social
media posts, often without detection De Giorgio et al. 2025 Papageorgion et al. 2024 Telenti et al. 2024
Törnberg et al. 2023).
As LLM-generated outputs grow more sophisticated and luman-like, detecting them has become an increas- ingly difficult challenge (Abdali et al.) 2024 Gameiro et al.] 2024 Wu et al., 2025 Zhang et al. 2024 Reliable AT-text detection is crucial for mitigating risks such as misinformation, AT-assisted academic dis- honesty, professional misconduct, and the inadvertent suppression of authentic human writing. Traditional approaches to this problem rely on supervised detectors (Shukla et al.) 2024 Tolstykh et al. 2024 Wang et al. 2024b) trained annotated datasets of Al and human-authores text. These models often incorporate rich features, ranging from stylometry and structure to information-thccrrtii metrics, and achieve high per- formance within the domain they were trained on. However, such methods struggle to generalize to unseen models or domains (Doughman et al. 2024 Gameiro et al. 2024), especially as new LLMs are frequently released. In contrast, zero-shot detectors (Bao al. [2024 Gehrmann et al.] 2019 Mitchell al. 2023 Wang al., (2024a) offer promising alternative by avoiding model-specific training. These approaches either extract statistical cues from language model probability distributions use LLMs themselves

---

Published in Transactions on Machine Learning Research (02/2026)
? DivEya Extraction
Compu tation of Surprisal a
Feature Concatenation Classification
Distribution Variance,
Feature Concatenation
Human Text
May o "ti: 772
8(x)) log P(zs spa)
First- Order Features
Ay,, Aa
Extraction Surprisal per Token
Input Text
Frozen LLM
Second-Order Features
Binary Classifi (KGBoost)
0_0
Variancerll, Entropy, Autocorneiation d2t. Pat
Split compute S(x,) per token 21
Framework Bocsted
Externel Al-Detection Framework (if any)
Figure Overview of DivEye. DivEye extracts diversity-based features (see Section 3 Equation token-level surprisal patterns. These features can be used in two ways: (1) as standalone detector, (2) as enhancement to existing detectors, improving their performance.
from
inference-time detectors, enabling model-agnostic delection at scale. Given the increasing deployment of unknown or fine-tuned LLMss in the wild, zero-shot detection has become essential tool for maintaining platform integrity and addressing the forensic needs of AI-era communication.
Contributions. We introduce DivEye lightweight classifier trained on features extracted from off-the- shelf LLMs in a zero-shot manner. These features capture diversity- statistics of token-level surprisal, which we leverage to improve Al-text detection. Our approach focuses on capturing the distributional irregularities in Al-generated text that arise from differences in the generative process compared to human writing.
Zero-shot diversity detection: We propose DivEye, a lightweight classifier trained on zero-shot features derived from token-level surprisal diversity metrics. These metrics capture fluctuations and patterns that reflect the constrained and often repetitive generation process of LLMs. We provide a principled motivation for each feature, connecting them to known properties of human vs. machine text generation, and demonstrate how DivEye can improve Al-text detection using these features.
Language & Model-agnostic detection: DivEye leverages zero-shot features, requiring no access to the generator model's 's internals or any fino-tuning. It operates purely on token probability sequences from an off-the-shelf language model and generalizes across different languages and model families.
Complementary to evisting delectors: We show that DivEye captures statistical patterns that are dis- tinct from those used by traditional detectors, which often rely on fine-tuned language representations or classifier- basod signals. When combined with these approaches, DivEye significantly boasts overall ro- bustness, particularly against challenging high-quality generations and paraphrased adversarial examples.
Strong generulization across domains and allacks: Extensive evaluations across three benchmarks and varied testbeds reveal that DivEye not only achieves state-of the---r accuracy in standard settings but also remains robust when tested on unseen domains and language models.
2 Background and Problem Formulation
The emergence of LLM has led to new era of machine-generated text that closely mimic human writing across range of tasks. These models are trained to approximate the true conditional distribution of natural language, denoted as Puuman (x2 Fet), by learning from massive corpora of Himan-written text (Chn et al. 2024 Lu et al. (0025). The LLM's learned distribution is represented as PL (2(), and during inference, the model generates text by sampling tokens sequentially from this distribution. While modern LLMs achieve remarkable fluency, they still constitute an imperfect approximation: PLAM + Phuman
The code of method and experiments available https://github.comt com/IBM/diveye
2

---

Published in Transactions on Machine Learning Research (02/2026)
in general. At inference time, LLM selects tokens by sampling from this learned distribution Zhou al. 2024h. which remains approximation of the true distribution that governs human text generation (Ippolito et al. 2020 Jones et al. 2024). This approximation gap, subtle as it may be, is the crux of Al text detection.
From a theoretical standpoint, prior works Ghhosal et al. 2023 Sadasivan et al. 2025 highlight the fun- damental limitations of AT text detection: as generative models approach the ideal of human-like language modeling, distinguishing their outputs from real text becomes increasingly difficult, if not impossible. Yet, as Chakraborty et al. (2023) point out, even models arbitrarily close to optimal remain statistically de- tectable under certain conditions, particularly when multiple samples or robust features are available. This theoretical detectability provides a foundation for practical detection methods that capitalize on the subtle imperfections in current LLM outputs.
In practice, existing detection approaches fall into two broad categories: watermarking and zero-resource detection. Watermarking techniques (Block et al.] 2025 Gloaguen et al.; 2025 Kirchenbauer et al., 2024: Liang et al. 2024 Liu et al., 20244) embed distinct patterns in generated text. but necessitate access to model internals or fine-tuning capabilities, rendering them unsuitable for black-bax adversarial settings, well for practical cases of watermark-free Al text detection. In contrast, zero-resource detection methods require no prior knowledge of the target model, instead relying on statistical or learned discrepancies between human and AT text. These methods can be further categorized as statistical and maining-basee approaches.
Statistical / Zero-shot detection methods refers to identifying Al-generated text without task-specific training, either by leveraging LLM probability cues or prompting LLMs directly as detectors. For example, methods like Entropy (Lavergne al., 2008). LogRank (Ghosal et al. 2023). DetectGPT (Bao et al.) 2024 Mitchell al. 2023 and Binoculars Hans et al. 2024) use off-tho-shelf LLMs to evaluate the consistency of token predictions under masked or perturbed inpuls. These methods assume that Al-generated texts are sampled from narrower, more concentrated conditional probability distribution than human writing, resulting in greater token-level confidence and reduced lexical diversity.
Training-based / Fine-tuned detection methods (Chen al. 2023 Mao el al., 2024 Hu al. 2023 train classifiers, such as fine-tuned transformers on labeled corpora of human and AI text. While these models be accurate, they often fnil to generalize across domains or against adversarial paraphrasing, especially when trained on specific generators or prompts. We discuss all related works in more detail in Appendix A
Despite significant progress, no existing method fully resolves the problem of detecting Al-generated text in the wild. Our work addresses this gap by approaching the problem from a new angle: instead of analyzing individual token probabilities (Solaiman et al. 2019) isolation, we propose to measure statistical diversity over token sequences, quantifying how text fluctuates in its use of surprising or predictable tokens. This provides more global signature of the generative process that is robust to paraphrasing, domain shifts, and even partial text corruption.
3 DivEye: Methodologies
DivEye is built on the central observation that fluctuations in token-level surprisal provide a strong signal for distinguishing machinc- and human-generated text. By systematically analyzing the statistical variation of surprisal across a sequence, DivEye capt ures distributional and temporal pallerns that go beyond traditional likelihood- beadeoood-base metrics. The name DivEye thus reflects our method's focus on aiversity-awar analysis of language generation behavior.
3.1 Design Hypothesis
A central challenge in detecting Al-generated text (Ghosal et al. 2023 Sadasivan et al. 2025 lies in the fact that current models, though fluent, often prioritize coherence and consistency at the cost of variability and umpredictability. By contrast, human writers naturally introduce irregularities, such as unexpected lexical choices structural shifts, that make their text inherently more diverse.
3

---

Published in Transactions on Machine Learning Research (02/2026)
Our hypothesis is that human-written text inherently exhibits greater stylistic diversity and unpredictability than Al-generated text. In everyday writing, humans make creative, spontaneous choices, sometimes using unexpected words or phrases, that introduce bursts of surprise amid more routine language. Our approach centers on the premise that AI-generated text, despite its fluency, often lacks the inherent diversity observed human-written language. This divergence slems from the fundamental objective of LLMs: to maximize the likelihood of generated sequences within their learned probability distributions (Park & Choi 2024p. Consequently, Al-generated text tends to exhibit higher degree of predictability, resulting in lower variability and surprisal compared to human-authored content. We support this hypothesis through both intuitive reasoning and empirical evidence, as detailed in Remark 1
Remark Consider text sequence X model M. The language model defines a probability distribution Px((X) - II== Par(z; Teee) where each token is chosen to maximize overall likelihood.
= (I1, I2,...; .n) generated either by a human or by a language
Humans, however, produce language through complex, multi-layered cognitive process that balances infor- mativeness, creativity, and contextual appropriateness, rather than strictly maximizing statistical likelihood.
Formally, the surprisal of token 2, under model M defined as:
(x,) - log Pas (Te (ee))
Since M is trained assign high probability to plausible continuations, its outputs tend minimize surprisal on average, implying that maximum likelihood generation compresses diversity:
Ex~Px (xx) & Ex-P, [SM (xx)]
where PH denotes the distribution of human generated text.
Similarly, human language exhibits higher variance in surprisal due to spontancous creative choices, idiomatic expressions, and stylistic variation, causing:
Varx (x))) < Var
We validate this theoretical intuition through empirical experiments detailed below, which confirm statis- tically significant differences in surprisal and diversity metrics between Human-writtee and Al-generatod texts.
We collect 200 human-written essays and 200 by BiScope (Guo et al.) 2024). For each essay, we computed the token-level surprisal scores using a fixed language model evaluator (GPT-2) and then calculated the mean and variance of these surprisal values per essay. Figure shows the histogram of mcan surprisal scores across the two sets, while Figure2b display the histogram of surprisal variances. The human-wriiten lexts exhibit a noticeably wider spread and heavier tails in both metrics, indicating greater unpredictability and stylistic variability. In contrast, the Al-generated essays cluster around lower mean surprisal and exhibit significantly lower variance. These results empirically confirm our theoretical claim: human language inherently reflects higher diversity and surprise, whereas A Al-generated language, optimized for likelihood, tends toward more predictable and homogeneous patterns.
essays comparable topics, provided
Rather than treating token-level surprisal in isolation, DivEye analyzes how it varies across entire text to capture higher-level stylistic patterns. By extracting global statistical features from surprisal sequences, our method reveals differences in the rhythm and variability of unpredictability, traits that distinguish human writing from the more uniform outputs of LLMs, as illustrated by the clear class separation in predicted probabilities shown in Figure
These features are theoretically grounded in the Uniform Information Density (UID) hypothesis, which posits that optimal language speakers strives to distribute information uniformly Jaeger & Levy (2006). However,
4

---

Published in Transactions on Machine Learning Research (02/2026)
2.0
0.6
GPT-4-Turbo
GPT-4-Turbo
0.5
1.5
0.4
Density
Density
1.0
0.3
0.2
0.5
0.1
0.0
0.0
2.0
2.5
3.0
3.5
4.0
5
8
9
10
11
12
Mean Surprisal
Surprisal Variance
(a) Mean Surprisal Distribution
(b) Surprisal Variance Distribution
Figure 2: Distribution of token-level surprisal metrics for human-written vs. SPPT-4-Turoo-genraatte essays. The left plot shows the hislogram mean surprisal per essay, while the right plot shows the histogram of surprisal variance. Human-writter texts exhibit higher dispersion and heavier tails in both distributions, suggosting greater linguistic unprodictability and stylistic diversity. In contrast, GPT-4-Turbo outputs are more concentrated and predictable, aligning with the likelihood- maximizaiaiiiiiio objective of language models.
Leckere
Eychinettit
Bronnar
Detect LM
02
0.25 038 0.55
8.75
055
3.0
05
Prddeded Probabiity officman Authoretip
Figure 3: Distributions of predicted class probabilities for diverse Al-lext detectors. Trained and evaluated Testbed 4 of the MAGE benchmark, DivEye shows stronger separation between human- written and Label Al-generated texts, indicating greater confidence and discriminative power.
human-written
while human speakers aim for this efficiency, empirical evidence shows that natural language remains highly nonstationary (Piantadosi et al. 2011) and exhibits characteristic "burstiness' due semantic shifts (Alt- mann et al. 2009). In contrast, LLMs, driven by seeeihood-maximizattin objectives, often approximate the theoretical UTD optimum loo strictly, resulting in unnaturally flat probability curves devoid of these organic fluctuations (Holtzman et al.) 2020). Consequently, we introduce first-order and second-order derivatives of surprisal, as essential metrics to quantify this 'rhythmic unpredictability" capturing the distinct structural variance that separates human writings from LLMs.
3.2 Mathematical underpinning of DivEye
human-written To robustly distinguish Al-generated text from human- written text, it is insufficient to rely solely single
measure such as perplexity (Xu et al. 2024). Perplexity summarizes average token likelihood, but overlooks how unpredictability fluctuates within a text. To better capture these patterns, DivEye computes higher- order statistical features over surprisal sequences, revealing structural signals beyond aggregate likelihood.
Surprisal. Human language is inherently diverse and unprodictable, balancing consistent patterns with bursts of creativity, often introducing novel expressions, grammatical deviations, and stylistic vari- ation. These deviations result in varying levels of token predictability, which can be quantified using surprisal (Kuribayashi et al. 2025 well-known information-theerreii measure defined the negative log-probability of a token under language model:
S(xx) log P(rx 31, -...... .,,
5

---

Published in Transactions on Machine Learning Research (02/2026)
Given text sequence X {a11 #2, ................................ ........,,,,,, J. surprisal measures how 'unexpected' each token is in context. It can be computed directly from model's bog-probabilitis, providing a principled way to quantify the local uppredicabilityy of text.
Rather than examining individual token surprisals in isolation, we summarize their behavior through ag- gregate metrics. The mean surprisal serves as a coarse indicator of how "expected" text on average: Lower values suggest higher conformity to the model's learned distribution, whereas higher values point to more frequent unpredictability. However, as stated before, human writing is not merely unpredictable in aggregate; it also exhibits fluctuations in predictability that correspond to stylistic variation, topic shifts, bursts of creativity. This motivates analyzing not just the mean but also the variance of surprisal, which captures the extent of variation in token-level surprise throughout the text. Formally, this can be represented as:
Mean: js
n Σ S(x)); Variance: o
Σ (S(x,) - -ps)²
(1)
Mean and Variance are not sufficient. While mean and variance capture the central tendency and spread of surprisal values, they overlook deeper structural signals that differentiate human and Al text. Al-generated text is optimized for consistency, producing more symmetrical distributions centered around high-probability tokens ([ppolito et al. 2020). Skewness (ti) quantifies this asymmetry: a positive skew suggests the presence of rarc, surprising tokens typically found in human writing. Similarly, kurtosis (72) captures the frequency of extreme deviations from the norm. A high kurtosis indicates heavy-tailed behavior, another hallmark of authentic, stylistically diverse writing. These higher-order moments allow DivEye detect subtle irregularitics and stylistic outliers that can be missed by detectors focusing only average behavior.
n
S(xx) - pS 3 0s
1 n n Σ (S(z)) us
Skewness: 71
Σ 71
Kurtosis: 72
-3
(2)
Static metrics still miss temporal structure. While static surprisal statistics (mean, variance, skew- ness, kurtosis) describe the overall distribution of token-level unpredictability, they fail to capture how this unpredictabilityy evolves throughout sequence, a key trait distinguishing human and Al-generated text. To model these temporal dynamics, we compute the first-order difference AS, = S(x,) S((,,)), which reflects immediate changes in surprisal. The mean (An) and variance (Ao²) of AS, quantify the typical magnitude and variability of these shifts, capturing stylistic volatility such as abrupt topic or tone changes commonly found in human writing.
We further analyze the second-order difference A = AS, AS,-1, which tracks fluctuations in the rate of change of surprisal. From this sequence, we extract three metrics: (1) variance (o)²), to capture the extent of rapid or erratic stylistic transitions; (2) entropy (HA*), which reflects the irregularity of these transitions; and (3) autocorrelation (p(A?S))), which measures whether bursts of unpredictability cluster together, often indicative of structured human creativity. These second-order metrics reveal rhythmic and non-stationary patterns in human text that are typically absent in the more homogencous output of LLMs, providing a richer signal for robust Al-lext detection. Mathematically, these can be defined as:
1
AS, S(z)) S(x-1))
Ap
AS,
Σ (AS: -2A² 2=
(3)
n-
2=2
A = AS; - -AS-11
02
1
AA S. - MA3)?,
Har =
Σ Pb Pog.o..
(4)
n
3
t=3
((ASS)) E (A MA:)(A - HA+)]
02
where MA* mcan of sccond-order differences, and Pa the empirical probability of value falling into bin after discretizing S for entropy computation. We provide empirical validation of these temporal features and their individual contributions to detection performance in Appendix B

---

Published in Transactions on Machine Learning Research (02/2026)
Combinations. Collectively, DivEye, formalized as (D) in Equation 6 encapsulates critical aspects of text generation that distinguish human creativity from algorithmically generated predictability, thereby serving as robust basis for our detection framework.
D 2 72
Ap, 2 Bo PA?
(6)
Distribution
Ist-Order
2nd-Order
Here, D is vector of statistical features with dimension of 9, including distributional properties firsi- order differences, and second-order differences of the text. We can apply any autoregressive LLM to generate these feature vectors by passing the text tokens through the model to compute the features listed above, which are then concatenated into the final vector D. We train a binary classifier using DivEye features, optionally combined with predictions from an Al-text detector. Implementation details are further explained in Algorithm ! and Appendix C
DivEye as booster. Existing detectors, whether fine-tuned classifiers zero-shot LLM-based methods, primarily rely on semantic or surface-level cues, and often falter against high-quality adversarial exam- ples that closely mimic human writing. DivEye offers a complementary signal by capturing statistical and temporal patterns of token-level unpredictability that are orthogonal to traditional features. We integrate DivEye into both settings by augmenting detector outputs with its feature vector and training a wight meta-classifier (e.g., XGBoost (Chen & Guestrin 2016) or Random Forest (Breiman 2001)) over the com- bined representation. Empirically, we find that this fusion siguificantly boosts performance, particularly adversarial and out-of-distribution examples, without requiring retraining modification of the original model.
4 Experiments
4.1 Experimental Settings
Datasets. We evaluate our DivEye framework across a comprehensive suite of datasets that encompass a wide range of generative models, domains, and adversarial strategies. Our primary benchmark is the RAID dataset (Dugan et al. 2024), which consists of carefully crafted adversarial examples designed evade standard detectors. To assess robustness under diverse generation conditions, we also evaluate on the MAGE benchmark [Li ct al., 2024), which spans cight distinct testbeds targeting various domains (c.g.. Yelp
(Zhang et al. 2015)), XSum (Narayan et al. 2018). SciXGen (Chen et al. 2021a CMV (Tan et al. 2016
and generator families (e.g., GPT [Radford et al. 2019). OPT (Zhang et al. 2022). Bloom (et al.
2023
This granular evaluation allows us to isolate and quantify the contribution of diversity metrics across specific domains and model types. Details about each testbed in RAID & MAGE are discussed in Appendix H
Additionally, we incorporate HC3 (Guo et al. 2023), large-scale, heterogeneous corpus of human and machine text, which includes both English and Chinese instances of human and Al-generated Q&A data. The inclusion of HC3 enables us to probe cross-linguistic generalization of our method.
Baselines. We compare DivEye with various baselines, including both traditional statistical detectors and recent fine-tuned models. These include RADAR (Hu et al.) 2023). LogRank (Ghosal et al. 2023), Entropy [Lavergne et al. 2008), FaTt Bao et al. 2024). DetectLLM (Su et al.] 2023), Binoculars (Hans
et al. 2024 RAÍDAR (Mao et al.) 2024 OpenAT Detector (Solaiman et al. 2019), Longformer (Beltagy 2020 , and BiScope (Guo et al. | 2024]. These basclines cover a range of tochniques, from token-level
ct al.
likelihood-based ranking to transformer--ased classification. Additionally, we evaluate our framework against several other open-source detectors listed on the RAID leaderboard, ensuring fair and broad comparison with state-of-the-art public tools across multiple detection paradigms.
Implementation Details & Metrics. Unless stated otherwise, we use GPT-2 to compute all DivEye fea- ture vectors. Regardless, we studied the effect of DivEye with different LLMs as base models and summarized the results in Scction 4.6 In scorc-only detection scenarios, predictions are based solely over concatenatod DivEye features. For both standalone and boosted setups, we train lightweight XGBoost (Chen & Guestrin 2016) classifier a meta-model, using only DivEye features in the former, and concatenating them with
7

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: Performance of zero-shot methods on diverse testbeds from MAGE. The OOD settings examine the detection capability on texts from unseen domains or texts generated by new LLMs.
Settings
Methods
HumanAcc
MachineAce
AvgAer:
AUROC
Testbed 2,3,4: In-distribution detection
LogRank
58.81%
63.94%
61.38%
0.67
Entropy
76.43%
76.84%
76.64%
0.83
DetectLLLM
66.36%
62.07%
64.21%
0.72
Arbitrary-domains & Model-spocific (GPT-J T05]
PastDetect
62.31%
50.49%
56.4%
0.59
Binoculars
60.11%
65.22%
62.67%
0.69
BiScope
89.62%
84.86%
87, 24%
0.93
DivEye
90.63%
88.56%
89.60%
0.97
LogRank
89.61%
56.15%
72.88%
0.76
Entropy
85.96%
60.4%
73.18%
0.78
DetectLLM
88.54%
80.77%
84.66%
Fixed-domain (WP 34
cunttrry-mmdder
FastDetecttPT
87.25%
54.08%
70.67%
Binoculars
80.80%
62.07%
BiScope
91.78%
95.27%
93.53%
DivEye
92.22%
96.88%
94.55%
LogRank
84.91%
44.47%
64.69%
0.68
Entropy
75.68%
50.01%
62.86%
0.67
DetectLLM
64.74%
69.02%
66.88%
0.75
Arbitrary-domans & mobelsary-models
Dete
93.65%
41.73%
67.69%
0.7
Binoculars
76.1%
54.89%
65.49%
0.71
BiScope
91.54%
58.70%
75.12%
0.86
DivEye
73.72%
82.57%
78.15%
0.88
Testbed 5,6,8: Out-of-distributtoo detection
LogRank
85.84%
19.82%
52.89%
0.52
Entropy
77.56%
34.74%
56.15%
0.59
DetectLLM
67.85%
58.5%
63.18%
0.68
Unseen Models (BLOOM-7B 31
PestDtttct
94.57%
13.81%
54.19%
0.54
Binoculars
76.10%
34.89%
65.50%
0.71
BiScope
76.72%
30.47%
63. 83.60%
0.72
DivEye
74.75%
77.06%
75.91%
0.86
LogRank
88.57%
49.8%
69.19%
0.74
Entropy
78.5%
58.16%
68.33%
0.74
DetectLLM
74.15%
71.52%
72.34%
0.79
Unseen Domains (WP 35
Pet
95.99%
47.17%
71.58%
0.74
Binoculars
78.93%
67.8%
73.37%
0.8
BiScope
80.1%
78.3%
79.2%
0.86
DivEye
94.64%
84.53%
89.59%
0.97
LogRank
83.87%
43.95%
63. 63.91%
0.68
Entropy
74.93%
30.18%
62.55%
0.66
DetectLLM
63.66%
67.40%
65.53%
0.73
Unseen Domains & Unseen Models
Dete
93.38%
41.50%
67.44%
0.70
Binoculars
77.85%
69.39%
73.62%
BiScope
86%
82.58%
84.24%
0.92
DivEye
69.75%
83.22%
76.49%
0.87
the original detector's prediction scores in the latter. Each testbed in MAGE & RAID provides predefined
training and test sets, which we use for model training and evaluation. We evaluate all models using Aver- age Accuracy (AvgAce), AUROC, TPR@FPR-5% and Fl score to capture overall, thresholdfinddependenn. and balanced performance, respectively. For all methods, we independently optimize any required deci- sion thresholds for AUROC and for TPR@FPR-5%, ensuring that each model is evaluated under its most favorable threshold for each metric.
4.2 Performance of DivEye
We evaluate DivEye across wide range of challenging testbeds to assess its robustness and adaptability both domain and model shifts. Table [ presents the performance DivEye on six distinct testbeds from the MAGE benchmark (Li et al. 2024): three in-distribution and three out-of-distribution. Across all testbeds.
8

---

Published in Transactions on Machine Learning Research (02/2026)
CMV
AUROC 0.95
85.44
BLOOM-7B
AUROC 0.88
79.08
Yelp
TLDR
AUROC 0.95
86.69
FLAN-T5-Small
AUROC 0.88
80.37
SQuAD
XSum
AUROC 0.98
93.25
GPT-3.5-Turbo
AUROC 0.93
84.10
HSwag
OPT-125M
AUROC 0.93
84.17
ELI5
AUROC 1.00
94.62
SciGen
94.70
GPT-J
AUROC 0.97
89.59
WP
AUROC 0.99
05.50
ROCT
GLB-130B
90.64
20
40
60
80
100
20
40
60
80
100
Average Accuracy (%)
Average Accuracy (%)
Figure 4: (a) Performance DivEye across different domains, generated by GPT-J-6B. (b) Performance of DivEye across various generator models. Results based the MAGE benchmark.
Table 2: Performance of zero-shot and open-source finetuned methods RAID. Results are aggregated over 8 domains, 12 models, and 4 decoding strategies. å denotes difference in AUROC from benchmark leader.
1.0
0.913
0.8
TPR@
Frameworks
Type
FPR=5%
AUROC
5
0.2
Chimee
65-small-lora
30
Fine-tuned
93.9%
0.986
-0%
0.0
Classifier
DivEye (Ours)
93.63%
0.984
-0.20%
(zero-shot)
Figure 5: F1 scores on HC3 show that DivEye perperrss GLTR 89
Desklib Al
25
Fine-tuned
94.9%
0.973
-1.32%
SuperAnnoaaee
95
Fine-tuned
70.3%
0.910
-7.71%
and OpenAI-Detector 93 with strong results across English and Chinese.
Binoculars
16
Zero-shot
79.0%
0.844
-14.40%
RADAR 49
Fine-tuned
65.6%
0.819
-6.99%%
GLTR
30
Zero-shot 59.7%
0.727
26.27%
DivEye consistently achieves superior AUROC of 0.92 on average and AvgAce compared to existing zcro-shot baselines, showeasing its ability to generalize effectively to both seen and unseen generation settings. We demonstrate that human-written and machine-generated text can be distinguished based on the hypothesis outlined in Section 3 We further include the corresponding TPR0FPR-53 performance in Table 5
Table benchmarks DivEye on the RAID dataset (Dugan et al. 2024 which includes suite of diverse models, domains, attacks, and decoding strategies. DivEye outperforms diverse set of zero-shot methods by 26.11% and matches the performance of generative detection baselines, reaffirming its robustness to evasive generation strategies. Figures & demonstrate the performance of DivEye across different domains and generator models, achieving competitive AUROC of 0.98 and 0.93, respectively. These results highlight DivEye's 's stability and high performance across heterogencous sconarios, underscoring its domain and model-agnosticc nature.
Moreover, Appendix D.5 reports DivEye's detection rates on all major models, including GPT-3.5-Turbo (Brown al. 2020 and GPT-4o (ct al. 2024b), Claude-3-Opus and Sonnet (Anthropic). as well as Gemini- 1.0-Pro (eel al. 2025), demonstrating highly competitive accuracies across the board. Collectively, these results confirm that DivEye provides robust and adaptable foundation for detecting Al-generated text.
4.3 Robustness to Adversarial Attacks and Multilingual Text
To evaluate robustness, we assess DivEye on a diverse set of adversarial attacks, including paraphrasing attacks from the MAGE datasct and transformation-baat jailbreak attacks from the RAID benchmark. Table 3 shows that DivEye consistently achieves strong detection performance under these challenging set- tings. On the MAGE benchmark, DivEye outperforms the fine-tuned Longformer baseline in both average

---

Published in Transactions on Machine Learning Research (02/2026)
Table 4: Integration with DivEye consistently boosts performance across detectors, particularly on diverse domains (Testbed 4) and paraphrasing attacks (Testbed 7).
Methods
HumanAcc McchineAce
AvgAcc
AUROC
å: Boost
Testbed 4: Arbitrary Domains & Arbitrary Models
RADAR
47.74%
74.86%
61.30%
0.62
DetectLLM
64.74%
69.02%
66.88%
0.75
PaT
93.65%
41.73%
67.69%
0.7
Binoculars
76.1%
54.89%
65.49%
0.71
BiScope
91.54%
58.70%
75.12%
0.86
DivEye
73.72%
82.57%
78.15%
0.88
DivEye + RADAR
74.69%
85.31%
80%
0.90
18.7%
DivEye + DetectLLM
75.44%
84.23%
79.34%
0.9
12.96%
DivEye + GeT
79.42%
83.90%
81.66%
0.91
13.97%
DivEye + Binoculars
69.81%
83.47%
76.64%
0.87
11.15%
DivEye + BiScope
80.69%
88.31%
84.5%
0.93
9.38%
Testbed 7: Paraphrasing Attacks
BiScope
48.80%
89.79%
69.30%
0.81
DivEye
69.75%
83.22%
76.49%
0.87
DivEye + BiScope
65.38%
90.84%
78.11%
0.89
8.81%
accuracy and AUROC by 10.15% and 0.11 respectively. On the RAID benchmark, which reports only accu- racy, DivEye achieves competitive results across a range of adversarial perturbations, outperforming several zero-shot detectors, most notably surpassing Binoculars by 11.2%. A more detailed breakdown of perfor- mance by individual atlack lype is provided in Appendix E. We also lest DivEye's robustness to diverse adversarial scenarios, including character- and word-level perturbations, paraphrasing via commercial tools, prompt obfuscations, and distributional shifts, and find that consistently achieves exceptional detection performance; consolidated overview is provided in Appendix
We further evaluate DivEye's multilingual gen- eralizability using both English and Chinese splits of the HC3 dataset. Figure 5 illustrates that DivEye performs consistently well and has higher Fl scores across both languages using GPT-2 (Radford et al.) 2019] & GPT-2-ChinesS [CKIPAA 2024h for English and Chinese spectively. This suggests that surprisal-baser bassd statistical features are not heavily language- specific and can generalize across languages.
Table 3: Performance of DivEye and baselines on adver- sarial benchmarks, MAGE & RAID. + RAID reports TPR&FPR-5% instead of AvgAcc.
Settings
Methods
AvgAcc
AUROC
[MAG]] Testbed Paraphrasing Attack
Longformer |T]
69.34%
0.76
GPT-3.5-Turbo BiScope 1b]
Paraphrased
via
69.30%
0.81
DivEye (Ours)
76.49%
0.87
[RAID] Adversarial Attacks
Desklib
23
91.2%
0.948
4.4 Efficiency Analysis
Paraphrase, Whitespace,
e5-small-lora
80
85.7%
0.968
DivEye (Ours)
80.52%
0.951
In addition accuracy, we analyze the com- putational efficiency of DivEye. Figure 6b il- lustrates the latency of our method, showing that DivEye requires little 0.01 seconds per sample while outperforming several finc- tuned and zero-shol detectors, achieving up to a 2971x x speedup compared to RAiDAR. Be- cause DivEye only requires single forward pass through a small GPT-2 model and performs lightweight statistical computations, it significantly faster and more resourco-efficiet than larger fine-tuned trans- formers. This enables deployment in latency-sensitive environments without. compromising performance.
Misspelling, moglyph, Article Deletion
Binoculars 20
99.32%
RADAR
63.9%
0.819
GLTR
39
51.5%
0.709
10

---

Published in Transactions on Machine Learning Research (02/2026)
Inference (s
GPT2
0.88
29.71
GPT2-XL
0.89
20
1.95
1.27
2.19
Falcon-7B
Liama-3.14
0.91
0.1T
0.0
0.02
0.01
0.01
0.07
0.08
0.07
Llama-3.2-1
0.87
LogRank
Entropy
DivEye
DetectLLM
FastDetectGPt
OpenAl Detector
BiScope
Binoculars
RAIDAR
RADAR
Mistral-7B-v0.3
0.7
0.8
0.9
1.0
Score (%)
(a) DivEye using diverse base models.
(b) Inference time comparison.
Figure 6: (a) Performance of DivEye across different base models (GPT-2, GPT-2-XL, Falcon-7B). (b)
Inference time (in sec) comparison of various methods.
4.5 Effectiveness of Boosting by DivEye
We empirically verify that DivEye-basee based diversity features can act as performance boosters for a wide range of detection models. To integrate DivEye, we concatenate its feature vector with the original model's prediction scores and train a lightweight XGBoost classifier as a meta-model.
Table illustrates improvements in AUROC and AvgAce when diversity metrics are appended existing frameworks such as RADAR, Binoculars, DetectLLM, BiScope and FastDetoctGGTT. Across all evaluated baselines, the inclusion of diversity features consistently leads to better detection scores by over 18.7%. Addi- tionally, existing frameworks in combination with DivEye demonstrate substantial performance gains when evaluated against paraphrasing attacks. This validates the hypothesis that static and dynamic surprisal- based features capture orthogonal information to traditional hcuristics, making them a valuable addition any detection pipeline. We also investigate how much DivEye contributes to the final prediction relative the underlying detector. Appendix DA4]] reports a detailed feature- importane analysis, highlighting the complementary signals captured by DivEye within the boosted ensembles.
4.6 Ablation Studies
DivEye's Performance on Different Base Models. To assess the adaptability of DivEye across different language model backbones, we evaluate its performance, on Testbed 4 of the MAGE benchmark, when instantiated with various base LLMs used to compute token-level surprisal. As shown in Figure (aa DivEye consistently performs well across all models, achieving an AUROC of 0.88 with GPT2, 0.91 with Llama-3. 1-8B, and 0.90 with Falcon-7B. Notably, even the smallest model, GPT2, performs competitively, and human classification accuracy improves with larger models, suggesting that higher-capacity LMs better capture stylistic diversity. These results highlight DivEye's robustness and efficiency across scales, making it suitable for resource-constrainel settings. Appendix D.3 further reports baseline and DivEye performance across various base models.
Relevance of DivEye's Features. DivEye's feature set (Equation 6 captures token-level surprisal pat- terns across multiple orders, including distributional moments, first-order shifts, and second-order dynamics. To assess the contribution of each group, we compute feature importances from trained XGBoost model. On average, second-order features contribute the most (39.4%), followed by distributional features (34.2%) and first-order differences (23.7%). The prominence of second-order features suggests that abrupt unnat- ural shifts in predictability are strong indicators of machino-generatee text. While traditional distributional statistios remain informative, they are insufficient on their own. These findings support DivEye's central claim about second-order features: modeling the evolution surprisal yields stronger detection capabilities than relying solely on static mcasures. Additionally, Appendix [.6 contains a detailed discussion about the contribution of each component in DivEye, including both ceauuro-mpoorancc estimates and statistical leave-one-out ablation results.
11

---

Published in Transactions on Machine Learning Research (02/2026)
5 Conclusion
We introduce DivEye, a lightweight classifier for Al-text detection that leverages zero-shot diversity features from token-level surprisal. Our method is modcl-agnostic, computationally efficient, and demonstrates strong generalization across detectors and datasets. Appendix J provides a detailed discussion of the method's limitations; broader impacts, and associated ethical considerations.
References
Aaditya Bhat.
Gpt-wikiiintro, 2023.
URL http:///hananggeeeeoo///attttttt
GPT-RikI-intro
Sara Abdali, Richard Anarfi, CJ Barberan, and Jia He. Decoding the ai pen: Techniques and challenges in detecting ai-generated text, 2024. URL httpp:://arxw..orgg///aaa//////s wttp:o////bb//...0//////
Fares Alahdab. Potential impact of large language models on academic writing. BMJ evidence-basee Medicinee 29(3):201-20) 2024.
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérouane Debbah, Etienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Maz- zotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. The falcon series of open language models, 2023. URL
abs/aaas..11...//// 16867
Eduardo G. Altmann, Janet B. Pierrehumbert, and Adilson E. Motter. Beyond word frequency: Bursts, lulls, and scaling in the temporal distributions of words. PLoS ONE, 4(11 (67788, November 2009. ISSN 1932- 6203. doi: 10.1371 10.1371/journal htwp://oooaalloooo.000...... 0007667. URL http://dtx.../ doi org/ 1371/journal, 000e0
Anthropic. Model Card Addendum: Claude 3.5 Haiku and Upgraded Claude 3.5 Sonnet. https://assets.
Micael Arman. Poems dataset.
2020.
Abinew Ali Ayele, Nikolay Babakov, Janek Bevendorff, Xavier Bonet Casals, Berta Chulvi, Daryna Demen- tieva, Ashaf Elnagar, Dayne Freitag, Maik Fröbe, Damir Korenčić, Maximilian Mayerl, Daniil Moskovskiy, Animesh Mukherjee, Alexander Panchenko, Martin Potthast, Francisco Rangel, Naquee Rizwan, Paolo Rosso, Florian Schneider, Alisa Smirnova, Efstathios Stamatatos, Elisei Stakovskii, Benno Stein, Mar- iona. Taulé, Dmitry Ustalov, Xintong Wang, Matti Wiegmann, Seid Muhic Yimam, and Eva Zangerle. Overview of pan 2024: Multi-author writing style analysis, multilingual text detoxification, oppositional thinking analysis, and generative ai authorship verification condensed lab overview. In Experimental IR Meets Multilinguality,, Multimodaliily, and Interaction: 15th International Conference of the CLEF Asso- ciation, CLEF 2024, Grenoble, France, September 9-12, 2024, Proceedings, Part Il, pp. 231-259, Berlin, Heidelberg, 2024. Springer-Verlat Verlag. ISBN 978-3-031-71907-3, doi: 10.0077/966-33-3---99969 11. URL https: 00tp:/00.0077911---------00-0000
David Bamman and Noah A. Smith. New alignment methods for discriminative book sunmarization, 2013.
Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yuc Zhang. Fast-dctectgpt: Efficient zero-
shot detection of machine-generain text via conditional probability curvature, 2024. URL https://
arxiv.org/abs/2220 05130
Iz Beltagy, Matthew E. Peters, and Arman Cohan. Longformer: The long-document transformer, 2020. URL
Meghana Moorthy Bhat and Srinivasan Parthasarathy. How effectively can machines defend against machine- generated fake news? an empirical study. Anna Rogers, João Sedoc, and Anna Rumshisky (eds.). Proceedings of the First Workshop on Insights from Negative Results in NLP, pp. 48 18-53, Online, November 2020. Association for Computational Linguistics. doi: 000653//11222..nnii/i 1.7. URL https://
Salanthology.org/2000
wtw.:o///22...iihtttte....... insights-1.7/
12

---

Published in Transactions on Machine Learning Research (02/2026)
Michal Bicń, Michal Gilski, Martyna Maciejewska, Wojeiech Taisner, Dawid Wisniewski, and Agnieszka Lawrynowicz. RecipeNT.G: A cooking recipes dataset for semi-structured text generation. In Proceedings of the 13th International Conference on Natural Language Generation, pp. 22 28, Dublin, Ireland, December 2020. Association for Computational Linguistics. URL hattss://cllanthoolgg http:/hhhoooooooog/2000 wttp:/000...nn1........
Adam Block, Ayush Sckhari, and Alexander Rakhlin. Gaussmark: A practical approach for structural walermarking of language models, 2025. URL
13941
Matyáš Boháčck, Michal Bravansky, Filip Trhlik, and Václav Moravec. Fine-grained ezech news article dataset: An interdisciplinary approach to trustworthiness analysis, 2022.
Leo Breiman. Random forests. Machine Learning, 45(1): :: 32, Oct 2001. ISSN 1573-0565. doi: 10.1023/A A: 1010933404324. URL httpp:://oi..oo org/10 1023/A A: 1010933404324
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Tya Sutskover, and Dario Amodei. Language models are few-shot learners, 2020. URL https: http:://awwwxx.o///////////s//// wttp./b/////....1..53333 14165
Souradip Chakraborty, Amrit Singh Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. On the possibilitics of ai-generated text detection. 2023. URL https: http:://wwwwwww...aaaaoo abs 2304. 04736
Bciduo Chen, Xinpeng Wang. Siyao Peng, Robert Litschko, Anna Korhonen, and Barbara Plank. "secing the big through the small": Can LLMs approximate human judgment. distrriuttions on NLI from a few explanation?? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 14499- Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/ /1/////112222 /1t.2/..inn//...----e.. 842. URL https://
aclanthology wrg/2024. oclarthologgy.orr2222..... windininnnnnnneeeee wmm...mmmmp........ 842/
Hong Chen, Hiroya Takamura, and Hideki Nakayama. SciXGen: A scientific paper dataset for context- aware text generation. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 1483-1492, Punta Cana, Dominican Republic, November 2021a. Association for Computational Linguistics. doi: findings emnlp 128/
10.18653/vl. v5565//v122211.11 http//////////...iinnnii-ee/// emaiiinggseeml..122. URL
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 785 794. ACM, August 2016. 2939672.2939788
doi: 10.1145/9336672.233777 URL http://dk /oo. /tgg/10.114//
Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. DialogSum: A real-life scenario dialogue summarization dataset. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 5062 5074, Online, August 2021b. Association for Computational Linguistics. doi: 10.18653/v1 indings-acl.4420 URL https://aclanthooogg) org/
2021. f www.iiinnnnaaaaoo 449/
Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li,, Rita Singh, and Bhiksha Raj. Token prediction as implicit classification to identify LLM-generated text. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 13112- 13120, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023. emnlp-main.810. URL httpss://aclaathhlllgee wwwp://y..rr........ emnlp-mallppppp 1t..pppmmai..111/...
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun
13

---

Published in Transactions on Machine Learning Research (02/2026)
Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason
Wei. Scaling instruction-fieetunee language models, 2022. URL
CKIPLAB. Gpt2-chinese. https://github.cco http://ituuub..ooo/taapaaaaaaa/a /ttpa//////////eoooooo.ooo//oaoa wra.sfffrrrrsrrrrrrrrrrr 2024.
NLP Team Cohere. World-class ai, at your command, 2024. URL
Accessed: 2024-02-02.
Andrea De Giorgio, Giacomo Matrone, and Antonio Maffei. Detecting large language models in exam essays. In 2025 IEEE Engineering Educalion World Conference (EDUNINE), pp- 1-6. IEFE, 2025.
Desklib. DeskLib: Al- A.Lees--ettecto--1.... http:://wwggaaaaeee..ooooeaaaaaa 01
Jad Doughman, Osama Mohammed Afzal, Hawau Olamide Toyin, Shady Shehata, Preslav Nakov, and Zeerak Talat. Exploring the limitations of detecting machine- geniirttde text, 2024. URL https://arxiy:
org/abs/De
aaaaabb//.......// 2406 11073
Liam Dugan, Alyssa Hwang, Filip Trhlik, Josh Magnus Ludan, Andrew Zhu, Hainiu Xu, Daphne Ippolito, and Chris Kallison-Burch. Raid: A shared benchmark for robust evaluation of machine-generated text detectors, 2024. URL https:
org/ htwg/abb//205507949
BigScience Workshop et al. Bloom: A parameer open-access multilingual language model, 2023. URL
05100
Gemini Team et al. Gemini: A family of highly capable multimodal models, 2025. URL https://aaxx: OTg/abs/233 22tp..232...121 2382.11805
Meta al. The llama 3 herd of models, 2024a. URL htll:pps//aaxxv.oo////////212112
OpenAl et al. Gpt-4 technical report, 2024b. URL httpps//aardrr.rr///aa/2223...00
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Arnaul Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 889 898, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1 //t////v/////////// 1082. URL https:://claatthololg wwt/:18-g11/11/ 1082/
Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. ELI5: Long form question answering. In Anna Korhonen, David Traum, and Lluis Mårquez (eds.), Proceed- ings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3558 3567, Flo- rence, Italy, July 2019. Association for Compulational Linguistics. doi: 10.18653/11/P19--1366 URL https://aclaanthooog: 1tw.:aahhoooooooor/----99---
Rinaldo Gagiano, Maria Myung-Hee Kim, Xiuzhen Zhang, and Jennifer Biggs Robustness analysis of grover for machino-generated news detection. In Afshin Rahimi, William Lane, and Guido Zuc- con (eds.), Proceedings of the 19th Annual Workshop of the Australasian Language Technology Asso- ciation, pp. 119 127, Online, December 2021. Australasian Language Technology Association. URL https://aclantthoogy orgg/021....... http:/222.altaa1..... alta-1.12/
Henrique Da Silva Gameiro, Andrei Kucharavy, and Ljiljana Dolamic. Llm detectors still fall short of real world: Case of llm-generated short news-like posts, 2024. URL
00291
Sebastian Gehrmann, Hendrik Strobelt, and Alexander M. Rush. Gltr: Statistical detection and visualization of generated text, 2019. URL
Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and Am- rit Singh Bedi. Towards possibilities & impossibilities of ai-generated text detection: A survey, 2023. URL ktttps///arxe wttpiiv.orraaa/////eee 15264
14

---

Published in Transactions on Machine Learning Research (02/2026)
Thiband Gloaguen, Nikola Jovanović, Robin Staab, and Martin Vechev. Black-box detection of language model watermarks, 2025. URL http://awxwi..ogaab/////sases//a http//aabb2205.2077575 20777
Derek Greene and Pádraig Cunningham. Practical solutions the problem of diagonal dominance in kernel document clustering. In Proc. 23rd International Conference Machine learning (ICML. '06), pp. 377-384.
ACM Press, 2006.
Jesus Guerrero, Gongbo Liang, and Izzat Alsmadi. A mutation- text generation for adversarial machine learning applications, 2022. URL https://arzivvorrg//aa httpg///b//222..11p1g 11808
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. How close is chatgpt to human experts? comparison corpus, evaluation, and detection, 2023. URL
http:://araxv..or////////000
Hanxi Guo, Siyuan Cheng, Xiaolong Jin, ZHUO ZHANG, Kaiyuan Zhang, Guanhong Tao, Guangyu Shen, and Xiangyu Zhang. Biscope: AT-generated text detection by checking memorization of preceding tokens. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: //openreview.co aereenreree.nnttfouumiiiii htte://oorummidcHeeellSSSSs
Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Gold- blum, Jonas Geiping, and Tom Goldstein. Spotting 1hms with binoculars: Zero-shot detection of machine- generated text, 2024. URL http:////axxxiv..g/bbbbeeee
22000
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degener- ation, 2020. URL
Jianhua Hu, Huixiang Gao, Qing Yuan, and Ganchang Shi. Dynamic content generation in large language models with real-time constraints. 2024.
Xiaomeng Hu, Pin-Yo Yu Chen, and Tsung- Yi Ho. Radar: Robust ai-text detection via adversarial learning, 2023. URL htttps://axxi... orgg abs 20w./2377.000 03838
Daphne Ippolito, Daniel Duckworth, Chris Callison-Burch, and Douglas Eck. Automatic detection of generated text is casiest when humans fooled. In Dan Jurafsky, Joyce Chai, Natalic Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Arnual Meeling of the Associalion for Compula- tional Linguistics, pp. 1808 1822, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/ wt/p://.111000lll/--/1 www.elmmin....... URL
wwwp:/o0oyorr//000oooog acl main.. 164/
T. Jaeger and Roger Levy. Speakers optimize information density through syntactic reduction. In B. Schölkopf, J. Platt, and T. Hoffman (eds.), Advances in Newral Information Processing Systems, vol- ume 19. MIT Press, 2006. URL
wttpssccc/paaaaee wwwpapppppppppcpcccc http://...........ppoppop/aa///p filess/aappr/200///iee.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra. Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023.
Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. PubMedQA: A datasel for biomedical research question answering, In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the Ith International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 2567 2577, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19 1259. URL https://aclaantholoog 1t1.://90/...//11--2221300
Cameron R. Jones, Sean Trott, and Benjamin Bergen, Comparing humans and large language models on an experimental protocol inventory for theory of mind evaluation (epitome). Trensactions of the Association for Computational Linguistics, 12:803 -190 06 2024. ISSN 2307-387X. doi: 10. 10.1162/tacl a 00674. URL
httpss///abb..orr/...11/////// a_064
15

---

Published in Transactions on Machine Learning Research (02/2026)
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models, 2024. URL https://arxxww.rrg// wwtp:///.wwrraa/./.......
Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Tyyer. Paraphrasing evades detectors of ai-generated text, but retrieval is an effective defense, 2023. URL httpp://arxiv.orr/
18408
Tharindu Kumarage, Joshua Garland, Amrita Bhattacharjee, Kirill Trapeznikov, Scott Ruston, and Huan Liu. Stylometric detection of ai-generated text in twitter timelines, 2023. URL httpps:/arxivv.rrg// 2303. 036977
Tatsuki Kuribayashi, Yohei Oseki, Souhaib Ben Taieb, Kentaro Imui, and Timothy Baldwin. Large language models are human-like internally, 2025. URL http::://arrt...orr/a///222/2/
Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Ab- dullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, Shahul ES, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmaann, Huu Nguyen, and Alexander Mattick. Openassistant conversations democratizing large language model alignment, 2023. URL https: https://arxias //tp://aaxvvvvvo htsp/bbb/220044 07327
Thomas Lavergne, Tanguy Urvoy, and François Yvon. Detecting fake content with relative entropy scoring. In Proceedings of the 2008 International Conference on Uncovering Plagiarism, Authorship and Social Software Misuse Volume 977, PAN'08, pp. 27-31, Aachen, DEU, 2008. CEUR-WS.org.
Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. Mage: Machine-generalie lext detection in the wild, 2024. URL https://aaaat http://aaxvvv.o../... ab3/2355 13242
Gongbo Liang, Jesus Guerrero, and Izzat Alsmadi. Nutation-based adversarial attacks neural text detectors, 2023a. CRL https://arxiv.ort org/ abs////////// /00p../222.0555
Weixin Liang, Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. Gpt detectors are biased against non-native english writers, 2023b. URL http::///aaxiv.og//bb////////
Yuqing Liang, Jiancheng Xiao, Wensheng Gan, and Philip S. Yu. Watermarking techniques for large language models: A survey, 2024. URL httpp:///axxiv.or//bbb/////////
00089
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Irwin King, Hui Xiong, and Philip S. Yu. A survey of text watermarking in the era of large language models, 2024a. URL https://arxixs http/abb/221..0000
Zeyan Liu, Zijun Yao, Fengjun Li, and Bo Luo. On the detectability of chatgpt content: Benchmarking, methodology, and evaluation through the lens of academic writing, 2024b. URL https://arxiv.org org/ abs/ 0bs/2306.05527
Cristóbal Lozano, Ana Díaz-Negrillo, and Marcus Callies. Lozano, Deaz-Negrill,, Callies (2020) Designing and compiling a leurner corpus of wrillen and spoken narralives: COREFL, pp. 21 46. 12 2020. ISBN 978-3-631-70185-0, doi: 10.3726 /91/99977856-5511111
Yuxuan Lu, Jing Huang, Yan Han, Bennet Bei, Yaochen Xie, Dakuo Wang, Jessic Wang, and Qi He. Lim agents that act like us: Accurate human behavior simulation with real-world data, 2025. URL
https:// /ttp:///axvvvvvi www.ii.or/abba........oo 2503. 207499079999999
Brady Lund, Ting Wang, Nishith Reddy Mannuru, Bing Nie, Somipam Shimray, and Ziang Wang. Chatgpt and a new academic reality: Artificial intelligenco-wiitten rescarch papers and the ethics of the large lan- guage models in scholarly publishing. Journal of the Association for Information Science and Technology, 74(5):550-55)) 2023.
16

---

Published in Transactions on Machine Learning Research (02/2026)
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142 150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. URL http://www. http://wwwwwwww...////.// aclweb. wwwgg/nnhhloogg/111aaa
Chengzhi Mao, Carl Vondrick, Hao Wang, and Junfeng Yang. Raidar: generative ai detection via rewriting, 2024. URL httpp::/arxxiv.or//bba.s 2ttp://44...997000
Jesse G Meyer, Ryan J Urbanowicz, Patrick CN Martin, Karen O'Connor, Ruowang Li, Pei-Chen Peng, Tiffani J Bright, Nicholas Tatonetti, Kyoung Jae Won, Graciela large language models academia: opportunities and challenges. BioDala mining, 16(1):20, 2023.
et al. Chatgpt and
George Mikros, Athanasios Koursaris, Dimitrios Bilianos, and George Markopoulos. Ai-writing detection using an ensemble of transformers and stylometric features. CEUR Workshop Proceedings, 3496, September 2023. ISSN 1613-0073. Publisher Copyright: @ 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).: 2023 Iberian Languages Evaluation Forum, IberLEF 2023; Conference date: 26-09-2023.
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. Detectgpt: Zero-shot generatee text detection using probability curvature, 2023. URL https://arxxi..org 02//
NLP Team MosaicML. Introducing mpt-30b: Raising the bar for open-source foundation models, 2023. URL
WWW www.mosaicml. c0tla///mo..cccmooo//mmm0030 Accessed: 2023-06-22.
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, pp. 839 849, San Diego, California, June 2016. Association for Computational Linguistics. doi: N0.18653/v1/N16-1
URL https://acllatthol.gg 1tw.://hhooooooooo111111111100 org/N16-1188///
Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don't give me the details, just the summary! topic- aware convolutional neural networks for extreme summarization. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun'ich ichi Tsuji (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1797-1807, Brussels, Belgium, October-November 2018. Association for Computational Linguistics. doi: 10. 11853//11/D188 1200 URL
htt.:/antttoooooooor-o-//------- 1trg/D18-1206)
Team Olmo, ;, Allyson Ettinger, Amanda Bertsch, Bailey Kuehl, David Graham, David Heineman, Dirk Groeneveld, Faeze Brahman, Finbarr Timbers, Hamish Ivison, Jacob Morrison, Jake Poznanski, Kyle Lo, Luca Soldaini, Matt Jordan, Mayee Chen, Michael Noukhovitch, Nathan Lambert, Pete Walsh, Pradeep Dasigi, Robert Berry, Saumya Malik, Saurabh Shah, Scott Geng, Shane Arora, Shashank Gupta, Taira Anderson, Teng Xiao, Tyler Murray, Tyler Romero, Victoria Graf, Akari Asai, Akshita Bhagia, Alexander Wettig, Alisa Liu, Aman Rangapur, Chloe Anastasiades, Costa Huang, Dustin Schwenk, Harsh Trivedi, Ian Magmusson, Jaron Lochner, Jiacheng Liu, Lester James V. Miranda, Maarten Sap, Malia Morgan, Michael Schmitz, Michal Guerquin, Michael Wilson, Regan Huff, Ronan Le Bras, Rui Xin, Rulin Shao, Sam Skjonsberg, Shannon Zejiang Shen, Shuyue Stella Li, Tucker Wilde, Valentina Pyatkin, Will Merrill, Yapei Chang, Yuling Gu, Zhiyuan Zeng, Ashish Sabharwal, Luke Zettlemoyer, Pang Wei Koh, Ali Farhadi, Noah A. Smith, and Hannanch Hajishirzi. Olmo 3, 2025. URL https: /tttps://arxivy. wrg. abs/ 13961
Long Ouyang, Jeffrey WU, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730 27744, 2022.
Eleftheria Papageorgiou, Christos Chronis, Iraklis Varlamis, and Yassine Himeur. survey on the use of large language models (Ims) in fake news. Future Internet, 16(8):298, 2024.
17

---

Published in Transactions on Machine Learning Research (02/2026)
Bumjin Park and Jaesik Choi. Identifying the source of generation for large language models, 2024. URL
https://arxiv. bttps:///axxi..rr//bbbs. ab0///////22.....// .28846
Sayak Paul and Soumik Rakshit. arxiv paper abstracts. https://www.ker wtw.://gg.....caaaaaaaaaa/aaaaaa 2021.
Steven T. Piantadosi, Harry Tily, and Edward Gibson. Word lengths are optimized for efficient commu- nication. Proceedings of the National Academy of Sciences, 108(9):3526 3529, 2011. doi: 10. pnas.
1012551108. URL https: //wpw. pnas org/ dwwddbii/baaa/ httoo//ab//ii 1073/0............ http:////aa...1255111111110 1012555108
Jiameng Pu, Zain Sarwar, Sifat Muhammad Abdullah, Abdullah Rehman, Yoonjin Kim, Parantapa Bhattacharya, Mobin Javed, and Bimal Viswanath. Deepfake text detection: Limitations and op- portunities. In 2023 IEEE Symposium on Security and Privacy (SP), pp. 1613-1630, 2023. doi: 10.1109/SP46215.200 10179387.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever. Lan- guage models are unsupervised multitask learners. OpenAI, 2019. URL https://cdns www.ao..nnaccoooo
Leeter-language-modell. 2024-11-15.
_acteee_mmelss._eee are a unsuperrvaed_mmltittaa www.tttakt.tttttmmmmmm learners. pdf Accessed:
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. In Jian Su, Kevin Duh, and Xavier Carreras (eds.), Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 2383-2392 Austin, Texas, November 2016. Association for Computational Linguistics. doi: 10.18653/v1/D16- 1264. URL https: V/aclantholoog.orrg/DDB...
Jie Ren, Han Xu, Yiding Liu, Yingqian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. A robust semantics-basee based watermark for large language model against paraphrasing, 2024. URL https://arxx) org/abs 0tt/abb/23311.077/
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can ai-generated text be reliably detected?, 2025. URL
11156
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafcai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Trishala Neeraj, Jos Rozen, Abheesht Sharma, Andrea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Te Scao, Stella Biderman, Leo Gao, Thomas Wolf, and Alexander M Rush. Multitask prompted training enables zero-shot task generalization. In International Conference Learning Representations, 2022. URL https://opeerreiieit wttprreewwweeerfreeeeeeeeeeeeeee wwtp:/eeoerdddd//eooeeee
Dielmar Schabus, Marcin Skowron, and Martin Trapp. One million posts: A data set of german online discussions. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '17, pp. 1241-1244, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 9781450350228. doi: 10. 10.1145/30771333300011 URL htttps://dat wwtooi..ogg//..0 1145/
3077136. 3080711
Abigail See, Peter J. Liu, and Christopher D. Manning. Get to the point: Summarization with pointer- generator networks. In Regina Barzilay and Min-Yer Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Compulational Linguistics (Volume 1: Long Pupers), pp. 1073 1083, Vancouver, Canada, July 2017. Association for Computational Linguistics. doi: 10. 10.18653/v1/P17-1U0 URL https: crcclanhholog..org/PPP...
Sanidhya Madhav Shukla, Chandni Magoo, and Puncet Garg. Comparing fine tunod-Ims for detecting 11m- generated text. In 2024 3rd Edition of IEEE Delhi Section Flagship Conference (DELCON), pp. 1-8. IEFE, 2024.
18

---

Published in Transactions on Machine Learning Research (02/2026)
Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert- Voss, Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang. Release strategies and the social impacts of language models, 2019, URL
Setps://arxivy http:/rxxvvv...raab// aa/aabbs/110 09200
Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov, Detectllm: Leveraging log rank information for zero-shot detection of machinc- -eahbiitegeerteddd text, 2023. URL httl://pe//aaxi..oo/////////6.
SuperAnnottte. Super Al-Detector. httpe:/s//anggggggcc..oooopppppr
Chenhao Tan, Vlad Niculae, Cristian Maaessuu-Niuulecu-MMiii and Lillian Lee. Winning arguments: In- teraction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th International Conference on World Wide Web, WWW '16. International World Wide Web Conferences Steering Committee, April 2016. doi: 11145/2277227.288800
URL http://áxx doi. oeei.orgg/1... 1145/
2872427 2883081
Amalio Telenti, Michael Auli, Brian L Hie, Cyrus Maher, Suchi Saria, and John PA Ioannidis. Large language models for science and medicine. European journal of clinical investigation, 54(6)=e14183, 2024.
Irina Tolstykh, Aleksandra Tsybina, Sergey Yakubson, Aleksandr Gordeev, Vladimir Dokholyan, and Maksim Kuprashevich. Gigacheck: Detecting lmm-generated content. arXiv preprint arXiv:24 10.23728, 2024.
Petter Törnberg, Diliara Valeeva, Justus Uitermark, and Christopher Bail. Simulating social media using large language models to evaluate alternative news feed algorithms. arXiv preprint 1xXiv:230000598, 2023.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anma Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. Llama: Open and efficient foundation language models, 2023a. URL
https://arxis org/abs//3022.aa..
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Fer- rer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosscini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anrs Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nic, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subra- manian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fino-tuned chat models, 2023b.
Brian Tufts, Xuandong Zhao, and Lei Li. A practical examination of ai-gencrated text detectors for large language models, 2025. URL https://aaxiv: http://aaavvorrg/bbaaaaa///a httggab//4412.0153
Don Tuggener, Pius von Däniken, Thomas Poetz, and Mark Cicliebak. LEDGAR:
A largo-scale multi-label corpus for text classification of legal provisions in contracts. In Nicoletta Calzolari, Frédéric Béchet, Philippe Blache, Khalid Choukri, Christopher Cieri, Thierry Declerck, Sara Goggi, Hitoshi Isahara, Bente Macgaard, Joscph Mariani, Hélène Mazo, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Proceedings of the Twelfth Language Resources and Evaluation Conference, pp. 1235 1241, Marseille, France, May 2020. European Language Resources Association. ISBN 979-10-995446-3444 URL https: //aclanthology. hwwp:/hhoooggyooo//2222222200 wttp:/2200.rrec1... 155/
Michael Völske, Martin Potthast, Shahbaz Syed, and Benno Stein. TL;DR: Mining Reddit to learn auto- matic summarization. In Proceedings of the Workshop on New Frontiers in Summarization, pp. 53-63, Copenhagen, Denmark, September 2017. Association for Computational Linguistics. doi: 10.18653/v1/ W17-4508. URL http:::/becaatthooooo.oo/g/
19

---

Published in Transactions on Machine Learning Research (02/2026)
Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https::/gitthb.ccom hwtm://winggllllo////mmmttteaaea transssfhe-taaaaaooeeemr.. jax, May 2021.
Hong Wang, Xuan Luo, Weizhi Wang, and Xifeng Yan. Bot or human? detecting chatgpt imposters with a single question, 2024a. URL httpss//arxii...rt org. wwtp://0wr../bb/0050662222
Rongsheng Wang, Haoming Chen, Ruizhe Zhou, Han Ma, Yaofei Duan, Yanlan Kang, Songhua Yang, Baoyu Fan, and Tao Tan. Lim-detector: Improving ai-generated chinese text detection with open-source Ilm instruction tuning. arXiv preprint arXiv:2402.011 01188, 2024b.
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions, 2023. URL https: /arxiv werxxiv.or//aae httgg/bb/2212..1000
Max Wolff and Stuart Wolff. Attacking neural text detectors, 2022. URL 11768
Junchao Wu, Shu Yang, Runzhe Zhan, Yulin Yuan, Lidia S. Chao, and Derek Fai Wong. A survey on Ilm- generated text detection: Necossity, methods, and future directions. Comput. Linguistics, 51(1):775-338 2025. doi: 10.1162/COLI, A\ 00549. URL https://doi. org/ 1ttp://..11//ooa..1e
Jinwei Xu, He Zhang, Yanjin Yang, Zeru Cheng, Jun Lyu, Bohan Liu, Xin Zhou, Lanxin Yang, Alberto Bacchelli, Yin Kia Chiam, and Thiam Kian Chiew. Investigating efficacy of perplexity in detecting Ilm- generated code, 2024. URL hhttps://axxii http///rxxvv..r/bbbssass/ org/abs/2413 16525
Ann Yuan, Andy Coenen, Emily Reif, and Daphne Ippolito. Wordcraft: story writing with large language models. In Proceedings of the 27th International Conference on Intelligent User Interfaces, pp. 841 852, 2022.
Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? In Anna Korhonen, David Traum, and Lluis Mårquez (eds.), 5 Proceedings of the 57th Annual Meeting of the Association for Computational Långuisties, pp. 4791-480M Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18653/v1/P19-115 //aclanthology.oo htaptehhooooyy.oo//111-1111 or//P19-142//
URL https:
Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, Weng Lam Tam, Zixuan Ma, Yufei Xue, Jidong Zhai, Wenguang Chen, Peng Zhang, Yuxiao Dong, and Jie Tang. Glm-130h, An open bilingual pre-trained model, 2023. URL
hts/abs//2111 02414
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chon, Christopher Dowan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott,, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. Opt: Open pre-trained transformer language models, 2022. URL
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text classification. C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015. URL https: htttpp::/prroceeddngas
www.tugg....uuuiee neur meurips... cc/ pdf
paper
Yuehan Zhang, Yonggiang Ma, Jiawei Liu, Xiaozhong Liu, Xiaofeng Wang, and Wei Lu. Detection vs. anti- detection: Is text generated by ai detectable? In International Conference on Information, pp. 209 222.
Springer, 2024.
Zixuan Zhou, Xuefci Ning, Ke Hong. Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, and Yu Wang. A survey on efficient inference for large language models, 2024. URL httth::///aat //tpxi//axvvv./// /tta/aaivvr//aaaaaaeeee org/abs//4000 14294
20

---

Published in Transactions on Machine Learning Research (02/2026)
A More Details on Related Work
In recent years, the challenge of identifying Al-generated text has garnered significant attention, giving rise to variely of detection approaches. These methods largely fall into two categories: watermark-based techniques and zero-resource detection.
Watermarking. Watermarking embeds traceable patterns in a model's outputs during training or gener- ation, enabling downstream identification of machine-generated content (Ren et al. 2024 Liu et al. 2024a While watermarking can be effective in controlled environments, it relics on access to or cooperation from the model's developers, an assumption that frequently fails in real-world or adversarial scenarios. Furthermore, it is inherently unsuitable for practical situations where Al-generated text lacks any embedded watermark. This limitation has led growing interest in zero-resource detection methods, which make no assumptions about access to the model's internals or training data. Instead, these methods analyze the output text alone, offering a more flexible and broadly applicable approach. Within this space, techniques can be further cate- gorized into fine-tuned methods, which rely on labeled datasets, and zero-shot methods, which generalize to unseen models without task-specific training.
Fine-tuned Detection. Fine-tuned detection methods represent major strand of zero-resource detec- tion, often leveraging fine-tuned classifiers built atop pre-trained language models (PLMs). A pivotal de- velopment was the Grover model, which demonstrated that models trained on text from specific generators can achieve high accuracy on in-distribution data, particularly when integrating Grover-specific layers. This inspired a wave of PLM-based detectors, most. notably OpenAT's GPT-2 detector (Solaiman et al., 2019). which uses RoBERTa classifier trained on GPT-2 outputs. However, such detectors often struggle to generalize across models, especially as newer LLMs introduce more fluent and coherent outputs.
To improve generalization and robustness, recent work has focused on feature augmentation. Stylometric approaches, for instance, introduce handcrafted features that capture writing style discrepancies between humans and machines (Mikros et al., 2023). These include measures of phraseology, punctuation, linguistic diversity, and journalistic standards, which have proven useful for detecting Al-generated tweets and news articles. Additional features such as perplexity statístics, sentiment, and error-based cues like grammatical mistakes further enrich detection pipelines (Kumarage et al., 2023
Parallel efforts have explored structural features, incorporating models that explicitly account for the factual or contextual structure of text. Techniques such as TriFuseNet combine stylistic and contextual branches with fine-tuned BERT models, while others employ Siveenivve-iLSSTV to replace standard feedforward layers, enhancing interpretability and robustness (Liu al. 2024b).
Despite these advancements, fine-tuned detectors still require labeled training data and model-specific tuning of PLMs, which can limit their scalability to novel proprictary LLMs. Although these detectors perform exceptionally well on data similar to their training sets, they face significant drawbacks, most notably, a tendency to overfit to specific domains and reliance on retraining for every newly emerging Al model, which is unsustainable in light of the fast-paced evolution of generative technologies. This motivates the development of methods, that leverage zero-shot features, such as DivEye, that aim to detect Al-generated text without relying on supervised learning or access to model internals.
Zero-shot Detection. Recent research has focused on zero-shot detection strategies that require no fine- tuning on labeled examples from the target generator. These methods typically leverage statistical cues from PLM's output distributions or repurpose LLMs themselves as delectors.
A prominent class of zero-shot detectors exploits the probability structure of text under language models. De- tectGPT (Mitchell et al. 2023) detects machine-generated text by mcasuring how strongly the lig-likelihood drops under small semantic perturbations, leveraging the hypothesis that AI text lies in regions of higher negative curvature than human text. On the other hand, FestDDttt (Bao et al. 2024) eliminates the nced for explicit perturbations by directly measuring curvature in conditional probabilities, observing that AI text typically exhibits sharper transitions between tokens compared human writing. These observations are refined in D (uu et al. 2023), which introduces the Log- Likelihhd Log-Rank Ratio (LRR) and
21

---

Published in Transactions on Machine Learning Research (02/2020)
ELI5
CMV
HellaSwag
ROC
LogRank
LegRank
LogRank:
LogRank
Entropy
Entropy
Entropy
Entropy
RestDedact
FactDatect
DestDDtect
RastDelecio
RADAR
RADAR
RADAR
RADAR
DetectLi
DesectLL
DetectLL
DetectLL
DivEyo
DivEye
OpenWl Detector
OpenAi Detector
OpenAl Detector
OpenAi Detector
SciXGen
SQUAD
TLDR
LogRank
LogRark
LogRank
Legtank
Entropy
Enkropy
DectDe
FastDesect
D
FastDeteciger
NADAR
RADAR
RADAR
RADAR
DetertLLNe
DetectLLK
DetectLL
DetectLL
yo
DivEyo
DivEye
DlvEyo
OpenAl Detector
OpenAi Detector
OpenAl Detector
Openál Detector
XSum
Yelp
LogFank
Loghurk
Entropy
Entropy
1.0
DectDe
FastDatectory
RADAR
RADAR
DivEye
DivEyc
OpenAi Detector
OpenAl Detector
Figure 7: AUROC performance profiles seven Al-detection tools evaluated on text generated by ten diverse domains generated by arbitrary LLMs. Each spider plot corresponds a specific domain, with radial axes representing the AUROC score (ranging from 0 to 1) and angular axes representing the detection tools: RADAR, Entropy, LogRank, FastDette DetectLLM, OpenAI Detector, and DivEye.
Normalized Perturbed log-Rank (NPR) metrics to quantify the distinguishability of Al-generated content using statistical features derived from token rankings.
Another line of work focuses on token predictability and entropy. LogRank (Ghosal et al. 2023 investigates the use of token rank distributions and demonstrates that log-rank statistics, such as the frequency of top-ranked tokens, are reliable signals of Al authorship. This builds on early work such as entropy- detection [Lavergne et al. 2008) and GLTR Gehrmann et al.) 2019), which showed that humans tend to use more surprising and diverse tokens, while LLMs often fall back on high-probabilltyy continuations.
Moving beyond single-directional statistics, BiScope (Guo et al. 2024) proposes bi-directional cross-entropy framework that measures how well a model's predicted logits align both with the ground truth next token (forward loss) and with the previous token (backward loss). The key insight is that Al-generated text often exhibits predictable forward progression but weaker backward association due to its autoregressive nature. A shallow classifier trained on the joint distribution of these losses can reliably detect AI text with zero-shot generalization.
Finally, Binoculars HHans et al. 2024) offers a model-agnostic strategy by comparing the statistical disagroc- ment between two LLMs on the same input. By contrasting the outputs of two diverse LLMs, the method detects anomalies in token distributions that are characteristic of synthetic text. This ensemble- based dis- agreement is found to correlate strongly with model-generated samples, providing a powerful signal without the need for training data from either model.
Collectively, these techniques demonstrate that zero-shot detection can be achieved by carefully analyzing how text aligns with the inductive biases and statistical signatures of language models, without any finctuming or access the original generator. They lay the foundation for our proposed method, DivEye, which further capitalizes on biversity-based statistical properties to robustly differentiatee AI- and human-writee content.
22

---

Published in Transactions on Machine Learning Research (02/2026)
BLOOM-7B
OPT-125M
FLAN-TS-Small
LegRank
LogRank
LogRank
Entropy
Entropy
Entropy
DastDetec
GPT
GPT
MDDRR
DetectLoM
DetectLAM
DetectLoM
DivEye
DivEye
DivEye
OpenAl
OpenAl
OpenAl Detector
GLB-130B
GPT-3.5-Turba
GPT-J
LogRank
LogRarik
Log Rank
Entropy
Entropy
Entropy
sestDetest
FastD ete GPP
FastDetecIGPT
RADAR
ADAR
RADAR
Detecti
DetectLOM
Detect
DivEye
DivEye
DivEye
OpenAl Detector
Open Detactor
OpenAl Detector
Figure 8: AUROC performance profiles of seven AT-detection lools evaluated text generated by six different LLMs. Each spider plot corresponds to specific language model, with radial axes representing the AUROC score (ranging from 0 to 1) and angular axes representing the detection tools: RADAR, Entropy, LogRank, Fast Detee DL,, OpenAl Detector, and DivEye.
100
Static Only
Temporal
88.00
82.00
80
78.15
74.25
Score Accuracy
60
Average Accuracy
AUROC (%)
Figure 9: Ablation results on Testbed 4 of the MAGE benchmark showing the impact of temporal surprisal features. Adding temporal dynamics to static surprisal statistics improves both accuracy (from 74 74.25% to 78.15%) and AUROC (from 0.82 to 0.88), demonstrating their complementary value for robust Al-generated text detection.
B Motivation Behind Temporal Features
While statie surprisal statistics such mean, variance, skewness, and kurtosis provide useful summaries of token-level unpredictaiiitty, they overlook the evolution of this unpredictability critical to distinguishing human and AT-generated text. Human authors naturally embed stylistic variability through temporal fluctuations, such as abrupt topic shifts, tonal changes, and bursts of creativity, which manifest as distinctive temporal dynamics in surprisal sequences.
time, dimension
Intuitively, these temporal features, as listed in Section expose rhythmic and non-stationary patterns characteristic of human creativity and coherence, typically absent in the more uniform output of large language models.
23

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: TPROFPR-5% on Testbed 8 (Unseen Domains Models).
LogRank
Entropy
DetectLLM
Fest
Binoculars
BiScope DivEye
TPR
47.1
30.1
51.3
76.3
68.7
99.8
99.6
Furthermore, through an ablation study on Testbed 4 of the MAGE benchmark (Figure we empirically show that augmenting static surprisal features with temporal metrics leads to a measurable improvement in classification accuracy. This highlights the complementary value of temporal dynamics in enhancing the robustness of Al-generated text detection. Moreover, analysis of feature importance (Appendix [.6] reveals that temporal features collectively contribute more than static features, consistently ranking among the most informative signals for distinguishing between human and Al-generated lext.
Overall, these findings motivate the inclusion of temporal surprisal features as integral components of our DivEye framework.
C Implementation of DivEye
We provide a detailed description of our DivEye implementation in Algorithm | This includes all steps from surprisal computation to feature extraction and final classification. We use an XGBoost classifier for binary classification as preliminary choice, without extensive comparison to other classifiers, leaving exploration of alternative models for future work. For completeness and reproducibility, we include all additional imple- mentation details, such as hyperparameter configurations, model architectures, and experimental testbeds, in Appendix and Appendix H
Algorithm DivEye: Algorithm for Feature Extraction & Training
Require: Text dataset D {(1,, &)) where r; is text input and & (0,1) indicates whether it is human-written (e; = 1) machine-generated (&, 0)
Require: Pretrained ruto-regresvvve language model 96 (e.g., GPT-2)
Require: XGBoost classifier with hyperparameters θ
Ensure: Trained binary classifier fo Initialize an emply Teature matrix [] for each (xy, 4,) do
Compute token-level log-likel:hoods: yi (x:)))
Convert to token-level surprisals: 8; f- y/1
5: Compute diversity features DivEye(z;) E R as described in Equation i using
6: Append (DivEye(z;),4, to F.
7: end for
Train binary classifier Se on feature set F using XGBoost with hyperparameters
return fo
D Additional Results
In this section, we present additional supporting experiments that demonstrate the generalizabilit, robust- ness, and complementary strengths of DivEye through various ablation studies.
D.1 Domain-Specific Performance of DivEye
Figure 7 presents the AUROC performance of seven detection methods evaluated across ten text domains (Testbed of the MAGE benchmark). DivEye consistently achieves the highost AUROC scores in every domain reaching up to 0.99 in WP, 0.97 in CMV, and 0.95 in SciXGen, outperforming other detectors by a notable margin. This highlights DivEye's adaptability and robustness in capturing domain-specific writing
24

---

Published in Transactions on Machine Learning Research (02/2026)
patterns that other methods frequently miss. These results reinforce the advantage of leveraging surprisal features for more generalizable and context-sensitive detection of Al-generated text.
D.2 Model-Specific Performance of DivEye
Figure g compares the AUROC performance of seven detection methods across text on generated by six different large language models (Testbed of the MAGE benchmark). DivEye achieves the highest AU- ROC scores across all models, demonstrating strong robustness (0.95 on GLB-130B, 0.89 on GPT-J, 0.85 on GPT-3.5-Turbo, This consistent performance highlights DivEye's effectiveness in capturing tempo- ral surprisal patterns that generalize well across different language model architectures, making it broadly applicable for reliable Al-generated text detection.
D.3 Performance against other base models
We evaluate the robustness of all detectors across different backbome models and report results in Table 6 These backbone models include GPT-2 RRaffrdd et al., 2019), GPT-2-XL (Radford et al. 2019), Falcon-7B (Almazrouei et al. 2023), Llama-3.22 2-BBB (et al., 20244),, Llama-3. 1-8B fet al. (0244a) and Mistral-7B-vo.3 (Jiang al. 2023). Competing methods such as Binoculars, BiScope, and DetectLLM show moderate variation with backbone choice, while GaT and LogRank generally underperform. These results highlight that DivEye maintains strong and stable detection capability regardless of the underlying base model.
Table Performance of different detectors across backbone models.
Backbone Model
DivEye
Binoculars
BiScope
LogRank
DetectLLM
FastDetectGPT
GPT-2
0.88
0.71
0.86
0.68
0.75
0.69
GPT-2-XL
0.89
0.73
0.86
0.68
0.76
0.70
Falcon-7B
0.90
0.73
0.89
0.70
0.80
0.72
Llama-3 3.2na-3.221B
0.87
0.71
0.87
0.70
0.76
0.71
Llama-3.1-80
0.91
0.77
0.90
0.72
0.81
0.73
Mistral-7B-v..3
0.90
0.76
0.90
0.71
0.80
0.72
D.4 Relative Importance of DivEye in Boosted Model
Figure T0 illustrates the ielative feature importance of DivEye when integrated into boosted ensembles with five existing Al detectors: BiScope (Guo et al.) 2024), OpenAl Detector (Solaiman et al., 2019), RADAR (Hu) ct al. 2023 DetectLLM (Su et al. 2023). and Binoculars (Hans et al. 2024). DivEye contributes significantly to the overall model, with particularly high importance when combined with RADAR (91.92%), OpenAl Detector (90.26%), and Binoculars (89.71%). Even in ensembles with more vanced detectors like BiScope, DivEye still adds valuable signal (32.93%). These results affirm the standalone strength of DivEye and its utility in hybrid detection frameworks.
D.5 Results with Different Proprietary LLMs
Table reports AUROC scores of DivEye on text generated by five proprictary LLMs, Claude-3 Opus, Claude-3 Sonnet, Gemini 1.0-pro, GPT-3.5 Turbo, and GPT-4 Turbo, using data provided in the BiScope paper Guo et al. 2024) across five domains. DivEye achieves consistently strong performance on the Normal dataset (c.g., 1.000 on GPT-3.5 Turbo for Essay) and remains robust under paraphrased inputs, with AUROC scores generally above 0.95. These results highlight DivEye's ability generalize across diverse generation models and domains, under text transformations.
25

---

Published in Transactions on Machine Learning Research (02/2026)
100
V1-
60
0
Niscese
Openni Detectir
RADA
Binocu
Figure 10: Feature importance of DivEye when integrated with various existing detectors. The plot shows how much DivEye contributes to the overall detection model when combined with BiScope, OpenAl Detector, RADAR, DetectLLM, and Binoculars. Higher values indicate stronger complementary impact from DivEye's diversity-based features.
Table 7: AUROC scores achieved by DivEye on five commercial LLMs across various domains. Results are shown for both the Normal and Paraphrased datasets.
Normal Dataset 1.0ppr
Paraphrased Dataset 0-geo
Domada
Clande-3 Opns
GPT-335
Opus
Clonde33
Code
025288
0,8577
0,7824
0.0077
0.0041
0.8455
0.9033
0.7521
0.929
0.5017
Croaiive
0.888
I.9773
0,9815
0,8651
0,3008
0.9500
0,9900
0,5957
0.9917
0.554)
Essay
0.9300
0,9885
0,9972
1,000
0,9825
0,9975
0.9877
0.5614
0.885
0.5629
Yelp
0.8855
0,8133
0,0220
0.5384
0,8942
0.0545
0.9730
0,9083
0.8221
0.5671
D.6 Feature Importance of DivEye
presents the relative importance of each the nine diversity-based features incorporated in DivEye, which derived from surprisal statistics as detailed in Equation equation G The feature importances, ranging from approximately 8.1% to 13.0%, indicate that all features contribute meaningfully to model decisions, with temporal features such as, Аμ, entropy of second derivatives Had, and autocorrelation PV² exhibiting the highest influence. This balanced contribution underscores the complementary nature of these statistical descriptors in enhancing DivEye's detection capability when combined with existing baseline detectors.
26
12.00%
11.255
12,00%
10.
0/2
Figure 11: Relative feature importances for the nine diversity-basee features used DivEye. The features, listed in Equation equation 6 represent distinct surprisal-based statistics. Higher percentages indicate greater influence in model decisions when combined with existing detectors.
26

---

Published in Transactions on Machine Learning Research (02/2026)
D.6.1 Statistical Relevance of DivEye
To evaluate the individual contribution of each component in our diversity vector D (Equation 6 we perform a leave-one-out ablation study. Each feature is removed individually from the 9-dimensional vector, the classifier is retrained, and the resulting performance measured by the drop in AUC on Testbed of the MAGE benchmark. To also assess statistical significance, the lest set to compute p-values under the null hypothesis that both models perform equally well. Table g summarizes both the AUC drop and the p-value for each feature.
conduct paired bootstrap test, resampling
Table 8: Leave-onc-out feature ablation and statistical significance for DivEye. Each feature's removal leads to measurable drop in AUC and is statistically siguificant (p < 0.05).
Feature
Category
AUC Drop
p-value
Ha²
2nd-Order
-0.0263
0.001
Y1
Distribution
-0.0239
0.004
PAz
2nd-Order
-0.0152
0.012
o?
Distribution
-0.0115
0.016
Ag²
1st-Order
-0.0103
0.017
722
Distribution
-0.0091
0.019
Ap
Ist-Order
-0.0056
0.027
He
Distribution
-0.0013
0.032
d
2nd-Order
-0.0008
0.034
Several key insights emerge from this analysis:
Second-order features are the most impactful. Removing second-order entropy Ha² results in the largest decline in AUC, followed by the socond-order autocorrelation Pa2, highlighting the importance of modeling higher-order dependencies in token-level surprisal dynamics.
Distributional features are significant. Skewness (71) and variance (a²) contribute meaning- fully, indicating that asymmetry and dispersion in surprisal values enhance DivEye's discriminative performance.
First-order features contribute consistently. The mean and variance of first-order differences (Ap, A produce mcasurable gains, reflecting local variation in surprisal between adjacent tokens.
Even features with small absolute AUC drops, such as and oλ2, are statistically significant. (p < 0.05). This demonstrates that each feature contributes non-redundant information, supporting our core hypothesis that diversity token-level surprisal; capturing both distributional asymmetrics and temporal patterns is essential for detecting machine-generated text.
D.7 Performance against same model
To investigate whether DivEye's detection ability relies on a distributional mismatch between the generator and the surprisal model, we conducted a controlled experiment using the same model for both purposes. Specifically, we computed diversity-based features and trained the DivEye classifier using three different LLMs: Falcon-7B, Llama-3.1 and GPT-2-XL.
We further evaluated generalization by performing out-of-distribation test using generations from 500 prompts drawn from the OASST (Köpf et al., 2023 and Self-Instruct (Wang et al. 2023 datasets. Table g reports the resulting AT detection accuracies.
Despite using the same model for both generation and surprisal estimation, DivEye maintains high clas- sification accuracy across all settings. This demonstrates that DivEye's effectiveness stems from intrinsic statistical patterns in the generated text rather than artifacts arising from model mismatch.
27

---

Published in Transactions on Machine Learning Research (02/2026)
Table 9: DivEye performance when using the same model for both generation and surprisal computation.
Model
AI Accuracy (%)
Falcon-7B
98.2
Llama-3 1-8B
96.1
GPT-2-XL
98.8
D.8
Performance against Longformer
Table 10: AUROC comparison of DivEye, DivEye (w/ BiScope), and Longformer across MAGE test settings.
Setting
Longformer
DivEye
DivEye (w/ BiScope)
Fixed-domain, Model-specific
0.990
0.994
1.000
Arbitrary-domains, Model-specific
0.990
0.972
0.991
Fixed-domain, Arbitrary-modee
0.990
0.993
0.998
Arbitrary-domaine Arbitrary-models
0.990
0.880
0.934
OOD: Unseen Models
0.950
0.859
0.952
OOD:
Unseen Domains
0.930
0.975
0.989
OOD:
Unseen Domains & Models
0.940
0.924
0.986
Paraphrasing Attacks
0.750
0.870
0.923
Longformer, being a fine-tuned detector, was not included in Table | since its setup differs fundamentally. Nonethcless, for completeness, we provide a detailed AUROC-basod comparison of DivEye against Long- former across the 8 challenging MAGE testbeds.
As shown in Table TO DivEye consistently matches or outperforms Longformer in most settings. Even under OOD scenarios and paraphrasing attacks, DivEye demonstrates strong generalization, often exceed- ing Longformer's 's performance. DivEye (w/ BiScope) further improves AUROC across nearly all testbeds, highlighting the benefits incorporating diverse zero-shot features.
D.9
Performance across edge-cases
D.9.1 Performance across non-native-written texts
To evaluate potential bias against non-native speakers, known vulnerability in many perplexity-based detectors, conducted targeted evaluation using the COREFL dataset (Lozano et al. 2020 This dataset consists of 1426 essays writlen by nalive German and Spanish speakers, explicitly calegorized by their English proficiency levels (ranging from A1 to C2).
We trained (or configured) DivEye, Binoculars and FastDeteetGPT using a balanced set comprising 75% of the COREFL human texts and an equivalent number of Al-generated samples drawn from MAGE Testbed 4. We evaluate performance on the held-out 25% to measure detection accuracy across proficiency brackets.
Table 11: Detection Accuracy on Non-Native English Writing.
Framework
Average Accuracy
Binoculars FestDettetGPTT
68.34%
71.71%
DivEye (Ours)
82.07%
As shown in Table 11 DivEye significantly outperforms existing basclines in identifying non-native human text, achieving robust 82.1% average accuracy compared to 68.34% for Binoculars and 71.7% for FastDe- tectGPT.
28

---

Published in Transactions on Machine Learning Research (02/2026)
6-4
6.
Dygishh Proticercy
Figure 12: Granular depiction of various frameworks performance various non-native texts.
granular analysis reveals that performance of all frameworks correlates with linguistic complexity: A1 texts (the most beginner class) are the most difficult to detect, while accuracy steadily improves as proficiency rises. As for DivEye, this suggests that detection becomes easier as the text exhibits the richer rhythmic signatures of advanced writing.
Crucially, illustrated in Figure 12
D.9.2 Performance across legal & professional documents
A common criticism in Al text detectors is their potential fragility when applied to highly formal human writing, such as legal contracts or technical specifications. In these domains, human writers often strive for low entropy and rigid structure, traits lypically associated with machine generation. To address this concern, we evaluated DivEye on the LEDGAR dataset (Tuggener et al. 2020 a corpus of legal provisions extracted from US SEC filings.
Table 12: Detection Accuracy on LEDGAR datasct.
Framework
Accuracy
Binoculars 91.67%
GPT
94.8%
DivEye (Ours)
99.49%
We adopted a robust evaluation protocol using a 75-25 train-test split to ensure generalization. As shown Table 12 despite the constrained and repetitive nature contract language, DivEye successfully outperforms other frameworks on this task. These findings provide empirical evidence that DivEye's core hypothesis holds even in challenging edge cases.
D.9.3 Visualizing texts under observation
Analyzing the probability distributions across domains reveals a distinct hierarchy detectability, We have already seen a distribution in Figure 3 for DivEye's effectiveness on MAGE Testbed 4.
Contrary to the concern that formal writing might resemble algorithmic generation, find that in Figure professional legal text (LEDGAR) is the most easily distinguishable from AI, with its probability distribution shifted furthest towards the human extreme.
This suggests that while professional writing is structured, it contains sharp, domain-specific peaks in in- formation density that are representative of human writing. Furthermore, Standard (MAGE Testbed 8) and Non-Native (COREFL) texts exhibit remarkably similar distributions, effectively overlapping in the high-probability region. We hope to investigate this further in future works.
29

---

Published in Transactions on Machine Learning Research (02/2026)
Distribution DivEye
(Human-Writte) Wrritenn Textts)
4.0
Damain Professional (LEDGAR)
hon-itatve (CORL)1)
Standare (MAGE)
2.5
Dendky
1.0
0.0
9.0
Predicted Probability Human Authorship
Figure 13: Predicted distributions of predicted class probabilities for diverse human written texts, We utilize the LEDGAR, COREFL and the MAGE Testbed 8 texts to plot this distribution.
0.8
Marber of Takknt
Figure 14: DivEye's performance on different text lengths (characterized by tokens) on MAGE Testbed 4.
D.9.4 Text sensitivity analysis of DivEye
To determine the minimum linguistic context required for reliable detection, we evaluated the performance of DivEye across varying text lengths. We truncated test samples to specific token counts ranging from N = 10 to N = 256 and measured the AUROC at each interval. As illustrated in Figure 14 performance follows a predictable scaling law relative to information availability.
Detection is constrained on extremely short fragments, yielding an AUROC of 0.62 at 10 tokens; this is expected, however, the system demonstrates rapid convergence: performance improves steadily as the context window expands, reaching a robust AUROC of 0.88 by 256 tokens. This confirms that while DivEye benefits from longer contexts, effectively captures the structural signature of human writing within standard paragraph lengths.
We discuss short length texts as a limitation in Appendix J
E
Additional Adversarial Attacks on DivEye
We evaluate DivEye under range of adversarial settings, including character-level perturbations, word- and phrase-level edits, paraphrasing, prompt obfuscations, and distribution shifts (temperature changes and degenerate sampling), to comprehensively assess its robustness.
30

---

Published in Transactions on Machine Learning Research (02/2026)
Table 13: Performance of DivEye and open-source baselines on all listed adversarial attacks on the RAID benchmark.
Settings
Methods
FPROTPR=5%
Desklib AT
93.0%
e5-small-Hora
93.5%
Settings
Methods
FPR&TPR=5%
DivEye (Ous)
92.1%
Number Attack
[RID]]
Adversarial Attacks
Binoculars
76.4%
Desklib
94.9%
RADAR
65.7%
c5-small-lora
93.9%
GLTR
57.3%
Whitespace Attack
DivEye (Ours)
79.8%
Desklib AI
94.9%
Binoculars
68.7%
e5-small-Jora
93.9%
RADAR
61.1%
DivEye (Ours)
92.2%
GLTR
43.1%
Insert Paragraph
Binoculars
70.7%
Desklib
57.2%
RADAR
68.2%
c5-small-lora
93.9%
GLTR
58.3%
Upper-Lower Attack
DivRye (Ours)
85.3%
Desklib AI
99.7%
Binoculars
72.8%
c5-small-lora
11.1%
RADAR
65.1%
DivEye (Ours)
61.6%
GLTR
45.3%
Homoglyph Attack
Binoculars
36.1%
Desklib AT
80.6%
RADAR
44.8%
eG-small-lora
GLTR
20.3%
DivEye (Our))
67.1%
Synonym Attack
Desklib AI
90.5%
Binoculars
42.1%
65-small-Jora
92.0%
RADAR
62.7%
DivEye (Ours)
88.0%
GLTR
28.7%
Article Deletion
Binoculars
73.3%
Desklib
83.7%
RADAR
63.0%
e5-small-lora
85.5%
GLTR
48.9%
DivEye (Ours)
74.4%
Paraphrasc Attack
Binoculars
N/A
Desklib A1
94.3%
RADAR
62.4%
e5-small-lora
93.4%
GLTR
43.0%
Alt. Spelling Allack
DivEye (Ours)
92.01%
Binoculars
77.6%
Desklib AT
92.9%
RADAR
65.5%
eG-small-lora
92.5%
GLTR
58.2%
DivEye (Ours)
90.6%
Perplexity Misspelling
Dinoculars
77.2%
Desklib AI
87.5%
RADAR
64.3%
e5-small-lora
93.9%
GLTR
57.0%
DivEye (Ours)
92.0%
Zero Width Space
Binoculars
98.4%
RADAR
78.4%
GLTR
97.9%
E.1 Adversarial Attack Analysis of DivEye
We evaluate DivEye against a wide range adversarial attacks using the RAID benchmark, reporting average
classification accuracies across all attack categories listed in Table 13 DivEye achieves performance par with the top-performing fine-tuned models reported by the benchmark. Notably, it consistently surpasses all zero-shot detectors by significant margin across every attack type, demonstrating strong robustness against both diverse adversarial attacks.
E.2 Detection against other diverse online paraphrasers
To evaluate the robustness of DivEye against paraphrasing allacks intended to 'humanize' Al-generated text, we curate set of 21 arXiv abstracts generated by Baadee3.5-Sommae and paraphrase each using three widely used commercial tools: ZeroGPT2 GPTin and " This results in 63 paraphrased texts (21 per tool), each aiming to evade AI detectors through stylistic and lexical variation. We provide this smaller dataset the supplementary materials and our anonymous repository.
T https://www httpps//sww.zrroggt. www.gpt.ooo.ccooo com/
bttps::/app.gptiee gptinf...pppp....mm.mmm..mi com/
https://quillbat wtt. wwwpparapprasing-tooo
31

---

Published in Transactions on Machine Learning Research (02/2026)
Detection performance of DivEye against paraphrased outputs generated by three commercial tools. Each model was tested on 21 samples per paraphrascr.
Table 14:
Paraphraser
MAGE Testbed Model
Hlaude-3.5-Sonmee Model
Blaude-3.5-Sonann (original)
20
21
21 /
21
GPTinf
18
21
19
21
ZeroGPT
20
21
17 21
QuillBot
20
17 21
We assess detection performance using two XGBoost classifiers trained exclusively on DivEye features: one trained on MAGE's Testbed 4 (Arbitrary Models & Arbitrary Domains), and another trained on 280 Claude- 3.5-Sonnet gencrated arXiv abstracts (from BiScope (Guo et al. 2024)). The results, presented Table 14 highlight DivEye's ability to maintain delection accuracy even in the presence of strong paraphrasing transformations.
E.3 Dependence of DivEye on Generation Temperature
We investigate the influence of gencration temperature on DivEye's detection performance to evaluate its robustness against variations in lext predictability. Specifically, we conduct two experiments on the MAGE benchmark: intra-model temperature variation and cross-mos model aariabl--tepperen detection.
E.3.1 Intra-Model Temperature Variation
Using GPT-2 as the zero-shot feature generator, we evaluate DivEye across wide range of sampling tem- peratures (default T 1.0). AUROC results for selected MAGE testbeds presented in Table 15
Table 15: DivEye AUROC across different temperatures for GPT-2 generated texts.
Testheds Temperatures
- 0.1
T. = 0.3
T 0.5
T' 0.7
T 1.0
T' = 1.2
T 1.4
T' 1.6
Arbitrary Domains & Arbitrary Models
0.8784
0.8760
0.8776
0.8886
0.8825
0.8767 0.8583
0.8842
0.8698
Unsoen Models (GPT-3.5/Turbe) OOD)
0.8473
0.8432
0.8595
0.8619
0.8617
0.8488
0.8567
Results indicate that DivEye's AUROC remains consistently high across all temperatures. Even at extreme sampling regimes, performance does not degrado, suggesting DivEye captures stable distributional signals across different entropy levels within the same generator.
E.3.2 Cross-Model, Variable-Temperane Temperature Detection
We further test DivEye on Llama-3. 1-8B, generating 50 samples per temperature (ranging T 0.1 to 1.6) using OASST prompis. This simulates an adversarial generator varying sampling temperature to evade detection. Table 16 summarizes Al detection accuracy.
Table 16: DivEye Al detection accuracy for Llama-3. 3--BBB across different temperatures.
Temperature
AI Accuracy (%)
T 0.1
94.0
T 0.3
96.0
T 0.5
100.0
T = 0.7
96.0
T 1.0
96.0
T 1.2
98.0
T 1.4
94.0
T 7=1 1.6
96.0
32

---

Published in Transactions on Machine Learning Research (02/2026)
These experiments demonstrate that DivEye maintains strong performance across both intra-model and cross-model Temperature variations, consistently achieving high AUROC and detection accuracy. Even at high temperatures (T'= 1.6), where generations are more diverse, DivEye remains robust, highlighting its resilience against temperaturo-based evasion strategics in real-world deployment.
E.4 Robustness of DivEye to low-quality LMs and Prompt-based attacks
We further evaluate DivEye's robustness against two challenging conditions raised by the reviower: degen- erale or less predictable generators, and prompt-level adversarial obfuscations.
E.4.1 Performance Less Predictable Generators
While DivEye's primary focus is detecting outputs from realistic, high-quality LLMs, we also assess its behav- ior on weaker or degenerate text sources to explore the method's boundaries. The RAID benchmark already includes a variety of degenerate and obfuscation-style perturbations such as synonym replacement, para- phrasing, number swaps, homoglyph substitutions, and zero-width spaces which DivEye handles effectively (see Table 13 for results).
To complement these benchmark results, we evaluate two baseline degenerate generators producing 500 samples each:
1. Random Token Generator: uniformly samples tokens from GPT-2's vocabulary to generate in- coherent sequences without semantic structure.
2. Keyword-Stufinn Generator: repeats high-frequency topical keywords in ungrammatical, repet- itive patterns.
Using GPT-2 and Testbed 4 of the MAGE benchmark, DivEye achieves near-perfect AI detection: 99.99% on the random token set and 99.95% on the keyword-stuffed set. These results indicate that DivEye confidently flags incoherent or low-quality text as mon-human-wwitlen suggest ing that it is sensitive to general non- human-likeness rather than relying solely on repetition or frequency patterns.
E.4.2 Robustness to Prompt-Based Adversarial Attacks
We also assess DivEye under adversarial prompt-based obfuscation, designed to disrupt GPT-2's predictability patterns. Specifically, we generated 250 texts from GPT-4o and L1ama-3.1-8 using the following system prompt to encourage unusual and unpredictable writing styles:
'You are highly creative and unconventional writer. Your goal is to respond with orig- inal, imaginalive, and surprising ideas that avoid clichés, common phrases, or predictable completions. Do not repeat yourself or follow standard templates. Instead, embrace novelty, abstract connectinss and lateral thinking. Ureusual melaphors, unexpected analogies, and rare vocabulary are encouraged."
These generations were sampled from different models and prompts, constituting a fully out-of-distribution evaluation. Table 17 shows DivEye's 's detection performance compared to Binoculars.
Table 17: AI delection accuracy under prompt-based adversarial attacks.
Model
DivEye Accuracy (%)
Binoculars Accuracy (%)
1llma-3.1-89
90.8
72.4
GPT-4o
92.4
78.0
Despite these obfuscation strategics, DivEye maintains high accuracy, substantially outperforming Binocu- lars. This demonstrates that DivEye's diversity-based signals remain robust under both low-quality genera- tors and adversarial prompting, highlighting its resilience in real-world deployment, scenarios.
33

---

Published in Transactions on Machine Learning Research (02/2026)
Overall, these experiments confirm that DivEye effectively detects both degenerate text and highly umpre- dictable LLM outputs, providing strong evidence of its general robustness.
E.5 Dependence of DivEye on Top-p Generations
To ensure DivEye's robustness extends beyond temperature scaling, evaluated its stability under varying nucleus sampling (top-p) thresholds. We conducted an evaluation using Llama-3.1 generating 100 text samples per threshold across broad spectrum of values p in. 0.3, 0.5, 0.8, 0.95). The detection results are summarized in Table 18
Table 18: Detection Accuracy of DivEye on Llama-3. 1-8B across varying Top-p thresholds.
Top-p Threshold
AT
p 0.1
96%
0.3
94%
0.5
97%
0.8
93%
= 0.95
95%
Average
95%
As evidenced by the results, DivEye demonstrates remarkable consistency across the entire range of p values, with an average accuracy of 95% and negligible variance. This stability can be attributed to the nature of the sampling mechanism: while top-p removes low-probability tokens and may affect the absolute mean perplexity, fails to emulate the unpredictability charactoristic of human writing.
F
Additional Discussions
F.1
Robustness of Binoculars under OOD Conditions
While Binoculars achieves high performance its original paper, with AUROC values consistently above 0.99 (Tables and 4), robustness under out-of-distribution (OOD) conditions substantially weaker. For example, Tufts al. (2025) (Table 13) shows noticeable drop in AUROC when Binoculars applied to datasets with distributions different from its training set, despite remaining competitive. Similarly, in the Voight-Kampff Generative AT Authorship Verification Challenge 2024 (Ayele et al. 2024). Binoculars un- derperformed significantly under highly OOD conditions, failing to replicate its originally reported accuracy and AUROC.
These findings are consistent with our observed AvgAce 79%, reflecting Binoculars' sensitivity to domain shift. Importantly, our evaluation deliberately includes diverse scenarios to rigorously test gencralization beyond training conditions. This approach better reflects real-world deployment, where robustness to OOD tasks is critical. Our results do not contradict prior work; rather, they reinforce the understanding of Binoculars' limitations under distribution shifts.
F.2 Practical constraints on adversarial attacks against DivEye
The possibility of (argeted attacks that attempl to evade DivEye by steering autoregressive models to generate tokens falling in low- www-robabillity regions under GPT-2's distribution seems like plausible idea to evade detection. While such attacks are theoretically conccivable, implementing them in practice is extremely challenging.
Autoregressive models do not natively support fine-grained constraints that enforce divergence from another model's token-level distribution without compromising fluency or coherence. Even for large models such as GPT-4o, generating text that simultancously plansible and systematically unpredictable specific detector like DivEye is highly nontrivial.
34

---

Published in Transactions on Machine Learning Research (02/2026)
0.884
Detection Performance
DivEye Cutoff (9 Features)
0.85
0.80
0.75
AUROC Score
0.70
0.65
0.60
0.50
MeantS)
Var(5]
Skew(S)
Kurt(S)
MeantA]
var(A)
Varld")
H(&?)
pld9
H(A]
Meanta?)
Meania')
Varld")
Haa
Meania?)
Varld))
Haa
Figure 15: Inclusion of additional features into DivEye results in diminishing returns AUROC. This experiment was rum on MAGE Testbed
Furthermore, DivEye is designed to generalize across diverse model backbones. Our best-performing variant, for example, uses Llama-3. 1-8B as the surprisal scorer, which has a larger vocabulary and greater expressive capability than GPT-2. This substantially increases the difficulty for an adversary to generate text that appears unpredictable to the detector while remaining coherent and human-like.
Taken together, these considerations suggest that although targeted, detector-specific attacks are theoreti- cally possible, they are rare and practically hard to execute in realistic generation pipelines.
F.3 Diminishing returns in expanding DivEye
DivEye was deliberately designed with concise set of nine features to balance computational efficiency with interpretability. While the architecture is flexible and allows researchers to integrate additional signals, our empirical analysis suggests that the current feature set captures the vast majority of the discriminative signal. To validate this design choice, we conducted a scaling law experiment evaluating the marginal utility of incorporating higher-order surprisal dynamics.
We extended the original feature vector by computing third-order (A3) and fourth-order (At) derivatives of the surprisal sequence, extracting the mean, variance, and entropy for cach. We also include missing features, including the mean of the second-order and entropy of the first-order of the surprisal sequence. This resulted in expanded candidate set of 17 features. We then trained series of XGBoost classifiers, iteratively adding one feature at time in hicrarchical order and mcasuring the resulting AUROC on MAGE Testbed 4.
As illustrated in Figure 15 the detection performance improves sharply with the inclusion of the first-order and second-order DivEye fcatures, effectively plateauing the ninth feature (sccond-order autocorrelation). The addition of third- and fourth-order features yields negligible gains, confirming a law of diminishing returns.
35

---

Published in Transactions on Machine Learning Research (02/2026)
Intruct
Think ISIT
Intract oowniieee
Futed
De0e/7tion Rale
Figure 16: Impact of post-training objectives detectability; DivEye consistently maintains or slightly euhances delectability compared to the unaligned Base model.
F.4 Robustness across post-training objectives
Modern LLMs undergo extensive post-training alignment to enhance instruction following and safety. critical question is whether these alignment steps alter the statistical surprisal signature of the generated lext, polentially degrading detection performance compared to the pre-trained base model.
To investigate this, we conducted controlled experiment using the OLMo-3-7B (Olmo et al. 2025 model family, which provides open access to checkpoints at every training stage. We curated a datasct (of 600 texts from diverse domains, different temperatures and top-p) comprising eight distinct model variants, including the unaligned Base model, Instruct-tuned versions, Reasoning models, and RLVR. baselines. We adopted unified training strategy: a single DivEye detector was trained on an aggregate containing 75% of the data from all variants (balanced with human text), and subsequently evaluated on the held-out 25% of each specific variant.
As detailed in Figure 16 DivEye demonstrates remarkable stability across all training objectives. While one might hypothesize that alignment (which constrains the output space) would significantly shift the surprisal distribution, our results show that the core detectability remains high.
Contrary to the intuition that aligned models might better mimic human stylistic nuances, we observe that alignment consistently enhances detectability. The pre-trained Base model yields the lowest (yet still robust) accuracy of 97.33%, whereas models fino-tuned with DPO and Thinking capabilities achieve perfect detection rates.
This trend suggests that post-training interventions constrain the model's output distribution to a highly structured subspace. This constraint likely dampens the natural entropic variance present in the pre-training data, making casier for DivEye to distinguish post-trained models from genuine human writing.
G Failure of other motivational methods: LookForward
A natural hypothesis we considered was that LLMs, being autoregressive in nature, lack global sentence-level planning due their left-to-right generation paradigm. Unlike humans who often write with sense of the sentence's future, autoregressive models generate one token a time conditioned only the preceding context. Based on this, hypothesized that detection features relying on this "lack of foresight' could effectively identify machine-generated text.
This suggests that the model never observes æst when predicting ot, whereas human writing may implicitly reflect. awareness of future tokens. Our idea was to define a Foowrd discrepancy by comparing model likelihoods under forward-only conditioning vs. bidirectional context.
However, our empirical evaluations demonstrate that this feature is ineffective, achieving '0.00 AUROC on diverse testbeds. As LLLSs undergo extensive training and optimization, they appear to develop strong
36

---

Published in Transactions on Machine Learning Research (02/2026)
internal planning capabilities, even in autoregressive setting. Despite the absence of access to future tokens during generation, LLMss approximate global coherence and structure remarkably well. This aligns with recent literature suggesting that transformers internalize hierarchical and global sentence structure across layers, even when trained autoregressively.
While this method is theoretically appealing, its failure in practice highlights the difficulty of quantifying planning behavior in black-box LLMs. We hope this limitation can be better understood in the future through more ine-grained interpretability analyses of autoregressive models, which may reveal how planning and coherence emerge despite the lack of explicit future context.
H Testbed Details
We evaluate DivEye on comprehensive testbed spanning three major Al-text detection benchmarks, MAGE [Li ct al. 2024). HC3 (Guo et al., 2023) & RAID (Dugan et al. 2024), covering diverse range of domains, language models, and adversarial attacks. These benchmarks allow to assess the generalizability and robustness of our method across realistic deployment scenarios. This section provides a comprehensive overview of the testbeds used in our evaluation, including all domains, language models, and adversarial attacks featured in the MAGE and RAID benchmarks, along with relevant configuration details.
Details about MAGE Benchmark. The MAGE benchmark (Li et al. 2024) comprises eight diverse testbeds designed for evaluating muchine-generated text detection. Testbeds through include standard train, validation, and test splits, while Test through 8 serve as dusto--dist (OOD) datasets, evaluated using models trained on Testbed 4. Notably, Testbed 4, Arbitrary Domains & Arbitrary Models, is the most comprehensive, enabling evaluation across the full range of domains and language models listed in the MAGE paper. Detailed information regarding dataset splits and sample counts is available in the original benchmark documentation.
MAGE covers a wide array of domains, including CMV (Tan et al. 2016) Yelp Zhhan et al. 2015). XSum (Narayan et al., 2018), TLDR, EL15 [Fan al. 2019) WP (Fan et al.
2018), ROC (Mostafazadeh et al.
2016), HellaSwag (Zellers et al., 2019), SQuAD (Rajpurkar et al., 2016);
and SeiXGen
(Chen et al. 2021a
The OOD domains include CNN CNN/DailyMai (See et al. 2017], DialogSum CCen et al.
2021b PubMedQA
Jin et al.
2019), and IMDb (Maas et al. 2011).
MAGE also incorporates text generated from over 27 different LLMs (Brown et al. 2020
Chung et al. 2022
et al. 2023 Sanh et al. 2022 Touvron et al. 2023a Zeng et al.) 2023 Zhang et al. 2022), enabling rigorous
and varied evaluations. For further implementation specifics, readers are encouraged to consult the MAGE
paper.
Details about RAID Benchmark. The RAID benchmark (Dugan et al.) 2024 comprises over 6.2 million samples, offering extensive coverage across domains, language models, sample sizes, and adversarial attacks. Tt provides a clear separation into training, validation, and testing splits to support rigorous evaluation. The benchmark spans a wide range of domains, including scientific abstracts (Paul & Rakshit,
2021j, book summaries (Bamman & Smith 2013 BBC News articles (Greene & Cunningham 2006), poems
(Arman 2020), recipes (Bien et al.] 2020), Reddit posts (Völske et al. 2017), movie reviews MMaas et al.
2011), Wikipedia entries (Aaditya Bhat, 2023 Python code, Czech news Boháčck ct al. 2022 and German
news articles (Schabus et al. 2017).
RAID employs text generated from 11 diverse LLMs Radford et al. 2019 MosaicML 2023
Jiang et al. 2023
Cohere 2024 Ouyang et al. 2022 Touvron et al. 2023b jt al.) 2024b). ensuring broad model representation.
Additionally, it includes over 11 adversarial attack stralegies [Liang et al. 2023b Wolff & Wolf 2022
Bhat & Parthasarathy [2020] Krishna et al.) 2023 Pu et al.] 2023 Gagiano et al., 2021] Guerrero et al, 2022), designed to test the robustness of detectors under challenging scttings. Comprehensive descriptions
and detailed results of these attacks are provided in Appendix E.1 with all results reported as of April 2025.
For further implementation specifics, readers are encouraged to consult the RAID paper.
37

---

Published in Transactions on Machine Learning Research (02/2026)
Details about HC3 Benchmark. The HC3 benchmark (Guo et al. 2023 offers large-scale, mul- tilingual dataset designed to evaluate the effectiveness of detectors in distinguishing human-written text from Al-generated responses. It encompasses both English and Chinese content, covering a wide variety of domains and question types. This bilingual sctup facilitates cross-linguistic performance analysis and underscores the difficulties of achieving generalization across different languages and cultural contexts. In our experiments, we adopt an 80-20 train-test split. For comprehensive dataset numbers, we refer readers to the original HC3 paper.
Hyperparameter Settings
Table 19 outlines the hyperparameter configurations used for our experiments. utilize the XGBoost classifier with standard but tuned settings to handle class imbalance and optimize detection performance. For our proposed method DivEye, set the number of bins for entropy computation to 20 and truncate input sequences maximum length of 1024 tokens. All experiments were run on single NVIDIA DGX A100 (40 GB), and reported results reflect the median of three runs.
Table 19: Hyperparameters used for the XGBoost Classifier and DivBye.
XGBoost Hyperparameter
Value
random_star _ttattettteeeeee
42
scale_pos_weiine
(len Yurain) Σ \train// Σ Train
max_ max_depth
12
n_estimators
200
colsample_ bytree
0.8
subsample min wwwpcchlllllieee weight
0.7
5
gamma
1.0
DivEye Parameter
Value
Entropy bins
20
Tokenizer Max Length
1024 + Truncation
J Limitations, Broad Impacts, Reproducibility & Ethical Considerations
Future Work & Limitations. While DivEye demonstrates strong generalization across domains and models in zero-shot settings, several limitations suggest promising directions for future work. Our approach relies on features derived from LLM token-level behavior, which may vary across model sizes, architectures, and tokenization schemes. Although our current performance is robust, it is unclear whether we are ap- proaching an optimal limit for Al-lext detection. Moreover, our diversity metrics are less elfective on very short texts, where statistical patterns are inherently limited. We hope to address these challenges in fu- ture work by exploring more adaptive teacher selection strategics and improving robustness in diverse text lengths.
Broad Impacts. This work introduces DivEye, model-agnostic, and scalable framework for detecting Al-generated text that remains robust across models, domains, and decoding strategies. By leveraging purely intrinsic statistical features, without requiring fino-tuning or access to the internals of large language models, DivEye is broadly applicable and easy to deploy in real-world settings. We envision this framework as a practical tool to support responsible Al usage, aiding in the detection of synthetic text across domains such as oducation, journalism, and online content moderation. However, we emphasize that detection results should be interpreted with care and recommend using DivEye as one component within broader, multi-layered content verification pipeline.
38

---

Published in Transactions on Machine Learning Research (02/2026)
We deliberately restrict DivEye to nine features, each of which is theoretically motivated and captures distinct aspect of surprisal diversity. While additional features could be engineered, our preliminary experiments indicated diminishing returns beyond this set. This preserves interpretability, efficiency, and robustness, while still providing strong empirical performance. We also discuss certain concerns about our results and the practicality of adversarial attacks in Appendix
Furthermore, our finding that lightweight model (GPT-2) can effectively serve as an observer for state-of- the-art gencrators has significant implications for the democratization and sustainability of AI forensics. By demonstrating that detection does not require computational parity with the generator, DivEye establishes strong lower bound for efficiency. However, for production environments where maximizing detection sensitivity is paramount, we recommend leveraging more capable LMs as the feature extractor.
We release all codc and evaluation scripts to ensure full reproducibility. Detailed train- ing, lesting and hyperparameter configurations are included Appendices and C
Reproducibility.
Ethical Considcrations. As with all AI-text detectors, DivEye not infallible and may produce incorrect classifications false positives. We emphasize that detection outputs should be treated probabilistie sig- nals rather than definitive evidence. When used in high-stakes settings, such academic integrity content moderation, additional human review and validation are essential. We encourage responsible deployment of DivEye to support large-scale analysis, but caution against its use in critical decision-making.
K Illustrative cases of DivEye with probabilities
We provide few representative examples for readers in Table 20 showcasing the probability scores assigned by DivEye to different text sources.
39

---

Published in Transactions on Machine Learning Research (02/2026)
Source
Text
ProbobilityAr (DivEye)
GPT-4-Turbo
For centuries, the pursuit of immortality was the ul- timate quest, a beacon drawing the brilliant and the mad alike. I, Dr. Elara Mendoza, fell somewhere in between, teetering on the precipice of genius and insanity. And after countless sleepless nights, fueled by an obsession that bordered on madness, finally did it. I unlocked the secret to immortality. In my laboratory bathed in the cold, metallic gleam of ar- tificial light, the hum of machinery breathed life into my creation, serum, translucent and iridescent, potion promising eternity. As the final drop fell into the vial, a silence descended, thick with anticipation. But in this moment of triumph, chill swept through the room, frosting over the warmth of victory...
0.97035
Claude-3- O-uus
Dressing for Success: Budgeting for Interview At- tire and Work Uniforms as Medical Office Assis- tant As medical office assistant, presenting professional image is crucial for success in both the job interview process and daily work life. Dressing appropriately demonstrates respect for the health- care setting, instills confidence in patients, and show- cases a commitment the role. However, building wardrobe suitable for the medical office can be financial challenge, especially for those just starting the field. By developing a strategic budget plan... Loved this tour! grabbed groupon and the price was great. It was the perfect way to explore New Orleans for someone who'd never been there before and didn't know lot about the history of the city. Our tour guide had tons of interesting tidbits about the city, and really enjoyed the experience. Highly recommended tour. actually thought we were just going to tour through the cemetery, but she took us around the French Quarter for the first hour, and the cemetery for the second half of the tour. You'll meet up in front of grocery store (seems strange at first, but it's not terribly hard to find, and it'll give you chance to get some water), and you'll stop a visitor center part way through the tour for bathroom break if needed. This tour was one of my favorite parts of my trip!
0.96279
Human- Unman--Writenn
0.09172
Table 20: Representative examples of texts from various sources with their predicted probability of being Al-generatod according to DivEye.
40


Reviews:
[{"review_id": "UOh7ygoltg", "reviewer": "Reviewer_4Fu1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The submission tackles the problem of model-agnostic zero-shot detection of  AI-generated text. This refers to the challenge of detecting whether some text was generated by *any* AI system, rather than detecting text generated by a particular AI system. \n\nThe main contribution is the demonstration of the usefulness of particular features which capture distributional and temporal patterns in the per-token surprisal of the text, as extracted from a frozen LLM. The authors refer to these features as diversity statistics and hence call their classifier DivEye (I don’t understand the ‘Eye’ part of the name).\n\nDivEye’s feature set consists of the first four moments of the distribution of surprisal values for a given piece of text, the mean and variance of the first-order difference of those surprisal values (like the first derivative, capturing the change in surprisal values over the course of the sequence), and the variance, entropy, and autocorrelation of the second order difference (the difference of differences) of the surprisal values. From these features they train a binary classifier on examples of human-written and AI-generated text. The performance of this classifier on its own and as a booster for existing detectors is characterised with several datasets and ablations.\n\nThe authors motivate these features by claiming that they capture aspects of natural human language that will be missing in LLM-generated text, given their training objective. They claim that human language shows greater stylistic diversity, resulting in variations and fluctuations in the predictability of tokens within a piece of text. \n\nThe idea of using first and second order difference features for sequence-based machine learning applications is not especially novel, but it is great to see them work so well in this particular application.\n\n**Strengths:**\n* The idea is simple, makes sense, and is clearly described\n* SOTA or competitive performance on three benchmarks\n* Thorough evaluation on several datasets including text generated by a variety of AI systems, several baseline comparisons, assessing in- and out-of-distribution generalization, efficiency, as well as multi-lingual, adversarial and base model robustness.\n\n\n**Weaknesses:**\n* Not always clear how exactly this approach relates to past work, which has also used features of the log probabilities of a frozen LLM for this task. \n* The use of first and second order difference features is motivated by speculation about the nature of human language. These claims are presented as fact without reference to any of the relevant (e.g., linguistics) literature. \n* The title does not clearly communicate what this paper is about\n* Sometimes imprecise/confusing language: using ‘diversity’ and ‘unpredictability’ interchangeably, using “diversity statistics”/”metrics” as an umbrella term for distributional and temporal difference features.", "claims_and_evidence_yn": "Yes", "claims_explanation": "Yes and No. The most important claims are supported by the empirical results of the paper. However, the authors motivate their approach with several claims about human language for which they cite no prior evidence.\n\nI especially appreciate the appendices which show the importance of each of the DivEye features.", "audience_yn": "Yes", "audience_explanation": "Robust detection of AI-generated text in the wild is very much needed. The approach described here is easy to incorporate into existing approaches. These findings may also point to alternative LLM training objectives to capture precisely these features that currently distinguish human and AI text.", "requested_changes": "**Critical requested changes:**\n* Figure 3: To make the figures easier to understand, figure legends should say “Human-written” and “AI-generated” instead of “True Label 0” and “True Label 1” and the x-axis label should say something to indicate that it shows how likely the classifier judges the text to be human-written\n* The set of 9 features seems somewhat arbitrary. Why not include the mean of the second order difference or the entropy of the first order difference (or any of the other missing statistics)? If other features were also explored and these 9 were found to be the most informative or some other feature selection was performed, then that should be included. \n* The title should be revised to more clearly communicate the contribution of the paper, which is focused on the inclusion of distributional and temporal features of surprisal.\n* There is little to no discussion at the end of the main paper. Some space should be reserved at the end to discuss how the approach builds on and differs from what's been done before, discuss limitations of the approach and any avenues for future work. \n\n\n**Optional suggested changes:**\n* If the authors wish to motivate their features by reference to properties of human language (stylistic diversity, burstiness, rhythmicity, etc.), it would be good to cite some literature to support these claims. \n* I found sections 1-3 somewhat repetitive. To make room for a lengthier discussion, you might find opportunities to make the intro + methods sections more concise. I don't see the need for a separate box for Remark 1: Proof sketch. This is not something to be proved analytically anyway. \n* Modern LLMs in use today have been trained on a variety of objectives, not merely to maximize the overall likelihood of the generated text, which the authors focus on when motivating their approach. How do these additional objectives affect the success of the DivEye? For example, if you generate text using the self-supervised pretrained, supervised instruction-tuned, and RLHF-trained versions of the same model, does DivEye work best on the pretrained model? I wonder if these post-training steps would affect the importance of the different features tested here. Some discussion of pre-training vs post-training objectives could be included.", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "ZLov2u9695", "reviewer": "Reviewer_4ge7", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces DivEye, a framework for detecting AI-generated text based on the hypothesis that human-written text exhibits greater variability in token-level surprisal (the log probability of the current token based on pass texts) compared to LLM outputs. The method extracts nine statistical features, including surprisal's distributional moments, statistics for first and second-differences. The authors train a binary classifier (XGBoost based) using these features. They also experimented with combining this feature vector with existing detector's output as a booster. \nExperiments with MAGE, RAID and HC3 show strong performance, robustness on adversarial attacks and multilingual texts, and good computational efficiency. \n\nStrengths\n- The underlying assumption - human writing has richer variability in unpredictability than LLM outputs, is well motivated, and nicely illustrated by the distribution plots in Figure 2.\n- The DivEye features representing the surprisal distributions is clearly explained. Combined with the experiments performances and the feature importance analysis, show that it is simple, intuitive yet effective design. They are also easy and cost-efficient to generate - require a single forward pass through a small model + lightweight statistics computations.\n- The authors perform thorough experiments on multiple benchmarks, domains, generator models, adversarial attack types. \n- Overall, it is a simple yet effective framework with good interpretability and practicality. With highlights of model-agnostic, low computational cost, and potential of serving as booster to existing detectors.\n\nWeakness\n- The novelty is limited. The DivEye features are standard statistics applied to a known signal (surprisal) - moments of a distribution and basic time-series statistics. The paper does not justify why these features sufficiently describes of the surprisal distributions, and the actual gain comparing to the statistics used in existing frameworks, which already discussed entropy, curvature methods, etc. \n- Potential brittleness of the core hypothesis. The assumption that human text is more creative and unpredictable may break in several scenarios, including non-native speakers who may use simpler vocabulary/grammar and more predictive structures, and formal/legal writing which is highly templated with strict choice of wordings.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are largely supported. On MAGE, DivEye attains top zero-shot AUROC/AvgAcc across ID and OOD testbeds (Table 1). On RAID, it nearly ties the best fine-tuned open baseline in AUROC (0.984 vs 0.986) and clearly beats other zero-shot methods (Table 2). Its robustness is further shown by adversarial transformations. The fact it further boosted existing detector's result show that it captures some orthogonal signals.", "audience_yn": "Yes", "audience_explanation": "AI-test detection will be an increasing important challenge, with growing difficulties as the LLMs getting better. The approach is simple, interpretable, resource-light, and can be combined with existing detectors. It could be easily tested and adopted by interested readers.", "requested_changes": "- Bias & false-positive analysis. Evaluate DivEye's performances on edges cases, in particular, non-native English human writings, and highly-professional and formal writings such as legal documents. (Critical)\n- Provide a more thorough theoretical/empirical decomposition that explains why the DivEye features add signal beyond the statistics captured by the existing models. Provide a head-to-head comparisons with identical evaluator LM - swap DivEye features out for the existing ones(entropy/log-rank/etc) computed from the same evaluator LM and run identical pipeline. (Critical). \n- Since core idea of DivEye is surprisal feature engineering, it would be nice to include further feature importance and significance analysis, such as leave-one-out/leave-group-out tests with error bars, SHAP, permutation importances. (Strengthening)\n- Provide more feature visualisations, such as distributions on various text types (including the edge cases). (Strengthening)", "broader_impact_concerns": "The most important consideration will be wrongful accusations - as mentioned above, non-native speakers may use less diverse writing style, and their writings have been falsely classified to AI-generated texts with higher rates. The paper should include impact and guidelines on using DivEye.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "NgLcWpDW0N", "reviewer": "Reviewer_onBt", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes DivEye, a detection method that uses statistical patterns in token-level surprisal to distinguish human-written text from machine-generated text. Instead of relying on a fine-tuned neural classifier or a large LM as a detector, the approach extracts a compact set of surprisal-based features—covering distributional metrics (mean, variance, skewness, kurtosis) and temporal dynamics (first- and second-order differences, entropy, autocorrelation)—and feeds them into a lightweight classifier. The core idea is that human text tends to exhibit richer and more irregular surprisal fluctuations, whereas LLM-generated text often reflects smoother and more predictable patterns.\n\nThe empirical evaluation is quite extensive. The method is tested across multiple datasets (MAGE, RAID, HC3), various generator families, adversarial paraphrasing settings, and multilingual scenarios. DivEye performs strongly as a zero-shot detector and consistently improves the accuracy of several existing detectors when used as an auxiliary signal. The experiments also examine different surprisal base models and include feature-importance analyses that help explain why the method works.\n\nOverall, the paper is clearly written, the core idea is intuitive, and the results support the main claims.\n\n**Strengths:**\n- Conceptually simple and well-motivated approach\n- Strong performance across diverse benchmarks\n- Works in zero-shot settings and with small LMs\n- Offers interpretability through surprisal features\n- Complements and improves existing detectors\n\n**Weaknesses:**\n- Heavy reliance on hand-crafted features may affect long-term robustness\n- Some inconsistency in out-of-distribution performance\n- Surprisal is primarily computed using GPT-2, which may not capture modern LM behavior\n- Missing analysis on sampling/temperature effects and text-length sensitivity\n- Real-world, organically sourced text evaluation is limited", "claims_and_evidence_yn": "Yes", "claims_explanation": "The evidence presented in the paper is generally strong and well-organized. The authors support their hypothesis with an initial empirical study showing meaningful differences in surprisal behavior between human and GPT-4-Turbo outputs, which helps ground the motivation. The experimental section covers a wide range of benchmarks—including both adversarial datasets and multilingual settings—and DivEye consistently performs at or near the top among zero-shot detectors. The improvement it provides when integrated with other detectors also reinforces the idea that it captures a complementary signal.\n\nThere are some areas that could benefit from additional clarification (e.g., how performance changes with different decoding temperatures or with very short inputs), but these do not undermine the validity of the main claims. Overall, the evidence is convincing and supports the conclusions drawn.", "audience_yn": "Yes", "audience_explanation": "AI-generated text detection is an active and increasingly important area, and the community is still exploring methods that are reliable in open, real-world settings. A technique that is zero-shot, computationally efficient, and interpretable is relevant to both researchers and practitioners. In particular, the idea of analyzing surprisal dynamics rather than solely average likelihood is interesting and may inspire follow-up work. The ability to “boost” existing detectors without retraining is also a practical contribution that many TMLR readers would appreciate.", "requested_changes": "**Important (but not blocking)**\n1. Clarify the role of the base LM used for surprisal.\n    The method primarily uses GPT-2, which is quite outdated relative to the generators being evaluated. A deeper discussion of mismatch  scenarios (e.g., when the surprisal model is much weaker than the generator) would help practitioners understand the limits of the approach.\n\n2. Provide more insight into out-of-distribution failure cases.\nIn some OOD testbeds, DivEye does not always outperform all baselines. Even a brief analysis of which text types or domains present difficulty would strengthen the generalization claims.\n\n3. Discuss the effect of decoding strategies.\nSince surprisal variance is central to the method, different sampling temperatures or top-p values could meaningfully affect detectability. A small study or at least a discussion would be helpful.\n\n4. Consider a short analysis of text-length sensitivity.\nMany real-world inputs are short (e.g., tweets, messages). Because the method aggregates statistics across tokens, performance likely varies with sequence length; understanding that behavior would be useful.\n\n\n\n**Suggestions / Nice-to-have**\n\n5. Add some evaluation on organically collected human text.\nThe existing benchmarks are strong but controlled. Even a small qualitative evaluation on real-world short-form text, essays, or forum posts would make the practical value clearer.\n\n6. Minor clarity improvements in figures and captions.\nA few figures are dense; slightly expanding captions or referencing them more explicitly in the text would improve readability.", "broader_impact_concerns": "I did not identify any major unaddressed ethical concerns. The intended purpose—detecting AI-generated text to mitigate misinformation, academic misconduct, and similar risks—is aligned with positive societal outcomes. However, as with all detection systems, there are a few considerations worth emphasizing:\n\n- **False positives** can disproportionately affect non-native speakers or individuals with atypical writing styles.\n- **Overreliance** on automated detectors by institutions may lead to unfair outcomes if results are treated as definitive.\n- **Adversarial dynamics** are inherent to this field; stronger detectors may motivate more sophisticated evasion strategies.\n\nThese points do not detract from the contribution but should be acknowledged explicitly when deploying such systems in high-stakes environments.", "additional_comments": "Overall, I found this to be a thoughtful and well-executed paper. The proposed approach sits at a nice intersection of simplicity, interpretability, and practical utility. The empirical results are extensive and demonstrate that surprisal-based diversity features are surprisingly effective at capturing differences between human and LLM text. I also appreciate the inclusion of ablation studies and multiple base models, as these help reveal which aspects of the method matter most.\n\nWith some additional discussion around limitations and the suggested clarifications, I believe the paper would be even stronger. But as it stands, it already makes a solid and useful contribution to the area of machine-generated text detection.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)