Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective
Authors: Chenwang Wu
Yiu-ming Cheung
Bo Han
Defu Lian
Venue: NeurIPS 2025 poster ()
Keywords: Machine-Generated Text Detection
Simple-to-Hard Supervision
Large Language Model
Primary area: general_machine_learning
TL;DR: Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective

Abstract: Existing machine-generated text (MGT) detection methods implicitly assume labels as the "golden standard". However, we reveal boundary ambiguity in MGT detection, implying that traditional training paradigms are inexact. Moreover, limitations of human cognition and the superintelligence of detectors make inexact learning widespread and inevitable. To this end, we propose an easy-to-hard enhancement framework to provide reliable supervision under such inexact conditions. Distinct from knowledge distillation, our framework employs an easy supervisor targeting relatively simple longer-text detection tasks (despite weaker capabilities), to enhance the more challenging target detector. Firstly, longer texts targeted by supervisors theoretically alleviate the impact of inexact labels, laying the foundation for reliable supervision. Secondly, by structurally incorporating the detector into the supervisor, we theoretically model the supervisor as a lower performance bound for the detector. Thus, optimizing the supervisor indirectly optimizes the detector, ultimately approximating the underlying "golden" labels. Extensive experiments across diverse practical scenarios, including cross-LLM, cross-domain, mixed text, and paraphrase attacks, demonstrate the framework's significant detection effectiveness. The code is available at: \url{https://github.com/tmlr-group/Easy2Hard}.

Advancing Machine-Generated Text Detection from an Easy to Hard Supervision Perspective
Chenwang Wu Yiu-ming Cheung Bo Han Defu Lian²
Department of Computer Science, Hong Kong Baptist University, Hong Kong, China Schooo of Computer Science, University of Science and Technology of China, Hefei, China {cscwwu, ymc, bhanml)&comp hkbu. edu.hk, .k, liandefudustc.. edu.
Abstract
Existing machine-generated text (MGT) detection methods implicitly assume labels as the "golden standard". However, we reveal boundary ambiguity in MGT detec- tion, implying that traditional training paradigms are inexact. Moreover, limitations of human cognition and the superintelligence of detectors make inexact learning widespread and inevitable. To this end, we propose an easy-to-hard enhancement framework to provide reliable supervision under such inexact conditions. Distinct from knowledge distillation, our framework employs an easy supervisor target- ing relatively simple longer-text detection tasks (despite weaker capabilities), enhance the more challenging target detector. Firstly, longer texts targeted by super- visors theoretically alleviate the impact of inexact labels, laying the foundation for reliable supervision. Secondly, by structurally incorporating the detector into the supervisor, theoretically model the supervisor as lower performance bound for the detector. Thus, optimizing the supervisor indirectly optimizes the detector, ulti- mately approximating the underlying "golden" labels. Extensive experiments across diverse practical scenarios, including cross-L..M,, cross-domain, mixed text, and paraphrase attacks, demonstrate the framework's significant detection effectiveness. The code is available at:
www.ti..cmmmmmmmmtemeeeee ttmr/mmlll--e group//asy2Hard
Introduction
High-quality machine-generated text (MGT) is increasingly prominent due to its potential in areas like content creation [1. intelligent education 2 and customer service B. However, its misuse presents significant challenges, including misinformation [4) phishing attacks [5], and malicious impersonation (6) Compounding this is research 7 indicating humans struggle distinguish MGT from cormann-generated text (HGT), performing little better than random chance. This tension between MGT's risks and limited human detection highlights the urgent need for effective detection methods.
Existing detection methods can be primarily divided into: (1) metric-based methods, which detect differences by capturing the intrinsic statistical properties between MGTs and HGTs, using statistics such as Likelihood g and Entropy [0]. In addition, a series of works represented by DetectGPT MT Fast-DetectGPT 12]. and DALD [3] utilize the probability curvature of text under LLMs as key detection features. (2) Model-based methods do not rely on explicit feature engineering. They input the full text into deep learning models, which automatically learn and extract discriminative implicit features end-to-end. This category includes energy- based models [14), GNN-based model
[1], LLM [16), and other methods such SeqXGPT [17] AI-Catcher [18, and RADAR [99. With the powerful text representation learning capabilities of deep learning models, model- based methods typically demonstrate higher effectiveness terms of detection performance and robustness.
"Corresponding author: Yiu-ming Cheung (yme@ komp.hkbb.edu.hk)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

15.0
Label
12.5
Label
Human Machine
330-
Human
10.0
320
Human
15.0
250
Machiin
7.5
250
Mochina
7.5
200
220
5.0-
2.5-
180-
150
2.5
Label
Machine
150
100
0.0
-25
Human
10
0.4
1.0
2.5 Probabilty
Probabiiity
(D)
Figure I: Boundary fuzziness evaluation between (mixed) MGT and HGT, which illustrates the latent space distribution and prediction confidence distribution under pure (Sub-Fig. & 2) and mixed (Sub-Fig. & 4) texts. The mixed text is obtained by replacing 1/4 of MGTs with HGTs.
ChatGPT
ChwiGPT
ChaiGPT
detector
Hard
ChetGPT
Claude
detector Hard Solt
Cloude
Claude
Soft
Claude
detector
Detector
Hand
Liama
Uama
Soft
Lisma
Liama
P8-8
TPRg FPR-16
(a) ChatGPT Text Training
(b) Claude Text Training
Figure 2: Performance comparison with and without using soft labels in mixed text (1/4 of MGT was replaced with HGT). The detector is ChatGPT-D (20].
The aforementioned methods often implicitly rely on the assumption of perfect data labels; however, this may not hold in MGT detection. Specifically, considering the prevalence of collaboration scenarios (1 and the powerful generative capabilities LLMs, MGTss may explicitly implicitly resemble HGTss leading blurred boundaries. For instance, MGT's and HGT's overlap significantly in the latent space, especially in mixed texts, as shown in Fig. T and 3.3. Besides, Fig. 2 T show that HGT prediction probabilities concentrate indicates MGT) while MGTs with human-like features exhibit a broad distribution across [0, 1], reinforcing their boundary ambiguity. To further validate the inexactness of hard labels, we trained a detector on mixed texts (1/4 MGT replaced with HGT) using label smoothing (22) (with a small smoothing factor of 0.05). Notably, even though we cannot access the underlying "golden" labels, the soft label with small smoothing factor is closer to the golden labels than hard labels in significant mixed texts. As shown Fig. the results clearly indicate that higher upper bound for label smoothing means it has the potential for enhancement. These results collectively indicate that the existing learning paradigm is inexact. See Appendix D.5 for more details and results.
0 (0 indicates HGT, while
Despite this clear insight, resolving the issue challenging. MGT detection task surpasses human cognition, rendering inexact label annotation widespread. Furthermore, existing MGT detectors often more capable than humans, making it difficult to reliably assess the detection quality, i.e., the supervisor may be weaker than the detector, rendering inexact supervision inevitable. In summary, unlike the limited accidental errors in noisy label learning, the fundamental difficulty in annotation and evaluation makes inexact supervision widespread and inevitable. This leads to ask:
it possible to effectively learn from the supervisor when the provided label is inexact and the detection quality is difficult assess?
Our work dedicated to exploring this question. To achieve this, some key issues need to be solved:
RQL. How can the supervisor provide more reliable supervision signals under inexact labels and unclear detection quality?
RQ2. How can the detector be improved based the feedback signals provided by supervisors?
To address this, we propose an easy-to-hard supervision framework to enhance detection. Its core idea is utilize a carefully designed supervisor, focused on the relatively easier task of longer- text detection, to provide reliable feedback signals for guiding a more challenging target detector. To ensure the supervisor's reliable supervision signals, we meticulously consider its data quality improvement and structure design (RQ1). Firstly, we construct longer texts supervisor data, which can theoretically alleviate the impact of inexact labels, laying the foundation for reliable supervision. Secondly, deviate from traditional approaches that rely soft labels to provide supervision

---

signals, because it is unclear whether the soft label is more informative under unclear detection quality. Instead, we structurally integrate the detector into the supervisor's design, establishing a connection between the supervisor and the detector, where the supervisor's performance serves as lower bound for the detector's s capabilities. This coupled structure allows the detector to be indirectly optimized via the supervisor (RQ2), finally encouraging convergence to the underlying "golden" labels. Our contributions can be summarized as follows:
We analyze the inexact supervision inherent in existing MGT detection methods and highlight this widespread and inevitable limitation critical direction for future research.
We propose an easy-to-hane ho-h supervision framework for enhanced detection, theoretically proving its optimization properties that facilitate convergence towards the underlying "golden" labels.
We demonstrate the effectiveness of the proposed framework with negligible latency in various practical scenarios, including cross-LLM, cross-domain, mixed text, and paraphrasing attacks.
2 MGT Detection from an Inexact Supervision Perspective
Traditional Learning Paradigm of MGT Detection. Let denote the set of all possible text sequences, and y = {00 } denote the hard-label space, where 0 represents HGT and represents MGT. Each sequence 8 E can be understood as multiple dependent sentences. Then, the dataset can be represented as 'D {a 3/: w))} }. where x; E S is text composed of n.; sequences from S, and y: E y is its corresponding hard label. This definition understands texts as a collection of multiple sequences, similar to the definition in the existing work (23) For the detector parameterized by f the learning paradigm uses the cross-entropy loss, which is usually as follows.
N
1 N
Larin
Σ 1=1
(ys log I (az, + +1 log(1 - f (a:,, 0))
Inexact Supervision Learning of MGT Detection. In the above learning paradigm, it is implicitly assumed that the label ys is "golden standard"; however, this may not hold in MGT detection.
First, the boundary between MGT and HGT is often unclear, leading to potentially inexact hard labels. This blurred distinction stems from (1) human-machine collaboration 24 25 where texts involve both human and machine contributions (e-g., LLM drafting followed by human editing); (2) the powerful generative capabilities of LLMs, where they trained on extensive human data are capable of producing highly human- like texts, especially for specific text types like short texts (26). Therefore, MGTs may (explicitly or implicitly) contain human-like features, rendering hard labels inexact. This is also confirmed by the empirical results (Fig. aand Fig. 2 shown in the Introduction above, and more results can be found in Appendix D.5
An intuitive solution like knowledge distillation 27 faces a key challenge: its effectiveness mainly relies on the target model being weaker than the strong teacher model. This is difficult for MGT detection because human cognition in distinguishing MGT is limited (e.g., ? reports that human accuracy for GPT-3 texts is 49.9%). Besides, current MGT detectors show super intelligence and even smarter than humans (e.g.. RADAR achieves 87.3% of accuracy on Essay), making it even difficult to distinguish the quality of two predictions. For example, it is unclear whether 90% or 95% confidence is better for an MGT. Therefore, in MGT detection, the supervisor may be weak, and even picking a strong teacher model is challenging.
is another consideration, but its classic assumption is that there exist true, clearly discrete labels with only limited random error occurring during the annotation process. Instead, in MGT detection, as emphasized above, limitations of human cognition in distin- guishing MGT lead to widespread and complex inexactnesses in labeling (with correct categories), rather than limited random errors. Consequently, existing NLL techniques may require significant redefinition to accommodate this structure of inexact labels. See Appendix AA.4 for more discussion.
Noisy label learning (NLL) 28 29
In summary, due to human cognition's 's limitations and detectors' superintelligence, inexact supervision learning in MGT detection becomes widespread and inevitable. Therefore, this paper focuses on designing effective supervised learning algorithms under conditions where labels are generally inexact and detectors are difficult to evaluate reliably.
Thhis is natural assumption where text often has multiple topics, and sentences for each topic dependent.
3

---

Dataset
Target Detector
Annotation
crnangenanttediie bartors, peising cor geand grit.
Target detecter
SE!
Man the sked shar rakkents
Detection model
Self Refire
Sapardkar's retwork
to mien wefor ng stotes of
Hell Nao the nid haik pagn
Roliablc feedbact
Machino generated toxts
Supervisor
Longer Text Detector
caliige, de rade with fr
Samplng
spiR
HOT1
Detection
gumbel
Ludter echied surmeh aon pirser
Long
Longer
Figure 3: The easy- posy-to-iee toy-ttohhaa hard supervision framework, which uses a carefully designed supervisor, focused on relatively simple task of longer-text detection, to guide more challenging target detector.
3 Easy to Hard Supervision Enhancement Framework
To effectively learn from the inexact supervision lens, supervisor capable of providing reliable supervision signals and detector capable of continuous learning from this feedback are essential. To this end, we propose an easy-to-hard supervision enhancement framework, as shown in Fig. 3 includes two key components: an "easy" supervisor, carefully designed for the simpler task of longer-text detection, and a "hard" target detector for the more challenging MGT detection. Briefly, the supervisor is carefully designed from both data and architecture aspects to provide more reliable feedback signals to the detector, while the detector continuously improves by integrating this feedback with its original learning paradigm. We will detail how they are designed to achieve this subsequently.
3.1 Supervisor: Providing Reliable Supervision Signals
In MGT detection, given the inexact labels and difficulty in reliably evaluating detection quality, the supervisor must provide as reliable supervision signals as possible. To achieve this crucial goal, our approach focuses on two key aspects: improving the supervisor's training data quality and designing its model architecture.
Data Quality Improvement. A supervisor's ability to provide reliable supervision is closely linked to its own performance; thus, effective supervisor learning a fundamental prerequisite for achieving reliable supervision. Recognizing the inherent imprecision in training data, improving its training data quality is a natural and feasible method. Drawing upon existing research [23], which indicates a positive correlation between text length and MGT detection and implies that longer texts are easier to detect accurately, we focus the supervisor on detecting longer texts by constructing new longer text its input data.
Specifically, let the longer text-label pair be denoted as (Itong: Morrg then its generation rule is: first sample ylong Y, and then the corresponding longer text Flong is constructed as follows:
Flong 3 (0)
(t) where each (()) ~ ((x. E Dly = yonng
Here, 2 denotes the text splicing operator. The rationale behind this splicing operation is that joining MGTs or HGTs does not alter their fundamental nature as machine- geneerteee human-written text, respectively. The following theorem will prove this design's rationality by revealing the relationship between the distribution difference and the length of the text.
Theorem 3.1 (Distribution Difference for Longer Text), Let h(s) and (s) be the distributions for human--enerateee generated and machine- -enerrateggerr sequences on s S, respectively, with the total variation distance TV TV(m,, h) i O. For the text contains sequences. let Z 0 denote the ratio of human-like component incorporated in MGT. For longer text formed by concatenating k independent length-n texts, the total variation distance between their distributions, TViong can be bounded by:
nk(1 22
1 - -ezepp
I Tooogg $ - (1 5)
2
The theorem indicates that increasing text number & (original 1) for longer text will amplify the distribution difference TViong between HGT and MGT. This leads to clearer classification boundaries

---

and reduced hard label ambiguity. Training supervisors with this more discriminative data is crucial for achieving better performance and providing reliable supervision. Furthermore, larger mixed text ratio tends to a smaller distribution distance, theoretically supporting the conjecture in Section regarding unclear boundaries due to (explicit or implicit) mixed text presence.
Reliable Feedback. Under the reliable learning of the supervisor, the core issue lies in providing reliable supervision signals to the target detector. An intuitive approach is mimic knowledge distillation by directly using the supervisor's soft labels as supervision. However, this faces challenges in MGT detection. First, the detector's 's superintelligence makes unclear if the supervisor's soft labels offer superior information. Second, while the supervisor also performs detection tasks, it focuses on longer texts with different distribution characteristics; these domain differences make ensuring soft label quality challenging when used for more difficult target detection.
Therefore, instead of directly providing soft label supervision, we cleverly integrate the target detector into the supervisor's s design. To achieve this, we reveal the theoretical upper bound of the supervisor's performance, and link factors influencing it to the detector's performance. This link allows the supervisor's ' performance to be viewed as a theoretical lower bound for the detector's capability. Thus, maximizing supervisor performance indirectly enhances detector performance. Specifically,
Theorem 3.2 (Detection Power for Longer Text). Under the assumption of Theorem 3.1 the supervisor AUROC supo. satisfies
1 AUROC S -(1 (1-8)2nk(1-a)
The theorem reveals that the supervisor's performance is the lower bound of 1 1 (1 - 8)2nk(( (1-)), If k is an optimizable variable positively correlated with detector performance, then optimizing the supervisor favors larger k, thereby indirectly enhancing detector performance.
In the framework, & represents the number of original texts x!1 concatenated to form the longer text for the supervisor, while the detector is aimed at identifying each original text 2 To establish a positive correlation between k and the detector's performance (i.e., better detector performance corresponds to larger k),, we revisit the detector as a gating mechanism acting on each original text z(6) within the supervisor's input: If the detector misclassfies text 2(6), that it is filtered out, reducing the concatenation length k. In this way, larger value of k corresponds to higher accuracy. When reaches its maximum value (no filtering), the detector achieves an accuracy of 100%. Formally, we change supervisor's 's input from Flong in Eq. [
@k (x(() O argmmax(f (2x0)) .0,))
if Yong 1,
Flong
(2)
0- (2(()) @ (1 argmax(J(1 0f))))
if
Mong 0.
Here, the element- wise multiplication @ is performed at the input embedding level. To ensure the supervisor's feedback could be back prop-pppppated the detector, we use Gumbel Softmax (30) to replace discrete argmax. Further, we simplify the y/ = 0 branch and simplify Eq. to
Blong = = (2 @ Gumbel (x) ()))))
i... let HGT distribution h(s) collapse to Dirac by distribution (3I, which has following benefits.
Theorem 3.3 (Distribution Difference after HGT Distribution Collapse). Under the assumption of Theorem 3.7 and assuming that m(0) + 0 then if h((s) collapses to a Dirac dn distribution, we have limm(u)-<d TV (h, m)
This theorem indicates that this simplification maximizes the distribution distance between MGT and HGT. Similar to data quality improvement, this reduces the learning difficulty for the supervisor.
In summary, the supervisor 9 parameterized by Og makes predictions for the long text Ttong as g g( fg), where V'ong is calculated by Eq. 3 Assuming that the longer text dataset is Drong- the supervisor's 's training loss Lsupe. using cross entropy is as follows,
Lrupe.
1
Σ
(Moong log (zling: 0g) + (1 - ylong) log (1 g (zlong 0g)))
iDiongl (Xieng-Minng)
'This is a mild assumption, where LLM usually does not correspond to zero vectors to ensure that the text has sufficient information and is non-trivial.
5

---

Alg. D Noutlines the proposed framework's process. For each training batch, the su- pervisor's training data, a longer text set D'g is first constructed based on the cur- rent batch data DR (Line 4). Then, the original detector loss Lori. (Line 5) and the supervisor's 's loss Laupe. (Line 6) are computed based DB and D/g, respec- tively. Finally, joint training is performed for the detector and supervisor (Line 7). Theorem 3.4 (The Effectiveness of the Proposed Framework), Under the assumption of Theorem 3.1 and assuming that the MGT's golden label is approximately the proportion of pure machine- generated content distinct from HGT, if the supervisor reaches the best possible one, the detector converges to the underlying golden labels.
See Appendix A.5 for the discussion of the rationality of this golden label approximation. Unlike traditional methods that merely fit binary hard labels, this theorem reveals how the supervisor guides the detector towards convergence with more accurate underlying "golden" labels.
3.2 Detector: Learning from Reliable Signals
The detector is the target optimization model, which will improve itself by learning category informa- tion from hard labels and feedback from the supervisor. First, an enhancement framework, do not alter the original detector's s structure and training paradigm, thus ensuring flexible extensibility and convenient application to various detectors. Therefore, the first loss is the original detector loss Cori, and the specific form depends on the implementation of the detector. Secondly, the design of the supervisor is closely linked the detector, allowing to indirectly optimize the detector, thus the second loss is supervisor loss Supp. shown in Eq. 4 the detector, and the overall optimization objective with a coefficient is follows:
In summary, we jointly train the supervisor and
of. 0g arg max Cort. + X Csaps.' 0,00
(5)
The target detector f is trained only on the original, natural text, which allows it to learn the true data distribution. Instead, the concatenated text used only to train the supervisor g. This intentional separation ensures the distribution shift (natural text vs. longer text) does not compromise the target detector's performance.
3.3 Overall Framework
Algorithm Easy Hard Supervision Framework
1: Input: Train data D = {(x,, (;))" the detec- tor fxx By). the supervisor g(5long 0g). training
epochs T, learning rate 17- do for each batch of samples DB D do Based on the texts of DB, Construct N longer texts by Eq. [ denoted as D'z-
2: for t- = 0 to
4:
5:
Calculate detector's 's original loss Cori.- Calculate supervisor's 's loss Lsupe. by Eq. Jointly optimize 0g and bg by Eq. end for
Notably, supervisor data is constructed from the current batch DB rather than the dataset D. This design enables the reuse of fz((5), 00). rendering the compu- tational delay for x" in Eq. 4 negligible. The main training delay stems from the supervisor's s forward and backward passes. Fortunately, by comprehensively considering performance and efficiency (see Appendix [..4). the supervisor uses a simple network (e.g., the three-layer fully connected network in our work) to achieve good supervision, resulting in computational costs trivial compared to the detector's typically complex pre-trained model. Therefore, the proposed framework introduces only minimal training latency, and the detector's inference time remains unchanged due to its unmodified nature. See Appendix for more discussion of the proposed framework.
7:
9: end for
10: Return the trained detector (xr,0g).
4 Experiments
Datasets and L.MSs. We conduct experiments on two public datasets, Essay [16] and DetectRI. (2]. to validate our effectiveness. The Essay dataset comprises MGTs generated by GPT4AIL, ChatGPT,

---

Table 1: Performance concerning TPR&FPR-I& (%) on DetectRL. The detection model trained on text generated by PaLM.
Method
Sentence-level
Paragraph-Ievel
PaLM
CharGPT
Claude
Llama-2
Avg.
PaLM
CharGPT
Claude
Llama-2
Avg.
Likelibood
1.83
1.58
0.72
5.54
3.17
25.66 25.66
10.21
1.78
18.39
19:01
Log Rank
4.8420.41
1.2320.05
0.72+9.13
8725
3.01
27.49 27.49
11,55
2.08g00.5
41.93
20.76
Entropy
a.47
0.36
0.521..32
0.49
0.46
6.95 6.95_0.78
0.25.0.16
1.51a03c
2.03
2.68
NPR
2.2420.24
1.72+0.20
1.03+0.06
3.95
2.25
6.80_1.69
3.71=2.45
3. 3.29
16.002a.50
7.68
DetecTGPT
0.72+0.17
0.38
0.25ga.13
178440017
5.51
7.00-1.41
11.10-2.40
5.27a0.45
7.22
FasGGPT
1.33
0.27
0.0920.06
1.60ga.83
0.84
18. 18.15
11.87=1.1
0.44go.17
29. 29.49g0.70
14.99
CharGPT-D
9.71
3.13+a.45
+3.91
8.99
1244.74
17.21
4.62
44.28 44.28
22.81
ChatGPT-E
11.52 20.
11.21
2.23
3. 3.1310039
15.72
10.48
27.81
22.52 29
5.56
25.83
MPU
27.
27.38-1.6 60
26.45-a.m 3.30
7.31
20.42+2.77
22.89
70.13+1.4
79.16-1.71
18.22
1.22
MPU-E
30.40
30.04
7.40
31.55
23.97
24.85
75.75+3.16
84.23
4.86
19.78
-222
90.31
67.52
RADAR
31.38
39.89
10.06
26.9811.7
27.98
40.67
83.51:2245
39.18-3.1
39.18
5.10
86.01
72.35
RADAR-E
38.60
45.32
11.71
31.00
31,65
82.15
81.10.3.44
44.25
86.5813.14
74.27
Table 2: Performance concerning TPR @ PPR-PP--1% (%) on Essay. The detection model is trained on text generated by GPT4AII.
Sctting
Method
GPT4AIl
CharGPT
ChatGPT-turo
CharGLM
Dolly
Claude
Avg.
Likelihood
9.18 56
14.05
5.46
26.04
1.118
2.82 17
10.73
Rank
8.98
13.35
4.97
29.0443.44
6.7841.66
2.32
10.91
Entropy
1.36 26
2.40
1.65
1.10
1.70
1.44
1.61
Sent.
NPR
6.13 66
7.13
4.04 74
14.1441.e8
4.7110.551
3.2612.45
6.57
level
DetectiPT
4..17
3.57 40.337
3.481.0.3
4.77
3.70
3.4410.17
4.01
FasGGPT
12.3811.60
10.3010.26
4.5810.cs
28.8612.71
9.0110.60
2.7210.43
11.31
CharGPT-D
15.64
11-50+0.71
9.94
23.96+1.69
4.66
2.30
11.33
ChatGPT-E
16.18-1.12
10.90
10.49
25.46-1.70
1.70
4.9611..11
2.40
MPU
18.99-1.09
14.38+1..70
12.88 2.888+2.00
30.91
6,00+1.05
2.95
14.35
MPU-E
22.90
17.06
15.30-2.43
33.53 -.007
7.12400.8
3.58 3.58-0.24
16.58
RADAR
28.85-2.25
30.5 30.8842.11
32.56 1.59
39.99+2.45
13. 13.79+1.00
11.06+1.22
26.19
RADAR-E
32.80+3.24
34.33 2..3
36.69 1.38
44.90 3.337
44.90
14.78
10.30+00.74
28.98
Likelihood
46.33
68, (88622.. 13.332213.3
73.60 14.66
92.86+4.84
20.67 20.67+10.79
12.36 6.32
$2.41
Log-Rank
63-71+12.95
79.47
81.29++11.2
96.61+2.49
25 25.9249.00
19.47
61.08
Entropy
3.78 91
11.07 +2.00
16.31 1.779
8.35- 4.44
3.91
6.22
8.27
Para. level
78.50+3.99
+1.56
9.027
13.13
58.52+7.27
8.40+1.117
55.91
DeteeiGPT
31.53L6.06
39 (3.477. 18.30
1.24
30.31 17.82
20 20.48
5.64
22.45
FastGPT
0.1840.17
0.40++0.26
68.27 68.27+4492
1.70=0.78
0.00
7.69
13.04
ChaiGPT-D
58. 58.13-3.64
49. 49.9147.78
86.71+3.65
16. 16.52+2.50
2.31
41.40
ChatGPT-E
59.82=4.82
51.2447.11
35.56+6.60
85.67-6.40
17.09422.86
2.98-
42,06
MPU
71.07+7.13
71.6447.01
48.98
94.78+2.a9
28. 45.69944..6
7.6442.46
$3.80
MPU-E
78.31=6.35
74.09+7.14
52.09=8.18
96. 96.88_0.58
32.0845.58
9.20-3.0a
57.11
RADAR
91.03-2.50
+3.27
54.22+4.28
97.10+1.75
48.02+8868
42.40+4.95
69.51
RADAR-E
93.76.1.88
88.53
59.24
97.86 97.8621.48
53.4118.30
55.87 25.09
74.78
ChatGPT-turbo, ChatGLM. Dolly, and Claude, The DetectRL dataset includes MGTs from PaLM, ChatGPT, Claude, and Llama-2. Furthermore, the DetectRL dataset contains mixed text, paraphrase attack text, and cross-domain text to simulate and evaluate the detection performance in various complex real-world scenarios. Please refer to Appendix for detailed deseriptions of these datasets and Appendix D.4 for the implementation details of the experiments.
Baselines. We selected the following representative baselines, including: (1) metric-based methods: Likelihood g Log-Rank [1I. Entropy 19). NPR (33. DetectGPT [1] and Fast-Detectten
(FastGPT) 34]; (2) model-based methods: ChatGPT-D 20), RADAR [19], and MPU (35). Notably, the proposed enbancement framework can be easily applied to model-based methods, and we denote the enhanced versions using the proposed framework ChatGPT-E, RADAR-E, and MPU-E, Please refer to Appendix D.2 for detailed information on these baselines.
Evaluation metric. Following [3. 37 38) considering the negative impact on users when HGT is misclassified as MGT, and thus, we typically expect a very low false positive rate (FPR). Therefore, we focus the true positive rate (TPR) low TPR. Specifically, we evaluate TPR when FPR is set at 1 %, and denote it as TPR @ FPR-1 %. Besides, we use the area under the receiver operating characteristic curve (AUROC), which is commonly used in MGT detection.
4.1 Performance Evaluation
We conduct extensive evaluations in various practical scenarios (39 sentence-level, paragraph-level, mixed, paraphrasing, and cross-domain texts. Their detailed descriptions are in Appendix D.3
7

---

ChatGPT-D
ChatGPT-E
MPU
MPL-E
RADAR
RADAR-
0.50
0.90
0.03
0.9
TPA@FPM-15
0.75
0.45
TPROFFR: 1.N
0.50
0.30
0.45
0.6
0.30
0.5
6.15
0,00
0.4
Mixed PaLM
Mixed ChatGPT Text
Mixed Claude
Mixed Llama-2
Figure 4: Test performance PPR@FPRR-TE %) under various LLM mixed texts. Detectors are trained on text generated by PaLM. For each sub-figure, the left group: detectors are trained on mixed text, and the right group: detectors are trained on original text.
0.90
0.75
0.75
6.90
0.60
0,90
0.75
0.45
0.15
Back Trarslaion
Pulsh
Back Transtation
0.00
Transiation
0.30
Transation
PaLM Text
ChaIGPT Text
Claude Text
Uame-2 Text
Figure 5: Robustness (TPR 19) against paraphrasing attacks (Back Translation and Polish). Detectors trained on the PaLM texts and tested on the paraphrasing texts of various LLMs.
Sentence-level Detection. Tables T (left) and 2 (top) present sentence-level detection performance w.r.t. IPR@FPR-1%. % for models trained on PaLM and GPT4AIl texts, respectively. See Tables 0 23 the Appendix for AUROC and results trained on other LLMs. Overall, the proposed enhancement strategy substantially improves the original models' performance. For example, on the DetectRL dataset, the average TPR @ FPR-1 % for RADAR increases from 27. 27.98% to 31.65% with the enhanced RADAR-E, while performance rises from 26.19% 19% to 28.98% on the Essay dataset. Furthermore, the results consistently show that model-based methods generally surpass metric-based methods. For instance, FastGPT, the best-performing metric-based method in our experiment, is only comparable to ChatGPT-D on the Essay dataset and notably inferior to model-based methods on the DetectRL dataset. This highlights the significance of enhancement targeted Interestingly, we also observe that cross-LLM detection performance is not always inferior to within- LLM detection. For example, Table ssooss RADAR performing worse texts generated by PaLM compared to ChatGPT. We reasonably suspect that this may be related to the quality of the generated text, and the generated by ChatGPT is more discriminative.
approaches.
Paragraph-level Detection. Tables D (right) and (bottom) show the paragraph-level detection per- formance in terms of TPR @ FPR-1% where detectors trained on PaLM and GPT4AIL, respectively. For performance on AUROC and additional results trained on other LLMs, please refer to Tables 6 and 24 34 in the Appendix. We observe enhancing effects similar to those the sentence level, highlighting the proposed method's benefit at this granularity. Moreover, by comparing detection performance the paragraph level and the sentence level, it is evident that sentence-level detection is significantly more challenging, similar to the finding in [17]. This underscores the significance of sentence-level detection and encourages it to be primary focus for future studies.
Mixed Text Detection. Fig- illlstraaes the test performance (TPR @ FPR- 1 1%) on explicit mixed text, and the corresponding AUROC performance can be found in Fig. [2] in the Appendix. Our experiments were conducted under two settings: training on mixed text and training on original text. Compared to the performance improvements shown in Table I (right), the enhancements in detecting mixed text are more pronounced. This may be attributed to the inherently less precise hard labels in mixed text, the core issue our proposed strategy directly addresses, thereby demonstrating greater potential for improvement and highlighting the rationale behind our design. Besides, comparing the detectors trained mixed text (left group) and original text (right group). it can be found that the former has better robustness in detecting mixed texts, which is akin to adversarial training (40).
Paraphrasing Text Detection. Recent studies have shown that MGT detectors are vulnerable to paraphrasing attacks [4] 42 where paraphrased text can bypass detection. To assess the robustness improvement in adversarial scenarios, explored the detection performance texts subjected to

---

100
Table 3: Running time (seconds).
6:58
1:21
15
3,07
3.43
1:00
Paragraph-level
Training Time
Inference Time
Trakk Dur
AUAGC
-259
2A4
|Essay
DetectRL
Essay
DetectRL
0.94
6:55
1.41
-122
3.69
Likelihood
9.3
12.1
41.5
54,2
-2.0
Log Rank
11.2
13.8
50.3
62.0
ye
Entropy
7.6
34.8
47.8
NPR
308.0
527.0
1384.7
2369.2
DetectGPT
299.2
577.3
1345.0
2595.5
FastGPT
32.1
37.1
144.2
166.6
CharGPT-D
18.2
31.7
4.7
8,4
Figure 6: The performance improvement of the enhanced
ChatGPT-E
20.9
32.8
8.4
RADAR-E compared RADAR. The figure shows their differences for AUROC (left) and TPR @ FPR-1 1R-1% (right). Positive values indicate performance improvement.
MPU
17.9
31.9
4.7
8.3
MPU-E
19.2
33.1
4.7
8.3
RADAR
29.8
46.1
7.9
13.3
RADAR-E
32.0
48.4
7.9
13.4
detector KD
0,7
detector
detector
detector
0.93
0.50
Durs
0.5
0.6
KD
Durs
U
KD Ours
Ours
KD
0.4
0.84
0.81
ChaRSPT
0.30
Method
Method
Mathod
Mathod
DatactRL Dataset
Essay
Figure 7: Performance comparison with Knowledge Distillation.
two common paraphrasing attacks: Polish and Back Translation 32 as illustrated in Fig. and Fig. in the Appendix. Notably, detectors employing the proposed enhancement framework exhibited greater robustness in these attack scenarios. Compared to non-attack performance (Table [ right), performance generally declined under attack. This not only reaffirms the vulnerability of existing MGT detection methods but also underscores the critical need to enhance detector robustness.
evaluated across four high-risk domains: arXiv Archive, Writing Prompts, XSum, and Yelp Reviews. Results are presented in Fig. [6] with additional results in Fig. 14 in the Appendix. In most settings, models employing the proposed strategy demonstrate significantly enhanced generalization capabilities. This is likely attributable to our framework reducing the model's reliance on inexact hard labels, thereby mitigating overfitting.
Cross Domain Detection. Cross-domain performance
Running Time. Table 3 presents the runtime comparison in paragraph-level settings (sentence-leva results are in Appendix Table 36). As discussed in Section 3.3 since the target detector is not modified, the proposed framework introduces no additional inference delay. Regarding training overhead, the longer text for the supervisor is constructed within the batch, and the intermediate results f(x(5), 0y) from the detector data preparation phase. The primary training delay is the supervisor's forward and backward passes. However, as the supervisor model (i.e., three-layer fully connected network in the experiments) is significantly simpler than detectors, the overall training delay is negligible.
be reused. This ensures negligible overhead during the
4.2 Comparison with Knowledge Distillation
Fig. presents the performance comparison between the proposed framework and Knowledge Distillation (KD), which is based on the "strong teacher enhances weak student" concept. Here, we selected RADAR, which has the best detection performance in the experiment, the teacher model to guide ChatGPT-D and MPU. The results indicate that knowledge distillation even led to a decrease in the performance the student models in most settings. This is primarily because the effectiveness of knowledge distillation largely depends on the quality of the teacher model (i.c., high-quality soft labels). However, obtaining such teacher model is challenging in complex MGT detection scenarios.
4.3 Supervision Quality Assessment
To further verify whether the supervisor guides the detector to learn more accurate "golden" labels for enhancement, we introduce a knowledge distillation-banl analysis method for verification. Specifically, we use the detectors before and after enhancement as teacher models, distill student

---

Table 4: The impact of teacher model (RADAR) enhancement on Knowledge Distillation.
Method
DetectRL
Essay
PaLM
ChatGPT
Claude
Llama-2
GPT4AIl
ChaiGPT
ChaiGPT-turbo
CharGLM
Dolly
Claude
Originn KD
66.01
73,28
15.77
83,66
69.34
41.29 42.53
93.97 93.97
24.92
64.00
5.47
Ours-KD
67.49
76.00
16.81
84.77
70.11
25.68
65.47
6.00
Table 5: Case study in RADAR detector.
MGTs that detected by RADAR RADAR-D
MGTS can only be delected by RADAR-D
Case
When thought of comment. wrote Milio's, s. and I thought meh. This is just anotber pizza shop. But after eating there. realized that Milio was not just another Pizza shop. The food is delicious, the service very good. and the atmosphere is warm and seductive.
The atmesphere excellent with lots of big screen TV's to watch the game. The service was devent as well. The waitress was very friendly and watchfull, but kitchen secmeed backed up because our food took over an hour to come cul.
Case
The two-dimensionall kagome lattiee has been identifled as a promising platform realize quantum spin liquid states duc to its unique geometry. In this paper, sepeet the syn- thesis and characterization of series of two-dirnensional kagome antiferomagnets, ZnxCu4 x(DD)6112 ...
Sussex turther improved their hopes of reaching the One- Day knockout stuges by beating Surrey for third consecutive regrouped win....
Case Scientists, intrigued by my resilience, devised daring plan to send me into the ooigmatic depths of a black hole. Embracing my inner badass, agreed, embarking on a journey that would test the ilmits of my extraoniiary ahilities...
This place should have scas of glowing reviews! My wife and I had great tife. It's hard believe that such grea restaurant could be strip mall....
model respectively. By comparing these student models' performance, we can verify the quality of the supervisory signal. If the student distilled from the enhanced detector outperforms the student distilled from the original detector, it indicates that our approach has indeed enabled the detector to learn better detection scores from the supervisor, thereby improving detection performance. Table 4 presents the distillation performance, where Origin-KD and Ours-KD denote the original RADAR detector and our enhanced RADAR (i.e., RADAR-E) guided MPU detector, respectively. By comparison, we observe that Ours-KD achieves superior performance, proving that the proposed method learns better detection scores from the supervisor to enhance detection.
4.4 Case Study
We prepare some examples for quantitative analysis, as shown in T Specifically, the table shows sequentially selected three MGTs from the DetectRL dataset. The text in the left column is accurately identified as "MGT" by both the bascline version of RADAR and our improved version RADAR-E. The text in the right column represents "win" cases for our approach, that is, they are misclassified by RADAR but correctly detected by RADAR-D. We can find that the easy-to-detect texts (left column) typically exhibit more complex idioms, formal vocabulary, and convoluted expressions, which are typical of machine- machine-generated text. Instead, the cifficull-to-deter texts (right column), which our method successfully detects, are characterized by short, plain, straightforward, and somewhat colloquial sentences, which much closer to typical human-like distribution. The characteristics of difficult-to-deted text and the effectiveness of the enhancement strategy intuitively demonstrate our advantages in processing ambiguous machine-generated text.
5 Conclusion
Existing detection methods are limited by inaccurate labels leading to inexact training, while the limi- tations of human cognition and the superintellienne of detectors make inexact learning widespread and inevitable. To address this, the paper has proposed an easy-to-hard supervision enhancement framework. This framework utilizes easier supervisors (weaker models trained on simpler long-text text detection tasks) to enhance the more complex target detection model. By structurally integrating the detector into the supervisor's design, we model the supervisor as lower performance bound for the detector. Optimizing the supervisor thus indirectly optimizes the detector, leading to improved alignment with the underlying golden labels. Extensive experiments across diverse practical scenarios demonstrate that this framework significantly improves MGT detection capabilities.
10

---

Acknowledgment
This work was supported by the RGC Senior Research Fellow Scheme under the grant: SRFS2324- 2S02, RGC Young Collaborative Research Grant No. C2005-24Y, RGC General Research Fund No. 12200725, and NSFC General Program No. 62376235.
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint 23xiv:2303.08774, 2023.
[2] Qingsong Wen, Jing Liang, Carles Sierra, Rose Luckin, Richard Tong, Zitao Liu, Peng Cui, and Jiliang Tang. Ai for education (ai4edu): Advancing personalized education with Ilm and adaptive learning. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6743-6744, 2024.
[3] Keivalya Pandya and Mehfuza Holia. Automating customer service using langchain: Building custom open-source gpt chatbot for organizations. arXiv preprint arXiv:2310.05421, 2023.
[4] Canyu Chen and Kai Shu. Combating misinformation in the age of Ilms: Opportunities and challenges. Al Magazine, 45(3):354-368, 2024.
[5] Sayak Saha Roy, Poojitha Thota, Krishna Vamsi Naragam, and Shirin Nilizadeh. From chathots to phishbots?: Phishing scam generation in commercial large language models. In 2024 IEEE Symposium on Security and Privacy (SP), pages 36-54. IEEE, 2024.
[6] Leonard Salewski, Stephan Alaniz, Isabel Kio-Torto, Eric Schulz, and Zeynep Akata. In- context impersonation reveals large language models' strengths and biases. Advances in neural information processing systems, 36:72044-72055 2023.
[7] Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah Smith. All that's 'human'i is not gold: Evaluating human evaluation of generated text. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1 1ihh International Joint Conference Natural Language Processing (Volume I: Long Papers), pages 7282-7296, 2021.
[8] Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Worbert-Voss.. Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps, et al. Release strategies and the social impacts of language models. arXiv preprint arXiv: 1908. 09203, 2019.
[9] Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. Gltr: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics,
2019.
[10] Hanxi Guo, Siyuan Cheng, Xiaolong Jin, Zhuo Zhang, Kaiyuan Zhang. Guanhong Tao, Guangyu Shen, and Xiangyu Zhang. Biscope: Ai-generated text detection by checking memorization of preceding tokens. Advances in Neural Information Processing Systems, 37: 17406655-000000 2024.
[11] Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher Manning, and Chelsea Finn. Detectgpt: Zero-shot machine-generated text detection using probability curvature. In Interna- tional Conference on Machine Learning, pages 24950-24962. PMLR, 2023.
[12] Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: Efficient zero-shot detection of machine-generatee text via conditional probability curvature. In The Twelfth International Conference on Learning Representations.
[13] Cong Zeng, Shengkun Tang, Xianjun Yang, Yuanzhou Chen, Yiyou Sun, Zhiqiang Xu, Yao Li, Haifeng Chen, Wei Cheng, and Dongkuan DK Xu. Dald: Improving logits-based detector without logits from black-box llms. Advances Neural Information Processing Systems, 37:54947-54973, 2024.

---

[14] Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng, Mare' Aurelio Ranzato, and Arthur Szlam. Real or fake? learning discriminate machine from human generated text. arXiv preprint arXiv:/006.00 1906.0335, 2019.
[15] Wanjun Zhong, Duyu Tang, Zenan Xu, Ruize Wang, Nan Duan, Ming Zhou, Jiahai Wang, and Jian Yin. Neural deepfake detection with factual structure of text. arXiv preprint 2020.
[16] Vivek Verma, Eve Fleisig, Nicholas Tomlin, and Dan Klein. Ghostbuster: Detecting text ghost- written by large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 1702-1717, 2024.
[17] Pengyu Wang, Linyang Li, Ke Ren, Botian Jiang, Dong Zhang, and Xipeng Qiu. Seqxgpt: Sentence-level ai-generated text detection. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
[18] Bushra Alhijawi, Rawan Jarrar, Aseel AbuAlRub, and Arwa Bader. Deep learning detec- tion method for large language models-generated scientific content. Neural Computing and Applications, 37(1):91-104 2025.
[19] Xiaomeng Hu, Pin-Yo Yu Chen, and Tsung- Yi Ho. Radar: Robust ai-text detection via adversarial learning. Advances in neural information processing systems, 16:75577-5 2023.
[20] Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding. Jianwei Yue, and Yupeng Wu. How close is chatgpt to human experts? comparison corpus, evaluation, and detection. arXiv preprint 1.Xiv:2301.07977, 2023.
[21] Ekaterina Artemova, Jason Lucas, Saranya Venkatraman, Jooyoung Lee, Sergei Tilga, Adaku Uchendu, and Vladislav Mikhailov. Beemo: Benchmark of expert-edited machine- machine-generate generated outputs. arXiv preprint arXiv:2411.04032 2024.
[22] Rafael Müller, Simon Korblithh and Geoffrey E Hinton. When does label smoothing help? Advances in neural information processing systems, 32, 2019.
[23] Souradip Chakraborty, Amrit Bedi, Sicheng Zhu, Bang An, Dinesh Manocha, and Furong Huang. Position: On the possibilities of ai-generated text detection. In Forty-first International Conference on Machine Learning, 2024.
[24] Katy Ilonka Gero and Lydia Chilton. Metaphoria: An algorithmic companion for metaphor creation. In Proceedings of the 2019 CHI conference on human factors in computing systems, pages 1-12, 2019.
[25] Katy Ilonka Gero, Vivian Liu, and Lydia Chilton. Sparks: language models. In Proceedings of the 2022 ACM Designing Interactive Systems Conference, pages 1002-1019,202
Inspiration for science writing using
[26] Shuhai Zhang, Yiliao Song, Jiahao Yang. Yuanqing Li, Bo Han, and Mingkui Tan. Detecting machine- machine-generatee generated texts by multi nopliipoppulttion aware optimization for maximum mean discrep- ancy. In The Twelfth International Conference on Learning Representations.
[27] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprini arXiv: 1003.0253, 2015.
[28] Bo Han, Quanming Yao. Tongliang Liu, Gang Niu, Ivor W Tsang, James Kwok, and Masashi Sugiyama. A survey of label-noise representation learning: Past, present and future. arXiv preprint arXiv:2011.044 1,04066 2020.
[29] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware min- imization for efficiently improving generalization. In International Conference on Learning Representations.
[30] Eric Jang, Shixiang Gu, and Ben Poole. Categorical reparametrization with gumble-softmax. In International Conference on Learning Representations (ICLR 2017). OpenReview. net, 2017.
12

---

[31] Sadri Hassani and Sadri Hassami. Dirac delta function. Mathematical Methods: For Students of Physics and Related Fields, pages 139-170, 2009.
[32] Junchao Wu, Runzhe Zhan, Derek Wong, Shu Yang. Xinyi Yang, Yulin Yuan, and Lidia Chao. Detectrl: Benchmarking llm-generated text detection in real-world scenarios. Advances in Neural Information Processing Systems, 37:100369-100401, 2024.
[33] Jinyan Su, Terry Zhuo, Di Wang, and Preslav Nakov. Detectllm: Leveraging log rank infor- mation for zero-shot detection of Computational Linguistics: EMNLP 2023, pages 2335-12442, 2023.
text. Findings of the Association for
[34] Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: Efficient zero-shot detection of machine generated text via conditional probability curvature. In ICLR, 2024.
machine-generated
[35] Yuchuan Tian, Hanting Chen, Xutao Wang, Zheyuan Bai, QINGHUA ZHANG, Ruifeng Li, Chao Xu, and Yunhe Wang. Multiscale cesitive-unlabeled detection of ai-generated texts. In The Twelfth International Conference on Learning Reprexentations.
[36] Brian Tufts, Xuandong Zhao, and Lei Li. An examination of ai-generated text detectors across multiple domains and models. In Neurips Safe Generative Al Workshop 2024.
[37] Kathleen C Fraser, Hillary Dawkins, and Svetlana Kiritchenko. Detecting ai-generated text: Factors influencing detectability with current methods. arXiv preprint arXiv:2406. 2024.
[38] Abhimanyu Hans, Avi Schwarzschild, Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting Ilms with binoculars: Zero-shot detection of machine-generaaae text. In International Conference on Machine Learning, pages 17519-17537, PMLR, 2024.
[39] Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang. Linyi Yang, Shuming Shi, and Yue Zhang. Mage: Machine generated text detection in the wild. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 36-53, 2024.
Machine-generated
[40] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
[41] Kalpesh Krishna, Yixiao Song. Marzena Karpinska, John Wicting, and Mohit lyyer. Paraphras- ing evades detectors of ai-generated text, but retrieval is effective defense. Advances in Neural Information Processing Systems, 36:27469 36:27469-27500, 2023.
[42] Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil
Feizi. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303./11/56 2023.
[43] Nagarajan Natarajan, Inderjit S Dhillon, Pradeep K Ravikumar, and Ambuj Tewari. Learning with noisy labels. Advances neural information processing systems, 26, 2013.
[44] Bo Han, Quamming Yao, Xingrui Yu, Gang Niu, Miao Xu, Weihua Hu, Ivor Tsang, and Masashi Sugiyama. Co-teaching: Robust training of deep neural networks with extremely noisy labels. Advances in neural information processing systems, 31, 2018
[45] Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimization. International Conference on Learning Representations, 2018.
[46] Kathleen Fraser, Hillary Dawkins, and Svetlana Kiritchenko. Detecting ai- ti-generated text: Fac- tors influencing detectability with current methods. Journal of Artificial Intelligence Research, 82:2233-203 -278,, 2025.
[47] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. watermark for large language models. In International Conference on Machine Learning, pages 17061-17084, PMLR, 2023.
13

---

[48] Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. Provable robust watermarking for ai-generated text. In The Twelfth International Conference on Learning Representations.
[49] CHEN Liang, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, and Kam-Fai Wong. Watme: Towards lossless watermarking through lexical redundancy. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024.
[50] Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. On the learnability of watermarks for language models. In The Twelfth International Conference on Learning Representations.
[51] Yepeng Liu and Yuheng Bu. Adaptive text watermark for large language models. In International Conference Machine Learning, pages 30718-30733 PMLR. 2024.
[52] Hanlin Zhang, Benjamin L Edelman, Danilo Francati, Daniele Venturi, Giuseppe Ateniese, and Boaz Barak. Watermarks in the sand: Impossibility of strong watermarking for generative models. In ICLR 2024 Workshop on Secure and Truslworthy Large Language Models.
[53] Nikola Jovanović, Robin Staab, and Martin Vechev. Watermark stealing in large language models. In International Conference on Machine Learning, pages 22570-22593, PMLR, 2024.
[54] Yang Xu, Yu Wang, Hao An. Zhichen Liu, and Yongyuan Li. Detecting subtle differences between human and model languages using spectrum of relative likelihood. In Proceedings of EMNLP, pages 0010--10211, 2024.
[55] Xianjun Yang, Wei Cheng, Yue Wu, Linda Ruth Petzold, William Yang Wang, and Haifeng Chen. Dna-gpt: Divergent n-gram analysis for training-free detection of gpt-generated text. In ICLR, 2024.
[56) Cong Zeng, Shengkun Tang, Xianjun Yang, Yuanzhou Chen, Yiyou Sun, Zhiqiang Xu, Yao Li, Haifeng Chen, Wei Cheng, and Dongkuan Xu. Improving logits-based detector without logits from black-box llms. CoRR, 2024.
[57] Hoang-Quee Quoc Nguyen- Son, Minh-Sov Dao. and Koji Zettsu. Simllm: Detecting sentences generated by large language models using similarity between the generation and its re-generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22340-22352, 2024.
[58] Xiao Yu, Kejiang Chen, Qi Yang. Weiming Zhang, and Nenghai Yu. Text fluoroscopy: Detecting Ilm-generated text through intrinsic features. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15838-15846, 2024.
[59] Shixuan Ma and Quan Wang. Zero-shot detection of Ilm-generated text using token cohesiveness. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17538 17553, 2024.
[60] Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavskii, Sergey Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Piontkovskaya. Intrinsic dimension estimation for robust detection of ai- ti-generated texts. Advances in Neural Information Processing Systems, 36:39257-39276, 2023.
[61] Yann LeCun, Sumit Chopra, Raia Hadsell, M Ranzato, Fujie Huang, et al. A tutorial on easeg learning. Predicting structured data, 1(0), 2006.
[62] Zhongping Zhang, Wenda Qin, and Bryan Plummer. Machine-generated text localization. In Findings of the Association for Computational Linguistics ACL 2024, pages 8357-8371, 2024.
[63] Hyunseok Lee, Jihoon Tack, and Jinwoo Shin. Remodetect: Reward models recognize aligned Ilm's s generations. In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
[64] Xun Guo, Yongxin He, Shan Zhang, Ting Zhang, Wanquan Feng, Haibin Huang, and Chongyang Ma. Detective: Detecting ai- ai-generated text via multi-level contrastive learning. Advances in Neural Information Processing Systems, 37:88320-8823 88347, 2024.
14

---

[65] David Aldous. Random walks on finite groups and rapidly mixing markov chains. In Séminaire de Probabilités XVII 1981/82: Proceedings, pages 243-297. Springer, 1983.
[66] Xinlei He, Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. Mgtbench: Bench- marking machine-generated text detection. arXiv preprint arXiv:2303, 14822 2023.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have summarized the position and key contribution of the paper in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have discussed the limitations in Appendix A.7
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
The assumptions and proof are provided in Section and Appendix C
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
The technical details of implementation are introduced in Appendix D.4 In addition, the source code is available at hhttps://ggiiiei wtw.luu...ooottte www/tmlrlmmm//e/... Caroup/Easy2Pie
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

Answer: [Yes]
Justification: The source files are publiclly available at https://githut com/
tmlr- tmir-grouph arrupa
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm ec/
for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
//nips. //nipscc//pie wwwppbblccgguiicuceccccccoeooopp wet.ggddddeeoeeebbiniinn
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The details of experiment setups and implementation in Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification:
We perform five independent repeated experiments and report error bar.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
18

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have provided the running time in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:p///..eeee....co//////////a
Answer: [Yes]
Justification: We have carefully checked the NeurIPS Code of Ethics and confirmed that our paper obey it.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: have discussed the broader impacts Appendix A.8
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We have cited the original paper that produced the dataset.
Guidelines:
The answer NA means that the paper does
existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper is not about crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper is not about research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: In MGT detection, LLM is used to generate MGT for training and testing.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: for what should or should not be described.
22

---

Appendix
Further Discussion
24
A.1
The solution to Inexact Supervision
24
A.2
Scalability of the Proposed Approach
24
A.3
Different from Knowledge Distillation
24
Challenge of Noisy Label Learning
25
A.5
Reasonableness of the Golden Label Assumption
25
A.6
Differences from Existing Theoretical Results
26
Limitation
26
A.8
Broader Impact
26
Related Work
26
Watermark baasdd Method
26
B.
2
Metric-based Methods
27
B. 3
Model-based Methods
27
Theoretical Analysis
28
Proof of Theorem
28
Proof of Theorem 3.2
30
Proof the Theorem
30
4 Proof of Theorem
30
Additional Experiments
31
Datasets
D.2
Baselines
31
D33
Evaluation Scenario
................................
32
D.4
Implementation Details
33
D.5
More Results of Boundary Fuzziness
34
D.6
More Performance Comparisons
34
D.7
Performance under Noisy Labels
36
D.8
Comparison with Noisy Label Learning
36
D.9
Sensitivity Analysis
36
D.10
Ablation Study of Supervisor
39
23

---

A Further Discussion
A.1 The solution to Inexact Supervision
In the introduction, emphasized that this paper aims to learn effectively from the supervisor when (1) the provided label is inexact, and (2) the detection quality is difficult to assess. Therefore, proposed easy-to-hard enhancement framework. Here, we systematically discuss how the proposed framework solves them.
To address the first challenge, we design supervisors to focus on the relatively easier task of longer text detection. Theoretically. longer texts have greater class distribution distance (Theorem 3.1). which helps mitigate the negative impact of inexact labels. Additionally, we collapse the HGT distribution as a Dirac bg distribution, which theoretically increases the distribution distance (Theorem 3.3 further aiding the supervisor's learning.
To address the second challenge, we do not employ the method of supervisors directly providing soft labels to detectors, as is common in knowledge distillation. This is due to the evaluation difficulty and the fact that the supervisor may be weak and cannot guarantee the quality of the provided soft label. Instead, we structurally integrate the detector into the supervisor's 's design, establishing a connection between the supervisor and the detector, where the supervisor's 's performance serves as a lower bound for the detector's capabilities. This coupled structure allows the detector to be indirectly optimized via the supervisor, avoiding the larger error caused by directly providing label-based signals. Accordingly, the enhancement roadmap is: maximize supervisor performance -> maximize k -> maximize detector performance.
A.2 Scalability of the Proposed Approach
We examine the scalability of approach to large-scale scenarios from the perspectives of computa- tional compatibility and framework generality.
Computational compatibility. key aspect of our framework its computational efficiency. Incorporating an auxiliary supervisor throughout the training process adds negligible overhead, as shown in Table 3] This supervisor can be designed as a lightweight component that avoids bottlenecks, ensuring seamless integration into standard large-scale training pipelines with large models and massive datasets. In essence, the bias is in the training objective (Eq. 5). not restrictive architectural change that would hinder scaling.
Framework generality. The proposed framework applies model-basee approaches that rely on data and computing power rather than metric-based methods centered on specific features. This reflects the philosophy of "The Bitter cesson"-instead instead of forcing explanations of the detection mechanism through specific features, we learn to construct the most effective representations for detection using data and computation. Moreover, the enhancement framework is not built narrowly designed knowledge; rather, it represents a general learning architecture under inexact supervision by using easier supervisor to refine labels for the target model. Its applicability may extend beyond text detection to any inexact supervision problem characterized by inherently ambiguous labels.
Essentially, our framework employs an efficient computational method (training the supervisor on longer texts) to provide better learning signals for another method (the detector), rather than imposing rigid, artificially designed bias that limits learning and scalable.
A.3 Different from Knowledge Distillation
The proposed framework exhibits significant differences from Knowledge Distillation (KD) 27 in several key aspects:
Model. KD is essentially "strong teacher- weak student" enhancement paradigm, where a typically more powerful and complex teacher model guides student model to improve performance or achieve model compression. In contrast, the proposed framework follows an easy-to-hard enhancement paradigm, enhancing the target detector through a supervisor that focuses on the relatively easier task of long text detection. Thanks to the relative simplicity of the task, the
24

---

supervisor model can often be designed in a more lightweight manner. For example, in the paper. it is modeled as a simple three-layer fully connected network, which significantly reduces complexity compared to detectors usually built upon complex pre-trained models.
Task. traditional KD, both the teacher and student models are typically trained and evaluated on the same task, with the teacher model providing guidance based on its superior performance in that task. However, in the proposed framework, although the overarching goal for both the supervisor and the detector is MGT detection, the tasks they focus on differ significantly in terms of difficulty and data characteristics. The supervisor deals with longer texts that are relatively easier to detect, while the detector handles the more challenging original texts.
Supervision Signal. The core of KD lies in using the soft labels output by the teacher model supervision signal to guide the student model, with the typical optimization goal being the minimization of the divergence between the output distributions of the student and teacher models. This to enable the student model to learn the decision boundaries and generalization capabilities of the teacher model. Its effectiveness depends on the teacher model providing high-quality soft labels. However, in MGT detection, the detector's superintelligence makes it unelear if the supervisor's ' soft labels offer superior information. Therefore, the proposed framework does not directly rely on the supervisor's soft labels. Instead, by structurally incorporating the detector into the supervisor, we consider the supervisor's performance as a theoretical lower bound for the detector's capabilities. Thus, optimizing the supervisor indirectly enhances the detector.
A.4 Challenge of Noisy Label Learning
is primarily designed to address incidental and limited labeling errors, whereas in MGT detection, the issue to be addressed is the widespread inexactness in labels (the categories are correct) due to limitations of human cognition. There are significant differences in both the problem and the techniques involved:
Noisy label learning (NLL) 29 28
Nature of the Problem. NLL assumes the existence of clear, discrete ground truth, viewing erroneous labels in the training set as occasional random errors, with its core focus being the correction of these errors, emphasizing label accuracy.
contrast, the imprecision in MGT detection stems from the ambiguity of the issue itself- the "machine-generated attribute of text is not always a clear binary judgment. The focus in MGT detection is on hard labels that do not sufficiently represent the sample, rather than labeling errors.
and sample selection [44), attempt to identify and prioritize training on "clean" samples while reducing or ignoring the influence of "erroneous" ones. This mechanism typically implies the assumption that noise is limited and identifiable, and the default fact that when the noise ratio > 50%, is impossible to learn without additional assumptions [44]. However, in MGT detection, due to the limitations of human cognition in recognizing inherent data ambiguity, these inexact samples are both prevalent and complex. Even when clearly distinguishable samples exist, it is challenging for human cognition to reliably filter them out. This makes it difficult for NLL techniques to adapt to MGT detection.
Technical Means. Classic strategies in NLL, such as robust loss functions [43 29
To further verification, this paper also experimentally evaluates the feasibility of noisy label learning, as shown in Appendix D.8
A.5 Reasonableness of the Golden Label Assumption
In MGT detection, traditional binary labels ("human-generated"" or "machine--eneratte" struggle to accurately reflect the degree of "machineness" in text. Considering the limitations of human cognition, perfect, clear golden labels might be impossible to define. To this end, recognizing that the degree of "machineness" largely depends on the explicit mixed text in human-machin collaboration scenarios, as well as the implicit mixed texts when LLMs generate human- like text, we naturally approximate MGT golden labels as the proportion of purely distinguishable from HGT (even if unknown). Compared to simple binary labels, this continuous proportional labeling can more finely and quantitatively capture the true characteristics of texts and their degree of "machineness". For example, piece of text might be 30% machine-edited, so using this proportion would approximate its golden label to 0.3, which indicates the degree of machine-likeness in this piece of text, information that cannot be represented by a binary label.
generrtte content
25

---

Undeniably, the text generation in practical scenarios can be highly complex and unstructured (e.g.. clause-level, word-level modification), making the simple use of proportion as an approximation of the golden label imperfect. However, in the current absence of more precise metries, this proportion- based label provides a relatively unbiased ground truth approximation that better reflects machine contribution. It aids models in understanding the distribution characteristics of data within con- tinuous label space, similar to Mixup [45]. Future research could further explore more precise quantification methods for defining the degree of "machineness" in texts for MGT detection.
A.6 Differences from Existing Theoretical Results
Our theoretical results build upon Chakraborty's 's foundational theory [23 but differ substantially in several aspects:
Lower bounds vs. upper bounds. While existing work 23 establishes critical theoretical lower bounds for detection, extend the theoretical upper bounds (Theorem and the right-hand side of Theorem 3.1)) to further explore the potential of detection. Although they are both related to text length, they measure different bounds.
Guidance for the proposed framework. In addition exploring the detection potential, it is more important to guide the design of the proposed framework: by coupling the influencing factors of the supervisor's upper bound with the detector, maximizing the supervisor's performance indirectly optimizes the detector, i.e., maximizing the supervisor's performance -> optimizing the influencing factors of the upper bound optimizing the detector. Instead, such guided optimization mechanism cannot be achieved with the theoretical lower bounds.
A.7 Limitation
The proposed framework in the paper is currently applicable enhancing model-based detection methods with relatively good performance. One future research direction could explore ways to effectively enhance metric-based MGT detection methods under inexact supervision learning conditions. Moreover, revvals that the proposed framework tends to lead the detector to converge predefined "golden" labels: the proportion of purely machine-generated content distinguishable from HGT. As discussed in Section A.5 in the absence of more precise metrics, although this approximate gold label provides relatively less biased ground truth compared hard labels, it is undeniably imperfect. Future research may focus on the quantification of more precise golden labels and the design of detection algorithms that approximate them.
A.8 Broader Impact
This paper presents work that aims to advance the field of trustworthy machine learning and large lan- guage models, specifically by improving machine- generated text detection. We not involve human subjects, potentially harmful insights, potential conflicts of interest and sponsorship, discrimination and bias concerns, privacy and security issues, legal compliance, or other ethical issues.
B Related Work
Existing MGT detection methods can be broadly categorized into three categories: proactive wntermarking-basee methods, post-hoc metric-based methods and model-based methods.
B.1 Watermark-based Method
Watermarks are embedded as subtle yet consistent information the inception of machine-generated text while maintaining expected grammar and semantics, which facilitate traceability by enabling
the detection of machine- gennrrtee text [46]. Kirchenbauer al. [47] randomly partition tokens into two categories, biasing the probability distribution to favor one category, resulting in higher frequency these tokens in watermarked text. This allows for watermark detection using statistical hypothesis testing. Furthermore, simplifying this approach to fixed word list has demonstrated greater robustness against paraphrase attacks (48). Chen al. [49] evaluated the impact of watermarks on different capabilities of large language models from cognitive science perspective, finding
26

---

that knowledge recall and logical reasoning are more adversely affected than language generation. To address this, they introduced Watermarking with Mutual Exclusion, dynamically optimizing token use during decoding by applying exclusion rules to recognized lexical redundancies. In addition to manually designing watermarks, leveraging the capabilities of language models to directly learn to generate watermarked text is promising. This includes training student models [50] and semantically invariant watermark models (51). While watermark-based methods exhibit strong detection capabilities, they are limited by the requirement for full access to the generation model, which hinders their practicality in detection tasks, as there is often no prior knowledge of the generation model, let alone the ability to modify its generation rules. Besides, they also vulnerable to attacks 52 531. which exacerbates the challenge of detection.
B.2 Metric-based Methods
Metric-based methods detect based on the statistical differences between generated and natural text. Mitchell et al. [I]] found that slight perturbations in generated text can result in rewritten text having a lower log probability than the original text, leading to the design of DetectGPT. Similarly, Solaiman et al. [8 performed detection based on the higher log probability of generated text compared to natural text. Additionally, using relative likelihood instead of absolute likelihood has proven to be more competitive than previous zero-shot methods [54]. Considering the intensive computational cost of DetectGPT, Fast-Detectte differences in token selection and used sampling to improve efficiency. DNA-GPT (55) divided the text into two parts and used them to generate the other part, performing detection by measuring the difference between the generated and the original. Since LLMs are often black boxes in practice 39), these methods use proxy model to extract features, leading to performance degradation due to distribution discrepancies between the proxy and target models. To address this, Zeng et al. [56] proposed a distribution-alige -aligned detection framework, ensuring enhanced detection capability and resilience against rapid model iterations with minimal training investment. Nguyen-Son et al. [57]
12 introduced conditional probability curvature to measure
observed that the similarity between original and generated texts is significantly higher than between generated and subsequently regenerated texts. Accordingly, they proposed SimLLM, which detects by estimating the similarity between input sentences and their generated counterparts. Yu et al. [58] captured intrinsic features of text by identifying layers with the greatest distribution differences when projecting onto lexical space, and using intrinsic rather than semantic features for detection has proven to yield better performance. Given that existing methods struggle with out-of-distribution data, token cohesiveness [59] has been shown to be good indicator. with LLM- ---enrerreed text often exhibiting higher token cohesiveness than suman-written text. Tulchinski et al. [60] discovered that the intrinsic dimension of text is a good metrie, generated text typically has average intrinsic dimension about 1.5 lower than natural text. Clearly, metrics-basee based methods are manually designed features based on limited data, casting doubt on their broader effectiveness.
B.3 Model-based Methods
Model-baset methods do not involve explicit feature extraction; instead, they leverage the powerful representation capabilities of deep learning models to implicitly learn distinguishable features by taking the entire text input. Energy-based based models [61], which perform well in continuous space, have been applied to text sequence detection, demonstrating better adaptability to changes in LLM architectures [14]. SeqXGPT [17] focuses on sentence-level detection, employing detection framework that uses the probability list from a white-box LLM as detection features. Al-Catcher [18] integrates multilayer perceptrons and convolutional neural networks learn statistical features and high-level semantic representations for detection. Addressing the challenge of limited information in short text detection, Zhang et al. [62] utilized contextual information simultancously predict whether multiple sentences were written by machines or humans. Zhong et al. (15) represented the factual structure of a given document as an entity graph, where adjacent nodes denote sentences with consistent relationships, and employed graph neural networks to learn sentence representations,
combining them into the document representation for prediction. Hu et al. [19 proposed RADAR. a robust detector resistant to paraphrasing attacks, by employing adversarial learning between a paraphraser and a detector. Lee al. (63) observed that LLM-ggeerated text has higher estimated preference alignment compared to human-written text, making it easily detectable by utilizing LLMSs trained to mimie human preference distributions. Guo et al. [64] posited that the key to detection lies in distinguishing different authors' writing styles, proposing a detection framework based on
27

---

multitask auxiliary and hierarchical contrastive learning. Considering the similarity between short texts and human texts, MPU [35 treated short texts as partially unlabeled and adopted multiscale positive-unlabeled strategy for training. These model-based methods, when supervised on specific datasets, demonstrate higher effectiveness and robustness 32
C Theoretical Analysis
C.1 Proof of Theorem 3.1
Theorem. 3.1 (Distribution Difference for Longer Text). Let h(8) and m(s) be the distributions for S, respectively, with the total variation
Staman-generaten and machine- machine-generated sequences distance (Vm,,,) 5 > O. For the text contains n. sequences, let a 2 0 denote the ratio of human-like component incorporated MGT. For longer text formed by concatenating independent length-n texts, the total variation distance between their distributions, TVoong can be bounded by:
nk(11 - =x) S long 11 11-11115)))) 5))
1 - --2epp(-
Proof. We will prove the upper and lower bounds of the total variance distance of longer texts TViang respectively.
Upper bound. First, we introduce necessary lemma to help our proof.
Lemma C.1 (Coupling Lemma of P and Q
Suppose that P and Q are given. For every coupling (X,Y Y)
TV (P.Q) = in * Y) (X,Y) is coupling of (P, (P,)).
direct corollary of this lemma that for any coupling (X,Y) of (P, Q), there is:
TV(P, Pr(X YY).
For the longer text containing k texts, each of which contains n sequences, therefore, it is natural that the longer MGT is i.i.d. sampled from the m xnk where m Xmk := m m ... m (nk times) denotes the product distribution. Similarly, longer HGT is i..d sampled from the h xnn In our setting, HGT with a ratio is mixed into MGT. Therefore, the longer MGT is revisited as m xl xank
Based on the Coupling Lemma, to obtain the upper bound, we need to construct random pair (xx(oo)nk xonk and Y ~ h xnn and
(X = ((X,,.... Xtn), (Yi,..., (E.))) so that then calculate Pr(X + Y). we can construct this coupling by coupling each component (X;, Yi) independently.
Specifically, by the Coupling Lemma, there exists coupling (U, V) of m and h such that Pr(U *f V) TV(m, h) 8. We can choose such coupling (U, V). Then, we construct coupling for (X, Y) as follows:
For the first (1 a)nk components (i I,..., (1 a)nk): Let (X,, Y.) be drawn independently and identically from (U, V). Thus, X; ~ m and Y, ~ h. and Pr (X, o Y:) = 6.
For the last onk components (i = (1 - a)nkk + 1,..., nk): we need X₃ ~ and Y, ~ h. The simplest coupling is to let X; Y; and draw this common value independently from the distribution h. Then, X; = Y ~ h and Pr (Xi + Y.) 0.
Now calculate the probability of X # Y under this coupling. X # Y iff there is at least one index i E (1....,nkk) such that X; + Y.
(x 4 Y) Pr (UPÉ (X: xt))
Using the probability of complementary events:
Pr(X # +Y)= - Pr (n {X, Y.))
28

---

Since (X,, Y1) pairs are constructed independently, the event {X, Y; is independent for different therefore:
n2
Pr (
(x,
According to our coupling construction:
For i 1, 1....., a)nk): Pr -X1==XX;=1111110 -
-. 8.
For i (1 a)nk I....,,kk nk Pr
1 =
Therefore.
(1-o)nk
) 1)-1 1
Pr(X YY=
II (1 66)
-1-16) -8)11-a)nk
(=(1-a)nk-1
Therefore, Pr(X * =)=1-(1 (1
8)(1-))kk and accordingly, TVtong Pr(X Y)
1 (1 - -8)01a)mm The upper bound is proved.
Lower bound. From the definition of total variance distance, we know that there exists a specific measurable subset A E S such that
Pr(s~mEA - =.
the probability of single sample drawn from falling into set i Pr h A) p. the probability of single sample drawn from m falling into A is Pr (r m A) p 8, where >
According to the proof of the upper bound above, the longer MGT can be considered as sampling set of (1 a )ank i.i.d. sequences (s: [+-[))n) from distribution m, and sampling a of onk i.i.d. nn ( (1-0)))k+++1 from distribution h. Then, the expected number of sequences belonging A is (q (1 )jkk.. Similarly, the longer HGT can be considered as sampling set of nı i.i.d. sequences 8,-=1 from distribution h, the expected number of sequences in A gnk. Then we can apply the Chernoff bound to have
sequences
Pr at least (a+ (1-a)) kn sequences of MGT are in. I exp
(1-0)28kn
2
and
Pr at most q (1 a))
a))
nk sequences of HGT are in A S exp
Now consider the even E' where nk sequences containing at least q ( (-aid) then have
sequences of A.
TViong > Pr LE|LLong MGT) - Pr (E) HGT)
1 e
exp
2exp
Therefore, the lower bound is proved.
29

---

C.2 Proof of Theorem 3.2
Theorem. (Detection Power for Longer Text). Under the assumption of Theorem 3.1 the
supervisor s SURROC supe. satisfies:
AUROC supe. 1
2 -1--1(nnht
2)2kk(1-a)
Proof. Invoking Proposition in existing work 23 we have
TV long
AUROCsupu S +
TVitng
6
Since the right-hand part is the monotonically increasing function of TV tong. combing the upper bound in Theorem 3.1 can bound
TV long
AUROC supr.
S
TV
long
(1 - (1 - ()(kkk)
- + 1 - (1-6) (2
2
=1 2 (1 5)2(1 2(1
The theorem is proved.
C.3 Proof the Theorem 3.3
Theorem. Distribution Difference after HGT Distribution Collapse). Under the assumption of Theorem 3.7 and assuming that m(0) - 0, then if h(s) collapses to Dirac delta distribution, we have lim (0) TV (h, m)
Proof. The assumption that h converges in distribution to Dirac measure ån implies that for the set A = (0). Pr(s E (0)) ) - do ((0)) )- = 1. We are given that Pr(s ~ m E {(}} - 0. Consider
the measurable A {0)- The difference in probabilities for this set tends to:
Pr(s h (0)) Pr(s m (0)) + 11 11 11
By the definition of the total variation distance supremum, TV(h, m) 2/ Pr(s h E {0}) Pr(s ~ m E {{}) J. Taking the limit, we get (0) TV(h, 2 1. Since the total variation distance between any two probability distributions at most (i.e., TV(h,m) I 1), we have limm(00)-00 TV(h, m) = 1. The theorem is proved.
C.4 Proof of Theorem 3,4
Theorem. 3.4 The Effectiveness of the Proposed Framework). Under the assumption of Theorem 3.1 and assuming that the MGT golden label is approximately the proportion of pure machine- generated content distinct from HGT. If the supervisor reaches the best possible one, the detector converges to the underlying golden labels.
Proof. According to our approximation of the golden label (i.e., the proportion of pure machine- generated content distinguished from HGT in MGT), the golden label of MGT is 1 a. In the proposed framework, we use Gumbel Softmax, whose mathematical expectation maintains the original prediction distribution of the detector. Therefore, under the expected behavior, for longer MGT, suppose that the predicted soft label of the detector is 1 a', then texts with a' proportion are filtered. Here we consider two cases:
a' > a. To maximize the supervisor's performance, it is necessary to retain the MGT part much as possible, that is, the HGT of proportion and the MGT with a' a proportion are filtered. After filtered, the longer MGT can be considered sampling set of a' nk i..d. sequences
30

---

(8t (8i)i=1 ( can be considered as any length, which is also set as (1 a' n'k i.i.d. sequences from distribution h. Similar the proof of Theorem 3.1 we have the total variance distance between longer MGT and HGT is TV long (1 h11-aa)na (nk) 1 (1 (1 Ink Furthermore, similar to the proof of Theorem 3.2 the supervisor's ' performance satisfies:
(1-a")nk from distribution For longer HGT, since it collapses to Direc dg distribution, it
1 (1=8))ak(1-a)))
AUROCsupe S 2
When the supervisor achieves the best possible one, a' should be minimum, i.e., a' a.
a' a. Similarly, to maximize supervisor's performance, the original HGT of a' proportion is fil- tered, and the MGT retains. Therefore, the total variance distance between longer MGT and HGT is
TV long = (TVmm(1-a)mt x (a=oa "jnk, hh(11-1 (I-aa))k (a-o' )nn) I TV(m* ((1-a)nk; h h(1- and the best supervisor also is AUROC supu. 1
8)2nk(1-a) when a'
In summary, when the supervisor is optimal, the detector's prediction probability for MGT is The theorem is proved.
D Additional Experiments
D.1 Datasets
A detailed description of the datasets used in the paper is as follows:
Essay [6]. The essay dataset comprises 1,000 text samples. The HGT samples are the original essays from IvyPanda, which spanned numerous subjects and educational levels (high school to university). To create the MGT samples, first employed ChatGPT-turbo to generate a specific prompt designed to align with the content of each original essay. This prompt then fed into various LLMs, including GPT4AIl, ChatGPT, ChatGPT--urbo, ChatGLM, Dolly, and Claude, which produced machine-generate essays in response. This generation strategy allowed for the generation of a diverse corpus of ALM-generated essays linked to the topics the initial source documents.
DetectRL [32]. It comprises human-written samples from four sources: arXiv academic abstracts (2002-2017), XSum news, Writing Prompts stories, and Yelp Reviews. These domains were specifically chosen because they are considered particularly susceptible to generating deceptive text when L.LMs are misused. For each source dataset, chooses 2,800 human-written samples HGT's. For MGTss, it selects four LLMs that widely used in the real world, including GPT-3.5- turbo (ChatGPT), PaLM-2-bison (PaLM), Claude-instant (Claude), and Llama-2-77b (Llama-2), to generate machine texts. In addition, the dataset includes various practical attack scenarios. The first is the paraphrase attack, which uses the Dipper paraphraser [4] and Google Translate's to rewrite the generated MGT. The second is the mixed text, which randomly replaces a quarter of the original machine- category.
machine-generated generated text with human-written written text, but its label
remains in the
D.2 Baselines
To verify the effectiveness of the proposed strategy, we compare it with metric- methods such as Likelihood, Log-Rank, Entropy, NPR, DetectGPT, and Fast-DetectGPT, as well model-based methods such as ChaIGPT-D, MPU, and RADAR.
Likelihood 8 It employs LLM to quantify the log probability at the token level. To derive detection score for a given text, these individual token log probabilities are then averaged. Notably, higher score suggests that the text is more likely have been machine-generater
Log-Rank [II]. It assigns a score to a text by averaging values derived from the log rank of each token. Specifically, for each token, based on the context that precedes a language model provides a rank for that word among its possible predictions. The logarithm of this predicted rank is then taken. The text's score is the mean of these log-rank values across words. It should be noted that lower score indicates higher probability of the text being machine-generated.
31

---

Entropy 19] Similar to the Log-Rank score, the Entropy score for a text is the average of the conditional entropy for each token, given its prior context. It worth noting that macchine-generatee texts are likely to have lower Entropy score.
DPt [1]]. It assesses text by quantifying how minor perturbations affect its log probability under the L.M. The core intuition posits that text generated by an LLM typically resides near local optima within the model's log probability landscape. Consequently, applying small alterations to model-generated text tends to result in a reduced log probability according to the model, compared the original text. Conversely, applying similar minor perturbations to human-written text does not consistently lead to a decrease and may result in the log probability being either higher or lower than that of the original text.
NPR [33]. Similar to DetectGPT, the Normalized Perturbed I Log-Rank (NPR) method also applies perturbations to the original text. The rationale behind NPR is that both MGTs and HGTs are susceptible to minor disturbances, which indicated by an increase in their Log-Rank score after such perturbations. However, this effect is notably more pronounced in MGTs.
DesttDete (FastGPT) 34] It reveals the limitation of DetectGPT's s intensive computational cost, uses the conditional probability curvature metric to identify the difference in token selection between LLMs and humans in given environment, and proposes to use a more efficient sampling step instead of the perturbation step of DetectGPT.
ChatGPT-D (20). It is built upon RoBERTa model that was fine-tuned using the HC3 dataset, and solely utilizes the pure answered text from the dataset.
The Multiscale Positive- Unlabeled (MPU) training framework attempts to solve the difficulty of short text detection without sacrificing long text. First, considers the similarity between short machine text and human text, regards this part of short text as partially "unlabeled", and reformulates Al text detection as partially positive unlabeled (PU) problem. Then, in this PU context, it uses length-sensitive multi-scale PU loss to train the detection model.
MPU 35
RADAR 19). It trains robust MGT detector through an adversarial learning strategy. Specifically, it includes paraphraser and a detector, where the paraphraser aims to generate real content to evade AI text detection, while the detector tries to detect this part of the content. Both sides improve the robustness of the model in this adversarial training environment.
The implementation of these bascline methods is mainly based on the MGT detection benchmark (66). while Gast-DetectGPT is based on the DetectRL benchmark (32). In order compare them fairly with our enhancement methods, we also fine-tune these model-based methods on the given dataset. The learning rate, training epochs and other parameter settings of the enhancement strategy are consistent with them, as shown in Appendix D..4 [.4
D.3 Evaluation Scenario
To comprehensively evaluate the effectiveness of the proposed framework, we conduct experiments
in the following practical scenarios:
Cross-LLM. We trained the detector on texts generated by one LLM and tested it on texts generated by various LLMs to evaluate its cross-LLM generalization performance. The main text presents our results of training on the DetectRL dataset based on PaLM and testing on other LLMs (see Table []. as well as the results of training on the Essay dataset based on GPT4All (see Tables 2 Additionally, we provide the complete results of training and testing texts generated by all LLMs in Tables 6 134Jin in the Appendix. These experiments cover both sentence-level and paragraph-level granularity, For sentence-level experiments, selected sentences with a token number of at least 5 as valid samples. For paragraph-level experiments, we used the original text without any modifications.
Cross-Domain The DetectRL dataset comprises four different domains: arXiv academic abstracts, XSum news articles, Writing Prompts stories, and Yelp Reviews. We use this dataset to evaluate the model's cross-domain performance by training the detector on texts from a source domain and testing it on texts from the remaining target domains. In this cross-domain evaluation setup, all MGTs are generated using the default PaLM model. Fig. and Fig. present the heatmap that visually illustrates the relative performance improvements of the proposed framework when applied to various baseline detection models.
32

---

Paraphrase Attack. It involves paraphrasing text to preserve its original meaning. We conduct experiments on the DetectRL dataset, which contains paraphrased data from Polish and Back Translation. As in the existing adversarial attack setting, we train on clean text and evaluate its robustness on paraphrased text. Specifically, the detector trained on the clean PaL.M texts and tested on the paraphrasing texts of various LLMs, with results shown in Fig. 5 and Fig. [3]
Mixed Text. In the real world, text is often not purely generated by humans machines indepen- dently, which makes mixed text very common in practice. The DetectRL dataset contains mixed text by replacing one quarter of the sentences in an LLM-generated text with human-writter text at random. In this scenario, we perform two settings: (1) The detector was trained on original mon-miiee) text and tested mixed text; and (2) The detector was trained and tested mixed text. Fig. and Fig. T2 show the performance under mixed settings, where for each sub-figure, the left group denotes detectors trained mixed texts, and the right group denotes detectors trained on original texts.
D.4 Implementation Details
Supervisor Implementation. a supervisor providing reliable guidance to the detector, we model it as a three-layer fully connected neural network in this work. Although it appears simple, this choice made after in-depth consideration of many aspects:
Performance. Firstly, providing reliable supervision signals depends the supervisor having satisfactory performance itself. Based on this consideration, there is a tendency to select models with higher expressive capacity as the supervisor. However, the core idea of the proposed enhancement framework is to improve the quality of input data by maximizing the supervisor's performance, thereby promoting the improvement of the detector. If the supervisor's model is werrparaaettriiedd it may prioritize adjusting its parameters to directly fit imperfect data, rather than effectively transmitting signals to the detector to improve data. This may weaken the enhancement effect. For example. if the number of original text segments that constitute longer text is 4 (i.e., k = 4), and the average accuracy of the current detector for these text segments is 50%, this means that the supervisor's 's long text has only 2 text segments on average, which is defective. In this case, an over-parameterizee supervisor model may directly this defective data through parameter memory, rather than improving this defective data through optimization (i.e., approaching the length of 4). Based on this consideration, the supervisor tends to simple model, which is inconsistent with the above considerations. Thus, as compromise, we adopt three-layer fully connected neural network. On the one hand, benefiting from the proposed two data quality enhancements (Theorem and and Theorem 3.3). even a three-layer network can achieve satisfactory performance, as shown in Appendix D On the other hand, the design of this simple model prevents the weakening of enhancement effects on the detector typically caused by over- over-parametee parameterized models.
Efficiency. As an enhancement framework, minimizing the introduced training delay is preferable. As discussed in Section 3.3 the longer text for the supervisor is constructed within the batch, and the intermediate results f ((,()) 0y) from the detector can be reused. This ensures negligible overhead during the data preparation phase. The primary training delay is the supervisor model's s forward and backward passes. This implies that simpler models can achieve reduced training delays. Therefore, chose a three-layer fully connected neural network, whose training delay is almost negligible, as shown in Table and Table[36]
In summary, using three-layer fully connected neural network as the supervisor is a comprehensive consideration of performance and efficiency. In the specific implementation of the supervisor, the size of the three hidden layers are 256, 64, and 2, respectively. The input of the supervisor needs to convert the text into the embedding, which is obtained by the tokenizer of the detector and using the embedding layer of the detector. Let e(x) represent the embedding of text ar then Eq. 3 rewritten as:
can be
e(x") = 33 (e(x)(0) @ (umbee(((((())))00)))
Here, the text slicing operator + is a vector concatenation operation.
Hyperparameter Setting. We fixed five different random seeds (1-5) and conducted five independent repeat experiments. For the DetectRI. and Essay datasets, we randomly selected 10% of the data as the training set, with the remaining 90% evenly divided into validation and test sets. To ensure
33

---

18.0
label
310
Labd
16.
Label
350
Labal
7.5
Nachine Human
Human
Machine
500
Humon
1.0
Machine
Human
Machine
2.5
150
150
-2.5
100
100
5.0
50
50-
-7.5
Probsbility
Probabillty
PbUM
Maxed PaLM Text
Figure 8: Boundary fuzziness evaluation between MGT (PaLM) and HGT, which illustrates the latent space distribution and prediction confidence distribution under pure (Sub-Fig, & 2) and mixed texts (Sub-Fig. & 4). The mixed text is to replace 1/4 of MGT with HGT.
15-
130
10
14-
300
250
Machine
500-
250-
2.00
10
0.0
0.2
0.4
0.6
0.5
1.0
10
5
10
0.0
0.2
0.4
2.6
0.5
Protublity
Probebilty
(4) Claude Tect
(b) Mixed Claude Teat
Figure 9: Boundary fuzziness evaluation between MGT (Claude) and HGT, which illustrates the latent space distribution and prediction confidence distribution of pure texts (Sub-Figure & 2) and mixed texts (Sub-Figure 3 & 4). The mixed text is to replace 1/4 of MGT with HGT.
fair comparison with baseline models, the enhanced models used the same hyperparameters as their corresponding base models. Specifically, all models set to 32. Regarding learning rates, we set 5e-6 for relatively smaller models like ChatGPT-D and MPU. For the larger RADAR model, found that a learning of 5e-6 led to unstable training, so a smaller learning rate of le-6 was chosen. For supervisor-related hyperparameters, the default settings
fine-tuned for epochs, with batch size
follows: the number of texts in longer texts (k 3), the number of longer texts per batch (N' = 128), and the weight (A 10). For performance analysis under more hyperparameter settings, please refer to Appendix D.9
D.5 More Results of Boundary Fuzziness
Continuing the discussion on the blurred boundaries between MGT and HGT mentioned in the intro- duction, Fig. Fig. 9 and Fig. 10 further show the boundary fuzziness evaluation results on more LLM texts. To enhance the persuasiveness of the visualization results, the analyses characterizing the latent space distribution and prediction confidence distribution performed best in our experiments. The analysis results across various LLMs consistently indicate that there is a general blurriness in the boundary between MGT and HGT.
based on the RADAR 119, which
Continuing the discussion in the introduction about the inexactness of hard-label -abeeebbb---edd training, Fig. 11 further presents experimental results mixed texts of additional LLMs. It can be observed that in most settings, even the simple application label smoothing can improve detection performance. This result also indicates that traditional hard-label-basee learning may be inexact.
D.6 More Performance Comparisons
Sentence-level Detection. Continuing the sentence-level detection setting discussed in Section (left) and Tables I provide detailed comparison of the performance of various detectors trained on texts generated by different LLMs the DetectRL and Essay datasets, respectively. From these tables, it can be observed that the experimental results are consistent with the main conclusions in the main text. Notably, in a total of 312 cross-LLM detection settings, the proposed enhancement strategy outperformed the corresponding baseline models in 87% of the cases. This widespread and consistent improvement highlights the general applicability and practical value of the proposed enhancement framework.
4.1
Tables 6 2
Paragraph-Ievel Detection. Continuing the paragraph-level detection setting Section 4.1 Table 6 12 (right) shows the performance comparison of detectors trained with various LLMs in the DetectRL
34

---

4.00
12-
Label
10
300
Human Machine
17.5
Human
Machina
200
12.5
Label
200
150
130
Probablitty
Lieme-2 Text
(b) Nised Llumar22 Teat
Figure 10: Boundary fuzziness evaluation between MGT (Llama-2) and HGT, which illustrates the latent space distribution and prediction confidence distribution of pure texts (Sub-Figure & 2) and mixed texts (Sub-Figure & 4). The mixed text is to replace 1/4 of MGT with HGT.
ChatGPT
ChatSPT
ChatsPT
Hard Scftt
ChatGPT
Solt
Claude
PaUM
FaLM
detector
FaLM
POUM
Detector
Hard
Hard Soft
H
Soft
3.0 0.1 0.2
0.3 D- 255
0.5 0.7
0.88 0.64
0.72
0:80
0.96
0.0
0.1
0.2
0.3
0.4
0.64
0.72
0.80 AUROC
0.88 0.96
TPR:FFF PR- 18
AUROC
TPRg (PR-
(a) PaLM Text Training
(b) Uama-2 Text Training
Figure 11: Performance comparison with and without using soft labels in mixed (1/4 of MGT was replaced with HGT). The detector is ChatGPT-D [20].
dataset, while Table 24 shows the performance comparison the Essay dataset. We can also get the same conclusion the main text. In addition, similar to the sentence-level setting, under the 312 cross-LLM settings, the proposed enhancement strategy outperforms the basic model in 85% the settings. This extensive enhancement effect is very valuable.
Mixed Text Detection. Continuing the mixed detection settings from Section 4.1 Fig. 12 presents performance comparison of various methods on mixed texts based on the AUROC metric. Similarly, compared the performance improvement on original texts (see the right part of Table (6, the proposed method demonstrates a more significant enhancement in detecting mixed texts, highlighting the rationality of our design.
Paraphrasing Text Detection. Section 4.1 has demonstrated robustness enhancement against paraphrase attacks in terms of TPR FIR-FPR-11% Here supplement with Fig. [3] showing AUROC performance under the same attack settings. Similarly, the AUROC metric also indicates that the proposed strategy enhances the robustness of the original detection models.
Cross Domain Detection. addition to the cross-domain performance on RADAR shown in Section 4.1 Fig. 14 i4iluustrssss the proposed strategy's enhancement of cross-domain performance for ChatGPP-D and MPU. Similar findings can observed, where the enhanced versions exhibit better cross-domain generalization capabilities in most settings.
Test on Newer LLMs. Based on the Essay prompts, we employed the latest LLMs GPT-4o, GPT- 4.1, ReepSee and Llama4 Maverick-to generate machine text. The detection results are shown Table 35] where detectors are trained on GPT4AIl texts from the Essay dataset. As can seen, our proposed enhancement strategy remains significantly effective these newer LLMss Moreover, compared to the LLMs used Table 2 the detection performance on these more advanced LLMs has declined, underscoring the ongoing arms race between detection and model development: detectors design stronger strategies based on current generation models, and then newly emerging LLMs produce higher-quality text that is harder to distinguish, thus driving further progress both sides.
Running Time. Table 36 presents comparison of the runtime (including inference and training time) of various detection models under the sentence-level setting. Similar to the paragraph-level setting, the additional training delay introduced by the enhancement strategy at the sentence level is minimal. This primarily due to the lightweight design of the supervisor model and the method of constructing long text data within batches during training.
35

---

Table Performance concerning AUROC (%) on DetectRL. The detection model is trained on text generated by PaLM.
Method
Senienoelovel
Paragraph-lovel
PaI.
ChaGGP
Claude
Llama-2
Arg.
Pal
ChaiGiT
Clonde
Lluma-2
Avg.
Likelihood
59.72+a
58 68.04+0.38
45.02-o.20
GT. 07.522
57.65
11.42+0.49
G8. 08.61+0.09
42.47 42.47+11.7
78.58-0.4
64.77
Log Rank
59.08+0
55 65.92en.62
44.28-0.21
67.71
56,74
71.64=0.49
65. 65.99+0.90
42. 12.1111.11
79.96-0.44
64.93
Entropy
49.4320.3 as
48.7335.78
49.37
28.08
60.73
63.08
61-73:13.56
66.21 06.2110.84
60.44
NPR
53.41+0.09
60.59+0.20
47.60-0.41
57.86 57.86+0.35
53.11
51.30+1.5a
52.81+1.35
42.88++.11
56.80-9.5s as
50.96
DetessGPP
53.08 61
54.311n.99
47.35_
56.93
52.91
57.92_0.72
50.0410.94
65.37
49.21_o.3.
55.63
Pa
62.43
65.8820.55
40.81
67.122...3
59.94:0.93
61. 10.26....3
24.14:1.00
70.39-0.44
53.93
ChatGPTDD
73.201
74.1412.01
6..72-1.73
78.18
73.06
82.61
82 32.23112.49
68.9311.89
92.43_1.00 1.61
81.55
ChatGPT-E
75.33
77.1222:91
68.33_0..8
79.0811..
75.09
83.98 28.07
83.50
69.81
92.50_9.30 38
82.45
MPI:
87 59
00.6661o.c
1..96_1.12 1.12
88.3310.55
97-11 10.02
98.07
3s
1b.51
98.98
00
93.02
MPL-E
89.54.0
92. 02.1822.:3
76.96-0..s
89.4320.8
87.03
97.880
98.7620
80.6210.0
99.27
94.11
RADAR
80.60
92. 72.0
26.29_0..2
67. 67.14.0.4
86.48
98.
98. 98.87
90.37.0079
98. 98.90_0.1
96. 59
RADAR-E
20.74
98.74_0.05
77.65
88.14
98.29
29.04.0.27
91.68-1.15
19.13-0.17
97.03
Table 7: Performance concerning AUROC (%) on DetectRL. The detection model is trained on text generated by ChatGPT.
Sentence-level
Paragraph-level
Method
PaLM
ChatGPT
Claude
Llama-2
Avg.
PaLM
CharGPT
Claude
Llama-2
Avg.
Likelibood
59.72.g
58.04_o.s
45.02.0.0
27.82
57.65
71.42
66.61
42.47.1.97
78.58
64.77
Log-Rank
59.06
55.92
11.28
67. 67.71
56,74
65.99 71.61
12.11gia
79. 99.961..45
64.93
Entropy
50.36 se
12.5620.15 65.89_o.0
56. 26.2820.45
53.77
60.73
63.08 os
51.7320.6
66.21
60.44
NPR
53.41
20
47.60
57 20.8
53.11
54.31 51. 51.69
88.4820.87
62.01
51.62
DetectGPT
63.06.2064
54.31=0.29
27.35
56 20.00
52.91
49.58 48.23
47.16-16.11
09.76
FastGPT
62.43
65.88_0.68
40.81
67. 12
56,56
61.26 59.91
21.14g1.06
70.3920.44
53.93
CharGPT-D
72.38
74.98_2.67
67.0
67.09=1.6a
1..8
73.07
84.04 79.36
69. 69.50=a.29
91.62+2.16
81.13
ChatGPT-E
73.64:096
76.91
68.
17.14
78.91 20.90
74.41
80.03:1.64
88.09
70.52:2.49
93. 13:11
82.94
MPU
84.21 1
91.25_00:0
28
89.11g0.48
86.61
99.14 91.22g0
18
79.14g1.02
99.15
MPU-E
85.24
95.15
15.15-011
79.5420.8
89.94
87.47
95.7840.35
90.58
82.6621.00
99.42..0.08
94.36
RADAR
85,00=0.43
95.05-..16
20.61
07.48
86.28
99.56 97.22::04:
11
86,43 86.13..079
99. 2..07
95.62
RADAR-E
85,91
95.63_0
78.28
88. 28.28
87.03
97.3420.3
99.65
86.87
99.361.013
95.80
D.7
Performance under Noisy Labels
Our framework assumes that hard labels are correct by default. To this end, this section verifies the detection performance in the presence of noisy labels. We randomly flipped 10% of the labels and then evaluated detectors trained these noisy data, as shown in Fig. [5] First, found that the proposed enhanced framework remained effective. Second, compared to the noiseless results in Table I our method is generally less affected by noise, verifying mitigation efforts.
D.8
Comparison with Noisy Label Learning
In addition to discussing the challenges of noisy label learning (NLL) techniques in Appendix
A.4
we also evaluated them experimentally. We selected two typical NLL methods: Co-teaching
44 and SAM (29). applying them to the ChatGPT-D baseline model for evaluation, denoted as
ChatGPT-Co and ChatGPT-SAM, respectively. The performance comparison is shown in Fig. [6
It can be observed that the direct application of these NI.L techniques not only failed to improve performance but, in most cases, even led decline in detector performance. This is mainly because the core objective of NLL techniques is to identify and correct limited erroneous labels in training data. However, in the context of MGT detection, the labels themselves are correct; the issue lies in widespread labeling inexactness. Therefore, NLL strategies designed based on the assumption of erroneous labels do not align with the nature of imperfect labels (ambiguity) in MGT detection, making them difficult to effectively apply to tasks aimed at enhancing MGT detection.
D.9
Sensitivity Analysis
D.9.1 About the Supervisor
Sensitivity w.rt. Original Text Number for Longer Text. core idea the design of the supervisor is to use longer texts to enhance data quality. Theorem 3.] theoretically demonstrates that
36

---

Table Performance concerning TPR& (%) on DetectRL. The detection model trained on text generated by ChatGPT.
Method
Sentence-level
Paragraph-Ievel
PaLM
CharGPT
Claude
Llama-2
Avg.
PaLM
ChatGPT
Claude
Llama-2
Avg.
Likelibood
1.83
1.58
0.72
5.54g0.00
3.17
25.66
10.21
1. 1.78
38.39
19.01
Log Rank
4.
1.23
0.72
5.25
3.01
27.49_1.13
11.55 1.
2.08
41.98 47
20.76
Entropy
8.62
0.58
0.67
0.77=0.15
0.66
6.95
0.25=0.16
1.51
2.03=0.78
2.68
NPR
2.24
1.72:0.20
1.03
3.9520.00
2.23
6. 6.33
2.87
1.19
20.000
7.60
DeteciGPT
0.72
0.38
0,25
0.84g0.17
0.54
3.76
4.82
5.88
6,28.1.35
5.19
FP
1.33
0.27
0.09
1.65g0.63
0.84
18.15_1.44
11.87
0.44
29.49
14.99
CharGPT-D
9.56
9.78
3:26
13.56
9.03
18.15-a.77
19.28
4. 4.23
41.98
20.91
ChatGPT-E
10. 10.52.0.66
10.86-1.9
3.31
16.37
10.07
22.79
32 11
6.01
49.37 27.72
27.62
MPU
31.441159
46,28_
11.15g0.00
31.12
30.75
60.59-1.07
89. 12
18 18.69
90.33
64.73
MPU-E
34.76.1.77
52.61_2.26
12.82
36.15
34.09
68.08_2112
94.49
23,49
92.78.1.00
69.70
RADAR
33. 33.11+2.::
56.45-2.08
14.1
33.11
34.19
74.24_4.05
93.00=111
28.6822.40
87.64=1.54
70.89
RADAR-E
34.70+1.: 62.24 1.22
14.02
34.41
36.49
74.26
94.61
28 28.6822.45
89.37
71.73
Table Performance concerning AUROC (%) on DetectRL. The detection model is trained on text generated by Claude.
Method
Sentence-leve)
Paragraph-Ievel
PaLM
ChatGP
Claude
Llama-2
Avg.
PaLM
ChatGPT
Claude
Llama-2
Avg.
Likelibood
40.28
41.96 58
32. 52
42.35
28.58
$3.39
57.53_1.77
21.42.a
35.23
Log-Rank Rank
40.94
44.08-0.02
55. 65.72+0.21
32.29
43.26
28.36-0.40
20.0
57 67.89-1.11
20.04=0:45
35.07
Entropy
65.89 50.3640.55
52 22.5660..1
56.28
53.77
43.13
11.96
48.5 48.50-1.29
39.87:1244
43.36
NPR
16.50 16.41_0:20
52. 10ga.4
42.14
46.89
48.31 1.15
15.69
61.52_0.a7
37.99.0.m
48.38
DeteciGPT
46.94.0.8 45.69
52. 32.6559.37
43.07
30.30
47.09
57.92_..7
50.04
65.37
49.21 ss
55.63
FasGPT
62.43g0.45
55.88_0.58
40.81g0.31
67.12
32
56.56
50.94_0.93
61.26
24.14_1.06
70.39
53.93
ChaGGPT-D
10.48
71.20_2..0
67 25
75.99
71.23
77.99
79.01
72.45=4.20
90.02:1.58
79.87
ChatGPT-E
71.15-1.92
06.91g1.5
76.01
71.13
77.09
80.08
71.11:..1
90.32:2
80.45
MPU
81.53g000 89.80
84.89
86.
18
85.60
97. 97.93 93.73_20.1
97.00 97.0020.55
98.80
96.87
MPU-E
82. 12.12
91.19-0.5a
87.3320.0
86.65_0.42
86.82
98.30 94,29_0068
98.76=0.17
98.86
97.55
RADAR
84.92 84.9220.45
02.62-0.49
84. 54.58
84.97
86.77
96.75-0.25
97. 27.80
90 08.26-20.22
98.28
98.02
RADAR-E
85.47
93.14 0.39
13:a.s
85.37 20.21
87.65
97.12 0.33
97. 27.98:0.55
99.38
98. 98.12:: 28
98.23
this strategy can lead to better feature discrimination, thereby mitigating the negative effects of hard labels and fostering effective learning. To empirically verify these theoretical findings, we analyze the impact of the number of original texts constituting longer texts (i.e., the value of k)) on supervisor performance, shown in Fig. 17 Experimental results align with theoretical predictions: increasing the number of original text segments that constitute longer texts helps improve the supervisor's learning performance.
Sensitivity w.r.t. the Number of Longer Texts N' Per Batch. Increasing the number of longer texts processed by the supervisor (reflected in the number of longer texts per batch, N') should enhance its detection capability. To investigate this, we experimented with different settings of N' evaluate the supervisor's 's's performance, shown in Fig. [8] The experimental results meet expectations: as the value of N' increases, the detection performance the supervisor indeed improves.
Sensitivity w.r.t. Supervisor Loss Coefficient X. In the joint training process shown in Eq. 5) the weight of the supervisor loss term directly determines the emphasis on supervisor training within the overall optimization objective. To explore its impact on the supervisor's performance, we experimented with different settings to evaluate the supervisor's performance, as illustrated in Fig. 19 it can be observed that in the relatively weaker ChatGPT-D, the supervisor requires larger A. contrast, for the relatively stronger MPU and RADAR, the supervisor's performance initially improves with an increase in À value but reaches saturation and does not show significant further improvement. This might be related to the implementation of the supervisor, which, as explained in Appendix D.4 uses the input embeddings from the detector as its embeddings. In the stronger MPU and RADAR, the higher quality of embeddings provides a better initialization for the supervisor's learning, alleviating the overly large focus on the supervisor.
37

---

Table 10: Performance concerning TPR@FPR-1% (%) on DetectRL. The detection model is trained on text generated by Claude.
Method
Sentence-level
Paragraph-level
PaLM
CharGPT
Claude
Llama-2
Avg.
PaLM
CharGPT
Claude
Llama-2
Avg.
Likelihood
0.50
0.35
1.30
0.25
0.60
0.72-0.21
0.30
3.73
0. 2.00
1.21
Log-Rank
0:44:0.19
0.35
1.14
0.23
0.54
0.51
0.22
3.31
0.02+a.0s
1.03
Entropy
0.62
0.58
0.67
1.
0.66
1.80
0.12:am1
1.24
0.37
0.93
NPR
2.06ggoam
1.9220.7
2.6740.0
1.84
2.12
7.32_00m
7.27
10.45
5.39
7.61
DetenGGP
0.80
1.12+0.20
1.62+0.45
0.55
1.05
-.21
7.00+1.41
11.1042.4
6.27
7.22
FasGGPT
1.33
0.2720.10
0.09
1.65
0.84
18.15g1.44
11.87=1
0.44g0.17
29.40 29.49
14.99
CharGPT-D
8. 8.24
7.6320.9
3.54g0.80
12.3420.50
7.94
15.76
13.47-5.88
4.89
35.65+6.84
17.44
ChatGPT-E
1.20
1.80
3.29
12.46:0.99
7.97
+9.11
20.00
6.58
38 88.4748.49
21.04
MPU
s0
21.95-100
18.46
26.91g1.1
22.60
51.61:257
76.66-4.7
09.155m.21
86.06
71.62
MPU-E
21.16 6a
31.80-294
22.34
29.00
26.08
56.91
78.12
85.44
85.5442.18
76.50
RADAR
23.93
42.08
19.6820.47
23. 20.08
27.27
68.13
67.60
07.66
93.13
73.03
75.49
RADAR-E
23.28
41.12_2.5
21.57
21.7811.7
26.94
67. 67.81
68.73_7.5
94.12.1.52
73.94.m.m1
76.15
Table 11: Performance concerning AUROC (%) on DetectRL. The detection model is trained on text generated by Llama-2.
Sentence-
Paragraph- level
Method
PaLM
CharGPT
Claude
Llama-2
Avg.
PaLM
CharGPT
Claude
Llama-2
Avg.
Likelibood
59.7220.4 su 59.06 se
58.04_0.6
45.0220.20
67.82
57.65
71.4220.49
66.61 90
42.47.1.27
78.58
64.77
Log-Rank
65.92_0.2
44.28=0.21
67.71 20.54
56.74
71.6440.4
65.99
42.11=1.a1
79.96
64.93
Entropy
50.36
55.89_030
12.56
56.2880.44
53.77
63.08 60.73
51.732000
66.2120a8
60.44
NPR
53..41 =0.60
47.60
27.86
53.11
51.6991.13
20.51
38.4820.87
22.01
DetecTGPT
53.06.0.e4
64..31-0:29
a7.35
56.
20.00
52.91
44.7445..7
49.35-20.4
41.06-12.61
50
50.41
46.39
FastGPT
62.43 48
55.88_.0.88
40.81go.3
67. 67.1220.37
56.56
59.9140sa
61.26.00.0 90
21.14g1.06
70.39
53.93
ChaGGP-D
71.95
73.82-2.64
65
65.92=1.8
78. 78.7
72.60
80.90
84.866.3..
70.41:2.05
93.66
82.46
ChatGPT-E
73.16-2.01
75.58_2..8
67.21g145
80.09
74.02
82.30
86.33 81
72.35
9:.322.0.95
83.82
MPU
83.85
90.38_0.4
74.43
00.85
84.88
94.0610
97.95
74.59-1.11
99.13
91.43
MPU-E
85.08 41
91.61
75.0920.99
21.90.0..11
85.92
95.43
98.79
77.8226.70
99.4220.14
RADAR
86.82.0.41
03.36-0.a7
75.32
29.82
86.33
97.57
97.
99.30..0.r
88.0991129
99.33+0.07
96.07
RADAR-E
87.88
94,24_0.28
76.47
20.98
87.39
97.77
99.44_0.10
89.47=1.08
99.432..04
96.53
D.9.2 About the Detector
Sensitivity w.r.f. Original Text Number l for Longer Text. According to our theoretical results (Theorem 3.T). longer text lengths help achieve greater distribution distance for text data, thereby simplifying the supervisor's learning difficulty and laying the foundation for providing reliable supervision to the detector (empirically proved in Fig. [7]. To this end, we explore the impact of different original text numbers & for longer text detector performance, shown in Fig. 20 The setup with A = 0 represents the original detector bascline without using the enhancement strategy. The experimental results align with the theoretical predictions: using longer texts for supervised learning enhances the supervisor's performance (Fig. 17). thereby providing more reliable supervisory signals and ultimately improving detector performance.
Sensitivity w.r.. the Number of Longer Texts N' Per Batch. The performance of the supervisor significantly impacts the target detector's performance, and the supervisor's 's own effectiveness largely depends on the amount of longer texts (proved in Fig. [8]. Here we aim to explore the impact of varying quantities of longer text data on the performance of the detector, as shown in Fig. 21 N' = 0 represents the original model without enhancement. The results are as expected: as the amount of long text data used for training the supervisor increases, the detector's 's learning is better supported, leading to improved performance. This can also be seen from the consistent trend of changes in Fig. 18 and 21 Although increasing the data volume might introduce additional computational overhead, as indicated in our previous runtime analysis, even with relatively large data settings (e.g., N=128), the additional training delay remains minimal.
Sensitivity w.r.t. Supervisor Loss Coefficient X. We also investigated the impact of the supervisor's loss term coefficient À on detector performance, illustrated in Fig. 22 We can also find that the detector's performance change curve is consistent with the change of the supervisor (Fig. 19). which also indirectly emphasizes the guiding role of the supervisor the detector.
38

---

Table 12: Performance concerning TPR@FPR-198 (%) on DetectRL. The detection model trained on text generated by Llama-2.
Method
Sentence-level
Puragraph--ovel
PaLM
ChatGPT
Claude
Llama-2
Avg.
PaLM
ChaiGIT
Claude
Llama-2
Avg.
Likelihond
4.83
1.58
0.72
2.54
3.17
25.66
10.21
1.78
38.39
19.01
Log Log-Rank Rank
4.84
1.23
0.72.0.13
3.01
27.49
11.55g1.0
2.08
41.98
20.76
Entropy
0.62
0.5840.11
0.67=0.12
0.77
0.66
6.95
0.25
1.5140.34
2.03
2,68
NPR
2.24
1.7240.20
1.03:6.ce
3.9540.6
2.23
6.33
2.87+1.5
1.19
20.00;1.61
7.60
DetectGPT
0.72+0.17
0.38
0.25
2.84
0.54
3.09
4.75
3.76gn.01
6.33
FastGPT
1.33g0.54
0.27+0.10
0.09
1.65:s0.6
0.84
18.15
11.87a1221
0.44
20.4
14.99
ChaGGPTDD
9.36
9.00
3.04
13.6941.91
8.77
22.322a.95
23.34g7.14
5.41
48.58.66.3
24.91
ChatGPT-E
10. 1.77
10.11=1.7
3.15
15.46 15.4682..8
9.75
24.00
24.55
5.69
52.41
26, 26.66
MPU
23.94 23.94g1.21
6.78
33.77
22.93
35.97
74.81g108
15.50g142
89.25
58.88
MPU-E
24. 24.82
20.96-2220
7.13.0.7
34.5841.5
24.12
62.30
81.58
17.1642.00
92.86
63.47
RADAR
31.27 13.70.3.08
11.38
1..44
30.76
75.4044.54
88 68
34.14a381
88, 15.03
71.56
RADAR-E
34.5821.92
49.46-a.36
12.39
39.59
34,01
76.1226.00
91.52g1.26
38.6420.29
90.14
74.10
Table 13: Performance concerning AUROC (%) on Essay under sentence-level settings. The detection
model is trained text generated by GPT4AIL.
Method
GPT4AIl
ChatGT
ChaiGPT-turbo
ChaGGLM
Claude
Likelihood
81.83
82,98
75.88
89.59
75. 12 15.42=0..3 -2..0.3
62. 02.022-2 -2.33
78.05
Log-Rank
80.82 m
81.5410.34
73.73 a3
89.4220.30 2..08
74.27 20.2
60.11
Entrupy
59.7710.44
64.67
65.8610
60.94.0.33
58, 38.29_2.44
59. 10 10.11000:00
61.44
NPR
72.1610.17
71.4610.36
68.9210 74
77.7810.00 0.
67.21 -0.59
14_0.
DetectGPT
73.01
72.1920.a8
70.721.
77.19 0.:0..0
68.59 _0.00
65. 75..000000
71.24
FasGPT
81.65+0.20
80.14+00.40
71.65+0 67
89.85=a0.2
77.29
63. 63.19 1.2000
77.29
ChatGPT-D
80.64
78.6310.70
74.95
86.73
66.33 1.21
60.58
74.64
ChatGPT-E
81.5641.24
79.21
75.39 1.
87.742
66.76_1.22
60.34_1
75.16
MPL
87.83+0.711
85,4440.79
83.60+ 97
91.58+a..d
73.46-0.68
69.08 09.98
81.83
MPU-E
89.541...4b
87, 37.254g..0
86.06
92,89
75. 75.82-0.24
72.
83.94
91.55
91.6210.34
91.8610 45
94.0220.00
_0.4
00.73
RADAR-E
92.39
92.44
92.76
94.81 20.33
83.99 83.99.0.45
80.50
89.53
D.10 Ablation Study of Supervisor
In our framework design, aim encourage the detector correctly classify each sample within longer texts, rather than strictly requiring the predicted probability of the correct class to approach I. This raises the question of whether directly using the loss function only focused on the class could achieve similar goals. Therefore, we conduct an ablation study using two class-only loss functions to highlight the role of the supervisor in enhancement.
First, we define the training loss of the baseline model after removing the supervisor module as follows:
N'
k
Lsupe.
Mlonggi log (fa ,0)) + Vlong.:) log(1 gumbel(f(x))) ,0)))
KN
To ensure that gumbel(f (o) 0y)) is meaningful, the non-hard-label version of Gumbel-Softme is used.
Second, Hinge loss to replace Csupe. follows,
N'
Caupu.
max (Mteggg. 2 2 1) (f(x +2 (((x
KN
These two variants are defined Gumbel and Hinge. Aside from the form of the supervisory signal, all other experimental settings (such as k, N', X) remain consistent. The experimental results are shown in Fig. 23 It can be observed that using these alternative loss functions focused solely on class can enhance detection performance in some settings, yet there are instances of instability. For example, applying the Hinge loss to the RADAR model on the Essay dataset resulted a performance decline. Furthermore, even though these alternatives provided performance improvements in certain
39

---

Table 14: Performance concerning AUROC (%) on Essay under sentence-level settings. The detection model is trained on text generated by ChatGPT.
Mothod
GPT4AII
CharGPT
ChaiPPT-tubb
CharGLM
Dolly
Claude
Avg.
Likelihood
81.83
82.9810.28
75.8810
89.59
75.42_00
75.42 0.00
62.62 22.0220.0
78.05
Log-Rank
80.82 42
81.5440.44
73.73+0 5a
89.42
T4.27=0
60. 60.11=0.0m
76.65
Entropy
69.77+0.44
64.67+0.21
65.86+0 26
60.94+0..8
58. 68.29-0.45
50. 10 10-000
61.44
NPR
72.16.0.47
71/463a.36
68.92
77.78:0..a
67.21
64.14 -0.09
70.28
DetectUPP
73.0110.34
72.1910.33
70.7210
77.1910.30
68.59
65. 65.74_0.05
71.24
FauGPT
81.65
80.14g0.40
71.6510 67
89.85g0.900
77.29
63.19_0.20
CharGPT-D
78.41
17
72.56
85.11
61.39-1..0
58.52-1.1
72.69
ChatGrTTE
79.75+2.68
78.43+199
73.31+3.00
86,8D+2.00
65.95-1.88 1.83
59.77 00.77-2.335
74.00
MPU
87.7840.84
87.10
83.7040
92.91
74.35=0.ed
69.62
82.58
MPU-E
89.28
88.5919.95
85.9610 a2
93.9310
76.39 76.39_0.06
71.87 11.87_1.16
84.34
RADAR
90.82+0.00
92.4740.08
91.89 64
94.58 46
83.77=0
50.75=1.63
89.05
RADAR-E
91.06-0.30
92.82 0.81
92.32 92.32+0.20
94.85 00.01
84.25 +0.41
81.31 +..40
89.43
Table 15: Performance concerning T @ PPR-PP (%) on Essay under sentence-level settings. The detection model is trained on text generated by ChatGPT.
Method
GPT4All
CharGPT
CharGPT-hurbo
ChaGGLM
Dolly
Claude
Avg.
Likelihood
9.18
14.05
5.46 46
26.04 as
6.86
2.82
10.73
Rank
8.98+1.11
13.35
4.97
29.04 44
6.78+1.66
2.32+0.05
10.91
Entropy
1.364a.26
2.40+
1.66+0.33
1.10+4..31
1.7040.45
1.44
1.61
NPR
6.1314.86
7.13
1.01g0 74
11.141
4.71:0.53
3.2611043
DetectGPT
4.11
30.77
3.48
5.77.1.47
3.70
3.44
4,01
FasGGP
#1.60
10.30
4.58
28.86
9.01
2.7220.48
ChatGPT-D
13.71
10.96
8.60
22.54:
4.26
1.9210.23
10.33
ChatGPTE
14.18
11.49
8.35
24.37 1
4.25110.0
2.11
10.79
MPU
20.4441.57
16.39+1.40
12.9840.94
34.36
5.42+0.88
2.58+0.04
15.36
MPU-E
24. 24.19
20.47
16.72+1.70
38.02+5.16
7.01+1.46
3.30
18.29
26.16:1.98
31.71g2.98
30.9012.33
13.90
11.99.1.3
10.09
RADAR-E
25.68 25.6812.13
32.45 13.06
30.97
39.1613.8
11.9811.8
9.82
25.01
settings, their enhancement effects were generally inferior to our proposed strategy. This because
our strategy not only focuses correct classification but, more importantly, guides the model's predicted probabilities to approximate the underlying true labels through the supervisor's signal (see Theorem 3.4), offering richer supervisory information.
40

---

Table 16: Performance concerning AUROC (%) on Essay under sentence-level settings. The detection model is trained on text generated by ChatGPT-turbo.
Mothod
GPT4AII
CharGPT
CharGPT-turbo
CharGLM
Dolly
Claude
Avg.
Likelihood
81.83 ab
02.98
75.88 46
89.592a.4:
62. -2.35
78.05
Log-Rank Rank
80.8210.0
81.5410.34
73.7310
89.4210.33
74.27
60. 60.11_0:42
76.65
Entropy
59.77=0.44
64.67
65.86 26
60.94
58.20=00 00.43
59.10=0.29
NPR
72.16+0.47
71.46+0.36
68.92+0.
77.78+0.36
67.21 07.21-0.00
64. 64.14-0.60
70.28
DeteeGGPT
73.01+t0.54
72.1944.33
70. 70.72:t0.0
77.1920.30
68. 68.59_0.39
65.74 -0..0a
71.24
FaRGPT
81.65
80.14g..mm
71.6510
89.85
77.29 20.0.
63.
77.29
ChaIGPT-D
77.63
76.49
75.88 2.2.....00 66
82.34
63.53 23:3_.000
60.98 60.98_1.8 1.002
72.81
ChatGPT-E
78.76+3.90
77.3442.58
78.00 90
82.83+3..8
64.52
62.95
74.07
MPU
86.63+0.16
85. 95.17++0.22
89. 00,77++0.11
88, 88.62+0.700
72.01 -1.09
74.73-0.41
82.82
MPU-E
87.65-20.85
86.2719.73
91.45:1
89.17
73.56 1650.0
1..9.92
84.26
RADAR
89.3910.19
90.5810.
94.7210
90.85
81.67 81.67-0.17
85.46
88.78
RADAR-E
89.94_0.48
91.17=0.41
20.24
91.43=1.00 1,00
81.83 1.228
20.68
Table 17: Performance concerning
(%) on Essay under sentence- settings. The detection model is trained on text generated by ChatGPT-turbo.
Mothod
GPT4AUl
CharGPT
ChatGPT-uubb
ChaIGLM
Dolly
Claude
Avg.
Likelihood
9.18-1.66
14.05+0.67
5.46+0..0
26.0444.85
6.86+1.11
2.82+0.17
10.73
Log-Rank
8.98-1.11
13.35
1.97
29.01 44
6. 6.78
2.3240.00
10.91
Entropy
1.36-0.20
2.40-0.56
1.65+0.33
1.10+0.51
1.70+0.48
1.44+0.18
NPR
6.13 6.13=0.es
7.13
4.04
14.14
4.71
3.26:t0.4
6.57
DetectGPT
2011
3.57
3.48
5.77
3.70
3.44
FasGPT
12.38
10.30
4.58
28.86g2.71
9.01
2.7220.43
11.31
ChatGPT-D
15.12+1.81
10.91:a..s
11.07
20.20
4.49
2.56
ChatGPTTE
15.58
11.1441.61
13.81
19.89 2.82
4.69
3.2410.90
11.39
MPLI
20.69
15.92
25.56+1
24.93
5.96
4..1
16.28
MPU-E
23.32 s0
18.1512.02
30.58
25.0512
6.87
6.2710.91
18.37
RADAR
24.82+1.18
26. 00
43.08
25.44=2.00
11.14=1.45
17.00
24.71
RADAR-E
2.449
-8.49-3.40
48.15+2.91
4.94
11.85+1.89
14.97-3.13
Table 18: Performance concerning AUROC (%) on Essay under sentence-level settings. The detection model is trained text generated by ChatGLM.
Method
GPT4All
CharGPT
CharGPT-urbo
CharGLM
Dolly
Claude
Avg.
Likelihood
81.83
82.98
75.88
89.59
75. 75.42-0.35
62.62-o.0
78.05
Log-Rank
80.82
81.5410.34
73. 73 53
59.4220.30 38
74.27 20.2
60. 60.11_o.44
76.65
Entrogy
59.77
64.6740.91
65.86 26
60.94
58.29
59.10
61.44
NPR
72.16+0.47
71.4640.36
68.92 0.2....4
77.78
67.21 =0.69
64. 14...0
70.28
DetectGPT
73.01
12.19+0.35
70.72+
77.19+0.30
68.59
65.74 65.74-0.85
71.24
PaRGGP
81.65..0.20
80.1 80.14ga..30
71.65
89.85gg0.2
77.29_0.34
63. 09.19_00200
77.29
ChaiGPT-D
79.2512.29
77.72.1.49
71.41 90
$6.9412.00
65.20_1185
11.5
ChatGPT-E
80.1441.73
18,53
72.22+1
88, 18,05
66.27=1.44
58. 18.24=1.44
73.96
MPL!
86.30+0.45
85.02+0.15
T6.39 47
93.71 +..18
71.60-0.18
40.96
MPU-E
87.30
86,01
77.94
94,36
73,00
64. 01.08-e.c
80.44
RADAR
90.6810.05
91.8610.31
88.95
95.30
82.51 82.51_0.61
73.63
87.16
RADAR-E
91,23_0.a3
92.33 0.20
89.03
96.07 20.28
83.04
73.33 _0.87
87.51
Table 19: Performance concerning TPR@ (%) on Essay under sentence-level settings. The detection model is trained text generated by ChatGLM.
Method
GPT4AIl
ChatGPT
ChaiGLM
Dolly
Claude
Avg.
Likelihood
9.1821.58
14.05
5.46
26.0124.35
6.8621.13
2.82
10.73
Log-Rani
8.98 11
13.35 10.58
4.97 so
29.04g3.48
6.78
2.32
10.91
Entropy
1.36
2.40-0.65
1.65 34
1.10
1.70
1.44
1.61
NPR
6. 1..000
7.13+1.64
4.04+0.74
14.14.1..5
4.71
3.2640.4a
6.57
DetectGPT
20.1
3.57 2..77
3.48-a.as
5.77:41.47
3.70.20.7
3.4440.17
4.01
FastGPT
12.38.1.00
10.30
4.58
28.8623.71
9.0 9.0120000
2.72.0-43
11.31
CharGPT-D
14.1642.18
11.23+1.72
7.85+1.12
24.64++4.12
4.29
1.66+0.a7
ChaiGPT-E
15.094
11.48
8.29
24.74-33225
4.47
1.7840.38
10.97
MPU
17.5441.45
12.96+1.71
6.29
35.50+2.60
5.06+0.43
L.440,
13.16
MPU-E
19 19.2942.83
14.51+1.89
7.73411.46
38.2122.09
6.50
2.1940.41
14.74
RADAR
24.8812.00
29. 29.56 29.56144115
19.96
44.89
11.41
5.9820.55
22.78
RADAR-E
1..8
30.30=2.71
20.12:2.60
50.01
12.28
5.6040.95 98
41

---

Table 20: Performance concerning AUROC (%) Essay under sentence-level settings. The detection model is trained on text generated by Dolly.
Method
GPT4AII
ChaiGT
ChaiGPT-turbo
ChaiGLM
Dolly
Claude
Avg.
Likelihood
81.83 ab
02.98
75.88 46
89.59
12-0.33
62. -2.5
78.05
Log-Rank Rank
80.82+0.0
81,54+0..44
73.73+
89.42+a.35
74.27 74.27-0.28
60. 90.111-..2
76.65
Entropy
59.77.2..4
64.67.10.21
65.86 26
60.94:20.35
58.20 -..00...3
59.10-0.29
NPR
72.1610.47
71.4610.36
68.9210
77.7810.30
67.21
64. 64.1440.00
70.28
DeteerGPT
73.01
02.19
70. 70.722:0 61
77.19
68, 59
65. 00000000
71.24
FastGPT
81.65+0.20
80. 80.1440..0
71.66+
89.85+0.2e
77.20
63. 19.0.0
77.29
ChaIGPT-D
75.66
74.66.0.0d
69.48 o
82.52 0.
62.39
56. 56.44_0.co
70.19
ChatGPT-E
75.59
74.58+0.44
68,87 6s
82.66+1.88
62.44
56. 13 13.0.4
70.04
MPU
82.92+0.04 81.044+0.35
75.68+
88.08+0..2
70. 70.79 0.29
63. 26.12--..22
76.96
MPU-E
84.33-20.p
82.361a.17
TT.78:11 20
89.25
T2.91
65.
RADAR
87.88+0.88
88.76+0.96
88,00+
91.28++0
84.46 0.2-
82.59 -0.95
87.16
88.66_0.08
89.71
20.40
92.19_0..44 20.44
85.37.20.28
84.12.20.81
Table 21: Performance concerning @ @FP-11PPP (%) on Essay under sentence-level settings. The detection model is trained on text generated by Dolly.
Method
GPT4All
CharGPT
ChatGPT-tubo
Cha1GLM
Dolly
Claude
Avg.
Likelihood
9.1811.50
14.05.
5.461
26.04g
6.86g1.13
2.82
10.73
Log Rank
8.98
13.35g0.6e
4.97
29.0443.44
6.78
2.32
10.91
Entropy
1.36+0.20
2.40++0.88
1.65+0.84
1.10+0.31
1.70+0..4
1.4440.18
NPR
6.13
7.13-20.54
0.
14.14
4.71
3.26::0.45
6.57
10.57
30.77
3.48
5.7741.47
3.7020.75
3.44
Fas:GPT
12.38
10.30+
4.58
28.86
9.01+0.06
2.72+0.43
11.31
CharGPT-D
12.57.00.9
9.75-20:25
7.11
19.86
1.01-t0.33
1.70
9.17
ChatGPT-E
12.39
9.79
6.7210
20.06 04
3.96
16441
9.09
MPU
11.01
10.96g0.70
6.522 69
26.85
4.22
1.6910
10.71
MPU-E
16.41+0.52
11.93+0.02
7.93+0.00
28,83+1.13
3.00+0.66
1.94:0.18
RADAR
18.59
21.01
20.25.1..7
26.77
11.59
9.84
18.01
21.17
23.22+1.71
22.34+1.80
1..87
:4.24
10.17+1.67
Table 22: Performance concerning AUROC (%) on Essay under sentence-level settings. The detection model is trained on text generated by Claude.
Method
GPT4A1
CharGP
ChaiGPT-turb
ChaiGLM
Dolly
Claude
Avg.
Likelihnol
81.83
82.98
75.88
89.5910
75.42_00
62.62
78.05
Log-
80.82
81.5440.34
73.73++ 6a
89.42
74.27 14.27=11
60. =0.42
76.65
Entropy
59.77+0.44
64.67+0.21
65.864 26
60.94+0.55
58.29
59.10
61.44
NPR
72.1610.47
71.16ga..6
68. 2S.9221.
77.78
67.2 67.21_0.55
61. 61.11_p..9
70.28
DetectGPP
73.01
72.1940.33
70.724 d1
77.19
35.74_0.c0
FasGPT
81.65
80.14
71.65
89.85
77.20
63. 63.19.0030
77.29
ChaiGPT-D
73.73+2.04
73.73+1.05
71.41+2202
79.08=1.75
61.00-1.68
18-2.22
ChatGPT-E
73.20+1.12
13
69.91
79.13
60.61
58. 58.82 02.8821.77
69.17
MPU
82.6319.6
81.3910.19
81.87 10.770.12
85.69 2..57
71.18
78, -1:35
80.17
MPU-E
83,91
82.1710.40
83,81
86.15
73.54
82,53
82.02
RADAR
82.3340.65
82.91 40.71
88.764. 65
81.58+0.65
79.47 79.47=0.68
91.48 91.48-0.33
84.42
RADAR-E
82.81+0.58
83.36-0.50 0.82
89.27 +0.50
82.05
79.90 =0.90
91.94.0.50
84.89
Table 23: Performance concerning TPR@ (%) on Essay under sentence-level settings. The detection model is trained text generated by Claude.
Method
GPT4All
ChatGPT
ChaiGLM
Dolly
Claude
Avg.
Likelihood
9.1821.58
14.05 20.57
5.46
26.0124.35
1.86
2.82+0.17
10.73
Log-Rani
8,98 11
13.35 10.58
4.97 so
29.04g3.48
6.7811.06
2.32g0.05
10.91
Entropy
1.3620.26
2.40=0.65
1.65 34
1.10
1.70
1.44
1.61
NPR
6.13-0.00
7.13+0.64
4.04+0.74
14.14+1.85
4.71+0.51
3.26+0.43
6.57
DeteciGPT
20.52
2.577
3.48 as
5.77+11.4
3.70+a.75
3.44
FasIGPT
12.38+1.60
10.30
4.58
28.86
9.01+0.66
2.72+
11.31
ChaGPT-D
13.22+1.4s
9.76+0.70
8.60+1.65
18.60+1.45
4.05++.42
2.24+0.40
9.41
ChaiGPT-E
12.31
10.15
7.51
19.42-2.11
3.93
2.04
9.23
MPU
14.80+0.16
11.66+0.49
12.58-1.49
22.34+1.54
4.3344.36
6.44+1.01
12.02
MPU-E
16.14-1.28
12.52=11.00
16.27=2.01
21.28
5.09
10.80 10.86
RADAR
9.80
10.87
19.40
6.9011.42
7.40 7.40
28. 28.8812277
13.87
RADAR-E
10.1241.64
11.3041..0
20.49 85
7.7141.61
7.2841.12
28.90
42

---

Table 24: Performance concerning AUROC (%) on Essay under paragraph-level settings. The detection model is trained on text generated by GPT4AII.
Method
GPT4AII
ChaiGP
ChatGPT-turb
ChaIGLM
Dolly
Claude
Likelihood
96.16
98.79
99.13
99.29
90.90 100000.000
92.76. 02.76_0.23
96.17
Log-Rank
96.65+0.81
98.9440.16
99.224.
99.48+0.21
90.68 1.288.2
92.04 92.04-0.19
96.17
Entropy
75.52+1.a1
20.33+0.22
94.52
85, 85.25+0.75
74.76-1.44 1.45
8G. -0.67
84.43
NPR
90.77
99.1610.12
47.82 01
99.55.0.00
95.23_o.55
49.50 19.50_0.67
DeteciGPT
95.01 10.08
97.08g0.30
46.03
96.19
92.64 92.64_0.01
46.94 16.94_1-16
78.98
FasGPT
67-33+1102
12.35+1.44
76.43
45.40=1.98
62.70=0.74
ChaiGPT-D
94.74+1.24
95.29+0.08
89.62+
99.05+a.1r
71.41-9.98
62.35
83.74
ChatGPT-E
96.07 40.07
96.05=2..34
90,48 me
99.14.:..2 20
71.92-2.00
59.01 -9.07
85.95
MPU
97.86
97.831..
95.4610 31
99.6310.11
85.28 1.16
77. 11.47
92.29
MPU-E
98.20
98.1510.88 33
96.0010
99.7320.10
86.74
80.30 -1..0.1.65
93.19
RADAR
99.60+0.07
99.36+0.10
96.32+ 27
99.89
93.58
95.75-0.85
97.75
RADAR-E
99.70=0.06
99.51 00.12
20.49
99.93 0000
94.45 #0.75
97.12 97.12-t0.38
98.20
Table 25: Performance concerning AUROC (%) Essay under paragraph-level settings. The detection model is trained on text generated by ChatGPT.
Method
GPT4AII
CharGPT
ChaiGPT-turbo
CharGLM
Dolly
Clande
Avg.
Likelihood
96.1640.50
18.79
99.13 19
99.29+a.28
92. 92.76-0.25
96.17
Log-Rank
96.6510.31
98.9110.16
99.2 99.2210
99.1810.21
90.68
92 02.04 02.01
96.17
Enirugy
90.33
94.52 46
85.25
86. 20.02
NPR
97.77
99.16+0.12
47.82+1.01
99.55
05.23_..
49.50-0.c0
81.50
DeteetGPT
+0.25
97.08+a.29
46.03+
96. 16.199...9
92. 02.04-0.61
46 46.94-1.10
78.98
FasGPP
67.33
1.
96.81 as
76.43 a.
45. 16.-10_...00
82.70_p.74
73.5
ChaiGPTDD
90.46112.18
93.5221.71
87.7311 74
98.79
61.00_1.00
35.55 26.00000
78.00
ChatGPT-E
91.14
94.17
89.20 es
98.68
63.31
37.87
79.06
MPLI
96.90+0.61
97.78+0.33
95. 10+0
99.56+
82.00 82.09-2.68
70.68 -3.87
90.35
MPU-E
97.35
98.11
75.764a as
99.61
83.89 1.63
74.72
91.57
RADAR
99.6110.10
99.7010.06
99.00
99.92.0.00 0..0
93.07
95.49 09.499..7
97.80
RADAR-E
99.68 20.00
99.81 20.04
20.21
99.95_0.00
90.67
96.46.0.64
Table 26: Performance concerning @ (%) on Essay under paragraph-level settings. The detection model is trained text generated by ChatGPT.
Method
GPT4AlI
ChaiGPT
ChatGPT-urbo
ChaGGLM
Dally
Claude
Likelibond
46.33_16.4
68.62
73.60
92.86
20.67 100079
12.36 26
$2.41
Log-Rank
63.74_12.04
79.47
81.29
96.61
25.92
19.47
61.08
Entropy
3.78
11.07 1:.07:2.000
16.31 21.79
8.35
3.91 3.91 1
6.22 6.22
8:27
NPR
78.50
85.91
9.02
95.13
58.52
8.40
55.91
DoseGPT
$1.53.1..0 te.s6
39.17 39.417ia..0
7.24:11.5
30.31 37.31
20.48 18.48aa.am ao
3.61
22.45
PasGGP
0.18+0.17
0.40++0.20
08.27=4.92
1.70+0.79
0.00+0..0
7.69+1.45
13.04
CharGPT-D
52.35
44.00
31.20 66
$4.02 0.
12.84
1/2020.04
ChatGPT-E
47.02-18.77
40.62+10.50
30.36
70:00-20
13.2740.71
1.24+0.70
33.75
69.1647.00 02
66.13
44.71
95.09 1.86
26.59 21
5.47
MPU-E
69.20
71. 16..61
50.27 18.08
08.2611.6
07.5415.99
7.1612.16
53.15
RADAR
92.9442..5
92.41gg..w
72.31:s.a1
u..5
17.38
RADAR-E
93.71+1.75
95.87
79,51
99.20
54.80+8.02
55.56+55.06
79.77
Table 27: Performance concerning AUROC (%) on Essay under paragraph-level settings. The detection model is trained on text generated by ChatGPT-turbo.
Method
GPT4AIl
CharGPT
ChaiGPT-uubo
CbuGLM
Dolly
Claude
Avg.
Likelihood
96.16+0.20
98. 79.0..
99. 99.13-4..
99.29+a.22
90.90+1.a4
92. 92.76.+0.2 17.++0.25
96.17
Log-Rank
96.65+4..0 a1
98. 91
99. 99.22 0.07
99.18=0.22 21
90.68 28
92.01+0.10
96.17
Entropy
75.52111
30.3311-91
94. 24..52.0.00
80.2510077
74.76 21
86.21
84.43
78.56 29
489.84
47.98_1.10
79.68 68
77.36
49.45
DetectGPT
76.83 2a0.
78. 1377777
16.78.1
77.54_37
75.83 Lo0..oo.00
47.33 1:331110000
67.07
FasGGPT
67.38.11.02
72.35 14.44
96.81 10.00.
76.43....w
10.17..7
CharGPTDD
87.78+2..00
92.25 13.22++1.0
88. 15.003-3..50
98.85+ 0.30
59.03+3.44
29.69 49.44..8
75.94
ChatGPT-E
88.7342..
08.76600.0 00
90.58=1.49
98.85=0.11 17
59.50 03.50
30.99
MPU
96.39+40.45
97.44+
97. 07.02-0.00
99.46+4..11
80.87 1387+2.000
68. 06..1.. +6..09
89.96
MPU-E
07.1840.44
97.96
20.96
97.74
99.58 14
83.69 21.7
74.41
91.76
RADAR
99.37
99. 10.14.
99.51 1.lgo.00
99.80 -0
92.65 16555...0
93.23 10.50
97.33
RADAR-E
99.43 99.43
99,56
99.66
99.87 10.0t
93.30
94.44
97.71
43

---

Table 28: Performance concerning TPR (%) on Essay under paragraph-level settings. The detection model is trained on text generated by ChatGPT-turbo.
Method
GPT4AII
CharGP
ChaiGPT-urbo
CharGLM
Dolly
Claude
Avg.
Likelibond
46.33=16..4
13..22
14.66
92.86.4.54
20.67 10.779
12.36 26.32
Bog-Rank
63.74 63.74_19.98
79. 19.4777.88
81.29
96.61
25, 25.9219
19.47 18
61.08
Entropy
3.78 3..810.91
1:200
16.31
8.35
3.91
6.22
NPR
63. 23,.28-1112
68. +34.43
7.29+5.08
75.76-37.90
46.54424..0
6.31+3.21
44,66
DeseoGGPT
23.23=12 60
42.80
5.73+s.12
24.6/1144 .e
16.66 18
4.31+2.22
FaStGPT
0.18+0.17
0.40+=0.20
68. 27, 27..27
1.70+0.75
0.00++..0
7.69+1.48
13.04
ChaiGPTDD
53.17.7.00
44.89
35.24
81.38
16.56
1.11
38,73
ChatGPT-E
54.0314.84
45.6413.52
39.82 1.2....0
81.12 23.99
15.04
1.29
39.49
MPU
68.20
69.82
61.82
95.00
25.16: 25.16144117
4.7
54.12
MPU-E
70.02+99.61
71.56+7.78
65.02-8.ap 2022-8.97
95.58+n.8.
27.83++
6.22+1.88
56.04
RADAR
$6.97.33.0
86.36
86.04.g
95.18
45.39
35.07
72.50
RADAR-E
87.33+2.62
89.87 +.877--3.25
40.31+2..64
96.96.0.64
4..61
39.56+6.88
Table 29: Performance concerning AUROC (%) on Essay under paragraph-level settings. The detection model is trained text generated by ChatGLM.
Method
GPT4Al
CharGPT
ChatiPPTtuuro
ChaILLM
Dolly
Claude
Avg.
Likelihool
96. 96.1610.30
98.79
99. 99.131o
99.29
90.90
92.76 02.76_2.23
96.17
Log-Ran Rank
96.65
98.94
00.1222.117
99.48
1.28
92. 02.04-0.11
Entropy
75.52
90.33+0.21
94.52+0
85.25+0.75
74.76
86. 30.21-0.67
84.43
NPR
20.21
99.162a.12
47.82
99.55 08
95, 15.23_0.. 23.2.......
19. -0.67
81.50
DetectGPP
95.01
97.0810.00
46.03 37
96,1910 1..000
92.64
46.94 46.94_1.16
78.98
FasGPT
67.33
12.35g1.44
96.81 as
76.43go.8
45.40=1.66
82.70 82.70_0.74
73.50
ChaiGPT-D
87.52+2.41
92.28+1.64
86.6842.15
99.03
59.47
28.50 -3.68
75.60
ChatGPT-E
87.3042.02
92.22+1.0s
$7.1441.47
99. 10.16+:..22
59. 72. 59.72-2.99 2209
29. 29.45-3.11
75.83
MPU
96.51
97.201a.20
93.34 as
99.75
80.68_1.11 1.12
MPU-E
96.94
97.51
93,8810
99.81
82,06
66.71 (6.77__1.00
89.48
99.53
99.45+0.16
96.44 47
99.93 .8
92.65 -6500.60
00.06
RADAR-E
99.60-0.07
99.58
98.57+0
99.95 99.95-0.02
93.30+0.67
95.47
97.74
Table 30: Performance concerning TPR@FPR-1% (%) Essay under paragraph-level settings. The detection model is trained on text generated by ChatGLM.
Method
GPT4All
CharGPT
ChaGGPT-uubo
ChaiGLM
Dolly
Claude
Avg.
Likelihood
46.33 -6.49
12:622+11.. 32
73.60=14.65
92.86+4.54
20 10.79
12.36-
$2.41
Log-Rank
63.74 -2..68 98
79.45 79.47+7.55
81.29 =1.29
96.61 +2.40
25.9220.00 0.00
19.4 19.47 19.47=8.24
61.08
Entropy
3.78+0.91
11.07 -7.00.000
16.31 +1.79
8.35-3.44
3.91+1.40
6.22+1.85
8.27
NPR
78.50
85.91 (1.91++) 1.00
9.02+
95.13+1.78
48.52
8.40
55.91
Detee1GP
31.53 66
7.24
30:31
20.48 80.0
5.6420.74
FP
0.1840.11
0.40
68.27+4..2
1.70=0.ra
0.00
7.69=
13.04
CharGPT-D
53.99
45-42
30.04:14.88
87.37
15.75
38.99
ChatGPT-R
53.99
47.69
30.67:4.66
89.02+3.70
15.47
1.56
39.73
MPU
69.07
65 65.96
43.07.0.7a
96.07
24.73
3.56
50.57
MPU-E
72.7114.50
70.67 13.19
47.24.1..9
1o.52
27.59
4.22
53.24
RADAR
90.52_1.18
88.71
63.16_1a.es
98.26
46.01
42.67
71.56
RADAR-E
92.44 11.44
90.71 12.
65.38
63.38_11.63
98.53
18.45
47.51 19.13
73,84
Table 31: Performance concerning AUROC (%) on Essay under paragraph-level settings. The detection model is trained text generated by Dolly.
Method
GPT4AII
ChaIGT
ChaiGPT.turbo
ChaiGLM
Dolly
Claude
Avg.
Likelihood
96.16+0.8
98.7940.19
99.13
99.29
90,90
92.76-0.22
96.17
Log-Rank
96.65+0.31
98.94+a.16
99,22
99.48+a.21
90.68 1.28
92.04 02.04-0.19 0.11
96.17
Entropy
7..52
90.3310.21
94.52
25.25
74.76
86.
84.43
NPR
97.77 L0.21
99.1610.12
47.82
99.55
95.23 2: 20.0
49.5 20000_.000
81.50
DelectGPT
95.01
97.08
46.03 37
96.194..00
92.64
46.94 16.194_.11
78.98
PasGPPT
67.33 1.00
72.35+1.44
96.81
76.43+0.55
45.40 45.40-1.98 1.99
82.70 82.70-0.74
73.50
ChaGGPTDD
93.43
92.97
86.10 as
98.81 0...1.00100 10
71.61 10ll2..0
49.83_9..o
ChatGPT-E
93.9441.20
94.18
88, 15.13411.
98.57
72, 12210=4..40
53.01
81.32
MPU
97.07
97.6210.30
94.71 79
99.63ga.3 01
87.51=0.72 =0.75
63.44g..00
92.50
MPU-E
97.35
97.82+0.10
95.25
99.58+ 0.
88.71=1.22
81. 18 18-3.22
93.37
RADAR
99.54+0.13
99.10+0.31
98.36+ so
99.79+0.11
96. 15..88-0.00 0.99
97. 07.81 0..0.
98.45
RADAR-E
99.64
99.18 0.22
98.35
99.83 20.0s
96.66
98.38 20.37
98.67
44

---

Table 32: Performance concerning TPR@FPR-I 1 (%) on Essay under paragraph-level settings. The detection model is trained on text generated by Dolly.
Method
GPT4All
CharGPT
ChaiGPT-tubb
ChaIGLM
Dolly
Avg.
Likelihood
16.33 16.33_16.49
68. 13.
73. 73.60 11.60011463
92. 5.
20.67 10.79
12.36 363366.
52.41
Log Rank
63.74_12.95 18
79.47
11.
96.61_a.o0
25.92 00
29.7
Entropy
3..81
11.07 12.00
16.31
8.35
3.91
6.22.1.88
8.27
NPR
78.50+a.99
65.91+1.06
9.02++0.57
15.13+1.73
58.52+7727
8.40+1.17
DetectGPT
31,53
39. 17 1..00
7.24
30.31
20.48 18.88..000
5.6422077
22.45
FastGPT
0.18+0.17
0.40++0.26
68.27:+4.92
1.70-u.7a
0.00+0..0
7.69+1145
13.04
CharGPTD
51.37
42. 1
30.19
82 1.:35
14.80
1.56
37.16
ChatGPT-R
50.71+2.40
39.42 09.442+0..7
30.09+4.112
79.60+2.87
12.94
1.91
35.78
MPU
70.75+7.12
69 4..1
18,58
95. 29.3.8
32.33
7.91
54.19
MPU-E
43.11=11.50
94.6443.29
31.07 +9.66
8.03+44.88
RADAR
91.48+8.50
79.1 79.11
$5.8220.60
96.21 96.21 #2.13
62.58
59 30.42
74.10
RADAR-E
92.12+4.11
81.07+4.52
55.33-+0.28
+1.39
63.48
66.13-9.38
75.72
Table 33: Performance concerning AUROC (%) Essay under paragraph-level settings. The detection model is trained on text generated by Claude.
Method
GPT4AII
ChaiGPT
ChaiGPT-urbo
CharGLM
Dolly
Claude
Ave.
Likelibond
96.1640.88
98.79.0.19
99.13.0.0
99.29
90.90 +:1 1.888
92. 02.76
96.17
Log-Rank
96.6510.a a1
98.941o.16
99.22.0.11
99.48
90.68 1:1
92.04 10
96.17
Entropy
75.52
90.3310:21
94.52
85.25
74.76
86.21 86.21 10
84.43
NPR
78.56
79.44+339.5
77.98-11.10
79.68
77.36-+36.0
49.45+00.0
68.74
DetecrPPT
58.991444
59.30 146.15
18.241.3
59.08
58.54..48
18.86
55.50
FastGPT
67.33+1.02
72.35+1.44
96.81+0..2
76.43+u.as
45.40-+1..8
82.70+0074
73.50
CharGPTDD
03.43
03.40
87.13 87.13=1.81
98.72
70.72
52.88
82,71
ChatGPT-E
93.8011.48
(6.6661100 .o1
88. 88.86111
98.5522.33 3.35
70.61 19..85
56.46.9.11
MPU
07.21
97.86
96.36_10.0
99.61
86.90
86.70
94.11
97.5110.91
10.39
99.59
88.27 .4
69.24
RADAR
99.7010.00
99. 19 19.2222
95. 98.59.1..0
99.92.u.04
96.1419.22
99.38 10.14
98.87
RADAR-E
99.74:0.0s
99.56-0.20
98.55--0.8
99.94:0.05
96.55+0.22
99.59 +.0.0
Table 34: Performance concerning TPR@FPR-1% (%) Essay under paragraph-level settings. The detection model is trained on text generated by Claude.
Mothod
GPT4AII
ChaiGPT
ChatGPT-tubo
CharGLM
Delly
Claude
Avg.
Likelibood
16.33=16.4
18..52
13.60
92.86
20.67
12.36
Log-Rank
63.74-12.00 11..9
79.47777.8
81.29 11.22
06.61+2.44
25.97 25.92++9.00
19.47 +8.24
61.08
Entropy
3.78
11.07
16.31=1.79
8.3543.44
3.91
6.22+1.86
8.27
NPR
63.28 2a1 87
68.80 134.04
7.29
75.76 01
46.54
6.31
44.66
DetectGPT
19.23_18.77
25.87
4.40
19.02_17.00
12.65
3.1112.5M
14,05
FasGPT
0.18+0.11
0.40
68.27 48.27
1.70+0.75
0.00=20.00
7.69
13.04
CharGPT-D
53.9913.50
45.73
32.67
83.12
14.70
2.4912.27
38.78
CharGPT-E
50.80+a,7e
42.5% +9..
32.40-4.47
79.42-47.66
13.17+1..6
2.71+2.11
MPU
71.4840.10
72.40
62.62
98.52
34.37
14.58
56.99
MPU-E
75.1715.85
70.27.16.4
01.8227.773
95.7129.18
33.1747.70
16.22:6.14
RADAR
95.40
90.09 90.0911..2
64.84 64.84115.00 115.09
98.48 98.4810.74
61.77
91.73113.0
83.72
11.78
91.24 2.66
64.31 115.02
98.84.0.00 10.50
62.86.6.00
1.2002
CharGPT-D
ChatGiT-E
MPL-E
RADAR-E
0.95
0.96
0.90
0.98
AUROC
0.92
0.98
0.88
0.88
0.94
0.84
0.84
0.80
6.72
Ton
0.68
Mixed PaLM
Mixod ChatGPT Text
Mixed Cloude
Mixed Liama-2 Toxt
Figure 12: Test performance (AUROC) under various LLM mixed texts. Detectors are trained on text generated by PaLM. For each sub-figure, the left group: detectors trained mixed text, and the right group: detectors are trained on original text.
45

---

ChatGPT-D
CharGiT-E
MPU
MPL-E
RADAR
RADAR-E
1.00
1,00
0.96
0.96
0.65
0,95
0.80
0.08
0.34
0.94
0.80
0.64
Back Transiaton
Polleh
Back Thanslation
Polish
Back Transiation
Po
Back Transaaionn 7o.lsh
PaLM Text
ChatEPT Text
Claude Text
Liama-2 Text
Figure 13: Robustness (AUROC) against paraphrasing attacks (Back Translation and Polish). Detec- tors are trained on the PaLM texts and tested on the paraphrasing texts of various LLMs.
6:06
115
+0:25
11.11
WUP enling smb
0.30
022
R75
-0.02
6:5
2:27
3.43
1.3
6:31
F25
3
Than
52
2/4
4.43
825
3.49
0:36
6.44
8.0
2.06
324
354
332
0.20
2.42
yd
Dum2n
Domain
1ol Domain
(a) ChatGFT.E
(B) MPU-E
Figure 14: The performance improvement of the enhanced ChatGPT-E and MPU-E compared to ChatGPT-D and MPU. The figure shows their differences for AUROC and TPR @ FPR-1. Positive values indicate performance improvement.
Table 35: Performance on newer LLMs.
The detection model is trained on text generated by GPT4AIL.
TRROFTR-IS
AUROC
GPT-4o
GPI-4.1
DeepSeck-RR RI
Liuna4 Maverick
Avg.
GPT-4o
GPT-4.1
DeepSeek--R
Liama4 Maverik
Arg.
ChaIGPTD
2.98
1.20
0.00
15.87
12.51
54.94
36.48
9.37
95:48
19.07
ChaiGPT-E
3.47
1.33
0.00
47.91
13.18
$7.68
38.14
12.00
95.90
50.93
MPL
3.07
0,13
59.00
17.39
76.37
63.5)
40.68
97.20
MPU-E
72.05
RADAR-E
75.18
99.67
Table 36: Running time under sentence-level setting.
Training Time
Test Time
Sentence-level
Essay
DetectRL
Essay
DetectRL
Likelihood
42.8
47.5
192.3
213.8
Log-Ram Rank
43.9
47.8
197.6
215.1
Entropy
42.6
47.0
190.5
213.3
NPR
1485.9
2396.7
6680.3
10774.8
DetectGPT
1769.6
2306.0
7955.6
10367.1
FastGPT
106.7
116.2
480.0
522.5
ChatGPT-D
28.7
29.5
5.9
6.3
ChatGPT-E
30.9
30.9
5.9
6.3
MPU
27.3
29.6
5.8
6.3
MPU-E
29.8
30.1
5.8
6.3
RADAR
76.7
78.2
16.9
18.4
RADAR-E
78.4
81.6
17.1
18.3
46

---

weo Enhanne
Erihance
15
TPR@ (RerFRR.3 1%
73
TPR
90
Chab.et
NPJ
RADAR
ChatGPT
MPU
MPU
RADAR
Google-PaLM
Figure 15: The Impact of noise label regarding TPR-FPR-1% on DetectRL dataset.
CharCPT-D
ChatGPT-Co
ChatGPTTSAM
ChatGPT-E
0.34
0.25
0.88
0.81
0.78
0.24
0.75
0.10
0.16
0.05
0.08
0.69
Maragraph
Peregrash
Peragrush
Setting
Setting
Setting
Setting
Essay
Figure 16: Performance comparison (AUROC and TPR@FPR-1S %) with NLL methods. The version applying Co-teaching and SAM ChatGPT-D is denoted ChatGPT-Co and ChatGPT-SAM. The supervisor is trained on PaLM texts on DetectRL. and GPT4AII texts on Essay.
ChatGPT-E
MPU-E
RADAR
0.4
0.8
0.2
TIPN FIRR-1
1.6
TPR@FPR FPR--1 9%
0.3
AUROC
0.96
0.4
0.2
0.1
0.3
0.3
0.8
(a) DetectRL Dataset
(b) Essay Dataset
Figure 17: Performance (AUROC and TPR @ FPR-11%) of the supervisor under different text number for longer text. The supervisor is trained on PaLM texts on DetectRL and GPT4AIl texts on Essay.
ChatGPT-E
MPU-E
- RADAR-E
0.10
TPR@FRR.I 156
AUROC
1PR@FPR:1K
0.45
AUROC
0.08
0.30
8.55
0.68
0.90
2
16 N'
32
64
128
16 N'
32
64
125
16
32
64
128
16
32
64
125
N'
11'
(a) DetectRL Dataset
(b) Essay Dataset
Figure 18: Performance (AUROC and TPR @ FPR-R99 1%) of the supervisor under different longer text numbers per batch. The supervisor is trained PaLM texts on DetectRL and GPT4AIl texts Essay.
47

---

ChalGPT-E
MPU-E
RADAR-E
0.80
0.75
0.75
18.100
AUROC
0.925
0.60
AUROC
TPROFPR
0.900
03
6.1
0.92
0.2
0.88
0.88
(a) DetectRL Dataset
(b) Essay Dataset
Figure 19: Performance (AUROC and TPR @ @FP-11%) of the supervisor under different supervision
loss coefficient The supervisor trained on PaLM texts on DetectRL. and GPT4All texts on Essay.
ChatGPT-E
MPU-E
RADAR-E
0.11
TPM8 FPR: 1.66
AUROC
0.87
0.08
0.20
0.72
TPREFPR- FP-
0.18
0.10
0.24
0.32
0.90
0.30
0.27
0.88
2
3
k
K
K
(a) DetectRL Dataset
(b) Essay Dataset
Figure 20: Performance (AUROC and TPR@FPR-I 1%) of the enhanced detectors under different text number & for longer text. The detector is trained on PaLM texts DetectRL and GPT4AIl texts Essay.
ChatGPT-E
MPU-E
RADAR-E
0.23
0.76
0.12
0.72
0.11
AUROC
0.87
1PA0FFR-1
0.27
AUROC
IPRGEPR
0.16
0.02
0.09
0.86
0.88
0.24
N'
N'
N
(a) DetectRL Dataset
(b) Essay Dataset
Figure 21: Performance (AUROC and TPR TPR@FPR-1%% of the enhanced detectors under different longer text numbers per batch. N' 0 represents the original model without enhancement. The detector is trained on PaLM texts on DetectRL and GPT4All texts on Essay.
ChatGPT-E
MPU-E
RADAR-E
0.76
0.11
0.76
AUROC
0.09
0.27
AUROC
0.73
0.84
PPR-FPR-2
0.10
0.88
0.82
0.90
0.32
0.28
0.88
0.88
A
(a) DetectRL. Dataset
(b) Essay Dataset
Figure 22: Performance (AUROC and T of the enhanced detectors under different supervision loss coefficient X. The detector is trained on PaLM texts on DetectRL. and GPT4AIl texts on Essay.
48

---

detector
0.7
detector
detector
detector orgin
0.96
oniinn
Finga
0.6
oniinn
Pingu
orgin Finge
Hingu
0.5
0.88
0.4
0.58
0.80
0.80
0.60
Original
Original
Original
Essay
Figure 23: Performance comparison with category-based supervision signals (Hinge and Gumbel). The detector is trained on PaLM texts DetectRL and GPT4AIl texts Essay.
49


Reviews:
[{"review_id": "Bq9YvvDtv4", "reviewer": "Reviewer_9f7L", "summary": "The paper proposes an easy-to-hard enhancement framework for machine-text detectors. It enhances the performance a detector by relying on supervision signals coming from another that is trained to distinguish longer texts. Relying on previous theoretical insights regarding the increase in distributional distance as the size of the text increases, the authors propose a framework where optimizing the supervisor indirectly optimizes the detector.", "questions": "•\tClarification – In Equation 3, there is $x^j$ multiplied by the Gumbel sample, however, if I understand it correctly the $x^j$ is in fact a string. Or, was it mean that the supervisor takes the set of Gumbel-softmax samples from the detector, and then predicts human or machine?\n\n•\tGenerally, I think a stronger Knowledge Distillation baseline is needed (W3), more ablations are necessary (W5), and the writing has to be polished (W1). Tackling those, in that order, would ease some of my concerns.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n•\tThe method is generally interesting and novel, and is a nice extension of Chakraborty’s work (https://arxiv.org/pdf/2304.04736) for enhancing the robustness of detectors. \n\n•\tThe author’s deal with the inherent uncertainty in the labels of human-written and machine-generated text, better focusing in on the more relevant case where the label is in fact smooth (partial machine and human).\n\nWeaknesses:\n\n•\tW1- I found the language of this paper very hard to follow, and it needs a lot of polishing before it’s published. \n\n•\tW2- Line 47 states that Figure 2 is clear in showing that label smoothing exhibits better performance. It’s not clear from the figure that the label smoothing variants do exhibit better performance. In fact, the variance of the results is very high, and in fact, many of the medians are worse than those that use hard labels (left hand side in particular). In general, a lot of the results on mixed texts tend to be quite marginal (Figure 4), with strong models (RADAR) remaining close to the performance of their enhanced counterparts. \n\n•\tW3- When comparing against knowledge distillation, it would’ve been good to also report distilling a version of RADAR that has been trained on long texts against a version of RADAR that hasn’t to more closely mimic your approach. \n\n•\tW4- I found that many of the theorems are quite similar to Chakraborty’s (https://arxiv.org/pdf/2304.04736) and distract from the most important one being Theorem 3.4 (the effectiveness of the approach). For example, increasing the length of the text  amplifies the distributional distance is something that was already shown by Chakraborty. Similar comments hold for Theorem 3.2, in that the larger the k (the larger the text), the better the AUROC is also something that is known from Chakraborty. \n\n•\tW5 – Ablations regarding the number of texts concatenated are missing. Also, although randomly splicing sentences does preserve the label as machine-generated or human-written, it does shift the distribution of natural text generated by humans or machines.", "was_revised": true, "final_justification": "The paper makes a significant contribution, extending the insight of Chakraborty's theorems into a recipe for training better machine-text detectors. The authors have run a good amount of experiments and addressed my concerns in their rebuttal.", "rebuttal": "Many thanks for your positive comments and constructive feedback. Please see the responses to your comments below.\n\n> **[W1]**  I found the language of this paper very hard to follow, and it needs a lot of polishing before it’s published.\n> \n\nThank you for your suggestion!  We will comprehensively revised the paper to make it clearer, including (1) streamlining the discussion of related work to only provide the essential background; (2) rewriting the introduction to consistently emphasize our contributions in each paragraph, e.g., highlighting the identification of the ‘inexact supervision’ in Paragraph 3 to provide clarity about the contribution of research scenario; (3) refining the notation for clarity, for example by updating $(x', y')$ to the more meaningful$(x_{long}, y_{long})$ and changing our model's suffix from '-E' to a more meaningful '-E2H' (Easy-to-Hard); and (4) correcting grammatical errors and typos, e.g., \"simple to hard\" was changed to \"easy to hard\". We are confident that these revisions will substantially improve the paper's readability.\n\n> **[W2]**  Line 47 states that Figure 2 is clear in showing that label smoothing exhibits better performance. It’s not clear from the figure that the label smoothing variants do exhibit better performance. In fact, the variance of the results is very high, and in fact, many of the medians are worse than those that use hard labels (left hand side in particular). In general, a lot of the results on mixed texts tend to be quite marginal (Figure 4), with strong models (RADAR) remaining close to the performance of their enhanced counterparts.\n> \n\nWe acknowledge that with such a coarse label smoothing strategy, even a small smoothing factor cannot prevent some labels from deviating further from the underlying ground-truth labels. This may introduce noise and lead to greater training instability, i.e., the high variance you pointed out. Nonetheless, it is undeniable that **the higher upper bound of label smoothing indicates its potential to enhance detection performance**. At the same time, **its instability highlights that addressing inexact supervision is a challenging task, which further highlights the necessity of our work.** To avoid confusion, we will revise the statement to: “a higher upper bound for label smoothing means that it has the potential for enhancement.” For the results shown in Figure 4, we believe that achieving consistent positive gains for state-of-the-art detectors, with negligible training overhead, constitutes a valuable contribution. \n\nThank you again for your feedback, which will further improve the clarity of our paper and better highlight our contributions.\n\n> **[W3]**  When comparing against knowledge distillation, it would’ve been good to also report distilling a version of RADAR that has been trained on long texts against a version of RADAR that hasn’t to more closely mimic your approach.\n> \n\nThanks for your insightful suggestion. A comparison between the distilled versions of RADAR and RADAR-E (our method) will provide better insight into whether our approach learns higher-quality underlying labels. Tables 1 and 2 below present the distillation detection performance on the Essay and DetectRL datasets, where Origin-KD and Ours-KD denote the original RADAR detector and our enhanced version (i.e., RADAR-E) guided MPU detector, respectively. Comparing Origin-KD and Ours-KD, we observe that the latter achieves superior performance, reflecting that our method yields higher-quality labels that are closer to the true underlying labels. Furthermore, when comparing distilled and original versions (Origin-KD vs. Origin, Ours-KD vs. Ours), the distilled versions perform worse. This indicates the limitations of knowledge distillation under inexact supervision and highlights the superiority of the proposed enhanced framework.\n\nTable 1. Performance (TPR@FPR-1%) of distilled MPU on DetectRL dataset.\n\n| Method | PaLM | ChatGPT | Claude | Llama-2 | Avg |\n| --- | --- | --- | --- | --- | --- |\n| Origin | 70.43  | 79.16  | 18.22  | 87.89  | 63.92  |\n| Origin-KD | 66.01  | 73.28  | 15.77  | 83.66  | 59.68  |\n| Ours-KD | 67.49  | 76.00  | 16.81  | 84.77  | 61.27  |\n| Ours | 75.75  | 84.23  | 19.78  | 90.31  | 67.52  |\n\nTable 2. Performance (TPR@FPR-1%) of distilled MPU on Essay dataset.\n\n| Method | GPT4All | ChatGPT | ChatGPT-turbo | ChatGLM | Dolly | Claude | Avg. |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Origin | 71.07  | 48.98  | 94.78  | 28.69  | 71.64  | 7.64  | 53.80  |\n| Origin-KD | 69.34  | 41.29  | 93.97  | 24.92  | 64.00  | 5.47  | 49.83  |\n| Ours-KD | 70.11  | 42.53  | 93.97  | 25.68  | 65.47  | 6.00  | 50.63  |\n| Ours | 78.31  | 52.09  | 96.88  | 32.08  | 74.09  | 9.20  | 57.11  |\n\n> **[W4]**  I found that many of the theorems are quite similar to Chakraborty’s (https://arxiv.org/pdf/2304.04736) and distract from the most important one being Theorem 3.4 (the effectiveness of the approach). For example, increasing the length of the text amplifies the distributional distance is something that was already shown by Chakraborty. Similar comments hold for Theorem 3.2, in that the larger the k (the larger the text), the better the AUROC is also something that is known from Chakraborty.\n> \n\nWe appreciate the reviewer’s careful observation regarding the connection with the foundational work [1]. Our theoretical results build upon Chakraborty’s foundational theory but differ substantially in several aspects:\n\n1. **Lower bounds vs. upper bounds.** While [1] establishes critical theoretical lower bounds for detection, we extend to the theoretical upper bounds (Theorem 3.2 and the right-hand side of Theorem 3.1) to further explore the potential of detection. Although they are both related to text length, they measure different bounds.\n2. **Guidance for the proposed framework.** In addition to exploring the detection potential, it is more important to guide the design of the proposed framework: by coupling the influencing factors of the supervisor’s upper bound with the detector, maximizing the supervisor's performance indirectly optimizes the detector, i.e., maximizing the supervisor's performance -> optimizing the influencing factors of the upper bound -> optimizing the detector. Instead, such a guided optimization mechanism cannot be achieved with the theoretical lower bounds.\n\nTherefore, although the findings revealed by these theorems appear similar, their bounds and contributions are fundamentally different. We will revise the manuscript to more clearly state that our work extends from lower bounds in prior theory to a guideable upper bound, thereby avoiding confusion with established theory in [1].\n\n> **[W5]**  Ablations regarding the number of texts concatenated are missing. Also, although randomly splicing sentences does preserve the label as machine-generated or human-written, it does shift the distribution of natural text generated by humans or machines.\n> \n\nThank you for your suggestion! We wish to clarify that the sensitivity analyses for the number of concatenated texts ($k$) have provided in the Appendix D.8: the influence of $k$ on the supervisor's performance is detailed in **Figure 16**, while its impact on the detector's efficacy is presented in **Figure 19,** where $k=1$ corresponds to the ablation condition without text concatenation. The results from both figures consistently indicate that an increase in $k$ leads to a corresponding performance improvement for both components. \n\nBesides, we fully agree that the distribution of concatenated sentences differs from that of natural text. Therefore, our framework takes this into account: **the target detector is trained only on the original, natural text,** which allows it to learn the true data distribution. Instead, **the concatenated text is used only to train the supervisor**. **This intentional separation ensures the distribution shift does not compromise the target detector's performance.** \n\n> **[Q1]**  Clarification – In Equation 3, there is $x_j$ multiplied by the Gumbel sample, however, if I understand it correctly the  $x_j$ is in fact a string. Or, was it mean that the supervisor takes the set of Gumbel-softmax samples from the detector, and then predicts human or machine?\n> \n\nIn our implementation, the product operator $\\odot$ is performed at the input embedding level, as shown in Line 187 of our provided code (https://anonymous.4open.science/r/Easy2Hard/methods/ChatGPT_D.py). Therefore, in Equation 3, $x^{(j)}$ should be revised with $e_{x^{(j)}}$, where $e_{x^{(j)}}$ denotes the input embedding corresponding to $x^{(j)}$. For example, if a longer text of length 3 is represented as $[x^{(1)},x^{(2)},x^{(3)}]$ and the detector gives the gumbel softmax value of them as $[1,0,1]$, then the input to the supervisor is $[e_{x^{(1)}},0,e_{x^{(3)}}]$. We will revise the notation accordingly in the updated version to avoid confusion.\n\n> **[Q2]**  Generally, I think a stronger Knowledge Distillation baseline is needed (W3), more ablations are necessary (W5), and the writing has to be polished (W1). Tackling those, in that order, would ease some of my concerns.\n> \n\nWe sincerely thank the reviewer for providing this clear summary of the main points for improvement. We have made relevant responses above, and we hope that our responses have addressed your concerns. If you have any further questions or suggestions, we welcome the rebuttal opportunity to communicate further.\n\n[1] Position: On the possibilities of AI-generated text detection. ICML 2024."}, {"review_id": "qaVeAfLxkM", "reviewer": "Reviewer_a1gn", "summary": "This paper tackles the challenge of detecting machine-generated text (MGT) under inexact or ambiguous labeling. The authors propose an easy-to-hard supervision framework where a simpler supervisor, trained on longer and more distinguishable texts, guides the training of a more complex target detector operating on shorter. The framework is theoretically justified and empirically validated across diverse datasets, tasks, and attack settings, showing consistent improvements over baseline detectors.", "questions": "1.\tThe paper builds on the premise that label ambiguity between human- and machine-generated text necessitates an easy-to-hard supervision framework. However, given the increasing fluency of modern language models, aren't there some case where human and AI texts are inherently indistinguishable? How does the proposed method address such ambiguity?\n2.\tHow do the authors justify that longer texts improve separability between human and machine-generated content? Given that prior work [1] [2] links their difference to long-tail distributions, what is the connection between longer-text construction and capturing those distinctive patterns?\n3.\tWhile the paper provides extensive quantitative results, are there any concrete case studies ？These case studies may conduct between the proposed method and baselines. Such comparisons would help illustrate how the method improves detection in practice.\n4.\tTo what extent does the effectiveness of the proposed method depend on the quality of the original labels? Given that even humans often fail to distinguish between real and machine-generated texts, how does the framework handle ambiguous?", "limitations": "1.\tWhile text length correlates with detectability supported theoretically, real-world MGT (e.g., headlines or tweets) often appear in short form, where this assumption may break down.\n2.\tAlthough the paper constructs longer texts from labeled data, if the original labels are flawed or biased, the framework may reinforce these issues.\n3.\tSome theoretical results assume idealized conditions (e.g., total variation distances, independence), which may not strictly hold in practice. These need some evidences empirically.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Weaknesses:\n1.\tFigure 1 is the estanblised information from model collapse papers[1] [2];\n2.\tTheorem 3.1 Distribution Difference for Longer Text lacks of emprical supports.\n3.\tThe Easy-to-Hard Supervision Framework seems a bit complex. The paper assumes the existence of underlying \"golden\" labels approximated by machine-likeness, but this definition may lack grounding in real-world annotation practices. I’m just wondering—would using only supervised fine-tuning (SFT) be sufficient to achieve good performance on your collected data?\n\n\nStrengths:\n1.\tThis paper provides solid formal theorems to justify why longer texts are more discriminative and how the supervisor’s performance bounds the detector.\n2.\tAuthors conduct comprehensive experiments across tasks, detectors, and attack scenarios, with consistent gains.\n[1] Shumailov I, Shumaylov Z, Zhao Y, et al. AI models collapse when trained on recursively generated data[J]. Nature, 2024, 631(8022): 755-759.\n[2] How to Synthesize Text Data without Model Collapse?, ICML, 2025", "was_revised": true, "final_justification": "I think they have well addressed my concerns, and I think it is a good paper.", "rebuttal": "Many thanks for your positive comments and constructive feedback.\n\n> **[W1]**  Figure 1 is the estanblised information from model collapse papers;\n> \n\nWe thank the reviewer for raising the interesting issue of model collapse. Although Figure 1 may resembel model collapse visually, the underlying mechanisms differ substantially:\n\n- **Model and data**. Model collapse is a degenerative process affecting generations of learned generative models, in which the data they generate end up polluting the training set of the next generation. However, MGT detector is a binary classifier (predicting the probability of AI-generated), not a generative model. Its training data is fixed and static, with no iterative generation-feedback loop.\n- **Phenomenon**. Model collapse leads to an accumulation of low-perplexity texts and a loss of the long-tail characteristic of human text. Instead, in the inexact supervision detection, the opposite occurs: predictions for human text concentrate at the head, while AI-generated text shows a pronounced long-tail distribution (see Figure 1 in our paper). This is because AI-generated texts possess human-like characteristics, resulting in a wide range of prediction probabilities.\n\n> [**W2**]  Theorem 3.1 Distribution Difference for Longer Text lacks of emprical supports.  \n[**Q2-1**] How to justify that longer texts improve separability between human and machine-generated content?\n> \n\nThanks for your suggestion. Due to the challenges of directly computing distributional differences, we follow prior work [1] and use detection performance (AUROC) as a proxy, leveraging its theoretical connection to the total variation (TV) distance between distributions (see Proposition 1 in [1]). We evaluate detector performance (ChatGPT-D and MPU) across varying maximum text lengths, as shown in Tables 3 below. Results demonstrate that longer texts enhance detection, indicating it tends to have better separability. \n\nTable 3. Performance (AUROC) under different maximum text lengths (32-192) on DetectRL.\n\n|  | 32 | 64 | 96 | 128 | 160 | 192 |\n| --- | --- | --- | --- | --- | --- | --- |\n| ChatGPT-D | 73.06 | 79.90 | 80.03 | 80.84 | 81.42 | 83.24 |\n| MPU | 85.64 | 90.77 | 92.21 | 92.03 | 92.32 | 92.42 |\n\n> **[W3]**  The Easy-to-Hard Supervision Framework seems a bit complex. … I’m just wondering—would using only supervised fine-tuning (SFT) be sufficient to achieve good performance on your collected data?\n> \n\nThank you for your valuable comment!\n\n- **Regarding the framework's complexity.** Our framework does not change the original detector, but incorporating an auxiliary supervisor throughout the training process adds negligible overhead, as shown in Table 3 in the paper. Therefore, this supervisor can be designed as a lightweight component that avoids bottlenecks, ensuring seamless integration into standard large-scale training pipelines. In essence, our framework introduces a training objective (Equation 5), instead of making restrictive architectural changes that would hinder scalability. This ensures the simplicity and flexibility of our framework.\n- **About using only supervised fine-tuning techniques.** We clarify that the baseline methods to be enhanced all use fine-tuning techniques. As specified in Line 955, ChatGPT-D, MPU, and RADAR are all fine-tuned on the collected dataset. Extensive comparisons demonstrate that our easy-to-hard supervision strategy outperforms theirs.\n\n> **[Q1]**  … aren't there some case where human and AI texts are inherently indistinguishable? How does the proposed method address such ambiguity?  \n**[Q4]**  To what extent does the effectiveness of the proposed method depend on the quality of the original labels? Given that even humans often fail to distinguish between real and machine-generated texts, how does the framework handle ambiguous?\n> \n\nWe agree that some texts are inherently indistinguishable (e.g., “Please sit down”) and thus, effective detection—relying on quantifiable differences in data—cannot address such cases, which are beyond the current scope of MGT detection. Notably, the proposed framework does not aim to magically detect such inherently indistinguishable texts. Instead, our strength lies in providing reliable likelihood scores for texts that are challenging yet still differentiable, rather than a simple binary “yes” or “no” judgment. This is achieved by coupling the detector and supervisor via a gating mechanism (Equation 3), where optimizing the supervisor aligns the expected Gumbel output with the true soft label (Theorem 3.4), thus enhancing detection. Experimental results (see our response to your Q3) further confirm that the proposed method can learn more generalized detection scores.\n\nFinally, we emphasize that MGT detection and LLM development is a continuous “arms race”: as detectors strengthen, LLMs evolve to generate higher-quality, less distinguishable text, driving further advances on both sides. Ideally, LLMs produce inherently indistinguishable text and detectors will ultimately fail. We look forward to seeing such advances in generated text, as they will benefit the development of LLMs.\n\n> **[Q3]**  While the paper provides extensive quantitative results, are there any concrete case studies？…  \n**[Q2-2]**  What is the connection between longer-text construction and capturing those distinctive patterns?\n> \n\nThank you for your comment! Traditional case studies are suitable for metric-based methods with specific interpretable features, but are challenging for complex, black-box model-based detectors which our work focuses on. To this end, we introduce a knowledge distillation-based analysis method, which can also intuitively demonstrate the effectiveness of our method. Specifically, since the supervisor guides the detector to learn more precise “golden” labels (Theorem 3.4), the detector’s improvement should stem from learning better soft scores from the supervisor. Thus, we compare student models distilled respectively from detectors before and after enhancement: if the student of the enhanced detector outperforms the original, it confirms that our approach has enabled the detector to learn better detection scores from the supervisor, thereby improving detection performance.\n\nDue to space limitations, the results refer to Table 1 and Table 2 in the response to Reviewer 9f7L. By comparison, we observe that Ours-KD achieves superior performance, proving that the proposed method learns better detection scores from the supervisor.\n\n> **[L1]**  While text length correlates with detectability supported theoretically, real-world MGT (e.g., headlines or tweets) often appear in short form, where this assumption may break down.\n> \n\nThank you for this important question. In MGT detection, short texts often fall into the “inherently indistinguishable” category you mentioned above and thus commonly excluded, as seen in MMD [2] (filtering texts <5 words) and M4GT-Bench [3] (filtering texts <50 words).\n\nNonetheless, the proposed Easy-to-Hard framework offers hope for short-text detection: our framework does not impose length restrictions on the target detector or the supervisor, suggesting potential for short-text detection. Although this is beyond our work, we consider this a valuable direction for future research.\n\n> **[L2]**  Although the paper constructs longer texts from labeled data, if the original labels are flawed or biased, the framework may reinforce these issues.\n> \n\nWe thank the reviewer for this critical question. The proposed framework may help mitigate, rather than reinforce. Assuming b% of the machine text is mislabeled (similar analysis for mislabeled human text), i.e., b% of  human text mixed in, this increases the mixed ratio $\\alpha$ in Theorem 3.1. Similar to the proof of Theorem 3.4, optimizing the supervisor encourages the detector to predict those noisy texts as 0 (i.e., correctly classified as human text); otherwise, the supervisor can be further optimized when $\\alpha’>\\alpha$.\n\nTo further verify, we randomly flipped 10% of the labels and then evaluated detectors trained on these noisy data, as shown in Table 4 below. First, we found that the proposed enhanced framework remained effective. Second, compared to the noiseless results in Tables 1 and 2 in the paper, our method is generally less affected by noise, verifying our mitigation efforts.\n\nTable 4. The Impact of noise label regarding TPR-FPR-1% on DetectRL dataset.\n\n| Method | Google-PaLM | ChatGPT | Claude-instant | Llama-2-70b | Avg. |\n| --- | --- | --- | --- | --- | --- |\n| ChatGPT-D | 20.77 | 14.26 | 3.44 | 38.15 | 19.15 |\n| **ChatGPT-E** | **21.98** | **20** | **4.45** | **42.03** | **22.11** |\n| MPU | 60.40 | 64.55 | 12.61 | 77.87 | 53.86 |\n| **MPU-E** | **64.52** | **70.80** | **14.81** | **81.51** | **57.91** |\n| RADAR | 71.92 | 77.13 | 31.55 | 80.42 | 65.25 |\n| **RADAR-E** | **80.05** | **80.87** | **39.73** | **84.08** | **71.18** |\n\n> **[L3]**  Some theoretical results assume idealized conditions (e.g., total variation distances, independence), which may not strictly hold in practice.\n> \n\nThanks for your question. First, we assume a nonzero total variation distance between generated and human text, which is practical: if $TV(m,h)=0$, any text are inherently indistinguishable and detection becomes meaningless. Second, our assumption of text comprising $n$ independent sequences is general, as it holds even for dependent text when $n=1$. We will clarify these points in the revised paper.\n\nDue to space limitations, we only provide DetectRL results in our reply. Similar findings are found in Essay, and we are happy to continue to provide full results during the discussion phase if needed.\n\n[1] Position: On the Possibilities of AI-Generated Text Detection. ICML’24.\n\n[2] Detecting machine-generated texts by multi-population aware optimization for maximum mean discrepancy. ICLR’24.\n\n[3] M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection. ACL’24."}, {"review_id": "PMvYpfuB1t", "reviewer": "Reviewer_7nNE", "summary": "In this work, the authors propose an easy-to-hard enhancement framework for training a detector on machine-generated text. It employs an easy supervisor targeting longer-text detection tasks to enhance the more challenging target detector. Extensive experiments validate the effectiveness of the proposed approach.", "questions": "See weakness", "limitations": "N/A", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n1. The paper is well-written and easy to follow.\n\n2. The proposed easy-to-hard claim seems to be reasonable and practical.\n\n# Weakness\n\n1. The authors claim that the label of machine-generated text is hard to obtain but I cannot figure out the reason. I can simply use the text generated by LLM as the machine generated text. Do your indicate that it might be hard to find some human generated text currently?\n\n2. It is unclear how the supervisor helps improve the detector using the same dataset as depicted in Algorithm 1.\n\n3. Why does a single weak model (supervisor) can lead to a powerful model (detection).\n\n4. It might be unfair for comparison by using the baselines trained on different datasets.", "was_revised": true, "final_justification": "I think my main concerns have been addressed and tend to raise my score.", "rebuttal": "Thank you for the valuable feedback! Please find the point-to-point responses below. Any further comments and discussions are welcome!\n\n> **[W1]**  The authors claim that the label of machine-generated text is hard to obtain but I cannot figure out the reason. I can simply use the text generated by LLM as the machine generated text. Do your indicate that it might be hard to find some human generated text currently?\n> \n\nWe fully agree that **obtaining a text’s categorical label (hard label) is not difficult. Instead, the paper emphasize the difficulty of obtaining the underlying “golden” label (soft label)**, which needs to be distinguished from the categorical label you mentioned: given the increasing generative capabilities of LLMs and the extensive human–machine collaboration that blurs the boundary between them (as shown in Figure 1 in the paper), the text’s inherent “golden” label may not simply be “yes” or “no”, but rather a nuanced score indicating its “machine similarity” (the effectiveness of label smoothing in Figure 2 verifies this). In other words, while we can easily acquire a categorical label by determining whether the text is generated by an LLM, its intrinsic golden label (e.g., might be 0.85) is difficult to obtain because assigning such a score lies beyond human cognition. To avoid confusion, we will revise the manuscript to explicitly differentiate between the easily obtained “categorical label” and the hard-to-obtain “golden label”.\n\n> **[W2]**  It is unclear how the supervisor helps improve the detector using the same dataset as depicted in Algorithm 1.  \n**[W3]**  Why does a single weak model (supervisor) can lead to a powerful model (detection).\n> \n\nThank you for this comment! We will clarify how the supervisor enhances detection from both the methodological and evaluation perspectives:\n\n- **Methodology**. Although the supervisor’s data comes from the same dataset as the detector, the supervisor is trained on longer reconstructed texts gated by the detector, thereby guiding the detector to learn more precise “golden” labels for enhancement. Specifically, Theorem 3.2 shows that the supervisor’s performance is the lower bound of $1-1/2\\cdot(1-\\delta)^{2nk(1-\\alpha)}$. This allows us to couple the detector’s performance to these factors (e.g., $k$), then optimizing the supervisor will optimize the relevant factors, thus improving the detector. To this end, we employ a Gumbel-based gating mechanism (Equation 3) linking $k$ to detector performance, such that a larger $k$ corresponds to higher detector accuracy. Accordingly, the enhancement roadmap is: **maximize supervisor performance → maximize $k$ → maximize detector performance**. More importantly, Theorem 3.4 indicates that **maximizing supervisor performance drives the mathematical expectation of the Gumbel distribution (i.e., the detector’s prediction) to converge to the underlying “golden” label.** In contrast, traditional supervised training forces the detector to converge to a hard label (i.e., 0 or 1). Given the inherent ambiguity of generated texts (Figure 1), this convergence to the underlying “golden” label is central to achieving enhancement.\n- **Verification**. To further verify whether the supervisor guides the detector to learn more accurate \"golden\" labels for enhancement, we introduce a knowledge distillation-based analysis method for verification. Specifically, we use the detectors before and after enhancement as teacher models, distill a student model respectively. By comparing the performance of these student models, we can verify whether the supervisor provides more accurate supervision signals. If the student distilled from the enhanced detector outperforms the student distilled from the original detector, it indicates that our approach has indeed enabled the detector to learn better detection scores from the supervisor, thereby improving detection performance. Tables 5 and 6 below present the distillation performance on the Essay and DetectRL datasets, where Origin-KD and Ours-KD denote the original RADAR detector and our enhanced RADAR (i.e., RADAR-E) guided MPU detector, respectively. By comparison, we observe that Ours-KD achieves superior performance, proving that the proposed method learns better detection scores from the supervisor to enhance detection.\n\nIn summary, we methodologically and empirically demonstrate that weak supervisors guide target detectors to learn more accurate “golden” labels for augmentation.\n\nTable 5. The Impact of Teacher Model (RADAR) Enhancement on Knowledge Distillation on DetectRL dataset.\n\n| Method | PaLM | ChatGPT | Claude | Llama-2 | Avg |\n| --- | --- | --- | --- | --- | --- |\n| Origin-KD | 66.01  | 73.28  | 15.77  | 83.66  | 59.68  |\n| Ours-KD | 67.49  | 76.00  | 16.81  | 84.77  | 61.27  |\n\nTable 6. The Impact of Teacher Model (RADAR) Enhancement on Knowledge Distillation on Essay dataset.\n\n| Method | GPT4All | ChatGPT | ChatGPT-turbo | ChatGLM | Dolly | Claude | Avg. |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Origin-KD | 69.34  | 41.29  | 93.97  | 24.92  | 64.00  | 5.47  | 49.83  |\n| Ours-KD | 70.11  | 42.53  | 93.97  | 25.68  | 65.47  | 6.00  | 50.63  |\n\n> **[W4]**  It might be unfair for comparison by using the baselines trained on different datasets.\n> \n\nApologize for the unclear experimental setup. In fact, the compared ChatGPT-D, MPU, and RADAR are all fine-tuned on the used datasets (i.e., Essay and DetectRL) in our experiments, as described in Line 955."}, {"review_id": "ttbNDYQUb6", "reviewer": "Reviewer_Vuke", "summary": "This paper presents a new algorithm for classification-based detection of AI-generated text.\n\nThe key intution in the paper's method is that longer chunks of text provide a more reliable supervision for detecting AI-written text, a fact that's been well established in prior literature.\n\nThis paper's converts this intuition into a training-time algorithm. During training, gumbel-softmax sampled predictions over continguous text chunks are aggregated based on the detector's current weights, and then sent to an external \"supervisor\" network. The supervisor and detectors are then jointly optimized, with a much higher weight (10:1) on the supervisor network.\n\nInstead of optimizing on potentially noisy individual text labels, the detector is incentivized to optimize for detecting a longer chunk more holistically --- potentially learning more generalizable features.", "questions": "See review above", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 4, "strengths_and_weaknesses": "*Strengths*\n\n1. This is a pretty interesting and novel formulation for AI-generated text detection modeling. It's a nice conversion of the \"long text is easier to detect\" to an operational learning algorithm. I couldn't think of any cleaner alternative formulations during the review.\n\n2. The proposed formulation seems to beat ablated baselines across all tested settings, proving its effectiveness.\n\n3. The paper has great experimental breadth, including experiments on multiple datasets, base LLMs, detector baselines. The paper also includes experiments on paraphrased outputs and mixed-text detection which strengthens the contribution.\n\n4. The proposed method has no extra latency at inference time, and only a slight increase in training time latency.\n\n*Weaknesses*\n\n1. The base LLMs (whose text is being detected) in this paper are quite old --> ChatGPT / GPT-turbo / Claude / PaLM are models from 2022-2023. Do these methods generalize to newer LLMs from 2024/2025, which likely have a very different distribution?\n\n2. The paper was lacking in qualitative analysis and the kinds of features being learnt by this detector (vs the baseline). I would imagine this detector has learnt more robust / generalizable features pertaining to AI-generated text rather than overfit to noisy datapoints. Is there any qualitative evidence / win-analysis over individual test datapoints to justify this?\n\n3. The paper's presentation could really be improved, it was really hard to understand the paper's contributions from the abstract/introduction. I felt the authors were underselling their work by talking mostly about previous efforts. The mathematical notation and naming is also confusing. For instance, the notation in L179, the use of `'` is too subtle to distinguish differences between detector and supervisor labels. Also in L244 the naming isn't great, looking at the table it's hard to understand `-E` refers to the method proposed in this work.\n\n4. As a thought experiment, would this method work at scale? While I agree that supervising long-form detection indirectly is a good inductive bias in this setup, tradtionally inductive biases have usually lost out to scaled training (http://www.incompleteideas.net/IncIdeas/BitterLesson.html).", "was_revised": true, "final_justification": "Nice new experiments on new LLMs which should be incorporated in the final version. Agreement with other reviewers.", "rebuttal": "Many thanks for your positive comments and constructive feedback. Please see the below responses to your comments.\n\n> **[W1]**  The base LLMs (whose text is being detected) in this paper are quite old --> ChatGPT / GPT-turbo / Claude / PaLM are models from 2022-2023. Do these methods generalize to newer LLMs from 2024/2025, which likely have a very different distribution?\n> \n\nThanks for your valuable suggestion. Based on the Essay prompts, we employed the latest LLMs—GPT-4o, GPT-4.1, DeepSeek-R1, and Llama4 Maverick—to generate machine text. The detection results are shown in Tables 7 and 8 below, where detectors are trained on GPT4All texts from Essay dataset. As can be seen, our proposed enhancement strategy remains significantly effective on these newer LLMs. Moreover, compared to the LLMs used in Table 2 of our paper, the detection performance on these more advanced LLMs has declined, underscoring the ongoing arms race between detection and large-model development: detectors design stronger strategies based on current generation models, and then newly emerging LLMs produce higher-quality text that is harder to distinguish, thus driving further progress on both sides.\n\nTable 7. Performance concerning TPR@FPR-1% (%). The detection model is trained on text generated by GPT4All.\n\n| Method | GPT-4o | GPT-4.1 | DeepSeek-R1 | Llama4 Maverick | Avg. |\n| --- | --- | --- | --- | --- | --- |\n| ChatGPT-D | 2.98 | 1.20 | 0 | 45.87 | 12.51 |\n| **ChatGPT-E** | **3.47** | **1.33** | 0 | **47.91** | **13.18** |\n| MPU | 6.76 | 3.07 | **0.13** | 59.60 | 17.39 |\n| **MPU-E** | **8.53** | **3.64** | 0.09 | **62.18** | **18.61** |\n| RADAR | 66.67 | 50.53 | 55.78 | 86.09 | 64.77 |\n| **RADAR-E** | **77.47** | **63.16** | **68.53** | **91.56** | **75.18** |\n\nTable 8. Performance concerning AUROC (%). The detection model is trained on text generated by GPT4All.\n\n| Method | GPT-4o | GPT-4.1 | DeepSeek-R1 | Llama4 Maverick | Avg. |\n| --- | --- | --- | --- | --- | --- |\n| ChatGPT-D | 54.94 | 36.48 | 9.37 | 95.48 | 49.07 |\n| **ChatGPT-E** | **57.68** | **38.14** | **12.00** | **95.90** | **50.93** |\n| MPU | 76.37 | 63.57 | 40.98 | 97.20 | 69.53 |\n| **MPU-E** | **79.02** | **66.81** | **44.84** | **97.54** | **72.05** |\n| RADAR | 98.79 | 97.27 | 98.55 | 99.50 | 98.53 |\n| **RADAR-E** | **99.18** | **98.09** | **99.01** | **99.67** | **98.99** |\n\n> **[W2]**  The paper was lacking in qualitative analysis and the kinds of features being learnt by this detector (vs the baseline). I would imagine this detector has learnt more robust / generalizable features pertaining to AI-generated text rather than overfit to noisy datapoints. Is there any qualitative evidence / win-analysis over individual test datapoints to justify this?\n> \n\nTraditional case studies are more suitable for metric-based methods, where detection are based on specific interpretable features. However, it is challenging for model-based detectors that our work focuses on. This is because they are highly complex black-box models that lack interpretable features. \n\nFortunately, we introduce a knowledge distillation-based analysis method to verify how the proposed method improves detection. Specifically, the core of our framework is that the supervisor guides the detector to learn more exact “golden” labels (Theorem 3.4). Thus, the detector’s improvement should stem from learning better soft scores from the supervisor. Accordingly, we can use the detectors before and after enhancement as teacher models and distill a student model respectively.  By comparing the performance of these student models, we can assess whether our method strengthens detection through this mechanism. If the student distilled from the enhanced detector outperforms the student distilled from the original detector, it indicates that our approach has indeed enabled the detector to learn better detection scores from the supervisor, thereby improving detection performance. Tables 9 and Table 10 below present the distillation performance on the Essay and DetectRL datasets, where Origin-KD and Ours-KD denote the original RADAR detector and our enhanced RADAR (i.e., RADAR-E) guided MPU detector, respectively. Comparing  Origin-KD and Ours-KD, we observe that the latter achieves superior performance, proving that the proposed method learns better detection scores from the supervisor, echoing Theorem 3.4.\n\nTable 9. The Impact of Teacher Model (RADAR) Enhancement on Knowledge Distillation on DetectRL dataset.\n\n| Method | PaLM | ChatGPT | Claude | Llama-2 | Avg |\n| --- | --- | --- | --- | --- | --- |\n| Origin-KD | 66.01  | 73.28  | 15.77  | 83.66  | 59.68  |\n| Ours-KD | 67.49  | 76.00  | 16.81  | 84.77  | 61.27  |\n\nTable 10. The Impact of Teacher Model (RADAR) Enhancement on Knowledge Distillation on Essay dataset.\n\n| Method | GPT4All | ChatGPT | ChatGPT-turbo | ChatGLM | Dolly | Claude | Avg. |\n| --- | --- | --- | --- | --- | --- | --- | --- |\n| Origin-KD | 69.34  | 41.29  | 93.97  | 24.92  | 64.00  | 5.47  | 49.83  |\n| Ours-KD | 70.11  | 42.53  | 93.97  | 25.68  | 65.47  | 6.00  | 50.63  |\n\n> **[W3]**  The paper's presentation could really be improved, it was really hard to understand the paper's contributions from the abstract/introduction. I felt the authors were underselling their work by talking mostly about previous efforts. The mathematical notation and naming is also confusing. For instance, the notation in L179, the use of is too subtle to distinguish differences between detector and supervisor labels. Also in L244 the naming isn't great, looking at the table it's hard to understand refers to the method proposed in this work.\n> \n\nThank you for your thoughtful suggestion, which provides a clear roadmap for improving the paper. We will address each of the points:\n\n- **Clarification of Contributions**. We will revise the abstract and introduction to focus on our contributions. In the abstract, we remove the discussion of knowledge distillation to more directly state our contributions regarding the research problem, technical framework, theoretical insights, and extensive evaluations. In the introduction, we remove the discussion on noisy label learning and streamline the second paragraph to provide the necessary background instead of underselling it. Furthermore, we emphasize our key contributions more: the third paragraph will highlight the significance of identifying the 'inexact supervision' problem, and the final paragraph will underscore the novelty of our proposed 'easy-to-hard supervisor' framework from the technical and theoretical aspects. These revisions are intended to give readers a clearer understanding of our work’s value and novelty.\n- **Clarification of Notation**. The supervisor’s input will be updated from $(x'',y')$ to the more meaningful $(x_{long},y_{long})$ to distinguish it from the detector input $(x,y)$ to avoid confusion. In addition, we change the enhanced version from the suffix “-E” to “-E2H” (i.e., easy-to-hard), and emphasize it in bold in L244.\n\nIn addition, we will comprehensively correct grammatical errors and typos in the revised paper. Thank you again for your careful review, and we are grateful for the opportunity to improve our paper with these valuable suggestions.\n\n> **[W4]**  As a thought experiment, would this method work at scale? While I agree that supervising long-form detection indirectly is a good inductive bias in this setup, tradtionally inductive biases have usually lost out to scaled training (http://www.incompleteideas.net/IncIdeas/BitterLesson.html).\n> \n\nWe thank the reviewers for raising questions regarding scalability in the context of “The Bitter Lesson.” We examine the scalability of our approach to large-scale scenarios from the perspectives of computational compatibility and framework generality.\n\n- **Computational compatibility.** A key aspect of our framework is its computational efficiency. Incorporating an auxiliary supervisor throughout the training process adds negligible overhead, as shown in Table 3 in the paper. This supervisor can be designed as a lightweight component that avoids bottlenecks, ensuring seamless integration into standard large-scale training pipelines with large models and massive datasets. In essence, the bias is in the training objective (Equation 5), not in a restrictive architectural change that would hinder scaling.\n- **Framework generality.** The proposed framework applies to model-based approaches that rely on data and computing power rather than metric-based methods centered on specific features. This reflects the philosophy of “The Bitter Lesson”—instead of forcing explanations of the detection mechanism through specific features, we learn to construct the most effective representations for detection using data and computation. Moreover, the enhancement framework is not built on narrowly designed knowledge; rather, it represents a general learning architecture under inexact supervision by using an easier supervisor to refine labels for the target model. Its applicability may extend beyond text detection to any inexact supervision problem characterized by inherently ambiguous labels.\n\nEssentially, our framework employs an efficient computational method (training the supervisor on longer texts) to provide better learning signals for another method (the detector), rather than imposing a rigid, artificially designed bias that limits learning and makes it scalable. Thank you again for this thought-provoking question, which provides a clearer positioning for our future exploration on larger-scale data."}]

Author rebuttal:

Meta-review: This paper proposes a novel perspective on machine-generated text detection through an easy-to-hard enhancement. Reviewers are uniformly positive, highlighting the strong technical contributions, solid empirical validation across diverse datasets, and clear presentation. The rebuttal further clarified minor questions and reinforced confidence in the work. Given the overall enthusiasm and the paper’s timely and well-executed contributions, I recommend acceptance.
Decision: Accept (poster)