DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
Authors: Yongxin He
Shan Zhang
Yixuan Cao
Lei Ma
Ping Luo
Venue: NeurIPS 2025 poster ()
Keywords: AI-generated text detection; hybrid text attribution; few-shot adaptation;
Primary area: evaluation

Abstract: Detecting AI-involved text is essential for combating misinformation, plagiarism, and academic misconduct. 
However, AI text generation includes diverse collaborative processes (AI-written text edited by humans, human-written text edited by AI, and AI-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex characteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. AI-involved) or multi-classification (treating human-AI collaboration as a new class).
We observe that representations of texts generated through different processes exhibit inherent clustering relationships.
Therefore, we propose DETree, a novel approach that models the relationships among different processes as a Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, a comprehensive benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes.
Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions, further demonstrating the promise of training-based approaches in OOD settings. 
Our code and dataset are available at https://github.com/heyongxin233/DETree.

DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
Yongxin He
Shan Zhang 2.3,4 Yixuan Ca Lei Ma Ping Luo!
State Key Laboratory of Al Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
"hhe Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China University of Chinese Academy of Sciences, CAS, Beijing, China
Abstract
Detecting Al-involved text is essential for combating misinformation, plagiarism, and academic misconduct. However, Al text generation includes diverse collabora- tive processes (AI-written text edited by humans, numan-written text edited by AI, and Al-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex char- acteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. Al-involved) or mulli-classification (treating human AI collaboration as new class). We observe that representations of texts generated through different processes exhibit inherent clustering relationships. Therefore, we propose DETree, novel approach that models the relationships among different processes Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, comprehensíve benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes. Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization ios, particularly in few-shot learning conditions, further demonstrating the promise of training- approaches in OOD settings. Our code and dataset are available at https:://// / /tti/giiitt thub. com com//heeong::nn233/EETree
disttiibtiiinnn scenar-
Introduction
With the widespread deployment of large language models (LLMs) 2 5)
Al-involved text generation has become increasingly diverse, encompassing LLM-assisted refinement of human drafts, human revision of LLM outputs, and collaborative generation involving multiple LLMs, among other strategies. In practice, tolerance for AI involvement varies across scenarios. For example, LLM-based editing may be acceptable in copywriting, whereas it is strictly prohibited originality- focused contexts. This necessitates detection methods that can identify not only whether Al is involved, but also how it involved and to what extent. Despite this need, existing Al-text detection are typically developed and evaluated specific datasets, focusing binary methods 66, 7 9
"qqual contribution.
Corresponding author: cacyixuan@ice ac. cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

classification between purely weitttn and purely machine- generated content. However, such approaches often struggle to generalize complex scenarios involving human Al collaborative text.
Although recent studies have attempted to detect human-Al collaborative text, such as by regressing the degree of L.M involvement [10] classifying text based content and style features from predefined prompts [T.. However, these methods rely on coarse-grained estimation of Al involvement or shallow statistical representations, thereby limiting their applicability real-world detection tasks.
To address these issues, we construct RealBench, a large-scale benchmark designed to reflect practical hybrid text detection settings, which encompasses diverse human- AA collaboration modes, spanning 1,204 text categories (e.g., Llama3 _aanss_pplshh___e 4P [2] [2], human Lalllpoollshhhaeeeeeeeee Geminil. ..5 (13) and approximately 16.4 million text samples. The dataset covers wide range of generation types, and its distribution closely reflects real-world usage patterns. We investigate the categorization of hybrid texts and find that, in human-Al collaborative writing, traces of AT involvement tend be more prominent than human characteristics.
We propose novel representation learning-based approach, which incorporates structured source modeling, going beyond traditional flat classification. We observe that texts produced by different generation mechanisms naturally exhibit varying degrees of relational similarity. For example, the similarity between texts labeled as "Llama3 "Llama3_polish_____ GPT-4o" and "Claude3.5_pp _aarpphh_aappaaaeee___a GPT- 4o" [14) is higher than their similarity with "human polishppolhhhhhheee Geminil.5", and lowest when compared to purely human- written text. To model this structure, we propose an adaptive algorithm that constructs Hierarchical Affinity Tree (HAT) from the inter-class similarity matrix, with the flexibility to incorporate task-specific structural adjustments. HAT captures intrinsic affinities across arbitrary categories and offers strong interpretability and scalability. Building upon HAT, we develop a Tree-Structured Contrastive Loss (TSCL) that explicitly aligns the embedding space with the hierarchical structure. This alignment enhances representation quality.
Moreover, with the rapid emergence of new LLMs, an effective detector should exhibit generalization ability to identify AI involvement from previously unseen LLMs under out-of of-distribution (OOD) settings. While many detectors achieve high accuracy on training distributions or under minor domain shifts, their performance degrades under more severe distribution changes [15]. To address this, propose a vetrieval-based few-shot adaptation paradigm that reformulates cross-domain detection as a matching problem under low-resource conditions. The proposed method adjusts the classification decision boundary using limited support samples and demonstrates strong adaptability in the presence of severe distribution shifts, achieving improvements in AUROC under OOD settings, including +15.55 on MAGE-Paraphres [16,, an average of +7.94 on DetectRL 7
and + +10.399415.01 on the
GPT4o-Edited and 1amaa.1-Edited subsets of Beemo [8]
In summary, our contributions are threefold: (I) proposing a novel representation Saaming-baser classification paradigm that constructs a Hierarchical Affinity Tree among text categories; (II) constructing a realistic benchmark dataset that simulates human-Al collaborative writing scenarios and conducting systematic analysis of hybrid texts; (III) providing effective solution and promising direction for caining-based strategies to detect Al-generated content under OOD settings.
2 Related Works
Detection of Purely-Generater Text. Transformers and language models have advanced rapidly in recent years, giving rise to a variety of techniques for detecting A Al-generated text. Prior work in this
area explores multiple paradigms. Watermarking methods 19
20
21
22
23 24 embed identifiable
patterns into generated content, which can be identified during downstream detection using specialized
31 exploit the characteristic probabilistic distributions inherent in the text generation process of language models. These methods employ manually crafted statistical features to differentiate
algorithms. Statistical methods 25 26 27
28
29
30,
text from Human--ritted content. With the growing availability of large-scale datasets, data-driven approaches 6 32 33
have achieved notable progress. Ghostbuster [34 extracts token-level features from multiple weak language models, conducts structured feature selection, and uses the selected signals to train a classifier for Al text detection. T5-Sentinel 35 leverages intermediate hidden representations from
the T5 model g to perform classification. DeTeCtive [36] proposes a supervised detection framework based on stylistic feature learning. To improve the generalization of detectors to emerging models and unseen domains, recent work has explored few-shot learning approaches for detection. OUTFOX 37

---

employs in- in-context learning by incorporating a small number of examples into the prompt, enabling the model to perform both classification and adversarial example generation. UAR 338 leverages limited set of text samples generated by specific LLM to quickly localize the model's position in the stylistic space for detection purposes.
Detection of Hybrid or Collaborative Text. With the widespread adoption of Al-assisted writing tools, an increasing amount of text is now collaboratively authored by both humans and machines, resulting in "hybrid" content. Several studies [39 40, 41 have explored sentence-level and boundary- level detection of such human-Al collaborative text. MIXSET [42] reveals the failure mainstream detectors in reliably identifying hybrid content. APT-Eval [T5] shows that existing detectors often suffer from high false positive rates, limited discrimination ability, and strong biases toward older and smaller models when faced with lightly Al-polished text. HART [ proposes hierarchical detection framework based on decoupling content and expression, enabling multi-level identification of Al involvement, such as distinguishing between Al-assisted refinement (low-risk) and fully AI- generated text (high-risk). Unlike Hierarchical Text Classification (HTC) [43] 44 45), where the label hierarchy predefined and multi- path annotation is explicitly supported. hybrid text detection involves inherently ambiguous and dynamically evolving labels that lack fixed structural definitions. Therefore, the strong structural assumptions of HTC, like rigid tree taxonomies, are poorly suited to the flexible and ambiguous nature of hybrid text categories.
3 Method
An overview of the proposed DETree framework is provided in Figure 1 which outlines the key components of our method. Section 3.1 presents the motivation and construction of RealBench. Section 3.2 introduces the construction of the Hierarchical Affinity Tree (HAT) based on class-level representation similarities. Section 3.3 formulates the Strucctued Contrastive Loss (TSCL), which leverages the learned HAT to guide representation learning. Section 3.4 further describes the implementation details of the inference with DETree.
3.1
Motivation and RealBench Construction
In this work, we focus on the detection of human-Al collaborative text. Given an input text sequence = =[W1, W2...., w the goal is not only to determine whether LLMs were involved in the generation process, but also to identify the specific form of involvement, such as single LLM generation, multi-LLM collaboration, human wuittn-ritten text edited by LLMs, LLM-generated text revised by a human. Variations in text generation processes can introduce semantic and stylistic signatures that reflect the underlying generation source. For example, the similarity between samples labeled as aalama33_polsst _ GPT-4o" 22 and "Claude3.5 "Claude3.._ppaaphrraee GPT-4o" [4] is expected to be higher than their similarity with "DeepSeek-V3" V3" [4], since both involve refinement using GPT-4o. Meanwhile, their similarity to human-written written text should be the lowest, as their generation processes are fundamentally different. These interactions give rise to a hierarchical similarity structure among generation sources. However, existing approaches predominantly rely on flat class modeling, failing to capture and exploit this structurally informative signal.
Meanwhile, given that existing datasets focus on purely human- wwrtttnn- or Al-generated text, we construct a large-scale hybrid text benchmark, RealBench, by aggregating original samples from MAGE 16), M4 [46 47. TunihgBench 488. OUTFOX 337. and RAID 40 Based on these samples, RealBench introduces a range of hybrid text construction strategies that reflect real-world human-AI collaboration patterns, including paraphrasing, extension, polishing, and translation. For example, a human-written text polished by Geminil.5 1.5 is labeled as "human_polin Lrinaannpolis__eeee Geathh_emmiiiS. Additionally, RealBench integrates 11 perturbation- based attack types. See Appendix for more details.
3.2 Hierarchical Affinity Tree Construction
We propose a Hierarchical Affinity Tree Construction Algorithm to formalize and leverage latent rela- tional structures among text categories. Specifically, we fine-tune an encoder fo(.) using supervised contrastive learning [50], treating each category as an independent class. The expected inter-class similarity between any pair (X, Y) is computed as the dot product of their corresponding centroid vectors, where each centroid is computed as the mean embedding across all samples within that
3

---

Learned Representation Space
Class Similarity Matrix
Hierarchical Affinity Tree
HAT Construction Algorithm
Root
Structured Contrastive
Tree-
Root
2:00
23
0:30
D:7
620
Prior1
Learning
Involved
2:56
0:99
0.70
0.99
367
0.96
222
0.0
6.0
Jama@_polish_Grr
3loude3.5 wewpss_prpssss_GPPPPP4P0
humor__olish_Gemiiiee
DeepSeak-V3
humon
The overall framework of DETree. Class similarity matrix computed from representations learned via supervised contrastive learning. with each class treated independently. Based on the similarity matrix, Hicrarchical Affinity Tree (HAT) is constructed. Guided by the HAT, Tree- Structured Contrastive Loss is introduced retrain the text encoder, aligning the representation space with the hierarchical relations defined by the HAT.
Figure 1:
category, as shown in equation 9 Although the encoder is trained maximize inter-class separability, it still exhibits latent clustering tendencies among related categories in the embedding space, as shown in Figure suggesting the presence of hierarchical relations.
Motivated by the observed latent structure in the embedding space, we construct Hierarchical Affinity Tree (HAT) explicitly represent inter-category relationships. In the HAT, leaf nodes correspond to specific categories, while internal nodes capture their associations. The depth of the lowest common ancestor between two nodes reflects the degree of similarity between the corresponding categories.
The initial HAT is constructed based on a class similarity matrix E RNXN We employ a agglomerative hierarchical clustering algorithm [51] generate initial binary structure. Since hierarchical clustering enforces a strictly binary tree structure, closely related categories may be unnecessarily split across branches.
In response, we introduce an editable top-down subtree reorganization algorithm. We first predefine three prior heads according to whether hybrid texts belong to the human category, the AI category, or form an independent category, and then use these heads to guide subtree restructuring. See Appendix [ for details. For each subtree under reconstruction, of the subtree, assess their clustering quality using the Silhouette Score [52), and select the partition with the highest score. This process recursively applied until predefined stopping criterion is met. The overall computational complexity of the algorithm is O(N2 log N), and further implementation details provided in Appendix C
enumerate all possible partitions
3.3 Tree-Structured Contrastive Learning
To align the learned representations with the hierarchical structure defined by the HAT, propose a pree-Strcttred Contrastive Loss (TSCL) that explicitly models the hicrarchical relations among categories in the representation space. Formally, let T denote the HAT constructed over the full set of categories. Each category label y corresponds to a leaf node c in T. where de denotes its depth. The 0 corresponding
i-th ancestor of node counting from the bottom (with p( Starting from c, we partition categories into de disjoint sets based on their lowest common ancestor (LCA) with c. The i-th hierarchical partition set associated with node is defined as:
itself), is denoted by
H!! = {= leaf(s!") \ leaf( ((
where leaf(x)) denotes the set of leaf nodes under the subtree rooted at node x. Thus, H!! contains all categories whose nearest common ancestor with is exactly _!()) By definition, H (c).
Under the tree T, categories sharing closer common ancestor with e are expected exhibit higher similarity and lie closer in the embedding space. To formalize this intuition, we introduce the following hierarchical similarity constraint.

---

Theorem 3.1 [Hierarchical Similarity Constraint] If in the tree T, for any leaf class X corresponding node c. the following holds:
(Xsim(X,Y Y) >> (X, vOξi<j de, YE H Z H
(2)
Then, for any leaf classes X, Y. Z, the following inequalities holds:
E [(sim((X Y)] > [[sim(X; (X, 2)}, if (fLCCA(XY)) dLCA(X,2)-
Where E[sim(:;)) .) denotes the expected similarity between categories.
By Theorem 3.1
inequalities
2 and are equivalent. The proof is provided in Appendix D
To encourage the embedding space better align with the hierarchical structure modeled by the HAT, we aim to optimize the inequalities Jurting during training, such that categories that are closer the HAT structure mapped more similar representations, while those farther apart are pushed away.
However, directly optimizing inequalities challenging. Therefore, based on the equivalence in Theorem 3.1 we transform the objective to the more tractable incqualities? 2 Accordingly, we adopt the following training objective:
max B2
(4)
()r;0) Σ osiciSde
y sim((o(z). 20))) (oo))))
(5)
Here in equation 4 DD denotes the distribution of text samples æ, denote its corresponding leaf
node in the HAT, and its depth. In equation 5 the function fo(x) IRd denotes the embedding of x computed by the encoder fa. The sets H are defined as in equation | and ((H))) denotes the distribution over !T! The similarity function sim(::") measures the pairwise similarity between embeddings. The objective (ax; 0) computes the expected similarity gap between all hierarchical pairs (i,j) of categories associated with node c.
Motivated by contrastive learning, we reformulate the hierarchical similarity constraint into a struc- tured negativ sampling scheme tailored to the tree hierarchy. For each hierarchical level
(0 S < de), we define the positive set P; H(()) and the negative set N; Uf H!! The complete Structurdd Contrastive Loss (TSCL) defined as:
d Σ
LTSCI (I; 0)
(x; 0),
(6)
de 1=0
where each hierarchical contrastive loss c. (z; is computed as:
exp [p: EkEp, sim (ff()((),
c. (x: -llo
exp [P EkEp, sim sim((a(2), + EkEN, exp (sim()(())
Where T > 0 is a temperature parameter, and sim(.,) denotes the similarity function (c.g., cosine similarity) between embeddings. This formulation reflects hierarchy similarity structure, encouraging representations to be increasingly similar to categories with closer shared ancestors in the tree, and more dissimilar to those located further apart in the hierarchy.
In practice, due the large number of categories C, single training mini-batch B may not contain sufficient instances to fully cover all categories. Consequently, We introduce the Virtual Class Prototype (VCP) mechanism: for each class e, learnable prototype vector V, E Rdd introduced to participate in contrastive learning persistent anchor, without incurring additional memory overhead. The entire TSCL. training process supports parallel computation and incurs similar runtime to standard contrastive learning.
5

---

3.4 Inference with DETree
K-Means Based Database Compression. the task of Al-generated text detection, samples typically originate from multiple domains. Even when the source is the same, substantial variation in domain may lead to multi-cluster distributions (see Figure 2 In such scenarios, K-Nearest Neighbors (KNN) classification methods [53] naturally adapt to complex distributions and allow flexible modification of the retrieval database according to task-specific requirements. However, their computational and storage costs grow linearly with the size of the database. Moreover, when class sizes are imbalanced or spatial distributions are non-unifone distorted. To address this, we propose a database compression method aimed at reducing the number of retrieval candidates while maintaining a balanced class representation. Specifically, for each class, apply K-means [54] clustering partition its instances into K clusters. The normalized direction of the mean vector of each cluster is then used as its representative, resulting in a compact class representation. More details provided in Appendix E
decision boundaries tend to become
Retrieval- -Based Few-Shot Adaptation. Distributional discrepancies arise between datasets due to differences in text sources, prompt types, and generation model configurations, reflecting domain shifts. Conventional binary classification methods tend to rely superficial features, which limits their ability to generalize across domains. As a structured source modeling approach, DETree relies on retrieval database constructed from training data. However, when this data lacks sufficient coverage, the resulting database fails to generalize to target domains, leading to performance bottlenecks under -itt---dittribttion (OOD) scenarios. To address this, incorporate small number of target- domain samples to rebuild the retrieval database, enabling more accurate decision boundaries in the embedding space and significantly improving detection performance under domain shift.
Experiments
4.1
Experimental Setup
Dataset. We conduct in-distribution supervised experiments on the MAGE (16), M4 46
TuringBench [48]. OUTFOX (37), and RAID [49 datasets. Given substantial differences across benchmarks in terms of generation models, human text sources, domain distributions, and prompting configurations, we treat unseen benchmarks (DetectRL [17], Beemo [18), HART [T], and the OOD setting of MAGE) out-of-distribution test sets for evaluating model generalization. Detailed dataset specifications are provided in Appendix Hnd
Comparison Methods and Metrics. We evaluate zero-shot methods including Past-DetectGPT 28
(shortly Fast-Detect), Binoculars [29]. and Glimpse [55]: hybrid text classification via HART 11
few-shot classification via UAR [38; and data-driven methods including SCL [56). MAGE
[66].
T5-Sentinel [35], DeTeCTive (36), and RADAR [9]. Since large language models in real-world scenarios are often black-box and inaccessible, we exclude watermarking methods. Evaluation metrics include F1, AvgRec (mean recall of human and machine text), and AUC-ROC. We additionally report TPR@5% FPR to assess detection performance under low false positive constraints.
Implementation Details. We fine-tune RoBERTa-large cosine annealing, a 3e-5 initial learning rate, and 2000 step linear warm-up. Training runs for 10 epochs on 8xRTX 4090 GPUs with batch size 64 and maximum input length 512 tokens. For inference, we use Faiss-GPU [60] for efficient K-means and K-Nearest Neighbors. The representation layer selected from layers 17-19, and the number of neighbors k set to or 50 based on validation performance. To compute class probabilities, we follow DINOv2 [61] and replace majority voting with umillaity-weighhee scoring over the retrieved neighbors. The contrastive- learning temperature T is set to 0.07.
using LoRA 58 with Adam W [59].
4.2 Results
We explore our method across five key dimensions: (1) HAT Analysis: effectiveness of TSCL and interpretability of the HAT structure; (II) Supervised Detection: model performance on in-distribution binary classification tasks; (III) Out-of-Distributic Generalization: model performance binary

---

Table 1: Comparison of supervised detection performance. w/ per dataser denotes DETree trained individually on each dataset. w/ prior1/2/3 indicates DETree trained on RealBench under prior assumptions prior 1/2/3 (see Figure 7 w/o TSCL. refers to supervised contrastive learning performed independently for each class, without the TSCI.. All baseline methods, except Binoculars, are trained fully supervised manner on the corresponding datasets. Best results are bolded.
MAGE
M4-monolingual
M4-multilingual
TuringBench
Average
Method
AvgRec
F1
AvgRec
F1
AvgRec
Fl
AvgRec
F1
AvgRec
F1
SCL
90.59
89.83
91.92
91.21
86.27
84.75
99,46
99.22
92.06
91.25
RoBERTa-Base
87.30
88.37
88.70
88.44
80.01
84.44
99.59
99.29
88.90
90.14
MAGE
90.53
89.76
80.99
81.42
84.68
83,00
99.40
98.95
88.90
88.28
T5-Sentinel
93.49
93.30
84.01
81.08
76,21
68.99
99.39
97.43
88.28
85.20
Binoculars
64.96
70.58
89.89
89.89
80.63
82.43
51.24
9.98
71.68
63.22
DeTeCTive
96.15
96.16
98.44
98.38
93.42
93.05
99.74
99.35
96.94
96.74
DETree (Ours)
w/ per dataset
96.97
96.98
98.96
98.92
93.62
93.35
99.62
99.39
97.29
97.16
w/ priorl
96.87
96,96
99.86
99.85
95.05
94.85
99.74
99.32
97.88
97.75
w/ prior2
95.45
95.63
98.17
98.02
91.77
91.72
99.29
98.75
96.17
96.03
w/ prior3
96.15
96.28
99.52
99.47
95.53
95.33
99.52
98.88
97.68
97.49
w/o TSCL
95.65
95.82
98.84
98.73
92.11
91.89
99.39
98.83
96.50
96.32
classification under OOD settings; (IV) Hybrid Text Detection: the ability to identify varying forms of Al involvement in collaborative text generation; (V) Practical Robustness and Deployability: model performance under real-world constraints, including adversarial perturbations, database compression, and limited category coverage.
HAT Analysis. A total of 99.32% of triplets satisfy the constraint defined in Theorem 3.1 after training with TSCL, confirming the effectiveness of the proposed loss. Appendix J provides an intuitive visualization of the intermediate clustering structures in the HAT construction process. We observe that HAT is capable of autonomously capturing correlations among text sources: within hybrid texts, samples from the same model using the same transformation strategy (e.g., paraphrasing, extension, translation) tend cluster together. Within such clusters, texts based on the same model family exhibit further aggregation, forming terminal leaf nodes. Moreover, some non-family models (e.g., LLaMA [62] and GLM-130B [63], FLAN-T5 [64] and TO (65l) are also grouped together, potentially reflecting similarities in their pretraining data pretraining model (T5) g Models from different datasets cluster separately, suggesting distributional discrepancies.
Supervised Detection. As shown in Table 1 our method demonstrates superior performance in supervised settings. Training with the Tree-Structured Contrastive Loss (TSCL) leads to the best performance under priorl, while removing TSCL results in noticeable performance drop. We additionally investigate the impact of different prior assumptions on model performance. Among the tested priors, priorl yields the best results, followed by prior3, while prior2 performs the worst. This outcome aligns with the intrinsic nature of hybrid text: categorizing human-Al collaborative texts as Al-involved proves to be more appropriate. Once AI is introduced into the generation process, minimally, its stylistic signals tend to dominate and are often more salient than human-authored traits. This observation is further supported by the dimensionality reduction visualizations in Figure 2
Outto--Distribution Generalization. As shown in Table 2 under the zero-shot setting, evaluation metrics vary across different database configurations, indicating distributional discrepancies among the datasets. RAID and RealBench yield relatively strong results. The model performs well on MAGE's Unseen setting and the DetectRL scenarios but shows weaker performance on Beemo, primarily due to mismatches between the retrieval database and the test distribution under OOD scenarios. To mitigate this issue, we incorporate a small number of target-domain samples to adjust the decision boundaries. Under the few-shot setting, the inclusion of such samples improves classification performance, particularly on datasets where zero-shot accuracy low. Beemo's No Attack setting remains challenging; however, detection performance improves substantially on texts that have undergone further L.M editing. This may be because the additional editing amplifies LLM-specific stylistic features, resulting more concentrated distribution that is easier to detect.
7

---

Table 2: OOD evaluation results, measured by AUROC. All methods evaluated in zero-shot setting without any training. DETree (database) denotes evaluations where different datasets are used as the retrieval database. DETree (few-shot) and UAR denote few-shot evaluations, where or 10 samples are randomly drawn for each source type from the target validation or test set. Each experiment is repeated five times, and the average performance is reported. Best results are bolded.
Dataset
Setting
Fast-Detect
Binoculars
MAGR
RADAR
Dree (database)
UAR
Trer (few-shot)
RealBench
MAGE
M4
RAID
5/10 shot
5/10 shor
MAGIF
Unseen
Paraphrase
80.76
53.34
96.81
83.35
95.20
71.11
$7.37
90.80
99.13
88.29
97.78
91.92
98.98
72.07
74.22
58.18
99.58 99.81
98.90/98.77
Multi- Domain
58.52
83.95
86.67
92.15
74
98.15
00.00
98.94
73.84/75.39
99.62 99.88
DesectRL
Multi
59.58
83.30
86.26
91.86
98.62
98,11
98.08
98.92
03.97773.99
99.52 99.87
Multi-Altack
60.70
88.52
91.50
98.21
98.15
73/49
99.87
No-Allack
75:46
83.90
73.72
51.41
81.59
75.76
79.09
79.99
62.73 64.73/64.40
7
Beemo
GPT4e-Edited
62.64
78.15
67.79
60.16
82.07
75.01
77 44
83.79
65 52. 66, 36
88. 23 88.54
Llama3.1 I-Edited
65.43
65.37
62.65
87.16
75.02
81.74
63.61/66.55
93.81
Hybrid Text Detection. Table 3 reports the results on the three-level Al risk detection task proposed by HART, with details in Appendix H. Level-1 identifies whether the text involves any Al participation, Level-2 distinguishes whether the base content is human-written or Al Al-generated, and Level-3 detects whether the text is solely produced by a single Al model. Results show that, given well-defined database split corresponding to each task, DETree consistently achieves effective discrimination across all three levels, demonstrating the fine-grained discriminative power of the learned representations.
Table 3: AUROC and TPR @ 5%FP (shortly TPR5% results for the Al risk detection tasks on HART. The column labeled ALL aggregates results across multiple English subdomains, including Essay, ArXiv, Writing, and News. DETree is trained on RealBench under different prior assumptions and evaluated using the HART development set as the retrieval database. HART(Fast-Deted / Binoculars) employs the development set to fit its two-dimensional binary classifier. Best results are bolded.
Level-3
Level-2
Leve-1
Detector
Essay
ArXiv
Wrining
ALL((TPR5%)
Essay
ArXiv
Writing
ALL(1PR555)
Essay
Writing
ALLL(TPRR55)
RADAR
0.617
0.728
0.566
0.811
0.630
0.687 (10%)
0.758 (20%)
Log- Perplexity
0.868
0.850
0.811
0.799 (339))
0.364
0,485
0.438
0.473 (11%)
0.769
0.530
0.625
0.576 (6%)
Log Rank
0.667
0.874
0.813
0.814 (39%)
0.380
0.460
0.441
0.465 (11%)
0.739
0.542
0.611
0.573 (8%)
LRR
0.355
0,909
0.797
0.810 (50%)
0.560
0.616
0.551
0.573 (25%)
0.616
0.576
0.558
0.568 (19%)
Glimpse
0.929
0.869
0.819
0.849 (58%)
0.754
0.757
0.625
0.676 (30%)
0.878
0.719
0.618
0.688 (2216)
Fast Deseet
0.883
0.840
0.862 (60%))
0.734
0.718
0.692
0 (47%)
0,877
0.740
0.778 (55%))
Rinccullas
0.897
0.847
0,870 (62%)
0.715
0.711 (44%)
0.740
(55%)
[LART Detect)
(61%)
0.785
0.915
0.890
0.855 (59%)
(59%)
(Bnocuis))
0.854
0.904
0.905
0.883 (61%)
0.746
0.913
0.895
0,848 (32%)
0,900
0.840
0.828
0.838 (58%)
DETree (Ours)
o/ prior?
0.964
0.991
0,983
0.988 (95.3%)
0.994
0,99%
0.990
0.992 (9W.5%)
1.00
0.999
0.996
0.998 (99 5%)
of prior2
0.976
0.985
0.959
0.972 (88. 7%5)
0.981
0.992
0.973
0.982 (93)68)
0.989
0.997
0.994
01994 (98 0%8)
ar prior3
0.983
0
0.973
0.979 (92.2%%)
0.993
0.997
0.988
0.992 (96.8%)
0.999
0.998
0.998
0.998 (99.7%)
To further investigate the interrelations among various types of hybrid texts and evaluate DETrec's ability to distinguish them, we perform a comprehensive analysis on the HART dataset, as shown in
Figure 2 Figure 3 and Figure g
Based on the dimensionality reduced representations, although the training objective encourages clustering of all human-written texts, the model still separates human texts from different domains. This indicates that semantic information implicitly influences the modeling of text provenance; even without explicit supervision, the model is able to autonomously capture and leverage such distinctions.
Except for the machine humanized human category, the other five types of texts exhibit well-formed clusters in the embedding space and are clearly separable in binary classification tasks. An interesting observation is that although the training data does not include samples humanized by commercial tools, the model successfully identifies such texts. Machine humanized human texts tend to be indistinguishable from those based machine-genest geenertte base texts, but are distinguishable when human-written. the base text is human- wriiten. Human edits may preserve aacchne-originated styles or reflect refinement patterns resembling tools or LLMs. This suggests that human editing is diverse but does not fundamentally alter the characteristics of the base text.
In summary, DETree can detects Al-involved content, identifies the provenance of the base text (human or machine), and identifies whether and how it has been humanized by specific methods. We provide further analysis of hybrid texts on HART, including multilingual settings, visualizations, and
8

---

HAT construction in Appendix three authors in Appendix G
we additionally present the detection of hybrid texts involving
MM
MII
M
HM
100
<Typ
H
100.00
99.58
100.00
90
HM
99.73
96.95
87.08
99.55
MI1
80
(Tap- 21
93.80
96.95
47.50
MD4
70
(Typ>
MM
81.40
Machia
Clppe
Machese humunter
Figure 2: UMAP- based unsupervised visualization of the HART dataset in the representation space of the DETree. Colors indicate different text source types, with type indices corresponding the hierarchical categorization defined in HART. Abbreviations of cat-
Figure 3: Pairwise binary classification per- formance heatmap the HART dataset, eval- uated by TPR @ 5%FPP. Rows and columns represent the two text types involved in each detection pair. In the legend, MT denotes machine-generaln
egory names defined in Figure
humanized by tools.
100
90
TPR
80
70
DETree
RoBERTE-lage
60
DFTree No Attack
RoBERTielepe Attack
50
No Attack
Symonym
Papleciy
Altenolive
Deletca
Para
Uppar Lower
Whitopper
Honeglyph
Num Shuffle
Maspeling
Figure 4: Robustness evaluation of different detection methods the attack- augmented MAGE test set, measured by TPR @ 5%FPR. The x-axis denotes perturbation types. "No Attack" indicates that the model was trained without adversarial samples, while others were trained with all attack types. DETree evaluated using a compressed 10K-sample version of RealBench as the database.
97
TPR:25%
98
96
95
Mage
RealBBench
94
5%
10%
20%
50%
100%
5K
10K
20K
50K
100K
ALL
Percentage of Training Classes
Compression Size
Figure 5:
Detection performance (FI) on the MAGE test set using MAGE (-500K) and Real- Bench (- 12M) as the database under different com- pression sizes. "ALL" indicates no compression.
Detection performance the MAGE test set (TPR@5% F5%FPR) when training on RealBench with randomly selected subsets of classes at different proportions.
Figure 6:
Practical Robustness and Deployability. As shown in Figure incorporating adversarial samples during training improves model performance under various perturbations. DETree demonstrates stable performance across all attack types when trained with such examples, while RoBERTa-large still suffers substantial degradation under Synonym, Perplexity, and Homoglyph attacks. Without adversarial training, RoBERTa-large becomes highly vulnerable to perturbations. In comparison,

---

DETree maintains strong performance even without adversarial exposure, consistently outperforming the adversarially trained RoBgRTa-lagge in most attack scenarios.
As shown in Figure 5 we investigate the impact of K-means based datahase compression on detection performance. On the RealBench dataset, compressing the full set of training samples into 10K learnable representatives not only reduces storage and computational overhead, but also leads to slight improvement in detection accuracy. To further assess the influence of category coverage on model performance, we conduct experiments under varying proportions of training categories. As illustrated in Figure 0 increasing the number of training categories enhances model performance. Notably, when trained on only 10% of the full category set, the performance drop remains within 3%.
5 Conclusion
In this study, we propose DETree, a novel representation learning-based detection framework designed to address the challenges of identifying human-AT hybrid text in complex real-world scenarios. We construct RealBench, a large-scale benchmark dataset that encompasses diverse modes of human-Al collaborative writing. By explicitly modeling the hierarchical relationships among text sources, DETree reveals that hybrid texts generated through human-Al collaboration exhibit stronger AI traces than human characteristies. Extensive experiments demonstrate that our method achieves state-of-the-and performance across multiple benchmark tasks and maintains strong generalization capabilities under superviivn conditions and severe distribution shifts.
6 Limitations
While our method demonstrates strong adaptability to real-world detection scenarios, it still exhibits minor limitations in certain aspects. We have not explored adversarial evasion via model fine-tuning intended to bypass detection. While this work focuses on hybrid texts involving three authors, extending the analysis scenarios with more collaborators remains an interesting direction for future investigation. Furthermore, when encountering entirely unseen and rare domains, the model still requires small number of in-domain samples to effectively adjust the decision boundaries.
Acknowledgments
The work supported by National Key Research and Development Program of China (No. 2022YFB270250)) the National Natural Science Foundation of China (No. 62206265, 62076231).
10

---

References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, lige Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023.
[2] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama
3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
[3] An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412. 15115, 2024.
[4] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024.
[5] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 2025.
[6] Xiaoming Liu, Zhaohan Zhang, Yichen Wang. Hang Pu, Yu Lan, and Chao Shen. Coco: Coherence-enhanced machine-genne generated text detection under low resource with contrastive learning. Proceedings of the 2023 Conference on Empirical Methods Natural Language Processing, pages 16167-16189 2023.
[7] Yutian Chen, Hao Kang. Vivian Zhai Liangze Li, Rita Singh, and Bhiksha Raj. Gpt-sentinel: Distinguishing human and chatgpt generated content. arXiv preprint arXiv:2300.00 009969, 2023.
[8] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of machine learning research, 21(140):1-6) 67, 2020.
[9] Xiaomeng Hu, Pin-Yu Chen, and Tsung-Y Yi Ho. Radar: Robust ai-text detection via adversarial learning. Advances in Neural Information Processing Systems, 36: I 5077-15095, 2023.
[10] Zihao Cheng, Li Zhou, Feng Jiang, Benyou Wang, and Haizhou Li. Beyond binary: Towards fine-grained Imm-generated text detection via role recognition and involvement measurement. In Proceedings of the ACM on Web Conference 2025, pages 2677-2688, 2025.
[11] Guangsheng Bao, Lihua Rong, Yanbin Zhao, Qiji Zhou, and Yue Zhang. Decoupling content and expression: Two-dimensional detection of ai-generated text. arXiv preprint 0rXiv:2503.00258. 2025.
[12] OpenAL. Gpt-4o system card, 2024.
[13] Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Saan-Bapistte Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11800 2023.
[14] Anthropic. Introducing claude 3.5 sonnet.
hattps:/wwwt wwt.ropic.com//eeeeeeee
claude 3-5 sonnet, 2024.
[15] Shoumik Saha and Soheil Feizi. Almost ai, almost human: The challenge of detecting ai- polished writing. arXiv preprint arXiv:2502.1566 2025.
[16] Yafu Li, Qintong Li, Leyang Cui, Wei Bi, Zhilin Wang, Longyue Wang, Linyi Yang, Shuming Shi, and Yue Zhang. Mage: Machine gennere-ge text detection in the wild. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 36-53, 2024.
[17] Junchao Wu, Runzhe Zhan, Derek F Wong, Shu Yang, Xinyi Yang, Yulin Yuan, and Lidia S Chao. Detectrl: Benchmarking llm-generated text detection real-world scenarios. arXiv preprint arXiv:2410.23746, 2024.
11

---

[18] Ekaterina Artemova, Jason Lucas, Saranya Venkatraman, Jooyoung Lee, Sergei Tilga, Adaku Uchendu, and Vladislav Mikhailov. Beemo: Benchmark of expert-edited machine- gennrratd outputs. arXiv preprint arXiv:241 1.04032 2024.
[19] Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. On the learnability of watermarks for language models. arXiv preprint urXiv:23/2.04469, 2023.
[20] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. On the reliability of watermarks for large language models. arXiv preprint arXiv:2306. 04634, 2023.
[21] John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In International Conference on Machine Learning, pages 17061-17084. PMLR, 2023.
[22] Hengyuan Xu, Liyao Xiang. Xingjun Ma, Borui Yang, and Baochun Li. Hufu: A modality- agnositc watermarking system for pre-trained transformers via permutation equivariance. arXiv preprint arXiv:2403.05844 2024.
[23] Abe Bohan Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. k-semstamp: A clustering- semantic watermark for detection of machine-generated text. arXiv preprint arXiv:2402. 11399, 2024.
[24] Yapei Chang, Kalpesh Krishna, Amir Houmansadr, John Wieting, and Mohit Iyyer. Postmark: robust blackbox watermark for large language models. arXiv preprint arXiv:2406.14511 2024.
[25] Sebastian Gehrmann, Hendrík Strobelt, and Alexander M Rush. Gltr: Statistical detection and visualization of generated text. arXiv preprint arXiv:1906 1906.04433 2019.
[26] Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. Detectlim: Leveraging log rank information for zero-shot detection of machine-generated text. arXiv preprint arXiv:2306.05550, 2023.
[27] Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning, and Chelsea Finn. Detectgpt: Zero-shot maechine-generatee text detection using probability curvature. In Interna- tional Conference on Machine Learning, pages 24950-24962. PMLR, 2023.
[28] Guangsheng Bao, Yanbin Zhao, Zhiyang Teng, Linyi Yang, and Yue Zhang. Fast-detectgpt: text via conditional probability curvature.
Efficient zero-shot detection of arXiv preprint 3r000022100005130, 2023.
[29] Abhimanyu Hans, Avi Schwarzschild. Valeriia Cherepanova, Hamid Kazemi, Aniruddha Saha, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Spotting 1lms with binoculars: Zero-shot detection of machine-generate text. arXiv preprint arXiv:2401.12070, 2024.
[30] Eduard Tulchinskii, Kristian Kuznetsov, Laida Kushnareva, Daniil Cherniavski, Sergey Nikolenko, Evgeny Burnaev, Serguei Barannikov, and Irina Piontkovskaya. Intrinsic dimension estimation for robust detection of ai- ai-generated texts. Advances in Neural Information Processing Systems, 36:3937779999997 2023.
[31] Shuhai Zhang, Yiliao Song, Jiahao Yang. Yuanging Li, Bo Han, and Mingkui Tan. Detecting machine-generater texts by multi-population aware optimization for maximum mean discrep- 16041, 2024.
ancy. arXiv preprint
[32] Kristian Kuznetsov, Eduard Tulchinskii, Laida Kushnareva, German Magai, Serguei Barannikov, Sergey Nikolenko, and Irina Piontkovskaya. Robust ai-generated text detection by restricted embeddings. arXiv preprint 4rXivv2410.08// 13, 2024.
[33] Mervat Abassy, Kareem Elozeiri, Alexander Aziz, Minh Ngoc Ta, Raj Vardhan Tomar, Bimarsha Adhikari, Saad El Dine Ahmed, Yuxia Wang. Osama Mohammed Afzal, Zhuohan Xie, et al. Lim-detectaive: a tool for fine-grained machine-generater text detection. arXiv preprint 2xXiv:2408.04284, 2024.
12

---

[34] Anadi Chaman, Jiaming Wang, Jiachen Sun, Haitham Hassanich, and Romit Roy Choudhury. Ghostbuster: Detecting the presence of hidden eavesdroppers. In Proceedings of the 24th annual international conference mobile computing and networking, pages 337-351, 2018.
[35] Yutian Chen, Hao Kang, Vivian Zhai, Liangze Li, Rita Singh, and Bhiksha Raj. Token prediction as implicit classification to identify Ilm-generated text. arXiv preprint arXiv:231 2023,
[36] Xun Guo, Shan Zhang, Yongxin He, Ting Zhang, Wanquan Feng, Haibin Huang, and Chongyang Ma. Detective: Detecting ai ai-generated text via multi-level contrastive learning. arXiv preprint arXiv:2410.20964, 2024.
[37] Ryuto Koike, Masahiro Kaneko, and Naoaki Okazaki. Outfox: LIm-generated essay detection through in-context learning with adversarially generated examples. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21258-21266 2024.
[38] Rafael Rivera Soto, Kailin Koch, Aleem Khan, Barry Chen, Marcus Bishop, and Nicholas Andrews. Few-shot detection of machine-generated text using style representations. arXiv preprint arXiv:2401.06711 2024.
[39] Zijie Zeng, Shiqi Liu, Lele Sha, Zhuang Li, Kaixun Yang, Sannyuya I Liu, Dragan Gašević, and Guanliang Chen. Detecting ai-generated sentences in human-ai collaborative hybrid texts: Challenges, strategies, and insights. arXiv preprint arXiv:2403.03506, 2024.
[40] Zijie Zeng, Lele Sha, Yuheng Li, Kaixun Yang, Dragan Gaševié, and Guangliang Chen. To- wards automatic boundary detection for human-ai collaborative hybrid essay in education. In P Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 22502-22510, 2024.
[41] Laida Kushnareva, Tatiana Gaintseva, German Magai, Serguei Barannikov, Dmitry Abulkhanov, Kristian Kuznetsov, Eduard Tulchinskii, Irina Piontkovskaya, and Sergey Nikolenko. Ai- generated text boundary detection with roft. arXiv preprint 0tti::2311.083444 2023.
[42] Qihui Zhang, Chujie Gao, Dongping Chen, Yue Huang, Yixin Huang. Zhenyang Sun, Shilin Zhang, Weiye Li, Zhengyan Fu, Yao Wan, et al. clm-as---coauthor: Can mixed human-writee and machine-genee geeninned text be detected? arXiv preprint arXiv:2401.05952, 2024.
[43] Haibin Chen, Qianli Ma, Zhenxi Lin, and Jiangyue Yan. Hierarchy-awart label semantics matching network for hierarchical text classification. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4379, 2021.
[44] He Zhu, Chong Zhang, Junjie Huang, Junran Wu, and Ke Xu. Hitin: Hierarchy-aware tree isomorphism network for hierarchical text classification. arXiv preprint 13X1: 2023.
[45] Zihan Wang, Peiyi Wang, Tianyu Liu, Binghuai Lin, Yunbo Cao, Zhifang Sui, and Houfeng Wang. Hpt: Hawrr prompt tuning for hierarchical text classification. arXiv preprint 2xX1:22004.14133. 2022.
[46] Yuxia Wang. Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Chenxi Whitehouse, Osama Mohammed Afzal, Tarek Mahmoud, Toru Sasaki, et al. M4: Multi-generator, multi-domain, and multi-lingual black-box machine-generate text detection. arXiv preprint 2rXiv:2305.14900 2023.
[47] Yuxia Wang. Jonibek Mansurov, Petar Ivanov, Jinyan Su, Artem Shelmanov, Akim Tsvigun, Osama Mohanned Afzal, Tarek Mahmoud, Giovanni Puccetti, Thomas Arnold, et al. M4gt- bench: Evaluation benchmark for black-box machine-generated text detection. arXiv preprint arXiv:2402.11115, 2024.
[48] Adaku Uchendu, Zeyu Ma, Thai Le, Rui Zhang, and Dongwon Lee. Turingbench: A benchmark environment for turing test in the age of neural text generation, arXiv preprint arXiv:2109, 13296, 2021.
13

---

[49] Liam Dugan, Alyssa Hwang, Filip Trhlik. Josh Magnus Ludan, Andrew Zhu, Hainiu Xu, Daphne Ippolito, and Chris Kallison-Burette Raid: A shared benchmark for robust evaluation of machine-generated text detectors. arXiv preprint 1rxxix:44050074400 2024.
[50] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607. PMLR, 2020.
[51] Joe H Ward Jr. Hierarchical grouping to optimize an objective function. Journal of the American statistical association, 58(301):236 244, 1963.
[52] Peter Rousseeuw. Silhouettes: a graphical aid the interpretation and validation of cluster analysis. Journal of computational and applied mathematics, 20:53-65, 1987.
[53] Thomas Cover and Peter Hart. Nearest neighbor pattern classification. IEEE transactions on information theory, 13(1):21-21 1967.
[54] James MacQueen. Some methods for classification and analysis of multivariate observations. In Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Volume I: Statistics, volume 5, pages 281-298. University of California press, 1967.
[55] Guangsheng Bao, Yanbin Zhao, Juncai He, and Yue Zhang. Glimpse: Enabling white-box methods to use proprietary models for zero-shot llm-generated text detection. arXiv preprint arXiv:2412.11506 2024.
[56] Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. Supervised contrastive learning for pre-trained language model fine-tuning. arXiv preprint 1oXiv:20110014000 2020.
[57] Yinhan Liu. Roberta: A robustly optimized bert pretraining approach. arXiv preprint urXiv:/907: I 1692, 364, 2019.
[58] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2/06.00668 2021.
[59]
Loshchilov. Decoupled weight decay regularization, arXiv preprint arXiv: 1xX1v:1711.05500,, 2017.
[60] Meta. Meta. https://faiss.aa/ aw/aadex.iiiieeeeaeeeeeiieeiiiii html 2024.
[61] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2334.077193, 2023.
[62] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timo- thée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.133977 2023.
[63] Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang. Yifan Xu, Wendi Zheng. Xiao Xia, et al. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:22 10.024 2022.
[64] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetunee language models. Journal of Machine Learning Research, 25(70): 2024.
[65] Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:21 10.08207 2021.
[66] Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 2025.
[67] Aixin Liu, Bei Feng, Bin Wang, et al. Deepseek-v2: A strong, economical, and efficient ofeepessof-experts language model. arXiv preprint arXiv:2406.12344 2024.
14

---

[68] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 33:1877-1901, 2020.
[69] Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.0100 2022.
[70] Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. Bloom: A open-access multilingual language model. 2023.
1
[71] Ben Wang and Aran Komatsuzaki. Gpt-j-6b: A 6 billion parameter autoregressive language model, 2021.
[72] OpenAL. Gpt-3.5 turbo. 2023.
htt.:///ooooooooooooooo////////a 3-rtrbbbo
[73] Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Advances in neural information processing systems, 32, 2019.
[74] Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. Ctrl: A conditional transformer language model for controllable generation. arXiv preprint arXiv: 05858, 2019.
[75] Alexis Conneau and Guillaume Lample. Cross-lingual language model pretraining. Advances in neural information processing systems, 32, 2019.
[76] Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. Xlnet: Generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32, 2019.
[77] Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit lyyer. Paraphras- ing evades detectors of ai-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems, 36:27469-27500, 2023.
[78] Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint trXiv:2212.08077: 2022.
[79] Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305. 10403, 2023.
[80] Kalpesh Krishna, Yixiao Song. Marzena Karpinska, John Wieting, and Mohit Iyyer. Paraphras- ing evades detectors of ai- ai-generated text, but retrieval is effective defense. Advances in Neural Information Processing Systems, 36, 2024.
[81] John Dang, Shivalika Singh, Daniel D'souza, Arash Ahmadian, Alejandro Salamanca, Made- line Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, et al. Aya expanse: Combining research breakthroughs for a new multilingual frontier. arXiv preprint arXiv:2412.04266, 2024.
[82] Fengqing Jiang. Identifying and mitigating vulnerabilities in llm-integrated applications. Mas- ter's thesis, University of Washington, 2024.
[83] Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internim2 technical report. arXiv preprint arXiv:2403. 17297, 2024.
[84] Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, et al. Olmo: Accelerating the science of language models. arXiv preprint arXiv:2402.008838 2024.
15

---

[85]
Team GLM, Aohan Zeng, Bin Xu, Bowen Wang. Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, Lindong Wu, Lucen Zhong, Mingdao Liu, Minlie Huang, Peng Zhang, Qinkai Zheng, Rui Lu, Shuaiqi Duan, Shudan Zhang, Shulin Cao, Shuxun Yang. Weng Lam Tam, Wenyi Zhao, Xiao Liu, Xiao Xia, Xiaohan Zhang, Xiaotao Gu, Xin Lv, Xinghan Liu, Xinyi Liu, Xinyue Yang, Xixuan Song, Xunkai Zhang, Yifan An, Yifan Xu, Yilin Niu, Yuantao Yang. Yueyan Li, Yushi Bai, Yuxiao Dong, Zehan Qi, Zhaoyu Wang, Zhen Yang, Zhengxiao Du, Zhenyu Hou, and Zihan Wang. Chatglm: family of large language models from glm- 130b glm-4 all tools. 2024.
[86] Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, al. Gemma 2: Improving open language models practical size. arXiv preprint arXiv:2408.000 18, 2024.
[87] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguisties: human language technologies, volume (long and short papers), pages 1171-4186, 2019.
[88] Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. Gpt-neo: Large scale autoregressive language modeling with mesh-tensorflow. If you this software, please cite using these metadata, 58(2), 2021.
[89] Stephen C Johnson. Hierarchical clustering schemes. Psychometrika, 32(3):2412254, 1967.
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer:
[Yes]
Justification: The contributions and scope of the paper summarized in the last paragraph section [
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
The limitations of the work discussed in section
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
17

---

Justification:
The proof of Theorem 3.1 is provided in Appendix D
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Implementation details are provided in Section And we commit to open-source code, model weights and dataset.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [Yes]
Justification: We describe the external datasets used in Section H and the construction of our own dataset in Section and we will publicly release both our dataset and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurlPS code and data submission guidelines (https /nipss.cc/pulli//uuui guwde//ddee/// Cot//ooodeeesiseeeeeeeeee wwwpobmmsssioooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental details are illustrated in section 4.1 Guidelines:
The answer NA means that the paper does include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: Given the massive amount of experiments conducted in this paper, providing error bars would be computationally prohibitive. For the few-shot experiments in Table 2 we report the average over five independent runs.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
19

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Compute resources are described in section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics htttps://hourips.cc// hwwpar/g....cppoaaa//////ccceses
Answer: [Yes]
Justification:
The paper conforms with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Potential societal impacts the work are discussed in Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system Iearns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper does not involve models datasets that pose high risk of misuse. All data constructed from publicly available sources and does not include sensitive or potentially unsafe content.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We cite and credit the code, data, models and vital libraries used in the paper, such as implementation details in section 4.1
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperswithcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
21

---

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce a new dataset in this paper. The dataset constructed based on publicly available sources, and we will release it under the MIT license with accompanying documentation that includes construction details, data sources, licensing information, and usage instructions. No personal data is involved, and an anonymized download link will be provided submission.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details of the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
22

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The development of the core methodology did involve the use of large language models.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM for what should or should not be described.
23

---

A Broader Impacts
This study aims to improve the detection of Al-involved text, particularly by enabling more fine- grained distinction in human-AI hybrid text scenarios. We believe that this technology can play positive role in enhancing text transparency and safeguarding academic integrity and media credibility, However, we also recognize that its application may carry certain potential risks. For example, if used for excessive surveillance or content censorship, it could negatively impact legitimate information expression and freedom speech. On the other hand, stronger detection capabilities may drive the development of generative models that are designed to evade detection. Therefore, when applying this method, we recommend taking the specific usage context into account, paying attention to the boundaries of its application, and promoting its development within an open and transparent framework ensure that produces positive social outcomes.
B Prior
To obtain prior structure suited to the task characteristics, consider all possible similarities between hybrid texts and other generation types, and propose three prior assumptions: prior assumes that hybrid texts closer to Al-generated content; prior 2 assumes that human features are more prominent; prior treats hybrid texts as a distinct category, separate from both Al and human texts. The three priors illustrated in Figure
Root
Root
Root
AI- Involved
Human- Involved
AI- Human
Human
AI
Human
prior1
prior2
prior3
Figure 7: Illustration of three prior assumptions used during training, each defining a different hierarchical placement for Auman-AI collaborative texts, prior I assigns them as subtree under Al-generated texts; prior 2 places them under Homan--ritten texts; and prior treats them as an independent branch parallel to both AI and human texts.
C HAT Construction
C.1 Preliminaries
Agglomerative Hierarchical clustering Agglomerative hierarchical clustering [1 is a bottom-up unsupervised method that constructs binary tree to capture the hierarchical relationships among samples. Starting with each sample as an individual cluster, the algorithm iteratively merges the two most similar clusters based merge step, a new parent node is created to link the two child clusters, and the corresponding merge similarity (denoted as the merge score) recorded. Common inter-cluster distance metrics include minimum, maximum, and average linkage; we adopt average linkage to encourage a more balanced tree structure.
predefined distance metric until single cluster remains. At each
Silhouette Score Silhouette Score
standard metrie for evaluating clustering quality by
assessing the balance between intra-cluster cohesion and inter-cluster separation. For each sample, let denote the average distance other points within the same cluster, and b the average distance to points in the nearest neighboring cluster. The silhouette score is defined ranging from -1 to Higher scores indicate that the sample rell-matched to its own cluster and well-separated from others.
b-o with values
max(a.6)*
24

---

C.2 Similarity Matrix
To capture inter-class similarity, we first apply supervised contrastive learning by treating each class independently. Specifically, for a given sample the contrastive loss is defined as:
[P] EkEp cos(So(z), So()))))
exp
((x;0)
log exp
(8)
[P Eken cow(fo(z)))))ooooooooo cos(fo()) + EKEN exp (co))))(
where fo(-) is parameterized encoder, cos(:;)) denotes cosine similarity, is temperature pa- rameter, the set of positive samples sharing the same label as z, and N is the set of negatives from other classes. This objective encourages intraclass compactness and interclass separation in the representation space.
We use the trained encoder fo to compute the expected similarity between any pair of classes (X, Y) based their sample representations
M E j=1
E[sim( X, Y)
ΣΣ E fo(x)) fo(y,) 1=1
1 N
fa(zi)
fo(us)
(9)
N X M
M
where class contains N samples T; and class contains M samples Vj- As shown in equation 9 the expected similarity can be simplified as the inner product between the mean embeddings of the two classes.
Algorithm 1 Hierarchical Affinity Tree Construction Algorithm
1: Input:
2: similarity matrix E N ,end score
3: Output:
Hierarchical Affinity Tree
5: Definitions:
6: SilhouetteScore: Clustering quality metric. Partition: Divide the subtree of node into subgroups whose root merge_score score is bounded below by r. Aggloomrraiive@luserrigg Hierarchical clustering method. node mmrge__sorre Subtree similarity at merge.
Stepl Hierarchical Clustering
8: Build dendrogram The f- AgglommeraiivCChustrrnngggg
9: Step2 Tree Reconstruction:
10: function RECONSTRRUTION((odd))
T +- node
12:
if mode.is leaf or node.mergee score 2 then return T 14: end if
13:
15: T t- set merge_see scores from descendant nodes of node.
16: Initialize tracker r" +- score" +- 00
17:
for each E do Compute the partition C +- Partition(node, r') score SilhouetteScore((,.5 if score score" then
18:
19:
20:
21:
T' + score" +- score
22:
end end for
23:
24:
children Partition(node, r")
25:
each child chridrren do subtree +- (econstrucctionchall)) T.add eddgenndee subtree) end for
26:
27:
28:
29:
return T
30:
end function
31: T + cooonstractiom(Th_rooo,,, 0)
25

---

C.3 HAT Construction Algorithm
Algorithm Details We first extract the set of categories to be merged within each prior- subtree based on the task-specific prior knowledge, along with their corresponding similarity matrix. Using this matrix, we then construct an initial binary tree via Agglomerative Clustering, where each internal node stores the similarity score at which its child subtrees were merged (denoted as merge_ mwerge_scrre). Since binary tree structures may forcefully separate classes that should reside at the same hierarchical level, we recursively reconstruct the tree in top-down manner. A stopping threshold & introduced to prevent further partitioning of nodes with high internal similarity.
Specifically, for each node, we collect the merge_scorees of all its descendant nodes as a set of candidate thresholds. For each candidate threshold т. extract all subtrees within the current subtree whose root merge_ scores are exactly no less than and treat them as partition. We then compute the Silhouette Score for each partition and select the one with the highest score as the optimal structure. The node is then divided into multiple substructures accordingly, and the reconstruction process is applied recursively to each child. This results in a multiway tree structure, forming the final Hierarchical Affinity Tree (HAT). The detailed algorithm can be found in Algorithm
Time Complexity of HAT Construction For the hierarchical clustering step, heap-based opti- mization reduces the time complexity to O(N log N). The top-down reconstruction process has complexity of O(max dep x N²), where max dep denotes the maximum depth of the constructed HAT tree and N is the number of categories. Since the resulting tree is typically balanced, max dep can be regarded constant or at most Olog N). Therefore, the overall construction complexity O(N2 log N).
C.4 HAT Depth Ablation
We control the gramularity of HAT construction with silhouette- baaoouettte-beeddd termination threshold smaller threshold encourages further node splitting and thus deeper trees. To quantify its impact, we sweep E {0.30, 0.25, 0.20), which in our implementation correspond to maximum subtree depths 3, 6, and 9, respectively. We train DETree on RealBench and evaluate on MAGE.
Experimental results in Table 4 indicate that DETree is robust across different depths. When the tree too shallow, fine-graie fine-grained category structure is lost and accuracy drops; increasing depth improves performance and peaks around a maximum depth of 6. Pushing the depth further brings only slight decline. Unless otherwise stated, the main paper reports results using a maximum depth of 6 (i.c., 0=0.55).
Table 4: Effect of maximum HAT depth on DETree performance.
Max depth
6
96.48
96.96
96.84
AvgRecall
96.38
96.87
96.76
D Proof of Hierarchical Similarity Constraint
In this section, we prove Theorem 3.1
D.I Restatement of the Theorem
Theorem (Hierarchical Similarity Constraint) Let T be tree, and let X, Y, Z be arbitrary leaf classes (corresponding to leaf nodes). Define dLCA(X,Y) as the depth of the lowest common ancestor of X and Y (assume the root node has depth 0, and larger depth value means closer the leaf nodes). Then the following two inequalities are equivalent:
1. For any leaf class X (corresponding node c).
E[sim((X XX Y) E [[sim(X,
vosi<js de, Y H Z H().
(10)
where de is the depth of c, and H!! is the i-th hierarchical partition set of (definition see Section 3.3
26

---

2. For any leaf classes X,Y, Z,
E sim x, Y)
> E sim X, Z),, dELCA(X.))) > dLCA(X.2)-
(11)
D.2 Proof
To prove that inequalities 10 and inequalities 11 are equivalent, it is necessary prove the two-way implication: inequalities
inequalities and inequalities) TT inequalities TO
Based on the tree T, for any fixed leaf node c, we denote its ancestor sequence (fee)
0, o, de) satisfying Je (depth de), fe") is its parent node (depth de 1), and so on, fe is the root node (depth 0).
(de)
D.2.1 Part I: inequalities 10 inequalities 11
Assume inequalities T0 holds. It required to prove that for any leaf classes X, Y, Z, if dLA(( d1,cA(X,2). then (ssim,X, Y)] [[sim(XX 2)].
Fix any leaf X, its corresponding node is c.
Let LCA(X, Y) = f((), then Y E H and dLCA(X.Y) = depth(Je = de - k.
Let LCA(X, Z) f!()) then Z H!!"), and dI.CA(X.Z) depth( f(( de m.
From the condition dLCA(X,Y) > dLCA(X,2) we get:
d > de m
m.
Since < m and Y ( Z E mm according to equation (x) (take i = k, j = m)) we have: E Eisim X, Y)
E simmm, (0,mm(X2)))) 2)
This is exactly the conclusion required by inequalities 11 In summary, inequalities 10 inequalities 11 holds.
D.2.2 Part II: inequalities
11
inequalities 10
Assume inequalities holds. is required to prove that for any leaf class (corresponding node c),
and any 0 < i < S de, Y E H % H!!)) we have [[sim(X,XY) > E [sim(( X, 2)]
Fix any leaf X, its corresponding node is c.
By the definition of E ! ICA(X, = 1! hence dLCA(X.Y) = depth (f( = de 1.
By the definition of Z E H!!) LLCAAX,X,, = ,! hence dLCA(X.2) = depth(S( ({(5)) d. j.
From j, we obtain:
de 4-i> do -j dLCA(X.)) > dLCA(X.2).
According to inequalities 11 we have:
[simm( X, Y) > [[sim(( 2)]
This is exactly the conclusion required by inequalities 10 In summary, inequalities
inequalities I0 holds.
E K-Means Based Database Compression
The inference time and memory consumption of KNN-based classification scale linearly with the size of the retrieval database, making it increasingly costly as the database grows. Moreover, we observe a large number of redundant highly similar samples within the database, which not only increases
27

---

storage and computation costs, but also biases the decision boundary toward sample-dense regions, resulting in distorted classification surfaces.
To address this issue, we introduce database compression strategy based on K-Means clustering. The goal is to replace original class-specific samples with small number of representative vectors, thereby reducing computational overhead while smoothing the decision boundary. We assume each representative sample approximates local spherical region within the original data distribution. Specifically, we apply K-Means clustering to each class independently, partitioning its samples into clusters.
Consider one such cluster C = {I1, T2..... Fn C R where all samples z E C, are l no-rormalezed embeddings (i.c., ||>||| = 1). We aim to find a representative vector žy E Rd that maximizes the total cosine similarity with all samples in the cluster. The objective is formulated
max ER
cos x))
(12)
JFC
DEC,
Since we only care about the direction of žj, we constrain it to lie on the unit sphere, i.e., || 1. This simplifies the optimization to:
max Σ ||
3
Σ
x
(13)
zEC;
DEC,
The optimum achieved when žj aligned with the direction the sum vector, yielding:
ž = Exec, x
(14)
|nnec, 21)
We thus use the normalized mean vector of each cluster as its representative. As shown in Figure this compression strategy not only reduces classification cost but also improves performance, particularly in scenarios with highly imbalanced class distributions such RealBench.
F
More Results on Hybrid Text Detection
5 presents the model's 's performance on hybrid text detection under OOD settings across different languages in the HART dataset. Despite being trained primarily on English data, with only limited multilingual samples from M4, the model demonstrates strong lingulingual generalization. This suggests that modeling the generation mechanism helps overcome the limitations of language scarcity,
Figure presents the domain-wise visualization, where pure human-written texts consistently cluster within each domain. In contrast, Figure 2 shows the joint projection across all domains, where numan--rittee texts form four distinct clusters, while Al-generated texts form two clusters. This suggests that text semantics influence generation patterns, with a stronger effect on human writing than on AI generation.
Figure 9 shows the visualization the HAT structure for fine-graine grainer categories the essay domain of the HART dataset. Even the OOD dataset HART, the HAT retains the properties observed in Section 4.2] Moreover, the HAT structure provides an intuitive and interpretable view of the relationships among categories.
G Three-Author Text Detection
To better align with realistic scenarios involving more than two authors, we have expanded the HART dataset include three authors and conducted the following experiments.
used models not seen during training, such as gemma-3-12b (66) and deepseekv2 67 along with new prompts to introduce a third author for text editing in the HART dataset. We designed experiments to explore the impact different editor sequences on the final text. DETree is trained on RealBench and evaluated using the development set the retrieval database.
28

---

M
(Tp
Nm
NT
A01
11pe
MM
IM
TTy
TTy
Oype
Oppe
(a) Arxiv
(b) Essay
TTy
(Tye
Oxpe
Orge 20
21
(xpe:)
(c) News
(d) Writing
Figure 8: UMAP-based unsupervised visualization illustrating the distribution of four distinct domains (Arxiv, Essay, News, Writing) from the HART dataset in the representation space of DETree. Colors indicate different text source types, with type indices corresponding to the hierarchical categorization defined in HART. Abbreviations of category names are defined in Figure
medt Unann Lire
Figure 9: Visualization of the Hierarchical Affinity Tree (HAT) for the detailed categories in the essay domain of the HART dataset. In the figure, Level x represents the corresponding depth level within the HAT. If the current classes have not been split particular level, they are colored uniformly. For a class that has already become leaf node at an earlier level and thus has no corresponding label subsequent level in the actual HAT, inherits both the label and color from its immediate parent node.
29

---

Table 5: Results CCNews of HART. covering five languages. The best AUROC and TPR5% are marked in bold. The column 'ALL' denotes a mixture of languages. DETree is trained on RealBench under priorl and evaluated using the HART development set as the retrieval database. HART(Fast-Deteec) Binoculars/ / Glimpse) employs the development set to fit its two-dimensional binary classifier.
Detector
English
Chinese
French
Spanish
Arabic
ALL (TPR5%)
Level-3
LRR
0.8466
0.8625
0.8706
0.8744
0.6117
0.7651 (21%
Fast-Detect
0.8551
0.8655
0.8662
0.8310
0.5871
0.8118(48%)
Binoculars
0.8698
0.8698
0.8814
0.8474
0.5754
0.7990(48%)
Glimpse
0.8310
0.8868
0.8793
0.8382
0.7950
0.8323(51%)
Har(Fast-Detect)
0.8600
0,8459
0.8538
0.8397
0.5879
0.8065 (48%)
Hart(Binoculars)
0.8698
0.8495
0.8587
0.8548
0.5476
0.7924 (49%)
Hart (limppe))
0.8257
0.8681
0.8853
0.8729
0.8031
0.8481 (53%)
DETrree(Ours)
0.9888
0.8513
0.9525
0.9468
0.9026
0.9305 (74%)
Level-2
LRR
0.5296
0.8748
0.7118
0.7644
0.4777
0.6380 (11%)
Fast-Detect
0.6665
0.8361
0.7728
0.6961
0.4658
0.7007 (37%)
Binoculars
0,6770
0,8383
0.7779
0.7115
0.4543
0.6929 (37%)
Glimpse
0.5953
0.8123
0.7511
0.7269
0.6813
0.6921 (33%)
Hart(Fast-Detect)
0.8242
0.8295
0.8344
0.7837
0.5867
0.7793 (42%)
Hart(Binoculars)
0.8310
0.8234
0.8464
0.7955
0.4978
0.7515 (38%)
Hart Glimpse)
0.7094
0.8258
0.8257
0.8083
0.7969
0.7776 (41%)
DETree(Ours)
0.9940
0.9538
0.9806
0.9832
0.9374
0.9702 (82%)
Level-1
LRR
0.5009
0,8309
0,6480
0.7288
0.4760
0.6070 (08%)
Fast-Detect
0.6897
0.8349
0.7510
0.7331
0.4359
0.7032 (30%)
Binoculars
0.6969
0.8394
0.7484
0.7461
0.4286
0.7053 (33%)
Glimpse
0.5600
0.7928
0.6933
0.7034
0.6673
0.6596 (24%)
CartFast-Deteed)
0.7770
0.7997
0.7749
0.7669
0.4798
0.7288 (32%)
Hart(Binoculars)
0.7843
0.8041
0.7637
0.7657
0.4639
0.7264 (33%)
Hllmmp)))
0.6386
0.7904
0.7336
0.7634
0.7638
0.7302 (24%)
DETree(Ours) 0.9961
0.9964
0.9945
0.9968
0.9875
0.9936 (99%) %
First, we investigated the influence of the first author the final text, while keeping the second and third authors constant. The results in the first author impact section of Table show that when the third author is introduced, the model's ability to distinguish the first author is reduced. However, the model can still effectively differentiate between the two text types.
Second, we also examined the influence of the middle editor on the final text. The results in the second author impact section of Table show significant decline in detection accuracy when the third author is introduced.
Finally, the third editor is easily distinguishable.
The initial creator of the text sets the general direction and ideas, so these features are difficult to mask through editing. However, the middle editor's features, which are based on rephrasing the initial text, are easily masked by the third editor, leading to significant decrease in detection accuracy.
This demonstrates that the DETree's encoder is capable of capturing fine-grained, generalizable features. Even when three authors are introduced into the test text (while the model was only exposed to two authors during training), and the third author's 's model, and prompt are unseen, the encoder can still extract key features distinguish between different authors.
30

---

Table 6: Three-author detection results on the expanded HART dataset using DETree. Each row reports the AUC-ROC of DETree when distinguishing Sourcel from Source2. Results are grouped by scenario: First author impact- second and third editors are fixed while the first editor varies (ii) Second author impact first and third editors are fixed while the middle editor varies (iii) Third author impact first and second editors are fixed while the third editor differs.
Sourcel
Source2
DETree(AUC-ROC
First author impact (GPT-40 human)
gpt-4o_hhmmniine www.hhummmeeaaaeeeeee humanize-ggtt-35-ttrree -pt--..t 3eissa33ema-3
human human_rephrase volish_gemmma:
gpt-3..-tturo__
96.99
s.r-4o_humaizee_ge Betecepptttt.ttbbbbb deiis_ddepseek222
pepphaaee_gpt-3333 silis_ddeessekk22
96.21
gpt-4o humanize gpt-3.5-turbo
human rwwprase_.ptt..55ae... gpt-3.5-turbo
100,00
human jumann_polsh
99.43
gpt-4o
human hateipoooossss_seeeeeeeeee
99.37
First author impact (Claude-3.5-somm) vs human)
claude-3-5-somen humanize
human_ aaanan_epprrace_.om rephrase_gemini-lle -e--mmi-mm.--ppr.. polish_gemma-3
84.17
1.5--loo__popissiie 3
- humanize h-soannet_humaniee.
human Becrann_ephrr__eeee gemini-1.5 1.5-pn-1..--po... polish deepseekv2
84.05
gemini-L.5-pro. polish
_oolii-spopppposhshhhacpcee deepseckv2
glaude-3-5-somnet, humannize gemini-1.5-pp
human rephrase Setirase_gemii---e gemini-1.5-pmm
98.14
human_poliine Bainan_polis_ggemma---
98.81
human_polish__eepeeeee
98.98
Second author impact (GPT-3.5-turbo vs Claude-3.5-sonnett
gpt-4o_ saall__humaizeeeem hww.ahize_lauue--333333333333232 c-laude-3---sooeee
76.50
polish_gemma-33
--12 3-12b
htt__hmmaize_ggt--211 2iiss_ddepseekv2
gpt-4o_humanize.
htr__uuunnizzcllaade------5- claudd-3----sonnet dolishdeepsseek22
82.05
got-4o humanize 3atimiiiee_ppp3333333 gpt-3.5-turbb
gpt-4o humanize gpt-4o_humanize humanize_chaude-3555aee. claude-33--soonet
98.78
Second author impact (GPT-40 Gemini-1.5-Pro)
human rephrase gpt-4o
human rephrase 115-ni
68.71
polish_gentn Beiis__eemm--
Baanaan_rephhae__ee_ gpt-4o polish_deepseekie
human Beapann_eepraee___mm Bechhaase_gemnii--he gemini-1.5-pro- polish_ 2eiis__deepscek22
68.02
human gamma__ephrase_ggtee gpt-40
human rerrannseerraaeeeeeeeee rwwphrasegeemiiii-- www.-eeeeminn---. 1.5-pro
99.49
Third author impact (Deepseekv2 vs Gemma-3)
pwen2.5-72h_humaan
qwen2.5-72h, 2rnnh---72h_hummnizee_ s-see----soonnppoossshhhcoc 2oodiss_ddopseck22
100.00
llama-3.3-22 3.33770.
Hamma-3.3-70b h...mmuuuuu.
97.69
gemini-1.5 1.5-p-- polissh gemma-3
gemini- 1.5-pro polispooshhhhh deepseekv2
human duman_rephrase, setpun_r_hrarepttt....
human heaptnrrpppphpppp_e gpt-44____ molishddeppseeky.
99.60
wilish_gemmas
31

---

H Benchmark
In this section, we provide a detailed overview of the benchmark datasets used in our experiments.
MAGE 16
MAGE is a large-scale benchmark for Al-generated text detection, comprising texts
from 27 diverse large language models, including families such as OpenAl GPT [68], LLaMA [62]. GLMM-13B [63] FLAN-T5 [64]. OPT [69]. BigScience [70]. and Eleuther [7]]- It spans 10
domains and includes 332K training samples and 57K test samples. Additionally, MAGE provides subset for out-of-domain generalization testing, generated by GPT-4 [12] across four new domains (CNN/DailyMail. SialogSum, PubMedQA, and IMDb), to evaluate model performance in the "Unseen Domains Unseen Model" scenario. Based on this OOD text, a paraphrasing attack test set was also created using GPT-3.5-turie (72).
The M4 dataset is a large-scale collection spanning multiple domains, models, and languages, consisting of text generated by 8 large language models across 6 domains and languages. Built upon M4, the M4GT benchmark is designed for Al-generated text detection, with its test set paraphrased using the OUTFOX method to increase task complexity. M4GT defines two task settings: monolingual and multilingual. The monolingual setting contains 120K training and 34K test samples, while the multilingual setting includes 157K training and 42K test samples. In this study, we adopt the M4GT splits for training and evaluation.
M4, M4GT 46 47
RAID is large-scale benchmark dataset for robust machine-generateee text detection, comprising over 600K original texts generated by 11 language models across 8 domains. These texts are further expanded to 6.2M samples through 11 types of adversarial attacks. In this study, we use the 605K unperturhed samples released by RAID to construct the training and validation splits for our RealBench dataset.
RAID 49
TuringBench [48]
TuringBench, released in 2021, is one of the earlier datasets for Al-generated
text detection. It focuses on political news headlines and content within a single domain, incorpo- rating texts generated by 19 large language models, including the GPT series [68], GROVER (73). CTRL (74), XLM [75), and XLNet (76). The dataset consists of 12K training samples and 37K test samples.
OUTFOX 37
OUTFOX constructs student essay detection dataset comprising 15.4K human- written essays and 15.4K LLM-generated essays. also includes adversarially paraphrased samples generated using DIPPER (77 and OUTFOX attack methods.
DetectRL is a benchmark designed to evaluate Al-generated text detection in real-world scenarios. It covers four domains with high LLM usage and includes GPT-3.5 (72), Claude (78), PaL.M2 [79). and LLaMA-2 [62]. along with four types of attacks. We use DetectRL to study the generalization performance of detection models.
DetectRI. 17
texts generated by
Beemo serves evaluation benchmark, consisting of 19,683 texts, including edited versions produced by experts, 1amaa.1177BB (2) and GPT-4o [12]. covers five representative task types: open-ended generation, rewriting, summarization, open-domain QA, and closed-domain QA.
Beemo [18
HART is multi-level test set for hybrid text detection, covering four domains (student essays, arv introductions, creative writing, and news) and five languages (English. Chinese, French, Spanish, Arabic). It consists of 32K samples generated by six recent LLMs: GPT-3.5-Turbo [72). GPT-4o [2], Claude 3.5 Sonnet (78), Gemini 1.5 Pro [3, LLaMA 3.B-70B Instructt 2 and Qwen 2.5-72B-Instruct (3. HART categorizes samples based on the extent of AI involvement in text generation into four types: human-written (Type O), human content rephrased by AI (Type 1), AI- generated content humanized through various methods (including LLM paraphrasing, human editing, and commercial tools) (Type 2), and fully Al-generated content (Type 3).
HART 11
Based on this labeling scheme, HART defines three detection task levels: Level detects AI involvement (Type 0 vs. Type 1/2/3), Level 2 detects whether the core content Al Al-generated (Type 0/1 vs. Type 2/3), and Level 3 detects whether the text is fully generated by AI (Type 0/1/2 vs. Type
32

---

3). In this study, we strictly adhere to these task definitions to evaluate the model's 's out- of-mmnn generalization and performance in hybrid text detection scenarios.
I RealBench
RealBench is constructed from the unperturbed samples of MAGE, RAID, TuringBench, and OUT- FOX. with additional augmentation via Hybrid Texts. We further apply various perturbation attacks to enhance training and assess model robustness during evaluation. struction of Hybrid Texts, and Section I.2 details the perturbation attack strategies. The category composition and statistics of Bench are summarized in Table 7 and the distribution of sample counts across different augmentation types provided Table
the con-
Table 7: Composition of the RealBench dataset, where Basic text and Hybrid text represent the number of texts in basic and hybrid categories respectively, Basic categories and Hybrid categories denote the number of basic and hybrid categories. In the test set, the number of categories presented as "x/y", with x being the corresponding category number and representing the number of categories that only exist in the test but not in the validation and training sets.
Basic text
Hybrid
Total
Basic categories
Hybrid cafegories
Total categories
Dataset Name
Split
text
text
train
1,433,025 255,164
4,128,555
5,561,580
28
419
447
MAGE
valid
809,730
1,064,894
28
419
447
test
257,232
412,137
669,369
29/1
223/140
252
train 735,497
1,714,893
2,450,390
6
74
80
M4
valid 25,879
71,122
97,001
4
29
33
test 199,419
209,768
409,187
9/3
48/31
train 548,756
414,293
963,049
20
299
319
TuringBench
valid 93,036
53,087
146,123
20
299
319
test 182,714
169,884
352,598
20/0
139/80
159
train 1,553,366
1,943,413
3,496,779
12
191
203
RAID
valid 173,269
218,985
392,254
191
203
train 166,808
624,319
791,127
4
56
60
OUTFOX
test 2,956
1,956
4,912
Total
all 5,627.121
10,772,142
16,399,263
66
1.138
1,204
Table 8: Composition of adversarial attack samples in RealBench. Each column represents the amount of text generated by different adversarial attacks (including synonym replacement, perplexity attacks, paraphrase generation, etc.), excluding simple format attacks which are dynamically incorporated during training via on-the-fly augmentation.
Split
Synonym
Perplexity
Paraphrase
Translate
train 319,071
261,089
852,865
156.467
1,679.724
1.679.724
612,640
5,561,580
MAGE
valid
56,792
46,218
152,154
19,333
340,752
340,752
108,893
1.064,894
58,381
152,492
19,075
170,457
669,369
train 192,174
113,532
329,791
50,045
718,542
718,542
227.764
2,450,390
M4
valid 9,000
4,587
12,292
1,795
30,000
30,000
9,327
97,001
76.650
96,269
15,888
59,976
65,360
train 112,204
110,794
325,758
38.981
107.352
107,352
160,608
963,049
TuringBench
valid
19,051
18,833
55,152
5,718
17,550
17,550
12,269
146,
37.357
108.413
11.178
74.714
74.714
9,278
352,598
train 544,929
$11.376
372,989
856,726
215,964
215,964
778,831
3,496,779
valid 60,781
57,054
41,781
95,887
24,714
24,714
87,323
392,254
train 57,600
56,483
52,725
259,200
259,200
105,919
791,127
OUTFOX
1,000
988
968
1,956
4,912
Total
1,644,9900
1,290,757
2,553,6449
1,271,093
3,698.945
3,650,694
2,289,135
16,3999263
33

---

I.1 Hybrid Text Construction
The construction of Hybrid Texts involves four main approaches: paraphrasing, translation, continua- tion, and polishing.
DIPPER Paraphraser
Using fine-tuned T5-11B model (DIPPER (0) for paraphrasing. The where orgname denotes the original source.
output is labeled as
Adversarial Paraphraser We adopt paraphrasing strategy similar that proposed in OUT- FOX 37 to perform adversarial rewriting of LLM-generated texts. For each input, we first use the encoder model trained by Detective [36] retriever retrieve the top 5 most similar LLM and Human texts from its database. We then construct adversarial prompts using the template shown in Figure I0| and generate the rewritten texts with Qwen2.5-77 5B--trrttttt [3]. The output labeled as {orgname) wpwppeeeppppppppppppeeee wwt.aa.eeeeeee_.____ee.. 5_7b.
Translation Paraphraser Texts are first translated into Chinese using Frren2.5-7BIInsteeee and into German using Aya-23-8B (8I), then translated back into English. The output is labeled as (orgname)__ranssaate. eaaee__rranne (transname), where transname indicates the translation model used.
Text Continuation We retain the first 128 tokens of each text, constrained to no more than half of the original length, and use them as the prefix for continuation. The prompt templates shown in Figure ar used to guide generation. Three templates are used for training and validation, while two
additional templates are reserved for testing to enhance generalization. Continuations are generated using LLaMA-3.1- 1-BB (2), Mistral-8B [82), InterLLM--5778 [83] Olmo2-7B [84), GLM4-9B [85). and Qwen2.5-7B for training and validation, and Gemma-2 2-9B [6 and DeepSeek [67] for testing. The output is labeled as [orgname)_extend_ _etendd_________ee_eee________ the continuation model used.
( (extendname), where extendname denotes
Text Polish We apply the prompt templates shown in Figure 10 polish the texts. Each sample in the training and validation sets is processed with one of 13 randomly selected templates. while two distinct templates are used in the test set for evaluation. The polishing models are identical to those used in the Text Continuation setting. The output is labeled as {orgname)_ polisshhhhhh.. {pol shname), where pol i shname denotes the polish model used.
1.2 Perturbation Attack
Perturbation attacks categorized into types: Word Attack and Format Attack. For Word Attack, perturbed samples are pre-generated as part of the dataset. In contrast, Format Attack samples generated on-the-fly during training due their lower computational overhead.
Word Attack This strategy perturbs words that are potentially influential to the detection model by performing targeted replacements. We implement the following methods:
Synonym Attack:
Synonym substitution is performed using a BERT-based 87 language
model. For each selected token, replace it with candidate word from the top-ranked synonyms based on semantic similarity.
Perplexity Attack: Inspired by the hypothesis DetectGPT 27 that model-generated
texts reside near local minima of the language model's 's loss surface-we we generate adversarial samples by perturbing tokens to increase the negative log-likelihood (NLL). Specifically, we use GPT-Neo-2.7B (88) to compute token-wise probabilities, suppress high- high-probability tokens, and resample from the modified distribution to produce higher-NLL variants.
To constrain semantic drift, compute the semantic similarity between the original and perturbed texts using a BERT-based model, and discard similariy samples.
Format Attack This strategy perturbs textual formats rather than content. We adopt attack schemes primarily derived from the RAID dataset, including:
Alternative Spelling: Replacing American English spellings with their British equivalents.
34

---

Article Deletion: Removing articles (e.g., "the", "a", "an").
Add Paragraph: Inserting paragraph delimiters (e.g.. "\n") between sentences.
Upper-Lower: Swapping the case of letters within words.
Zero-Width Space: Inserting zero-width spaces (Unicode U+200B) between characters.
Whitespace: Adding whitespace between characters.
Homoglyph: Replacing characters with visually similar Unicode homoglyphs (e.g., replac- ing "e" with Cyrillic "e" (U+0435)).
Number Shuffle: Randomly shuffling digits in numeric tokens.
Misspelling: Introducing common spelling errors.
Prompt Type
Prompt Description
Here the results of detecting whether cach essay from each problem statement is generated by
Human or a Language Modcl(LM).
Text: (LLM textl)
Answer: LLM
Text: (Human [Human_textl] texti)
Answer: Human
Adversarial Paraphraser
Your Task: Please rephrase the input LLM-generated text ensure is indistinguishable from human- written content. It essential maintain the original meaning, tone, and details of the text, while keeping the word count within finput text len) words and expressing clear opinion.
Input Text: fimput_ text]
English
in +
finput_ text]
Chinese
#
Chinese
to
Please translate the following text from Chinese English without any additional output.
English
Chinese text: (input text) English
Translation Paraphraser
Bitte übersetzen Sie den folgenden Text ohne zusätzliche Ausgabe Englischen ins Deutsche.
English
German
Englischer Text: (input_ text] Deutscher Text:
German
lo
Please translate the following text from German to English without any additional oulput.
English
German text: (input text] English text:
Here is a picce of text. Please contimue writing from where it ends, maintaining the same tone, style, and context while making the continuation coherent and engaging. Input Text: finput text]
Train& Valid
Please continue the following text, expanding on its ideas a way that maintains consistent and style. The expansion should be coherent, logically structured, and to enrich the original content. Avoid using transitional phrases such 'firstly.' 'secondly.' or 'then.' Instead, opt for smoother transitions that flow naturally from one thought to the next. Use punctuation carefully, particularly minimizing the overuse of commas.
Text
Continuation
Test
Input Text: finput text)
Please refine the following paragraph to improve flow and clarity. Ensure that the original meaning structure are preserved, while enhancing sentence construction expression for better readability: sinput_text)
Train& Valid
Text Polish
Please adjust the language style of the following paragraph make it infornal. Maintain the core meaning and structure while ensuring that the tone aligns with more casual audience.
Test
Input Text: finput text)
Figure 10: Prompt templates used for Hybrid Text construction, covering four strategies: Adversarial Paraphraser, Translation Paraphraser. Text Continuation, and Text Polish. For Text Continuation and Text Polish, only one representative template is shown.
35

---

HAT Visualization
This section presents the intermediate and final results of the HAT construction. After the first stage of supervised contrastive learning, we compute pairwise class similarities using equation 9 Figure TT visualizes the similarity matrix for all 1,109 classes in the RealBench dataset. While the training objective aims to separate class representations, certain classes remain highly similar due to overlapping textual styles-such as shared sources or common post-editing models. This underscores the importance of modeling inter-class stylistic relationships in Al-generated text detection.
89 construct initial dendrogram (binary tree), as shown in Figure 12 under Prior 1. We then incorporate task-specific priors via Algorithm [ to transform the binary tree into a semantically coherent multiway HAT structure. Figures 13 [4 and and[3iillustrad the resulting HAT trees under Priors I, 2, and 3, respectively. Due to space limitations, we retain all numan-related classes and randomly sample 20% of the remaining classes for HAT construction and visualization.
Using the similarity matrix, we apply hierarchical clustering algorithm
In Figures 13 14 and [3 each Level represents depth layer in the HAT tree. At each level, all categories belonging to the same node are shown in the same color. For nodes that became leaf nodes earlier levels and their original color for consistency. For illustration purposes, only the top levels are shown.
further split, replicate their information in subsequent levels and retain
0.9
0.8
...
Figure 11: Similarity matrix of 1,109 classes in the RealBench training set after Stage supervised contrastive learning. Brighter regions indicate higher similarity.
36

---

Figure 12: Dendrogram constructed from the similarity matrix under Prior based on randomly
selected 20% subset of categories from the RealBench training set.
37

---

Figure 13: Visualization of the HAT constructed under Priorl using 20% randomly sampled Real- Bench training categories.
38

---

Figure 14: Visualization of the HAT constructed under Prior2 using 20% randomly sampled Real- Bench training categories.
39

---

Figure 15: Visualization of the HAT constructed under Prior3 using 20% randomly sampled Real- Bench training categories.
40


Reviews:
[{"review_id": "KujNXEu3yN", "reviewer": "Reviewer_UQj1", "summary": "DETree encodes texts with a supervised contrastive objective, constructs a Hierarchical Affinity Tree (HAT) from class centroids, and re-optimises representations with a tree-structured contrastive loss; at test time it performs k-NN retrieval over a K-means–compressed bank of only 10K prototypes, cutting memory and latency while preserving accuracy.\nAcross the HART hierarchy, RealBench and MAGE benchmarks, DETree attains up to 0.988 F1 / 99.5% TPR@5 % FPR, beating the strongest published detector by 30–40 pp on several sub-tasks. The model maintains high accuracy when trained on 10% of categories and gains ~30 pp in few-shot out-of-distribution transfer. Without any adversarial training DETree remains markedly more robust to diverse perturbations than an adversarially-trained RoBERTa baseline.", "questions": "* It would be nice to show how “human-like” the automatically generated edits are. For example: measuring the stylistic difference between AI-polished text and human-polished text (perhaps using a language model evaluator or some linguistic metrics).  \n* For the most part the paper is clearly written, but a few points could use clarification. The hierarchical prior assumptions are introduced in text and Figure 7, but the main paper only briefly references “prior1/2/3” in results without deep explanation. Readers must dig into Appendix B to understand them. Also, some implementation details, such as the exact temperature used in TSCL or how the *K*\\-NN search is accelerated (if at all), are not obvious in the main text. The authors could consider adding them as footnotes.  \n* Lots of prior work on detection uses TPR@1% FPR as the metric, as keeping the number of false positives low is critical. Could the authors compute and report some results with this metric instead of TPR@5% FPR?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n* The paper introduces a Hierarchical Affinity Tree (HAT) to model relationships among text generation processes, moving beyond flat binary or multi-class detectors. This is novel in that it does not treat \"hybrid\" text as just another class or rely on coarse metrics of AI involvement.  \n* The proposed RealBench dataset automatically curates \\~16.4 million samples across 1,204 categories of human-AI collaborative text, which is far larger and more diverse than prior benchmarks. RealBench covers a wide spectrum of collaboration modes (human drafts refined by AI, AI outputs edited by humans, AI outputs refined by other AI models, etc.) and concrete transformation strategies including paraphrasing, extension, polishing, and translation.  \n* Experimental scope is broad, covers baselines from 12 detector families and 8 distinct benchmarks (including in- and out-of-distribution).\n\nWeaknesses\n\n* One concern is the manual intervention used in shaping the HAT. The authors acknowledge that purely agglomerative clustering can yield \"unnatural\" splits, so they employ an editable top-down reorganization using task-specific prior knowledge (Appendix B). In practice, they experimented with three hand-crafted priors (hybrid text closer to AI, closer to human, or separate) and chose the one yielding best performance. While this analysis is interesting, it does mean the final tree is partly guided by human assumptions rather than learned entirely from data.  \n* Two-stage training on a 16M-sample corpus plus k-NN retrieval at inference raises memory/latency concerns despite prototype compression.  \n* The paper focuses on texts generated through one-step human-AI collaboration, e.g. a human draft then one AI edit, or one AI generates then one human edit, etc. It mentions not covering scenarios with more than two \"authors\" due to data scarcity. This is a reasonable limitation, but it means very complex yet realistic writing processes (iterative back-and-forth edits or multiple AIs) are out of scope.", "was_revised": true, "final_justification": "The rebuttal addressed my points raised, and I choose to maintain my original positive rating.", "rebuttal": "*We sincerely thank the reviewer UQj1 for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.*\n***\n> Weakness1: One concern is the manual intervention used in shaping the HAT. The authors acknowledge that purely agglomerative clustering can yield \"unnatural\" splits, so they employ an editable top-down reorganization using task-specific prior knowledge (Appendix B). In practice, they experimented with three hand-crafted priors (hybrid text closer to AI, closer to human, or separate) and chose the one yielding best performance. While this analysis is interesting, it does mean the final tree is partly guided by human assumptions rather than learned entirely from data.\n\nAs mentioned, the use of manual intervention in constructing the HAT is guided by task-specific prior knowledge to make slight adjustments, enhancing the model's fit for the Human-AI Collaborative Texts detection task. However, it is important to note that the tree structure is primarily driven by the similarity matrix learned in the first stage of the model. We believe that one of the key strengths of DETree lies in its ability to balance data adaptiveness with task flexibility, allowing for moderate optimization that aligns with the task while preserving the underlying hierarchical relationships from the data.\n\n> Weakness2: Two-stage training on a 16M-sample corpus plus k-NN retrieval at inference raises memory/latency concerns despite prototype compression.\n\nDuring the training phase, we use a RoBERTa-large encoding model with approximately 0.5B parameters. Given the large dataset size, training can be completed in about 40 hours using 8×RTX4090 GPUs.\n\nIn the inference phase, we utilize a smaller, non-autoregressive encoding model, which is faster compared to methods that calculate text probability distributions using large language models. When we compress the database to 10k samples for k-NN retrieval classification, the computational overhead is equivalent to attaching a 10k-sized classification head (~10M parameters) after the encoder.\n\nBinoculars and Fast_DetectGPT run inference on two RTX 4090 GPUs, while RADAR, MAGE, and DETree run inference on a single RTX 4090 GPU. We tested inference speed with 1k samples and report the average inference time per sample.\n\n||Time Pre Smaple|\n|-|-|\n|Binoculars|136.5ms|\n|Fast_DetectGPT|132.9ms|\n|MAGE|47.6 ms|\n|RADAR|18.9ms|\n|DETree|26.9ms|\n\n> Weakness3: The paper focuses on texts generated through one-step human-AI collaboration, e.g. a human draft then one AI edit, or one AI generates then one human edit, etc. It mentions not covering scenarios with more than two \"authors\" due to data scarcity. This is a reasonable limitation, but it means very complex yet realistic writing processes (iterative back-and-forth edits or multiple AIs) are out of scope.\n\nThank you for highlighting this limitation. Due to resource constraints, we have not previously explored scenarios involving more than two \"authors.\" However, when sufficient data is available, our method can be directly extended to handle more complex writing processes, such as iterative back-and-forth edits or collaborations involving multiple AIs. To better align with realistic scenarios involving more than two authors, we have expanded the HART dataset to include three authors and conducted the following experiments:\n\nWe used models not seen during training, such as gemma-3-12b and deepseekv2, along with new prompts to introduce a third author for text editing in the HART dataset. We designed experiments to explore the impact of different editor sequences on the final text.\n\nDETree is trained on RealBench and evaluated using the development set as the retrieval database. First, we investigated the influence of the first author on the final text, while keeping the second and third authors constant. The results show that when the third author is introduced, the model's ability to distinguish the first author is reduced. However, the model can still effectively differentiate between the two text types.\n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_gemma-3|human_rephrase_gpt-3.5-turbo_polish_gemma-3|96.99|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_deepseekv2|human_rephrase_gpt-3.5-turbo_polish_deepseekv2|96.21|\n|gpt-4o_humanize_gpt-3.5-turbo|human_rephrase_gpt-3.5-turbo|100|\n|gpt-4o_polish_gemma-3-12b|human_polish_gemma-3-12b|99.43|\n|gpt-4o_polish_deepseekv2|human_polish_deepseekv2|99.37|\n||||\n|claude-3-5-sonnet_humanize_gemini-1.5-pro_polish_gemma-3|human_rephrase_gemini-1.5-pro_polish_gemma-3|84.17|\n|claude-3-5-sonnet_humanize_gemini-1.5-pro_polish_deepseekv2|human_rephrase_gemini-1.5-pro_polish_deepseekv2|84.05|\n|claude-3-5-sonnet_humanize_gemini-1.5-pro|human_rephrase_gemini-1.5-pro|98.14|\n|claude-3-5-sonnet_polish_gemma-3-12b|human_polish_gemma-3-12b|98.81|\n|claude-3-5-sonnet_polish_deepseekv2|human_polish_deepseekv2|98.98|\n\nWe also examined the influence of the middle editor on the final text. The results show a significant decline in detection accuracy when the third author is introduced.\n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_gemma-3|gpt-4o_humanize_claude-3-5-sonnet_polish_gemma-3-12b|76.50|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_deepseekv2|gpt-4o_humanize_claude-3-5-sonnet_polish_deepseekv2|82.05|\n|gpt-4o_humanize_gpt-3.5-turbo|gpt-4o_humanize_claude-3-5-sonnet|98.78|\n||||\n|human_rephrase_gpt-4o_polish_gemma-3|human_rephrase_gemini-1.5-pro_polish_gemma-3|68.71|\n|human_rephrase_gpt-4o_polish_deepseekv2|human_rephrase_gemini-1.5-pro_polish_deepseekv2|68.02|\n|human_rephrase_gpt-4o|human_rephrase_gemini-1.5-pro|99.49|\n\nFinally, the third editor is easily distinguishable. \n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|qwen2.5-72b_humanize_claude-3-5-sonnet_polish_gemma-3|qwen2.5-72b_humanize_claude-3-5-sonnet_polish_deepseekv2|100|\n|llama-3.3-70b_humanize_gemini-1.5-pro_polish_gemma-3|llama-3.3-70b_humanize_gemini-1.5-pro_polish_deepseekv2|97.69|\n|human_rephrase_gpt-4o_polish_gemma-3|human_rephrase_gpt-4o_polish_deepseekv2|99.6|\n\nThe initial creator of the text sets the general direction and ideas, so these features are difficult to mask through editing. However, the middle editor's features, which are based on rephrasing the initial text, are easily masked by the third editor, leading to a significant decrease in detection accuracy.\n\nThis demonstrates that the DETree's encoder is capable of capturing fine-grained, generalizable features. Even when three authors are introduced into the test text (while the model was only exposed to two authors during training), and the third author's model, and prompt are unseen, the encoder can still extract key features to distinguish between different authors.\n\n> Q1: It would be nice to show how “human-like” the automatically generated edits are. For example: measuring the stylistic difference between AI-polished text and human-polished text (perhaps using a language model evaluator or some linguistic metrics).\n\nWe supplement analysis by using language model evaluation metrics (Log-Perplexity, Log-Rank) on the HART dataset to measure the stylistic similarity between AI-polished and human-polished texts. Specifically, for the two types of texts, we use these evaluation metrics as the basis for classification. If the two types of texts cannot be separated effectively (i.e., low classification accuracy), it indicates that the distributions of the texts are highly consistent under the given metric.\n\nHART uses the commonly used model GPT-Neo-2.7B to provide the conditional probability of each token. Log-Perplexity is calculated as follows:\n\n$\\text{Log-Perplexity}=-\\frac{1}{N}\\sum_{i=1}^{N}\\log p(w_i\\mid w_{<i})$\n\nLog-Rank measures the position of the correct token in the predicted ranking. For each token, we record its rank $r_i$ in the probability-sorted list, then take the logarithm and compute the average:\n\n$\\text{Log‑Rank}=\\frac{1}{t}\\sum_{i=1}^{t}\\log r_{\\theta}\\bigl(x_i\\mid x_{<i}\\bigr)$\n\nWe report the classification metric TPR@5% FPR. The detailed description of the HART dataset can be found in lines 749-761 of the paper. The differences between AI-polished and human-polished texts using these metrics are visible in Level-2 of the HART dataset.\n\n||Level-3|Level-2|Level-1|\n|-|-|-|-|\n|Log-Perplexity|33|11|6|\n|Log-Rank|39|11|8|\n|DETree|95.3|98.5|99.5|\n\n> Q2: For the most part the paper is clearly written, but a few points could use clarification. The hierarchical prior assumptions are introduced in text and Figure 7, but the main paper only briefly references “prior1/2/3” in results without deep explanation. Readers must dig into Appendix B to understand them. Also, some implementation details, such as the exact temperature used in TSCL or how the KNN search is accelerated (if at all), are not obvious in the main text. The authors could consider adding them as footnotes.\n\nRegarding the hierarchical prior assumptions, we will provide a more detailed explanation of these in the final version to ensure that readers can better understand our assumptions. Additionally, we will include further details on the temperature (0.05) used in TSCL and the Faiss-GPU acceleration of the K-NN search, providing clear explanations as footnotes.\n\n> Q3: Lots of prior work on detection uses TPR@1% FPR as the metric, as keeping the number of false positives low is critical. Could the authors compute and report some results with this metric instead of TPR@5% FPR?\n\nSince the HART paper uses TPR@5% FPR as the comparison metric, we adopted TPR@5% FPR for consistency in our evaluation. We have now included the performance of DETree with different priors on the HART dataset using TPR@1% FPR.\n\n||Level-3||Level-2||Level-1||\n|-|-|-|-|-|-|-|\n||TPR@5%FPR|TPR@1%FPR|TPR@5%FPR|TPR@1%FPR|TPR@5%FPR|TPR@1%FPR|\n|prior1|95.3|88.9|98.5|88.4|99.5|96.7|\n|prior2|88.7|76.0|93.6|58.7|98.9|85.8|\n|prior3|92.2|83.1|96.8|84.1|99.7|94.0|"}, {"review_id": "FfPCX3xTIu", "reviewer": "Reviewer_1CCF", "summary": "The paper proposes hierarchical contrastive training to detect collaborative text between humans and AI. The idea is that there are different degrees of similarity between human text, AI-generated text, modified human text using AI, and modified AI-generated text using AI, etc, so the effectively discriminate all cases, it is natural to arrange those classes hierarchically and use this hierarchy when training an embedding models with a contrastive loss.", "questions": "- Please fix my comments regarding Theorem 3.1. It is possible that this theorem does not contribute much, and perhaps better to remove it. \n- Please explain if you allow few-shot adaptation when evaluating competing methods. This is crucial to understanding whether the improvement is the result of the proposed hierarchical contrastive methodology.", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The representation approach for authorship identification using contrastive learning has drawn significant attention recently. The current paper proposes an interesting extension (hierarchical contrastive). \n- The evaluations appear comprehensive as far as I can tell. They demonstrate well that the proposed method outperformed other methods in identifying collaboration cases. \n\nWeaknesses:\n- While it appears from the results that we can detect collaboration on datasets different from those in training, the process of creating these simulated test cases is so similar to that in the training that it is difficult to argue that the approach can be applied in new scenarios. This problem is not necessarily technical in the sense that test cases are difficult to gather; it is more related to an attempt to tackle too broad a scenario. To clarify, the data science aspect of the work is solid; this weakness is related to the motivation for detecting human-AI collaboration. Perhaps you can consider different situations where hierarchy is natural, so the methodology can be useful? \n- Theorem 3.1 is not stated as a theorem. The sentence \"categories with closer LCA are expected to be more similar in the embedding space\" is problematic because it is unclear with respect to what distribution the expectation is evaluated. The word \"should\" in the second line is confusing. Can you remove it? \nIn Line 247 you refer to the \"constraint defined in Theorem 3.1\". What constraint? Do you mean \"condition\"?\n- In Retrieval-Based Few-Shots adaptation, it is unclear what you mean by \"superficial features\" and why their \"ability to generalize across domains\" is limited. Of course, the performance bottleneck due to differences between train and test cases is the fundamental challenge in learning. It is unclear how the approach taken here is different than standard training. Also, do you allow few-shot adaptation in competing methods?\n- Related works: It is worth noting that the problem of detecting collaboration between authors existed before the AI era, and some methods were developed there (for example, Savoy, Jacques. \"Machine learning methods for stylometry.\" Cham: Springer (2020) and related references). Additionally, related to AI text, [Kashtan & Kipnis, Harvard Data Science Review. 2024] proposed a method to address \"sparse human-AI collaboration\": a few human edits of AI text or vice versa. In general, these references discuss methods that are more explainable as they do not rely on representation learning, as in the current paper.", "was_revised": true, "final_justification": "The authors responded satisfactorily to my inquiries. I believe that the changes will improve the paper's clarity. \n\nI increased the paper's rating.", "rebuttal": "*We sincerely thank the reviewer 1CCF for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.*\n***\n> Weakness1: While it appears from the resutls that we can detect collaboration on datasets different from those in training, the process of creating these simulated test cases is so similar to that in the training that it is difficult to argue that the approach can be applied in new scenrios [...]\n\nThank you for your positive feedback. In our generalization analysis, we did not solely rely on our own dataset, RealBench, but also incorporated challenging datasets like DetectRL, Beemo (Table 2), and HART (Table 3), which are recently published. Notably, HART contains data generated by real human-AI collaboration. The three levels of testing proposed by HART are particularly relevant to real-world complex application scenarios. \n\nThe motivation for our \"human-AI collaboration detection\" task inherently encompasses a wide range of scenarios and is highly challenging. While existing research and published datasets may not yet cover all real-world collaboration types, the methodology we propose is a general detection method with multi-class hierarchical modeling ability and task adaptability. We believe our approach has the potential to be extended to more complex scenarios.\n\nTo better align with realistic scenarios involving more than two authors, we have expanded the HART dataset to include three authors and conducted the following experiments:\n\nWe used models not seen during training, such as gemma-3-12b and deepseekv2, along with new prompts to introduce a third author for text editing in the HART dataset. We designed experiments to explore the impact of different editor sequences on the final text.\n\nDETree is trained on RealBench and evaluated using the development set as the retrieval database. First, we investigated the influence of the first author on the final text, while keeping the second and third authors constant. The results show that when the third author is introduced, the model's ability to distinguish the first author is reduced. However, the model can still effectively differentiate between the two text types.\n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_gemma-3|human_rephrase_gpt-3.5-turbo_polish_gemma-3|96.99|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_deepseekv2|human_rephrase_gpt-3.5-turbo_polish_deepseekv2|96.21|\n|gpt-4o_humanize_gpt-3.5-turbo|human_rephrase_gpt-3.5-turbo|100|\n|gpt-4o_polish_gemma-3-12b|human_polish_gemma-3-12b|99.43|\n|gpt-4o_polish_deepseekv2|human_polish_deepseekv2|99.37|\n||||\n|claude-3-5-sonnet_humanize_gemini-1.5-pro_polish_gemma-3|human_rephrase_gemini-1.5-pro_polish_gemma-3|84.17|\n|claude-3-5-sonnet_humanize_gemini-1.5-pro_polish_deepseekv2|human_rephrase_gemini-1.5-pro_polish_deepseekv2|84.05|\n|claude-3-5-sonnet_humanize_gemini-1.5-pro|human_rephrase_gemini-1.5-pro|98.14|\n|claude-3-5-sonnet_polish_gemma-3-12b|human_polish_gemma-3-12b|98.81|\n|claude-3-5-sonnet_polish_deepseekv2|human_polish_deepseekv2|98.98|\n\n\nWe also examined the influence of the middle editor on the final text. The results show a significant decline in detection accuracy when the third author is introduced.\n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_gemma-3|gpt-4o_humanize_claude-3-5-sonnet_polish_gemma-3-12b|76.50|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_deepseekv2|gpt-4o_humanize_claude-3-5-sonnet_polish_deepseekv2|82.05|\n|gpt-4o_humanize_gpt-3.5-turbo|gpt-4o_humanize_claude-3-5-sonnet|98.78|\n||||\n|human_rephrase_gpt-4o_polish_gemma-3|human_rephrase_gemini-1.5-pro_polish_gemma-3|68.71|\n|human_rephrase_gpt-4o_polish_deepseekv2|human_rephrase_gemini-1.5-pro_polish_deepseekv2|68.02|\n|human_rephrase_gpt-4o|human_rephrase_gemini-1.5-pro|99.49|\n\nFinally, the third editor is easily distinguishable. \n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|qwen2.5-72b_humanize_claude-3-5-sonnet_polish_gemma-3|qwen2.5-72b_humanize_claude-3-5-sonnet_polish_deepseekv2|100|\n|llama-3.3-70b_humanize_gemini-1.5-pro_polish_gemma-3|llama-3.3-70b_humanize_gemini-1.5-pro_polish_deepseekv2|97.69|\n|human_rephrase_gpt-4o_polish_gemma-3|human_rephrase_gpt-4o_polish_deepseekv2|99.6|\n\nThe initial creator of the text sets the general direction and ideas, so these features are difficult to mask through editing. However, the middle editor's features, which are based on rephrasing the initial text, are easily masked by the third editor, leading to a significant decrease in detection accuracy.\n\nThis demonstrates that the DETree's encoder is capable of capturing fine-grained, generalizable features. Even when three authors are introduced into the test text (while the model was only exposed to two authors during training), and the third author's model, and prompt are unseen, the encoder can still extract key features to distinguish between different authors.\n\n> Weakness2&Q1: Theorem 3.1 is not stated as a theorem. The sentence \"categories with closer LCA are expected to be more similar in the embedding space.\" is problematic because it is unclear over whay is the epxectation. The word \"should\" in the second line is confusing. Can you remove it? In Line 247 you refer to the \"constraint defined in Theorem 3.1\". What constraint? do you mean \"condition\"?\n\nWe acknowledge that Theorem 3.1 was not clearly stated as a theorem, and we will revise it to a more formal mathematical expression. Specifically:\n\nIf in the tree $\\mathcal{T}$, for any leaf class X corresponding to node c, the following holds:\n\n$\\mathbb{E}\\bigl[\\mathrm{sim}(X, Y)\\bigr] > \\mathbb{E}\\bigl[\\mathrm{sim}(X, Z)\\bigr],  \\quad \\forall\\, 0 \\le i < j \\le d_c,\\; Y \\in H_c^{(i)},\\; Z \\in H_c^{(j)}.$\n\nThen, we can equivalently deduce that for any leaf classes X, Y, Z, the following inequality holds:\n\n$\\mathbb{E}\\bigl[\\mathrm{sim}(X, Y)\\bigr] > \\mathbb{E}\\bigl[\\mathrm{sim}(X, Z)\\bigr], \\quad \\text{if } d_{\\mathrm{LCA}(X, Y)} > d_{\\mathrm{LCA}(X, Z)}.$\n\nIntuitively, we aim to optimize the second set of inequalities, but it is difficult to optimize directly. Therefore, we transform it into the equivalent first set of inequalities for optimization. \n\nWe would appreciate your feedback on whether this form is appropriate, or if presenting it as a theorem may not be necessary.\n\nIn line 247, the \"constraint\" refers to the inequality constraint defined by Equation 2 in the original paper. We will update the wording accordingly to avoid confusion and ensure clarity.\n\n> Weakness3: In Retrieval-Based Few-Shots adaptation, it is unclear what you mean by \"superficial features\" and why their \"ability to generalize across domains\" is limited. Of course, perforamcne bottlneck due to differences between train and test cases is the fundamental challenge in learning. It is unclear how the approach taken here is different than standard training? Also, do you allow few-shots adaptation in competing methods?\n\nWe thank the reviewer for pointing this out. The term \"superficial features\" is indeed imprecise. We refer to features learned by supervised binary classification methods, which are optimized solely for binary discrimination and may capture spurious but separable features in the training data, leading to poor generalization in OOD settings. We performed dimensionality reduction on features extracted by MAGE (a supervised binary method) on the Beemo dataset (OOD) and observed significant distributional overlap, indicating a lack of effective separation. This is what we mean by the \"limit ability to generalize across domains.\" We will include the corresponding visualization in the final version.\n\nIn contrast, DETree captures intrinsic affinities across arbitrary categories, guiding the model to learn the fine-grained differences and relationships between categories. For example, even when texts are generated by different models from the same prompt, or when the same model generates texts through paraphrasing, polishing, or continuation, we aim for the model to understand the inherent relationships and distinctions between them. Thus, even with a small number of support samples, our method still achieves robust source matching and classification. \n\nIn Table 2 (right), we report the performance of all models with few-shot capability under the same support set; in Table 2 (left), we report the zero-shot results of all models without such capability under the same test setting.\n\n> Weakness4: Related works: It is worth noting that the problem of deecting collaboration between authors existed before the AI era, and some methods were developed there [...]\n\nIndeed, author collaboration detection has been an active area of research prior to the AI era. We will incorporate discussions on text stylometry and author attribution in the related works section of the final version. Our approach, however, specifically focuses on detecting a wide range of human-AI collaborative text patterns in the context of the AI generation era. While we acknowledge the interpretability advantages of stylometry methods, our approach demonstrates superior generalization and robustness, offering a valuable complement to traditional methods.\n\n> Q2 Please explain if you allow few-shot adaptation when evaluating competing methods. This is crucial to understanding whether the improvement is the result of the proposed hierarchical contrastive methodology.\n\nIn Table 1, we provide multiple sets of supervised experimental results, including training individually on each dataset and training with different priors on RealBench, which clearly demonstrates the performance improvements attributed to our hierarchical contrastive learning method.\n\nTo ensure fairness, in Table 2, for models that support few-shot adaptation, we report their performance under the same support set. For models that do not support few-shot adaptation, we also provide results under the same testing environment without few-shot adaptation."}, {"review_id": "cXerYSTbtU", "reviewer": "Reviewer_U1pr", "summary": "This paper introduces DETree, a method for detecting AI-generated text that is both interpretable and effective. The authors propose to build an interpretable decision tree classifier trained on features derived from both token-level perplexity signals and semantic embedding similarities.", "questions": "how does DETree behave when model-generated text has been post-edited by humans or blends human and AI content (i.e., in mixed authorship settings)?\n\nHow much does each category of features—local perplexity, global perplexity, embedding similarity—contribute to the model’s performance? \n\ncould you compare DETree to other lightweight or interpretable models such as logistic regression, rule-based classifiers, or shallow decision forests?\n\nHow do you balance the depth and complexity of the decision tree against interpretability? Does a deeper tree improve performance substantially, or introduce interpretability costs?", "limitations": "yes", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n(1) The authors carefully design a suite of features that capture both local (e.g., token-level perplexity) and global (e.g., embedding similarity) signals.\n(2) DETree is evaluated across multiple settings—cross-domain, cross-LLM, and adversarial attacks (paraphrasing)—and consistently performs competitively with black-box detectors.\n(3) The use of a decision tree as the classifier provides transparent, step-by-step reasoning behind predictions. The authors also provide visualizations of decision paths and feature importance to illustrate interpretability in action.\n\nWeakness:\n\nCertain feature definitions, especially around how local and global perplexity are computed and normalized, could be further clarified clearer mathematical notation or concrete examples.", "was_revised": false, "final_justification": "", "rebuttal": "*We sincerely thank the reviewer U1pr for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.*\n***\n> Weakness:Certain feature definitions, especially around how local and global perplexity are computed and normalized, could be further clarified clearer mathematical notation or concrete examples.\n\nThank you for pointing out the lack of clarity in the expression. You mentioned that \"local and global perplexity are computed and normalized, and could be further clarified with clearer mathematical notation or concrete examples.\" However, we did not use the terms \"local and global perplexity\" in the paper. To avoid any misunderstandings, we would appreciate further clarification on which specific part you are referring to, so we can make the necessary improvements and additions.\n\n> Q1: how does DETree behave when model-generated text has been post-edited by humans or blends human and AI content (i.e., in mixed authorship settings)?\n\nIn mixed authorship settings, we investigate the behavior of DETree in the Hybrid Text Detection section of the paper, based on the HART dataset. HART categorizes samples based on the extent of AI involvement in text  generation into four types: human-written (Type 0), human content rephrased by AI (Type 1), AI generated content humanized through various methods (including LLM paraphrasing, human editing,  and commercial tools) (Type 2), and fully AI-generated content (Type 3).  Based on this labeling scheme, HART defines three detection task levels: Level 1 detects AI  involvement (Type 0 vs. Type 1/2/3), Level 2 detects whether the core content is AI-generated (Type  0/1 vs. Type 2/3), and Level 3 detects whether the text is fully generated by AI (Type 0/1/2 vs. Type 3). \n\nDETree is trained on RealBench and evaluated strictly according to the testing setup of the HART dataset. As shown in Table 3 of the paper, the model performs significantly better than the results reported in the original paper under this setup.\n\nTo better align with realistic scenarios involving more than two authors, we have expanded the HART dataset to include three authors and conducted the following experiments:\n\nWe used models not seen during training, such as gemma-3-12b and deepseekv2, along with new prompts to introduce a third author for text editing in the HART dataset. We designed experiments to explore the impact of different editor sequences on the final text.\n\nDETree is trained on RealBench and evaluated using the development set as the retrieval database. First, we investigated the influence of the first author on the final text, while keeping the second and third authors constant. The results show that when the third author is introduced, the model's ability to distinguish the first author is reduced. However, the model can still effectively differentiate between the two text types.\n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_gemma-3|human_rephrase_gpt-3.5-turbo_polish_gemma-3|96.99|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_deepseekv2|human_rephrase_gpt-3.5-turbo_polish_deepseekv2|96.21|\n|gpt-4o_humanize_gpt-3.5-turbo|human_rephrase_gpt-3.5-turbo|100|\n|gpt-4o_polish_gemma-3-12b|human_polish_gemma-3-12b|99.43|\n|gpt-4o_polish_deepseekv2|human_polish_deepseekv2|99.37|\n||||\n|claude-3-5-sonnet_humanize_gemini-1.5-pro_polish_gemma-3|human_rephrase_gemini-1.5-pro_polish_gemma-3|84.17|\n|claude-3-5-sonnet_humanize_gemini-1.5-pro_polish_deepseekv2|human_rephrase_gemini-1.5-pro_polish_deepseekv2|84.05|\n|claude-3-5-sonnet_humanize_gemini-1.5-pro|human_rephrase_gemini-1.5-pro|98.14|\n|claude-3-5-sonnet_polish_gemma-3-12b|human_polish_gemma-3-12b|98.81|\n|claude-3-5-sonnet_polish_deepseekv2|human_polish_deepseekv2|98.98|\nWe also examined the influence of the middle editor on the final text. The results show a significant decline in detection accuracy when the third author is introduced. The model finds it much harder to distinguish between the two text types when the middle editor is involved.\n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_gemma-3|gpt-4o_humanize_claude-3-5-sonnet_polish_gemma-3-12b|76.50|\n|gpt-4o_humanize_gpt-3.5-turbo_polish_deepseekv2|gpt-4o_humanize_claude-3-5-sonnet_polish_deepseekv2|82.05|\n|gpt-4o_humanize_gpt-3.5-turbo|gpt-4o_humanize_claude-3-5-sonnet|98.78|\n||||\n|human_rephrase_gpt-4o_polish_gemma-3|human_rephrase_gemini-1.5-pro_polish_gemma-3|68.71|\n|human_rephrase_gpt-4o_polish_deepseekv2|human_rephrase_gemini-1.5-pro_polish_deepseekv2|68.02|\n|human_rephrase_gpt-4o|human_rephrase_gemini-1.5-pro|99.49|\n\nFinally, the third editor is easily distinguishable. \n\n|source1|source2|DETree(aucroc)|\n|-|-|-|\n|qwen2.5-72b_humanize_claude-3-5-sonnet_polish_gemma-3|qwen2.5-72b_humanize_claude-3-5-sonnet_polish_deepseekv2|100|\n|llama-3.3-70b_humanize_gemini-1.5-pro_polish_gemma-3|llama-3.3-70b_humanize_gemini-1.5-pro_polish_deepseekv2|97.69|\n|human_rephrase_gpt-4o_polish_gemma-3|human_rephrase_gpt-4o_polish_deepseekv2|99.6|\n\nThe initial creator of the text sets the general direction and ideas, so these features are difficult to mask through editing. However, the middle editor's features, which are based on rephrasing the initial text, are easily masked by the third editor, leading to a significant decrease in detection accuracy.\n\nThis demonstrates that the DETree's encoder is capable of capturing fine-grained, generalizable features. Even when three authors are introduced into the test text (while the model was only exposed to two authors during training), and the third author's model, and prompt are unseen, the encoder can still extract key features to distinguish between different authors.\n\n> Q2: How much does each category of features—local perplexity, global perplexity, embedding similarity—contribute to the model’s performance?\n\nIn our approach, embedding similarity plays a central role in computing the class similarity matrix, constructing the HAT structure, and performing the final retrieval classification. As for local perplexity and global perplexity, these features were not used in the paper.\n\n> Q3: could you compare DETree to other lightweight or interpretable models such as logistic regression, rule-based classifiers, or shallow decision forests?\n\nIn our experiments, we compared DETree with rule-based classifiers designed using perplexity and its variants, such as LRR, Glimpse, Fast-Detect, and Binoculars. These classifiers are lightweight and interpretable. As shown in Tables 1, 2, and 3, DETree outperforms these models in both in-domain, out-of-domain, and hybrid text test environments.\n\n> Q4: How do you balance the depth and complexity of the decision tree against interpretability? Does a deeper tree improve performance substantially, or introduce interpretability costs?\n\nThank you for your insightful comment. In the construction of HAT subtrees, we control the degree of node splitting using a termination threshold. A smaller threshold results in deeper trees. To assess the impact of tree depth on model performance, we tested three different termination thresholds (0.3, 0.25, and 0.2), corresponding to maximum tree depths of 3, 6, and 9, respectively. The model was trained on RealBench and evaluated on MAGE.\n\nThe experimental results show that DETree performs well across different tree depths. Specifically, when the tree depth is shallow, fine-grained category information is lost, leading to a performance drop. As the tree depth increases, model performance improves, reaching an optimal level at a depth of 6. Further increasing the depth only results in a slight decrease in performance. The results reported in the paper correspond to the experiment with a maximum tree depth of 6.\n\n|max dep|3|6|9|\n|-|-|-|-|\n|F1|96.48|96.96|96.84|\n|AvgRecall|96.38|96.87|96.76|\n\nWe will include additional ablation studies in the final version to further clarify this aspect."}, {"review_id": "D5UijwEmzG", "reviewer": "Reviewer_DGKn", "summary": "The work \"DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning\" introduces DETree, a detector for hybrid (human-AI collaborative) text based on representation learning. The paper tackles an already very important problem of identifying how large language models (LLMs) participate in a text's creation rather than merely deciding \"human vs. machine\" as a classification task. The key contributions are: \n* RealBench, a new benchmark with 16.4M samples spanning 1204 fine‑grained categories that encode generation pipelines (e.g., Llama‑3 draft polished by GPT‑4o);\n* a new approach to identifying these categories as follows:\n    * construct a Hierarchical Affinity Tree (HAT) over 1200+ source categories that arise in RealBench;\n    * train the encoder with a Tree‑Structured Contrastive Loss (TSCL) that forces embeddings to respect that hierarchy, \n    * then perform k‑NN classification with a compressed, centroids‑only database.\n\nExperiments cover in‑distribution, OOD, adversarial and few‑shot settings. DETree convincingly beats a strong set of baselines (RoBERTa, T5‑Sentinel, DeTeCTive, Fast‑DetectGPT, Binoculars, HART, UAR) on four public benchmarks and in several new scenarios, showing strong empirical gains.", "questions": "1. How many RealBench samples are verbatim or near‑duplicate across training/validation/test? Please provide a de‑duplication analysis.\n2. Did you test DETree with an entirely unseen LLM that does not appear in RealBench?\n3. What is the exact setting of the Silhouette‑based subtree edit heuristic (threshold, max depth)? An ablation varying these parameters would clarify its importance.\n4. Could TSCL be combined with a lightweight classifier (e.g., logistic regression on frozen embeddings) to remove the dependency on k‑NN?", "limitations": "The authors mention the scarcity of >2‑author collaborations and rare domains , but should also discuss:\n* the synthetic nature and potential style bias of RealBench;\n* relatively heavy GPU and memory footprintl;\n* how easy it would be for sophisticated attackers to fine‑tune their style to evade tree‑aware detectors (in the context of the detection-generation arms race).", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths:\n* [*significance*] a large carefully engineered new dataset, which is likely to become a new and important benchmark;\n* [*originality*] as far as I know, this is the first paper to explicitly model hierarchical provenance for hybrid text detection; this is also a good point for significance as hybrid provenance detection is of high practical and societal relevance;\n* [*originality*] TSCL is a non‑trivial extension of supervised contrastive learning;\n* [*quality*] the results are complemented by theoretical proofs;\n* [*significance*] the experimental study is convincing, with a detailed ablation study;\n* [*clarity*] the writing is generally clear and well organized; the figures (UMAP, HAT visualizations) are intuitive and helpful.\n\nWeaknesses:\n* [*quality*] RealBench is largely synthetic (paraphrase/translation of existing corpora); the authors do not quantify annotation noise or genre imbalance;\n* [*originality*] the approach uses well‑known building blocks (contrastive learning, agglomerative clustering, k‑NN), so novelty is incremental;\n* [*significance*] RealBench is also used as the retrieval database at test time, so in‑distribution and some OOD scores may be optimistic.", "was_revised": true, "final_justification": "I really liked the detailed author responses (to me and to other reviewers) but my score is already at \"Accept\", and it's not *that* groundbreaking a paper, so I'll leave it at that.", "rebuttal": "*We sincerely thank the reviewer DGKn for the thoughtful and detailed comments. Below, we provide point-by-point responses to each of the concerns.*\n***\n>Weakness1: [*quality*] RealBench is largely synthetic (paraphrase/translation of existing corpora); the authors do not quantify annotation noise or genre imbalance;\n\nThank you for your valuable feedback regarding annotation noise and genre imbalance. Our annotation process is based on publicly available datasets and is programmatically generated, ensuring determinism and the absence of additional subjective bias. Therefore, it theoretically does not introduce new annotation noise. In the final version, we will include a detailed explanation of the annotation quality in the original datasets and further examine whether any noise analysis has already been conducted. Additionally, we will provide a breakdown of the genre distribution in RealBench in the appendix.\n\n>Weakness2: [*originality*] the approach uses well‑known building blocks (contrastive learning, agglomerative clustering, k‑NN), so novelty is incremental;\n\nWe appreciate the reviewer's comments on the originality of our approach. Our method models the hierarchical relationships between different sources in human-AI collaborative texts using the Hierarchical Affinity Tree(HAT) structure. Building on this, we introduce Tree-structured Contrastive Loss (TSCL), which innovatively applies contrastive learning to the modeling of HAT structures, aligning the embedding space with the HAT structure. This method provides a general and interpretable solution for handling scalable author categories and can be conveniently applied to other hierarchical tasks. Additionally, we propose a K-Means-based compression algorithm and a retrieval-based few-shot method, which significantly enhance both classification speed and accuracy.\nOur approach demonstrates state-of-the-art performance in AI text detection across in-domain, out-of-domain, and hybrid text scenarios. It offers an effective detection solution in the context of the rapid emergence of new LLMs.\n\n>Weakness3: [*significance*] RealBench is also used as the retrieval database at test time, so in‑distribution and some OOD scores may be optimistic.\n\nWe would like to provide some additional clarifications regarding the experimental setup:\n\nFirst, in Table 1, we present multiple sets of supervised training results, including training individually on each dataset and training with different priors on RealBench.\n\nSecond, although DTree in Table 2 uses RealBench as the training set, it is tested on multiple retrieval databases, including RealBench, MAGE, M4, and RAID. The results show that even without using RealBench as the retrieval database, the models perform well in OOD environments. Additionally, we conducted few-shot experiments, reporting performance on the same support set for models that support few-shot learning, and providing results without few-shot support for other models to ensure fair comparisons.\n\nFinally, in Table 3, we trained the model solely on RealBench and evaluated it according to the original test setup from the HART dataset, where the retrieval database was not the RealBench training set but the dev split of HART.\n\n> Q1: How many RealBench samples are verbatim or near‑duplicate across training/validation/test? Please provide a de‑duplication analysis.\n\nThank you for your reminder. We performed a duplication analysis on the texts in RealBench using a hash algorithm. There are no duplicate texts in the original datasets- MAGE, M4, RAID, OUTFOX, or in their corresponding augmented versions. However, in the TuringBench dataset, 663 texts are identical between the training and test sets, and 311 texts are identical between the training and validation sets. A small number of duplicates were also found in the augmented data derived from TuringBench.\n\n> Q2: Did you test DETree with an entirely unseen LLM that does not appear in RealBench?\n\nLLaMA 3.1‑70B and GPT‑4o in Beemo, Claude and PaLM‑2 in DetectRL, GPT‑4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, LLaMA 3.3‑70B‑Instruct, and Qwen 2.5‑72B‑Instruct in HART are all models not seen in RealBench.\n\n> Q3: What is the exact setting of the Silhouette‑based subtree edit heuristic (threshold, max depth)? An ablation varying these parameters would clarify its importance.\n\nThank you for your insightful comment. In the construction of HAT subtrees, we control the degree of node splitting using a termination threshold. A smaller threshold results in deeper trees. To assess the impact of tree depth on model performance, we tested three different termination thresholds (0.3, 0.25, and 0.2), corresponding to maximum tree depths of 3, 6, and 9, respectively. The model was trained on RealBench and evaluated on MAGE.\n\nThe experimental results show that DETree performs well across different tree depths. Specifically, when the tree depth is shallow, fine-grained category information is lost, leading to a performance drop. As the tree depth increases, model performance improves, reaching an optimal level at a depth of 6. Further increasing the depth only results in a slight decrease in performance. The results reported in the paper correspond to the experiment with a maximum tree depth of 6.\n\n|max dep|3|6|9|\n|-|-|-|-|\n|F1|96.48|96.96|96.84|\n|AvgRecall|96.38|96.87|96.76|\n\nWe will include additional ablation studies in the final version to further clarify this aspect.\n\n> Q4: Could TSCL be combined with a lightweight classifier (e.g., logistic regression on frozen embeddings) to remove the dependency on k‑NN?\n\nWith TSCL training, the fine-grained features learned by DETree are capable of distinguishing between various text categories. As a result, any classifier can be used for classification, and this can be combined with our K-Means Based Database Compression method to further reduce training overhead.\n\nWhile we chose to use k-NN for classification due to its ability to fit complex decision boundaries and to avoid introducing additional training overhead for the few-shot method.\n\nThe following experimental results report that DETree was trained on the MAGE training set, with the K-Means compression method used to reduce the training set to 10K samples for use with traditional classifiers, and tested on the MAGE test set.\n\n||F1|AvgRecall|\n|-|-|-|\n|KNN|96.96|96.87|\n|SVM|96.63|96.57|\n|Logistic Regression|96.60|96.53|\n|Random Forest|96.60|96.49|\n\n> Limitations:\n>\n> The authors mention the scarcity of >2‑author collaborations and rare domains, but should also discuss: 1.the synthetic nature and potential style bias of RealBench; 2. relatively heavy GPU and memory footprintl; 3. how easy it would be for sophisticated attackers to fine‑tune their style to evade tree‑aware detectors (in the context of the detection-generation arms race).\n\nAs for the synthetic nature and potential style bias of RealBench, we employed different paraphrasing models and prompts during both the training and testing phases of RealBench. Additionally, we introduced multiple external datasets with significant differences in style and generation methods (such as MAGE, M4, DetectRL, Beemo, HART, etc.) during evaluation to ensure diversity and generalization.\n\nDuring the training phase, we use a RoBERTa-large encoding model with approximately 0.5B parameters. Given the large dataset size, training can be completed in about 40 hours using 8×RTX4090 GPUs.\n\nIn the inference phase, when we compress the database to 10k samples for k-NN retrieval classification, the computational overhead is equivalent to attaching a 10k-sized classification head (approximately 10M parameters) after the encoder, with minimal memory cost.\n\nRegarding the concern about sophisticated attackers fine-tuning their style to evade tree-aware detectors, this adversarial issue has not been addressed in the current work. We acknowledge the importance of this challenge and will discuss it in the limitations and future work sections of the final version.\n\n> Paper Formatting Concerns:\n>\n> The text on figures 3, 10, 12-15 is tiny to the point of unreadability (especially in the Appendix). Consider breaking them up into more figures to increase the font size. Minor issues with text consistency, e.g,. capiltalized \"in Equation 3\" on l.174 but immediately \"in equation 4\" on l.175.\n\nThank you for the valuable comments on formatting. We will address the above issues in the final version."}]

Author rebuttal:

Meta-review: This paper tackles the timely and practically important problem of detecting hybrid human–AI authorship by learning a hierarchical provenance structure (HAT) and aligning embeddings with a tree‑structured contrastive loss (TSCL). While the building blocks (contrastive learning, clustering, k‑NN) are familiar, their combination for hierarchical provenance is apt and yields strong empirical gains across supervised, OOD, few‑shot, and hybrid evaluations. The release of RealBench (16.4M samples, 1204 categories) is a significant resource that will likely catalyze follow‑up work. Main concerns are (i) manual priors and subtree edits during HAT construction (risk of bias/selection), (ii) the synthetic nature of RealBench and confirmed cross‑split duplicates in one source dataset, and (iii) reliance on a retrieval database at inference that can blur OOD boundaries if not carefully set. The rebuttal addressed measurement and fairness questions. Reviewer consensus is favorable (three accepts, one low‑confidence borderline with a clear misreading).
Decision: Accept (poster)