Title: DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: AI-generated text detection; hybrid text attribution; few-shot adaptation;
Abstract: Detecting AI-involved text is essential for combating misinformation, plagiarism, and academic misconduct. 
However, AI text generation includes diverse collaborative processes (AI-written text edited by humans, human-written text edited by AI, and AI-generated text refined by other AI), where various or even new LLMs could be involved. Texts generated through these varied processes exhibit complex characteristics, presenting significant challenges for detection. Current methods model these processes rather crudely, primarily employing binary classification (purely human vs. AI-involved) or multi-classification (treating human-AI collaboration as a new class).
We observe that representations of texts generated through different processes exhibit inherent clustering relationships.
Therefore, we propose DETree, a novel approach that models the relationships among different processes as a Hierarchical Affinity Tree structure, and introduces a specialized loss function that aligns text representations with this tree. To facilitate this learning, we developed RealBench, a comprehensive benchmark dataset that automatically incorporates a wide spectrum of hybrid texts produced through various human-AI collaboration processes.
Our method improves performance in hybrid text detection tasks and significantly enhances robustness and generalization in out-of-distribution scenarios, particularly in few-shot learning conditions, further demonstrating the promise of training-based approaches in OOD settings. 
Our code and dataset are available at https://github.com/heyongxin233/DETree.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe work \"DETree: DEtecting Human-AI Collaborative Texts via Tree-Structured Hierarchical Representation Learning\" introduces DETree, a detector for hybrid (human-AI collaborative) text based on representation learning. The paper tackles an already very important problem of identifying how large language models (LLMs) participate in a text's creation rather than merely deciding \"human vs. machine\" as a classification task. The key contributions are: \n* RealBench, a new benchmark with 16.4M samples spanning 1204 fine‑grained categories that encode generation pipelines (e.g., Llama‑3 draft polished by GPT‑4o);\n* a new approach to identifying these categories as follows:\n    * construct a Hierarchical Affinity Tree (HAT) over 1200+ source categories that arise in RealBench;\n    * train the encoder with a Tree‑Structured Contrastive Loss (TSCL) that forces embeddings to respect that hierarchy, \n    * then perform k‑NN classification with a compressed, centroids‑only database.\n\nExperiments cover in‑distribution, OOD, adversarial and few‑shot settings. DETree convincingly beats a strong set of baselines (RoBERTa, T5‑Sentinel, DeTeCTive, Fast‑DetectGPT, Binoculars, HART, UAR) on four public benchmarks and in several new scenarios, showing strong empirical gains.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* [*significance*] a large carefully engineered new dataset, which is likely to become a new and important benchmark;\n* [*originality*] as far as I know, this is the first paper to explicitly model hierarchical provenance for hybrid text detection; this is also a good point for significance as hybrid provenance detection is of high practical and societal relevance;\n* [*originality*] TSCL is a non‑trivial extension of supervised contrastive learning;\n* [*quality*] the results are complemented by theoretical proofs;\n* [*significance*] the experimental study is convincing, with a detailed ablation study;\n* [*clarity*] the writing is generally clear and well organized; the figures (UMAP, HAT visualizations) are intuitive and helpful.\n\nWeaknesses:\n* [*quality*] RealBench is largely synthetic (paraphrase/translation of existing corpora); the authors do not quantify annotation noise or genre imbalance;\n* [*originality*] the approach uses well‑known building blocks (contrastive learning, agglomerative clustering, k‑NN), so novelty is incremental;\n* [*significance*] RealBench is also used as the retrieval database at test time, so in‑distribution and some OOD scores may be optimistic.\n\nQUESTIONS:\n1. How many RealBench samples are verbatim or near‑duplicate across training/validation/test? Please provide a de‑duplication analysis.\n2. Did you test DETree with an entirely unseen LLM that does not appear in RealBench?\n3. What is the exact setting of the Silhouette‑based subtree edit heuristic (threshold, max depth)? An ablation varying these parameters would clarify its importance.\n4. Could TSCL be combined with a lightweight classifier (e.g., logistic regression on frozen embeddings) to remove the dependency on k‑NN?\n\nLIMITATIONS:\nThe authors mention the scarcity of >2‑author collaborations and rare domains , but should also discuss:\n* the synthetic nature and potential style bias of RealBench;\n* relatively heavy GPU and memory footprintl;\n* how easy it would be for sophisticated attackers to fine‑tune their style to evade tree‑aware detectors (in the context of the detection-generation arms race).", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-02"}

comments: SUMMARY:
This paper introduces DETree, a method for detecting AI-generated text that is both interpretable and effective. The authors propose to build an interpretable decision tree classifier trained on features derived from both token-level perplexity signals and semantic embedding similarities.

STRENGTHS AND WEAKNESSES:
Strength:
(1) The authors carefully design a suite of features that capture both local (e.g., token-level perplexity) and global (e.g., embedding similarity) signals.
(2) DETree is evaluated across multiple settings—cross-domain, cross-LLM, and adversarial attacks (paraphrasing)—and consistently performs competitively with black-box detectors.
(3) The use of a decision tree as the classifier provides transparent, step-by-step reasoning behind predictions. The authors also provide visualizations of decision paths and feature importance to illustrate interpretability in action.

Weakness:

Certain feature definitions, especially around how local and global perplexity are computed and normalized, could be further clarified clearer mathematical notation or concrete examples.

QUESTIONS:
how does DETree behave when model-generated text has been post-edited by humans or blends human and AI content (i.e., in mixed authorship settings)?

How much does each category of features—local perplexity, global perplexity, embedding similarity—contribute to the model’s performance? 

could you compare DETree to other lightweight or interpretable models such as logistic regression, rule-based classifiers, or shallow decision forests?

How do you balance the depth and complexity of the decision tree against interpretability? Does a deeper tree improve performance substantially, or introduce interpretability costs?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes hierarchical contrastive training to detect collaborative text between humans and AI. The idea is that there are different degrees of similarity between human text, AI-generated text, modified human text using AI, and modified AI-generated text using AI, etc, so the effectively discriminate all cases, it is natural to arrange those classes hierarchically and use this hierarchy when training an embedding models with a contrastive loss.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The representation approach for authorship identification using contrastive learning has drawn significant attention recently. The current paper proposes an interesting extension (hierarchical contrastive). \n- The evaluations appear comprehensive as far as I can tell. They demonstrate well that the proposed method outperformed other methods in identifying collaboration cases. \n\nWeaknesses:\n- While it appears from the results that we can detect collaboration on datasets different from those in training, the process of creating these simulated test cases is so similar to that in the training that it is difficult to argue that the approach can be applied in new scenarios. This problem is not necessarily technical in the sense that test cases are difficult to gather; it is more related to an attempt to tackle too broad a scenario. To clarify, the data science aspect of the work is solid; this weakness is related to the motivation for detecting human-AI collaboration. Perhaps you can consider different situations where hierarchy is natural, so the methodology can be useful? \n- Theorem 3.1 is not stated as a theorem. The sentence \"categories with closer LCA are expected to be more similar in the embedding space\" is problematic because it is unclear with respect to what distribution the expectation is evaluated. The word \"should\" in the second line is confusing. Can you remove it? \nIn Line 247 you refer to the \"constraint defined in Theorem 3.1\". What constraint? Do you mean \"condition\"?\n- In Retrieval-Based Few-Shots adaptation, it is unclear what you mean by \"superficial features\" and why their \"ability to generalize across domains\" is limited. Of course, the performance bottleneck due to differences between train and test cases is the fundamental challenge in learning. It is unclear how the approach taken here is different than standard training. Also, do you allow few-shot adaptation in competing methods?\n- Related works: It is worth noting that the problem of detecting collaboration between authors existed before the AI era, and some methods were developed there (for example, Savoy, Jacques. \"Machine learning methods for stylometry.\" Cham: Springer (2020) and related references). Additionally, related to AI text, [Kashtan & Kipnis, Harvard Data Science Review. 2024] proposed a method to address \"sparse human-AI collaboration\": a few human edits of AI text or vice versa. In general, these references discuss methods that are more explainable as they do not rely on representation learning, as in the current paper.\n\nQUESTIONS:\n- Please fix my comments regarding Theorem 3.1. It is possible that this theorem does not contribute much, and perhaps better to remove it. \n- Please explain if you allow few-shot adaptation when evaluating competing methods. This is crucial to understanding whether the improvement is the result of the proposed hierarchical contrastive methodology.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nDETree encodes texts with a supervised contrastive objective, constructs a Hierarchical Affinity Tree (HAT) from class centroids, and re-optimises representations with a tree-structured contrastive loss; at test time it performs k-NN retrieval over a K-means–compressed bank of only 10K prototypes, cutting memory and latency while preserving accuracy.\nAcross the HART hierarchy, RealBench and MAGE benchmarks, DETree attains up to 0.988 F1 / 99.5% TPR@5 % FPR, beating the strongest published detector by 30–40 pp on several sub-tasks. The model maintains high accuracy when trained on 10% of categories and gains ~30 pp in few-shot out-of-distribution transfer. Without any adversarial training DETree remains markedly more robust to diverse perturbations than an adversarially-trained RoBERTa baseline.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n* The paper introduces a Hierarchical Affinity Tree (HAT) to model relationships among text generation processes, moving beyond flat binary or multi-class detectors. This is novel in that it does not treat \"hybrid\" text as just another class or rely on coarse metrics of AI involvement.  \n* The proposed RealBench dataset automatically curates \\~16.4 million samples across 1,204 categories of human-AI collaborative text, which is far larger and more diverse than prior benchmarks. RealBench covers a wide spectrum of collaboration modes (human drafts refined by AI, AI outputs edited by humans, AI outputs refined by other AI models, etc.) and concrete transformation strategies including paraphrasing, extension, polishing, and translation.  \n* Experimental scope is broad, covers baselines from 12 detector families and 8 distinct benchmarks (including in- and out-of-distribution).\n\nWeaknesses\n\n* One concern is the manual intervention used in shaping the HAT. The authors acknowledge that purely agglomerative clustering can yield \"unnatural\" splits, so they employ an editable top-down reorganization using task-specific prior knowledge (Appendix B). In practice, they experimented with three hand-crafted priors (hybrid text closer to AI, closer to human, or separate) and chose the one yielding best performance. While this analysis is interesting, it does mean the final tree is partly guided by human assumptions rather than learned entirely from data.  \n* Two-stage training on a 16M-sample corpus plus k-NN retrieval at inference raises memory/latency concerns despite prototype compression.  \n* The paper focuses on texts generated through one-step human-AI collaboration, e.g. a human draft then one AI edit, or one AI generates then one human edit, etc. It mentions not covering scenarios with more than two \"authors\" due to data scarcity. This is a reasonable limitation, but it means very complex yet realistic writing processes (iterative back-and-forth edits or multiple AIs) are out of scope.\n\nQUESTIONS:\n* It would be nice to show how “human-like” the automatically generated edits are. For example: measuring the stylistic difference between AI-polished text and human-polished text (perhaps using a language model evaluator or some linguistic metrics).  \n* For the most part the paper is clearly written, but a few points could use clarification. The hierarchical prior assumptions are introduced in text and Figure 7, but the main paper only briefly references “prior1/2/3” in results without deep explanation. Readers must dig into Appendix B to understand them. Also, some implementation details, such as the exact temperature used in TSCL or how the *K*\\-NN search is accelerated (if at all), are not obvious in the main text. The authors could consider adding them as footnotes.  \n* Lots of prior work on detection uses TPR@1% FPR as the metric, as keeping the number of false positives low is critical. Could the authors compute and report some results with this metric instead of TPR@5% FPR?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}