Deep Kernel Relative Test for Machine-generated Text Detection

Authors: Yiliao Song, Zhenqiao Yuan, Shuhai Zhang, Zhen Fang, Jun Yu, Feng Liu
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Large language models, Machine-generated text detection, Relative test

Abstract:
Recent studies demonstrate that two-sample test can effectively detect machine-generated texts (MGTs) with excellent adaptation ability to texts generated by newer LLMs.  However, two-sample test-based detection relies on the assumption that human-written texts (HWTs) must follow the distribution of seen HWTs. As a result, it tends to make mistakes in identifying HWTs that deviate from the seen HWT distribution, limiting their use in sensitive areas like academic integrity verification. To address this issue, we propose to employ non-parametric kernel relative test to detect MGTs by testing whether it is statistically significant that the distribution of a text to be tested is closer to the distribution of HWTs than to the MGTs' distribution.  We further develop a kernel optimisation algorithm in relative test to select the best kernel that can enhance the testing capability for MGT detection. As relative test does not assume that a text to be tested must belong exclusively to either MGTs or HWTs, relative test can largely reduce the false positive error compared to two-sample test, offering significant advantages in practice. Extensive experiments demonstrate the superior performance of our method, compared to state-of-the-art non-parametric and parametric detectors. The code and demo are available: https://github.com/xLearn-AU/R-Detect.

Decision: accept (poster)
Reviewer scores: 6, 6, 6

Reviews and discussion:
<thread>
creation date:
1730171616085

thread_type:
official_review

<thread_object>
creator: reviewer_er1t

<content>
summary:
This paper examines the issue of how to detect machine-generated texts. It suggests that previous methods are unable to identify human-written texts when there is a change in distribution. It proposes to employ non-parametric kernel relative test to detect machine-generated texts by testing whether it is statistically significant that the distribution of a text to be tested is closer to the distribution of human-written texts than to the distribution of machine-generated texts. A kernel optimization algorithm is proposed to select the best kernel that can enhance the testing capability for machine-generated text detection. Some experiments support the effectiveness of the method proposed in this paper.

strengths:
1. The motivation of the paper is reasonable, and the problem it addresses has practical application value.

2. The method proposed in the paper is simple and effective, with a coherent logic, and the description of the algorithm is very clear.

3. The paper has a solid theoretical foundation, its arguments are logically sound, and the method is highly interpretable.

4. The experiments indicate that the algorithm performs well.

weaknesses:
1. The assumptions on which the method is based are too strong and do not align with reality. Firstly, there is significant overlap between machine-generated language and human language; they do not exist in completely separate domains. Additionally, the subspace assumption is overly idealized and lacks a solid foundation, which greatly undermines the paper's validity.

2. The method proposed in the article resembles a general text anomaly detection approach and is not closely related to large language models or machine-generated language detection. It appears to be a universal solution for text anomaly detection rather than a targeted one, as the specific characteristics of the problem have not been discussed or reflected in the method's design.

3. The comparative methods in the experiments are relatively few and not comprehensive enough. Many anomaly detection solutions could also address this issue, so more comparisons and discussions should be provided.

questions:
Please see weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730299216291

thread_type:
official_review

<thread_object>
creator: reviewer_4sdf

<content>
summary:
This submission proposes to address the limitation of existing non-parametric LLM detector approach that tends to make mistakes in identifying HWTs that deviate from the
seen HWT distribution. The authors suggest to employ non-parametric kernel relative test to address the issue. Basically, the idea has some novelty to some extend. The paper highly follow the method in paper “DETECTING MACHINE-GENERATED TEXTS BY
MULTI-POPULATION AWARE OPTIMIZATION FOR
MAXIMUM MEAN DISCREPANCY” from both methodology style and the experimental design.

strengths:
1). Basically, the idea has some novelty to some extend. 

2). The problem to address is important.

3). The writting is somehow clear and easy to follow.

weaknesses:
My major concerns are as follows:

1). The limitation of these MMD-based approach is obvious. The appraoch by nature is not Zero-Shot aaproach. As the approach needs to prepare HWT and MGT in advance, I would say the approach is not training free by nature. Especially, the paper proposes kernel optimisation algorithm to select best kernel. This by nature is somehow “training”. Thus, the approach only compare times with Bino is not sufficient, as it also needs to compare time with training/classifier based approach (Table 2)

2). The experiments are far from complete. Bascially, the datasets used are obviously too simple. Why not use the RAID datasets (in the following paper) that includes 22 different LLMs with different settings?
“RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors, ACL’24”

3). Need to validate the robustness of the detection algorithm under adversial attack. 

4). Need to provide non-english dattasets’ results to see the effectiveness on non-english settings.

5). As mentioned in 1), the paper needs to provide comparisons to other classification based approaches as well other metric-based/ logits-based approaches. Such as:
Classification-based: 
FEW-SHOT DETECTION OF MACHINE-GENERATED TEXT USING STYLE REPRESENTATIONS. ICLR’24. https://arxiv.org/pdf/2401.06712
Threads of Subtlety: Detecting Machine-Generated Texts Through Discourse Motifs. ACL’24. https://arxiv.org/pdf/2402.10586. 
Logits-based:
DetectGPT (ICML’23) 
Fast-DetectGPT (ICLR’24) 
DNA-GPT [ICLR’24]
DALD: Improving Logits-based Detector without Logits from Black-box LLMs. [NeurIPS’24]

6). The paper also highly depends on the feature extractor, that is “fine-tuned a RoBERTa model on GPT-2-generated”. What’s the performance for other feature extractors? (need experiements). Need to explain the major different and advantage of proposed approach over training-based approaches.

7). The literature review is far from sufficient. Pl refer to the recent surveys such as:
A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization
A Survey on Detection of LLMs-Generated Content, EMNLP’24.
On the Possibilities of AI-Generated Text Detection

questions:
refer to the weakness
</content>
</thread_object>

</thread>
<thread>
creation date:
1730416441870

thread_type:
official_review

<thread_object>
creator: reviewer_hh24

<content>
summary:
This paper proposes a non-parametric kernel relative test to detect machine generated text (MGTs) by testing whether it is statistically significant that the distribution of a text to be tested is closer to the distribution of human written text (HWTs) than to the MGTs’  distribution. It improves the current two-sample test-based detection methods, which assumes that HWTs must follow the distribution of seen HWT. The authors further develop a kernel optimization algorithm in relative test to select the best kernel that can enhance the testing capability for MGT detection.

strengths:
The proposed relative test can reduce the false positive rate that has been observed in current two sample tests. 

It also proposes a novel method to optimize kernels in relative tests for MGT detection, which significantly improving the effectiveness and efficiency of the detection process.

weaknesses:
While this paper presents some innovative ideas, its significance appears limited. There has been some similar studies such as the ICLR'24 paper "Detecting machine-generated texts by multi-population aware optimization for maximum mean discrepancy.". 

The proposed method requires the preparation of both the MGT and HWT datasets. In that case, simply comparing the method with Bino in the experiment is insufficient, and may even be unfair because Bino is a zero-shot method. It is essential for the authors to engage in a comprehensive experimental study, comparing their proposed method with various other detection algorithms.

The method leverages Roberta based GPT-2 as feature generator (line 200).  Please explain the reason, and provide a comparison of this choice with other potential alternatives in both text explanation and experimental results.

questions:
The method relies on several assumptions coming from empirical results (line 88, line 280-). The authors are encouraged to provide additional clarifications, both intuitive and theoretical, to further explain those assumptions.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732511890072

thread_type:
official_comment

<thread_object>
creator: area_chair_r7lt

<content>
comment:
Dear Reviewers,

Thank you for your time and effort in reviewing for ICLR'25.

This is a gentle reminder to read and respond to the authors' rebuttals. Please submit your feedback in time. Thank you very much!

Best regards,

AC
</content>
</thread_object>

</thread>
<thread>
creation date:
1732561937199

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Newly added experiments: result summary, design and notations

comment:
We run more experiments. The performance among methods/data is presented in the following tables, using AUROC as the metric. We list the summary at the beginning, then design, and the detailed results are followed.

### Summary:
1. **Table r1**: our method performs the best on complex MGT detection tasks (using tasks suggested by Reviewer `[4sdF]`).
2. **Table r2**: our method is robust to adversarial attacks (using tasks suggested by Reviewer `[4sdF]`).
3. **Table r3**: while our method does not perform well in non-English detection tasks in a zero-shot setting, detection effectiveness improves significantly with a properly optimised kernel (e.g., from 0.3 to 0.61 on mix-1, and 0.35 to 0.43 on mix-2).
4. **Table r4**: we highlight the best extractor for each data. It can be seen that when we use a powerful extractor, such as falcon-rw-1b (noting that Bino uses falcon-7b), the performance can be further improved.
5. **Table r5**: The results demonstrate that our method performs the best compared to other baselines on the benchmark Beemo `[arXiv24]`. 
6. **Table r6**: The results demonstrate that our method performs the best compared to other baselines on the benchmark DetectRL `[NeuRIPS'24]`. 
7. **Table r7-8**: The results demonstrate that our method performs the best compared to Text Fluoroscopy `[EMNLP'24]` on the benchmark RAID, Beemo and DetectRL `[NeuRIPS'24]` with and without attack. 


### Experiments unavailable:
1. The GitHub of threads-of-subtlety `[ACL'24]` is still **empty** (https://github.com/minnesotanlp/threads-of-subtlety) and the authors haven't replied to our request. 

The same experiments will run with a token size of #512 if it is available. All the experimental results and analysis will be added to the main submission and Appendix.

>### Experimental design:
>**Zero-shot setting**: Use the previous trained kernel and reference.
>- **Table r1**: Comparison to more baselines on RAID data (non-attack).
>- **Table r2**: Comparison to more baselines on RAID data under attack.
>- **Table r3 (zero-shot part)**: Test on RAID non-English data with and without attacks.
>- **Table r5**: Test on Beemo benchmark.
>- **Table r6**: Test on DetectRL benchmark.
>- **Table r7-8**: Test and compare to Text Fluoroscopy `[EMNLP'24]` where one of the authors of Text Fluoroscopy `[EMNLP'24]` helped in implementation.
>
> **New kernel setting**: Optimise the kernel with English RAID.
>- **Table r3 (rows of new kernel)**: Test on RAID non-English data with and without attacks with re-trained kernel.
>- **Table r4**: Test with other feature extractors on RAID data with and without attacks with re-trained kernel.
>
> **New reference setting**: Optimise the kernel with English RAID and use unseen RAID data as reference.
>- **Table r3 (rows of new kernel+reference)**: Test on RAID non-English data with and without attacks with re-trained kernel as unseen RAID as reference.
>---
>
>### Notations of datasets:
>RAID provides MGTs generated by different LLMs under different adversarial attacks. We randomly sample MGTs and attacks to consist of the test data, and this test data will be used for testing all detectors. Specifically:
>- **mix1**: Represents a mixture of MGTs generated by Mistral 7B (Chat), GPT-2 XL, MPT-30B, MPT-30B (Chat), and GPT-4.
>- **mix2**: Represents a mixture of MGTs generated by Cohere, Cohere (Chat), GPT-3 (text-davinci-003), and GPT-2 XL.
>- **mix3**: Represents a mixture of MGTs generated by MPT-30B, GPT-3 (text-davinci-003), GPT-2 XL, and GPT-4.
>- **att**: Represents randomly shuffled attacks from Article Deletion, Homoglyph, Number Swap, Paraphrase, Synonym Swap, Misspelling, Whitespace Addition, Upper-Lower Swap, Zero-Width Space, Insert Paragraphs, Alternative Spelling. Namely, a misspelled sentence can be followed by a synonym swapped sentence.
>- **N/Eng**: Represents using random samples of non-English MGTs (Code, Czech, German) in RAID.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper explores the challenge of detecting machine-generated texts. It argues that existing methods fail to identify human-written texts when there is a shift in distribution. The paper proposes using a non-parametric kernel relative test to determine whether the distribution of a given text is statistically closer to that of human-written texts than to machine-generated texts. To improve detection performance, a kernel optimization algorithm is introduced to select the most effective kernel for this task. Experimental results demonstrate the effectiveness of the proposed approach. The paper presents a reasonable motivation, addressing a problem with significant practical value. The proposed method is both simple and effective, with clear logic and a well-explained algorithm. It is built on a solid theoretical foundation, with logically sound arguments and high interpretability. Furthermore, the experimental results demonstrate the algorithm’s strong performance. Therefore, I believe this paper should be accepted.