Glimpse: Enabling White-Box Methods to Use Proprietary Models for Zero-Shot LLM-Generated Text Detection

Authors: Guangsheng Bao, Yanbin Zhao, Juncai He, Yue Zhang
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: Machine-Generated Text Detection

Abstract:
Advanced large language models (LLMs) can generate text almost indistinguishable from human-written text, highlighting the importance of LLM-generated text detection. However, current zero-shot techniques face challenges as white-box methods are restricted to use weaker open-source LLMs, and black-box methods are limited by partial observation from stronger proprietary LLMs. It seems impossible to enable white-box methods to use proprietary models because API-level access to the models neither provides full predictive distributions nor inner embeddings. To traverse the divide, we propose **Glimpse**, a probability distribution estimation approach, predicting the full distributions from partial observations. Despite the simplicity of Glimpse, we successfully extend white-box methods like Entropy, Rank, Log-Rank, and Fast-DetectGPT to latest proprietary models. Experiments show that Glimpse with Fast-DetectGPT and GPT-3.5 achieves an average AUROC of about 0.95 in five latest source models, improving the score by 51\% relative to the remaining space of the open source baseline. It demonstrates that the latest LLMs can effectively detect their own outputs, suggesting that advanced LLMs may be the best shield against themselves. We release our code and data at https://github.com/baoguangsheng/glimpse.

Decision: accept (poster)
Reviewer scores: 6, 5, 6, 8

Reviews and discussion:
<thread>
creation date:
1729824806168

thread_type:
official_review

<thread_object>
creator: reviewer_qgg2

<content>
summary:
To enable existing white-box methods to proprietary models, the authors propose Probability Distribution Estimation (PDE), extending methods like Entropy, Rank, Log-Rank, and Fast-DetectGPT to latest closed-source LLMs. This work uses only the top-K probabilies returns by the API to estimate the necessary metrics (such as mean and variance in DetectGPT). They estimate the probs after rank K using parametrized distribution (which token each pk refers to is not important). Three algorithms are tested: Geometric, Zipfian, and MLP.

It is applied to ChatGPT (gpt-35-turbo) (OpenAI, 2022), GPT-4 (gpt-4) (OpenAI, 2023), Claude3 Sonnet (claude-3-sonnet-20240229) and Opus (claude-3-opus-20240229) (Anthropic, 2024), Gemini-1.5 Pro (gemini-1.5-pro). The results is quite promising, and suggests larger LLMs can be universal detectors.

strengths:
I think the idea of estimating unknown probs using Geometric, Zipfian, and MLP is very interesting.

The idea can be applied to different white-box methods.

The performance are quite promising.

weaknesses:
Comparing to this proprietary and estimation approach, why not use open-source large model such as LLaMA as the scoring model? This paper did not mention models like LLaMA at all. If authors can give me a convincing answer or add some comparison experiment, I'll consider raise score.

It still assumes some partial information from the API. But I do agree this setting is realistic and interesting.

Some parts of the writing can be improved (refer to my questions).

questions:
Line302 : I did not understand what's the difference between source model and scoring models… My current understanding is that the authors want to argue that a good scoring model can detect machine-generated text from a wide range of LLMs, is my understanding correct? I think if you can add some text to explain that will be helpful.

Figure1: I think the notation p(x|x) is weird.

Line293: What do you mean by "generate corresponding text from source LLMs for each sample"? Do you mean generate text with the same prefix?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730296819976

thread_type:
official_review

<thread_object>
creator: reviewer_nxva

<content>
summary:
The paper proposes Probability Distribution Estimation (PDE), a method for enhancing the capabilities of white-box text detection methods, such as Fast-DetectGPT, in identifying machine-generated text from proprietary large language models (LLMs) like GPT-4 and Claude-3. PDE extends traditional white-box methods to proprietary models by estimating full probability distributions from limited API access. The authors demonstrate that using PDE with Fast-DetectGPT achieves improvements in detection accuracy over previous methods.

strengths:
- A thorough explanation of the PDE approach, describing its implementation across different parameterization methods (e.g., Geometric, Zipfian).
- Extensive ablation studies and analyses (distribution choices, top-K probabilities, language variations). This adds depth and clarity to the method’s application across proprietary LLMs.
- Completing datasets with new models that authors will release

weaknesses:
- The evaluations are conducted on limited datasets (150 pairs), potentially restricting the findings’ generalizability across more diverse or larger data sources. There are more samples for other models in the original M4 dataset.
- While PDE is an effective enhancement, it primarily focuses on density estimation and integration with existing detection methods rather than introducing a fundamentally new detection approach.
-  Although PDE improves upon Fast-Detect, the accuracy increase is relatively modest, raising questions about the practical advantage of this added complexity in real-world applications. In table 2 you report Fast-Detect (GPT-J/Neo-2.7) which has relatively good. It hints that it may be better to use bigger or newer open models (like Qwen2.5-1.5B or Llama3) than to use PDE to proprietary models.

questions:
- Did you try to Fast-Detect with other open models rather than GPT-J/Neo-2.7? Could you please scores from newer/larger models for better comparison?
- The evaluation dataset is very small; did you try other datasets? I believe it would be better to perform testing on bigger datasets, for example the original M4. 

Small remarks:
- line 101-102: "(↑ more than 25%)" might confuse readers. Better to add "relative to the remaining space"
- "*" in equations looks strange
- line 297: gpt-35-turbo -> gpt-3.5-turbo
</content>
</thread_object>

</thread>
<thread>
creation date:
1730376859592

thread_type:
official_review

<thread_object>
creator: reviewer_gnc2

<content>
summary:
The paper considers the topic of detecting the texts, generated by closed (proprietary) LLMs. Authors suggest several ways to approximate the distribution of token probabilities of those closed LLMs accessible via API (such as GPT-4), knowing only the probabilities of top-K or even top-1 tokens. Using these distributions, authors managed to emulate several methods of LLM-generated text detection (i. e. Fast-DetectGPT, Entropy, etc) as if they had access to these LLMs. Using these methods upon approximated distributions, they overperformed analogous methods based on distributions of GPT-Neo-2.7B.

strengths:
- The topic of LLM-generated text detection is important.
- The idea of the paper is worth considering.
- Authors check how their detectors behave when they fix a low False Positive Rate, that is a good practice.
- Authors check their detectors against paraphrasing attacks and consider the influence of various prompts.

weaknesses:
The major drawbacks of the paper lie in its experimental setup.
- The amount of machine-generated texts in the datasets in Table 2 (which is supposed to contain the main results) is extremely small. E.g., as far as I understood from part 3.1 SETTINGS, for each sub-domain "XSum", "Writing", and "PubMed" in Table 2, only 150 machine-generated texts vs 150 human-written texts are used. It's hard to tell anything for sure from such a small amount of examples, and the results may be very noisy. The authors need to increase their datasets to get reliable results.
- The choice of baseline open LLM (i.e. GPT-Neo-2.7B) for applying methods like Fast-Detect and Likelihood seems unreasonable because GPT-Neo-2.7B is weak compared to the newer open LLMs of comparable size. I would recommend adding at least the Phi-2 model (it also has 2.7B parameters) as a base for Fast-Detect, Likelihood, and other similar methods because its perplexity scores seem to be good in LLM-generated text detection (AI-generated text boundary detection with RoFT, Kushnareva et. al, 2024). Besides, many bigger models look like fair candidates for a good baseline (e.g. LLaMA 3.2 and many others).
- AUROC scores in the Tables seem to be very high both for the best baseline and for the best method suggested by the authors. It means that the task statement is too easy. It would be good to add more complicated data, where all the methods will struggle, and the difference between methods will become more clear.

Minor remarks about the presentation:
- Overusing the appendices: appendices D.1 and E look more suitable for the main text;
- Typos: log p(x|x) in Figure 1 and Equation (2);
- Figure 1 contains an extremely bright red color, it is hard to look at it. Please, change it to a softer color.

**UPDATE**: The first two of my concerns were addressed, so I updated my main score from 3 to 5. I adjusted scores about "soundness" and "presentation" accordingly.

questions:
- Line 138-140, Equation 2: why do you call this $d(x, p_\theta)$ value "curvature"? It doesn't seem to have a meaning of a curvature. Maybe "skewness" would be better word?
- Only AUROC is reported as a metric for comparison in the Tables. It would be interesting to see, what are the accuracy scores for all methods (for some fixed threshold)?
- Are there any differences between "Likelihood" method and perplexity scores?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730691720900

thread_type:
official_review

<thread_object>
creator: reviewer_32jz

<content>
summary:
In this paper, the authors consider the task of artificial text detection (ATD); one commonly-used approach here is to use a model itself to detect its own output. This scheme is inapplicable to modern proprietary LLMs because they don’t give access to their internal states. To address this issue, authors introduce an extension to existing white-box detection methods that rely on the distribution of model-assigned probabilities to each token. The authors propose to substitute token probability distributions by their density estimations (PDE) based on probabilities of the few most probable tokens and a hypothesis on the shape of the rest of the distribution. They test various candidates for the hypothetical token probability distributions. On example of Fast-DetectGPT authors illustrate the scheme of PDE-approach application to ATD solutions. Authors perform numerous experiments on different types of texts produced by various generating models and show the high performance of their method. They also perform ablation study, investigate the effect of different prompt designs, and explore the effect of paraphrasing adversarial attack.

strengths:
**S1.** The proposed approach shows promising results at increasing the efficiency of previously existing Artificial Text Detection methods against modern generating models.

**S2.** The text of the article is well-written and was easy to follow; extensive mathematical explanations are given.

**S3.** Proposed method is robust against some common adversarial attacks (e.g., automatic paraphrasing of generated texts).

weaknesses:
**W1.** The proposed method still requires the internal information from the source models – model-assigned probabilities of K most probable candidates for each token. Some proprietary models might not provide such information.

**W2.** The novelty of the proposed method compared to `vanilla’ Fast-DetectGPT is somewhat ambiguous. Essentially, this work proposes to compute statistics not from probability distribution over the entire dictionary (as in Fast-DetectGPT), but from probabilities of K most likely tokens and some assumptions on the distribution. 

**W3.** Datasets used for experiments are small: 150 pairs of human-written/AI-generated texts each. In particular, there are concerns regarding the generalization capabilities of the proposed. Also, given data is not large enough to properly train the baseline methods (RoBERTa).

**W4.** AUROC is the only quality metric reported. It has a number of limitations, and alone it is not enough to accurately compare different models. This experimental setup does not allow properly accessing the generalization capabilities of the model and discriminating different types of their biases. 
For reference, one of the common practices for Zero-shot AI-content detector evaluation is to set a threshold (by fixing FP-rate) on some 'training’ data (i.e., the probability of falsely declaring human-written text as AI-generated) and then report metrics (TPR/TNR/detection accuracy/etc.) on the `evaluation’ data.

**W5.** It is unclear how good this method would generalize across different styles of texts. As it follows from the text, all reported experiments were performed only in “in-domain” setup. Datasets used in this work are very small and style-specific (e.g., scientific-like *PubMed* and free-form fictional *Writing*). It is well known that some classifiers (like RoBERTa-based) excel on `familiar’ data but experience a significant drop in performance when they face texts of styles or genres unseen during training. From the presented results it is impossible to say whether the proposed method will fall into the same pitfall or not.

questions:
**Q1.** DetectGPT (and many other methods) has a known limitation of a drastically high rate of false alarms on texts written by people who are not fluent in English; for example, by undergraduate students preparing for the IELTS exam [1]. Have you considered such scenarios?
This issue becomes more and more crucial as AI-content detection tools become part of automatic anti-plagiarism systems in education in countries where English is taught as second-language.

> [1] W. Liang, M. Yuksekgonul, Y. Mao, E. Wu, and J. Zou. GPT detectors are biased against non-native English writers. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models.

**Q2.** How big is the difference in performance between your method (PDE + Fast-DetectGPT) and the original Fast-DetectGPT on various data?

**Q3.** Table 3 states that you use RoBERTa-large model as a baseline for experiments on robustness over languages. This is incorrect because RoBERTa was trained only on English-language data. Did you mean XLM-RoBERTa- large, which is actually multilingual, there?

**Q4.** Could you please specify in more detail the experimental setup for assessing the robustness against paraphrasing attacks (Appendix E.1)?

**Q5.** What are the limits for cross-model transferability of your method? (i.e, the proposed method works well when transferred from ChatGPT to GPT-4 and vice versa, but what would happen when it faces older models like GPT-2 or XLNet as source models?)

**Other remarks:**

-	Table 2 caption: instead of “average accuracies”, there should be “average AUROC”. In general, please revise your usage of the phrase “detection accuracy”, because right now it is difficult to understand whether you mean Accuracy (that is not reported by value anywhere) or AUROC.

-	lines 485/511: technically, PHD is a "black-box" method. It does not require the knowledge of what model was used to generate text; it uses inner embeddings from a fixed auxiliary encoder transformer.

-	Code to reproduce the experiments was not provided (this could be done in attachments to submission in or via an anonymous Github repository).

-	Please fix lettercases in paper titles in the references (e.g., “Fast-detectgpt: Efficient zero-shot detection of machine-generated text via conditional probability curvature” should be “Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature” in line 552; and so for other articles).
</content>
</thread_object>

</thread>
<thread>
creation date:
1731935267656

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Reply to Two Common Concerns

comment:
We appreciate the reviewer for their comprehensive feedback! We would like to address the two main concerns as follows.

---
> **Common Concern 1:** Datasets used for experiments are small: 150 pairs of human-written/AI-generated texts each. 

**Reply:** This dataset setting is **commonly used by recent studies** like DetectGPT (Mitchell et al., ICML 2023), Fast-DetectGPT (Bao et al., ICLR 2024), DNA-GPT (Yang et al., ICLR 2024), FourierGPT (Yang et al., EMNLP 2024), and DLAD (Zeng et al., NeurIPS 2024). We believe that this setting is adequate for assessing a zero-shot detection method (which uses service API) for two main reasons. 

First, the total number of samples and API calls for evaluating each method are large in our experiments. We evaluate each method on (300 samples) * (3 datasets) * (5 source models) = **4500 samples** in our main experiments and on (300 samples) * (6 languages) = **1800 samples** in our experiments across languages. **Of ALL the experiments, PDE has the highest AUROC**, significantly outperforming the baselines. Additionally, we run each experiment three times and report the median. The total cost of all the experiments is (4500 samples) * (5 methods) * (3 times) + (1800 samples) * (3 methods) * (3 times) = **83700 API calls**. 

Second, compared to other metrics like ACC and Precision/Recall, AUROC is less sensitive to the number of samples because it is an average score (true positive rate) across various false positive rates. Our empirical observations on ChatGPT generations show that the setting yields stable AUROC results (+/-0.0021) when we sample 150 pairs using different random seeds from the entire dataset. The relative order between methods does not change and the **conclusion holds**.

In conclusion, given that re-running the full experiments on more samples could lead to tens of thousands of API calls, resulting in **unnecessary carbon-dioxide emissions and service costs**, we prefer to maintain the current setting for our experiments.

---
> **Common Concern 2:** It is better to use bigger or newer open models (like Phi2-2.7B, Qwen2.5-1.5B or Llama3) than GPT-Neo-2.7B as the baseline.

**Reply:** The official Fast-DetectGPT utilizes GPT-Neo-2.7B, which we have selected as our baseline. Nonetheless, to encompass the latest open LLMs, we have updated our paper to incorporate experiments on Phi2, Qwen2.5, and Llama3, as depicted in Tables 2, 3, 4, and 5. The main results are like:

```
| Method                      | ChatGPT| GPT4   | Sonnet | Opus   | Gemini1.5-Pro| Avg.   |
| Fast-Detect (GPT-J/Neo-2.7) | 0.9615 | 0.9061 | 0.9304 | 0.9519 | 0.8099       | 0.9119 |
| Fast-Detect (Phi2-2.7B)     | 0.7821 | 0.5727 | 0.7151 | 0.7649 | 0.6251       | 0.6920 |
| Fast-Detect (Qwen2.5-7B)    | 0.7937 | 0.7023 | 0.8180 | 0.8545 | 0.7034       | 0.7744 |
| Fast-Detect (Llama3-8B)     | 0.8299 | 0.7554 | 0.8792 | 0.9089 | 0.7979       | 0.8343 |
```

The results indicate that **recent open LLMs underperform compared to Neo-2.7B**. We speculate this could be due to two factors. First, as the study “Smaller language models are better black-box machine-generated text detectors (Mireshghallah et al., 2023)” reports, larger open-source models do not necessarily yield better results. Second, these open-source models may have been partially trained on synthetic data (machine-generated text), which could lead to a distribution more akin to machine text rather than human text. This could disrupt the underlying assumption of Fast-DetectGPT that the scoring model closely aligns with the distribution of human text.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733136390428

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Revisions

comment:
With the generous guidance from our esteemed reviewers, we have made the following major updates:
1. The **title** and **abstract** have been updated to highlight the contribution.  
2. Experiments are conducted on the domain-diverse dataset **Mix3 (450 pairs)** and the language-diverse dataset **Mix6 (900 pairs)** to achieve the desired scale and diversity (see Tables 1-5, which address **Common Concern 1**).  
3. Experiments are carried out on **Phi2**, **Qwen2.5**, and **Llama3** to include newer and larger open-source models (see Tables 2-5, addressing **Common Concern 2**).  
4. **"Out-of-domain" settings** are integrated to evaluate detection accuracy (ACC) using thresholds from out-of-domain datasets across **text domains** and **source models** (see Tables 6, 8, and 9).  
5. We present the **TPR at FPT=1%** in the "Robustness Under **Paraphrasing Attack**" section (see Table 10).  
6. Additional adjustments have been made for clarity.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This work addresses the problem of machine-generated text detection. Existing approaches fall into white-box methods, which require access to the underlying model and are generally more robust, and black-box, which don't require access to the model (meaning they can be applied to proprietary LLms through an API) but are less robust. This paper proposes applying white-box detection methods to closed-source models by estimating the probability distribution in the black-box model using their new method for Probability Distribution Estimation (PDE). The method is general to different forms of white-box detection, and the experiments show using PDE to apply detection methods to closed-source models improves detection performance over their baselines.

Strengths:
- This paper addresses an interesting issue, the application of propriety LLMs to the task of generated-text detection though approximating white-box detection methods (Gnc2), and the results show that the approach demonstrates promise in improving detection accuracy with these models (32Jz, qGG2).
- The paper provides useful ablations in terms of controlling for false positive rates and common adversarial attacks against detection models (32Jz, Gnc2, NxVa)
- The paper is well-written and easy to understand. (32Jz, NxVa)

Weaknesses:
The primary concerns with the paper come from the experimental setup. All of the reviewers note the very small sample size in the evaluation dataset. The paper provides an impressive number of different detection methods, domains, and models in the experiments; however, the implications of these results are limited given the limited sample size and small differences in performance between methods. In addition, one reason the authors list is that they use a small sample size to rerun each experiment and report the median result. However, the variance across these runs - or even if the differences are significant across settings - is not reported.

Another minor concern raised by the reviewers (32Jz, qGG2) is that the application score of the method is limited, as model APIs do not always provide the required information; the authors never discuss this in their response.

Other weaknesses raised by the reviewers were addressed in the author response.