Paper: Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature (ICLR-2024)

Reviewer comment (Theory):
The sampling step in Fast-DetectGPT is not clearly motivated and straightforwardly approximates an expected difference. The derivation could instead be replaced directly by the analytical solution.

Author response:
**A3**: We have revised the paper to introduce an individual segment titled “Conditional Independent Sampling” to discuss it in detail. 

The sampling process plays a pivotal role in guiding us toward the solution. To discern whether a token within a given context is machine-generated or human-authored, it is essential to compare it against a range of alternative tokens in the same context. By sampling a substantial number of alternatives (say 10,000), we can effectively map out the distribution of their $\log p_{\theta}(\tilde{x}_ j|x_{<j})$ values. Placing the $\log p_{\theta}(x_ j|x_{<j})$ value of the passage token within this distribution provides a clear view of its relative position, enabling us to ascertain whether it is an outlier or a more typical selection. This fundamental insight forms the core rationale behind the development of Fast-DetectGPT.

The analytical solution is computationally more efficient but intuitively harder to understand, which we did not anticipate at the early stage of the research. Furthermore, when we use a single model for both sampling and scoring, the analytical solution shows a close connection to the Likelihood and Entropy baselines as we described on page 5. This connection seems so simple as the numerator in Eq. 7 is just the sum of Likelihood and Entropy. However, the sampling intuition plays a key role in finding this connection, where the Likelihood and Entropy have been used since year 2008.

Reviewer comment (Experiments):
The paper includes supervised RoBERTa baselines from OpenAI; however, these are not state-of-the-art for supervised detection. The paper would be strengthened by comparison to state-of-the-art supervised methods, such as Ghostbuster (Verma et al. 2023) or GPTZero (commercial model), especially given the claims in Section 5 that supervised methods have limited generalization capabilities in LLM-generated text detection. Because the primary purpose of the paper is to evaluate and compare zero-shot methods, however, this does not affect the reviewer’s score or recommendation.

Author response:
**A1**: We have revised the paper to include GPTZero as a baseline as Table 3 on page 7 and Table 7 on page 17 show. In summary, Fast-DetectGPT obtains higher average AUROCs than GPTZero. Specifically, on GPT-3 generations, GPTZero demonstrates poor detection accuracy on all three datasets, suggesting that GPTZero may not be trained on GPT-3 generations. On Chat-GPT and GPT-4 generations, GPTZero performs better than Fast-DetectGPT on news (XSum) but worse on stories (Writing) and technical articles (PubMed), which suggests that GPTZero may mainly be trained on news generations.

Reviewer comment (Experiments):
The paper mentions both Rank and LogRank baselines in Section 3.1 but only provides LogRank in tables.

Author response:
**A4**: We have revised the paper to remove the Rank baseline. Since Rank generally performs worse than LogRank, we remove it from the table to save space.

Reviewer comment (Experiments):
Did you experiment with computing the difference between the probability of the top-ranked word according to an LM scorer and the observed word? This measure should be closely correlated with the metric proposed in this paper and could be a slightly more informative alternative to the Rank model.

Author response:
**A2**: That is an interesting idea. We just experimented with it on GPT-4 generations and obtained an average AUROC of 0.8542, which is better than the likelihood (0.8212) and the log-rank (0.8088) but worse than Fast-Detect (0.9061). The results suggest that the probability difference is a better criterion than previous simple baselines.

Reviewer comment (Novelty):
The discussion of prior work with respect to alternate detection strategies such as watermarking is shallow. The Kirchenbauer et al. 2023 paper is for example not cited. While this paper takes an orthogonal approach, it would be good to see some motivation or discussion around the tradeoffs of those strategies.

Author response:
We have revised the paper to add a deeper discussion about watermarking in Appendix E on page 21, where the content is as follows.

Another line of detection methodology is watermarking that deliberately embeds information within machine-generated text to trace its origin \citep{jalil2009review, kamaruddin2018review, abdelnabi2021adversarial, gu2022watermarking, kirchenbauer2023watermark}. In comparison, Fast-DetectGPT relies on the innate distinction between the texts generated by humans and by machines, which may further be strengthened by explicit watermarks as additional features. 

In practice, these two strategies could potentially be combined to provide a more reliable detection solution. On the one hand, watermarking can be used to authorize the content generated by a specific service. On the other hand, when the service is out of our control and we cannot enforce the watermarking or a potential attacker has a strong LLM to remove the watermarks, the watermarking approach fails in these situations but the general detector like Fast-DetectGPT can still provide a valid solution.

Reviewer comment (Evaluation):
The discussion of ethical considerations and broader impacts is lacking. Liang et al. 2023 has shown that LLM detection systems tend to exhibit higher false positive rates for non-native speakers. While this doesn’t invalidate the usefulness of this work, at the least it is worth engaging with that literature and acknowledging the potential problems at play with this task. At best there could be experiments on the relative performance of this system on text written by different demographics as compared to prior work. Granted there is some analysis of performance on languages besides English but this is also relatively shallow.

Author response:
We have revised the paper to add a segment “Ethical Considerations and Broader Impact” in Appendix E on page 21 with the following content.

Fast-DetectGPT, serving as a highly efficient detector for machine-generated text, holds promise in enhancing the integrity of AI systems by combating issues like fake news, disinformation, and academic plagiarism. However, akin to other methods reliant on Large Language Models (LLMs), it is susceptible to inheriting biases present in the training data. Notably, as emphasized by \cite{liang2023gpt}, LLM-based detection systems may exhibit an elevated false-positive rate when confronted with text from non-native English speakers. Given the widespread and diverse utilization of such technologies, this presents a notable concern.

An immediate suggestion is to substitute the underlying LLMs in Fast-DetectGPT with alternative models trained on more varied and representative corpora. Additionally, we advocate for community involvement in the ongoing efforts to develop more inclusive LLMs, a development that would benefit not only Fast-DetectGPT but also similar systems at large.

Reviewer comment (Writing):
Presentation should be made clear. In the intro, paragraph 4 talked about the algorithm, yet it is unclear what does \tilt mean, what does <j means, also, the insight on why conditional probability is better is missing here, especially given that this is an extension of DetectGPT.

Author response:
We have made revisions to enhance the clarity of the paper. Please refer to the improved paragraph 4 in the introduction and the newly added section titled "Conditional Independent Sampling" on page 4 for detailed explanations. 

Briefly speaking, $\tilde{x}$ is generated by sampling each token $\tilde{x}_ j$ independently from the conditional probability $p(\tilde{x}_ j|x_{<j})$, where the passage $x$ is fixed. The independence of $\tilde{x}_j$ given the passage $x$ is the key to the high efficiency of Fast-DetectGPT. Specifically, Fast-DetectGPT requires only a single forward pass to sample and evaluate 10,000 samples (our default setting), in contrast to DetectGPT, which typically necessitates 100 forward passes for a mere 100 perturbations. This results in a significant reduction in computation time and resources.

Reviewer comment (Evaluation):
Is there results for speed comparison?

Author response:
We compared the speedup in Table 1 on page 1 and we have revised the paper to include a new segment titled “Inference Speedup” on page 7, providing a detailed description. 

Briefly speaking, for the detection of XSum generations produced by the 5-model. DetectGPT total costs (14598s+13819s+14383s+15941s+20372s) = 79113s, which is about 22 hours. In contrast, Fast-DetectGPT total costs (34s+31s+34s+49s+85s) = 233s, which is about 4 minutes. The speedup is calculated by 79113s / 233s, which is around 340x.

Reviewer comment (Writing):
Can you elaborate how \tilt {x} is generated? The reviewer is still confused.

Author response:
We have made revisions to enhance the clarity of the paper. Please refer to the improved paragraph 4 in the introduction and the newly added section titled "Conditional Independent Sampling" on page 4 for detailed explanations. 

Briefly speaking, $\tilde{x}$ is generated by sampling each token $\tilde{x}_ j$ independently from the conditional probability $p(\tilde{x}_ j|x_{<j})$, where the passage $x$ is fixed. The independence of $\tilde{x}_j$ given the passage $x$ is the key to the high efficiency of Fast-DetectGPT. Specifically, Fast-DetectGPT requires only a single forward pass to sample and evaluate 10,000 samples (our default setting), in contrast to DetectGPT, which typically necessitates 100 forward passes for a mere 100 perturbations. This results in a significant reduction in computation time and resources.

Reviewer comment (Evaluation):
Where does the acceleration come from? DetectGPT samples 100 pertrubations, how could this method accelerate 340 times? How many sampling does this needs?

Author response:
We have made revisions to enhance the clarity of the paper. Please refer to the improved paragraph 4 in the introduction and the newly added section titled "Conditional Independent Sampling" on page 4 for detailed explanations. 

Briefly speaking, $\tilde{x}$ is generated by sampling each token $\tilde{x}_ j$ independently from the conditional probability $p(\tilde{x}_ j|x_{<j})$, where the passage $x$ is fixed. The independence of $\tilde{x}_j$ given the passage $x$ is the key to the high efficiency of Fast-DetectGPT. Specifically, Fast-DetectGPT requires only a single forward pass to sample and evaluate 10,000 samples (our default setting), in contrast to DetectGPT, which typically necessitates 100 forward passes for a mere 100 perturbations. This results in a significant reduction in computation time and resources.

Reviewer comment (Novelty):
The novelty is limited. The conditional probability curve has already been used by another zero-shot detector released 5 months ago [1]. However, the author neither cites this previous work nor discusses its differences. Considering the reference [1] work was released 5 months ago, I will not consider them as concurrent work.

Author response:
We **strongly disagree** with this statement given the following evidence.

First, **neither the term “conditional probability curvature“ nor terms with similar meanings are found in the paper** including both arXiv versions published on 5/27/2023 (https://arxiv.org/abs/2305.17359v1) and on 10/4/2023 (https://arxiv.org/abs/2305.17359v2), where the second version is even after our submission to ICLR 2024 on 9/28/2023. 

Second, **the basic idea of DNA-GPT is fundamentally different from that of Fast-DetectGPT**. DNA-GPT generates various completions given a truncated prefix of a candidate passage, where the completions are generated autoregressively (requiring a decoding process for each completion). In contrast, Fast-DetectGPT does token-level conditionally independent sampling, where all samplings are done in the predictive distribution (requiring only one forward pass instead of any decoding process). 

Third, **DNA-GPT costs 80x more inference time than Fast-DetectGPT**. Experiments on XSum generations produced by the 5-model show that DNA-GPT takes 19289s (about 5 and half hours) to run across the five models, while Fast-DetectGPT only takes 233s (about 4 minutes). DNA-GPT costs 82.8x the inference time of Fast-DetectGPT.

Last, **DNA-GPT has much worse detection accuracy than Fast-DetectGPT** on both 5-model generations and ChatGPT/GPT-4 generations.  As Tables 2, 3, 5, and 7 in the revised version show, DNA-GPT performs even worse than DetectGPT on 5-model generations. Although its detection accuracy on ChatGPT and GPT-4 generations is higher than DetectGPT but is still significantly lower than Fast-DetectGPT (0.8836 and 0.7648 for DNA-GPT vs. 0.9615 and 0.9061 for Fast-DetectGPT on ChatGPT and GPT-4, respectively).

Reviewer comment (Reproducibility):
It is not clear how the sampling process works. Given a passage x, how do you sample the alternative x’? Throughout the paper, I did not find any explanation for this.

Author response:
We mentioned it in various places and in various forms, for example, “the conditional probabilities of alternative tokens p(\tilde{x}_j|x_{<j})” and “our approach begins by **sampling alternative word choices at each token**” on page 2, the definition of conditional probability function “p(\tilde{x}|x)” and “conditional sampling” part of Algorithm 1 on page 4, and the analytical expression of the sample mean in Eq. 5 on page 5.

The sampling of alternative \tilde{x} is conditionally independent given a passage x that each token \tilde{x}_j is sampled from the conditional distribution of p(\tilde{x}_j|x_{<j}) independently. We have revised the paper to introduce an additional segment titled “Conditionally Independent Sampling” to illustrate the sampling process in detail.

Reviewer comment (Experiments):
How would the number of resampled instances influence the result? I did not find any result for this.

Author response:
As we mentioned in “the analytical solution achieves a detection accuracy **almost identical** to the sampling approximation with 10,000 samples” on page 5, sampling with the number of resampled instances from 10,000 to 50,000 produces NO obvious differences in the AUROC, and also NO obvious difference in the time cost.

Reviewer comment (Reproducibility):
What is your default setting for the number of resampled instances for all the experiments? There is no clarification at all.

Author response:
We by default use 10,000 samples for the experiments as we mentioned in “the analytical solution achieves a detection accuracy almost identical to the sampling approximation with **10,000 samples**” on page 5.

Reviewer comment (Experiments):
How do you compare the speedup of your result over DetectGPT? Since the setting of your number of samples is unclear, I am not sure how you did compare it.

Author response:
We mentioned it in the caption of Table 1 “Speedup assessments were conducted **using the XSum** news dataset, with computations **on a Tesla A100 GPU**”, and we further revised the paper to include a new segment “Inference Speedup” on page 7 to discuss it in detail.

Specifically, we use 10,000 samples for Fast-DetectGPT and the default setting of 100 perturbations for DetectGPT. The speedup is evaluated on XSum generations from the 5 models. DetectGPT exhibited substantial computational demands, with total processing times across five runs summing up to 79,113 seconds (14598s+13819s+14383s+15941s+20372s, approximately 22 hours). Conversely, Fast-DetectGPT demonstrated exceptional efficiency with total times of only 233 seconds (34s+31s+34s+49s+85s, about 4 minutes). The resulting speedup factor is approximately 340x, calculated by dividing the total time for DetectGPT by that for Fast-DetectGPT (79,113s / 233s ≈ 340).

Reviewer comment (Evaluation):
The number of relative improvements is confusing. For example, in Table 1, why is the relative improvement 74.7%? In my understanding, (0.9887−0.9554)/0.9554*100% = 3.48%. I do not understand why you report 74.7%.

Author response:
We have revised the paper to clarify it as Table 1 on page 1 shows. The relative improvements are calculated by (new – old) / (1.0 – old), where the denominator (1.0 – old) represents the maximum possible improvement from the old AUROC. The metric shows how much improvement has been made relative to the maximum possible improvement, which is especially useful when the values of new and old are already high (close to 1). For example, intuitively, an improvement from 0.9554 to 0.9887 looks so different from an improvement from 0.7225 to 0.9338. However, when we compare their relative improvements, we can see that the former 74.7% is close to the latter 76.1%, which reveals the inner consistency between the two experiments.