DetectGPT
Zero-Shot Machine-Generated Text Detection using
Probability Curvature

Hamed Mahdavi

April 26, 2023

Motivation: Misuses of LLMs

Students use LLMs to write their homework. This impairs
student learning and makes a fair assessment of their
homework hard.
LLMs make research plagiarism easier. Paraphrased
plagiarized articles are hard to detect using older methods.
LLMs can ease propaganda: assume a scenario when alarge group of online bot social media accounts starts togenerate incorrect information about a topic.
With these possibilities, one needs to ask, is it possible todistinguish outputs of LLMs with human written text?

Hamed Mahdavi

DetectGPTProblem Definition

Problem Definition: Generated Text Detection
Given a candidate passage and a source model, we want todetect if the candidate passage is generated by a source modelwith a high probability.

Hamed Mahdavi

DetectGPT

Possible Approaches

Training another network for binary classification ofgenerated text:
We need to train a new classification model for each sourcemodel.
The classifier might overfit to the topics it was trained on.

Zero-shot Approach (Solaiman et al. 2019): One can lookat model outputs such as the average log-probability ofgenerated results to decide for each sample.
Does not need to train other models or gather a dataset.
But ignores the local structure of the learned probability
function around a candidate passage.

Hamed Mahdavi

DetectGPT

Proposed Solution

Facts:
We know that sampling from Language models follows alocal search procedure (e.g. beam search) to findhigh-probability samples
Solaiman et al. (2019) method works well in practice.

Hypothesis:
LLMs generate samples near local maxima of
log-probability. Perturbing a sample significantly reduces its
log probability.

Hamed Mahdavi

DetectGPTFormal Statement of Hypothesis
We denote a perturbation function with q(·|x). q can be theresult of asking a human to rewrite one of the sentences ofx while preserving the meaning of x.
the notion of a perturbation function, we can define theperturbation discrepancy d (x, pθ , q) :d (x, pθ , q) ≜ log pθ (x) − Ex̃∼q(·|x) log pθ (x̃)
Hypothesis
If q produces samples on the data manifold, d (x, pθ , q) ispositive with high probability for samples x ∼ pθ . For
human-written text, d (x, pθ , q) tends toward zero for all x.

Hamed Mahdavi

DetectGPT

Proposed Solution: Visualization
Candidate passage:“Joe Biden recently made a move to the White House
that included bringing along his pet German Shepherd…”

DetectGPT(reword(1) Perturb withT5)(2) Score(3) Compareong”

ri“pet” → “dog”“madeamove”→ “mGPT-3...

e “bDelet...

xg alnginoved”

Yes🤖

from GPT-3No🤔

from other source

Figure: DetectGPT compares the log probability under p of theoriginal sample x with each perturbed sample x̃i . If the average log
ratio is high, the sample is likely from the source model.
Hamed Mahdavi

DetectGPT

log pθ(x)Hypothesis: Visualizationx real ∼ phuman(x)x fake ∼ pθ (x)x̃real2x̃ fake
2x̃ fake
1x̃ fake
3x̃real4x̃real1x̃ fake
4x̃real3…

Log likelihood

Fake/real sample

Perturbed fake/real sample

Figure: Human written text is not a result of greedy local search
optimization, so if we perturb it, the log-probability of the perturbed
text does not drop.

Hamed Mahdavi

DetectGPTTesting Hypothesis: Using other LLMs
gpt2-xl

1.0

EleutherAI/gpt-neo-2.7B
Human
Model

60

Frequency

0.8
4020
0.60
0.4
60

0.0

0.1

0.2

EleutherAI/gpt-j-6B

0.3

0.00 0.05 0.10 0.15 0.20

EleutherAI/gpt-neox-20b

40
0.2
20
0.00
0.0
0.00 0.05 0.10
0.2 0.15 0.200.40.25

0.0 0.6

0.1

0.80.2

Log Likelihood Drop (Perturbation Discrepancy)

0.3 1.0

Figure: One can test the stated hypothesis by modeling q with amasked language model with T5. Here 500 news articles from XSum
dataset have been used for real data.4 LLMs have been used tosample models from XSum’s articles based on their first 30 tokens
Hamed Mahdavi

DetectGPT

Interpretation of the Perturbation Discrepancy asCurvature
The perturbation discrepancy approximates a measure ofthe local curvature of the log probability function near thecandidate passage.
If zi s are i.i.d with mean zero and variance 1, then thefollowing is a random estimation of A matrix’s tracetr(A) = Ez z⊤ Az
We can approximate second-order directional derivativewith:z⊤ Hf (x)z ≈f (x + hz) + f (x − hz) − 2f (x)h2

Hamed Mahdavi

DetectGPT

Interpretation of the Perturbation Discrepancy asCurvature (cont.)

Combining two equations from the last slide we get:− tr(H)f (x) ≈ 2f (x) − Ez [f (x + z) + f (x − z)]
If the noise distribution is symmetric, then we have:− tr(H)f (x)≈ f (x) − Ez f (x + z)
2

Hamed Mahdavi

DetectGPT

Results: AUROC for Detecting Samples

Figure: DetectGPT’s AUROC consistently outperforms four previously
proposed criteria across various models and datasets, with 500
samples used for evaluation. Bold indicates the best AUROC, and an
asterisk (*) denotes the second-best. The final row shows
DetectGPT’s AUROC compared to the strongest baseline method ineach column, ranging from 1.5B to 20B parameters in GPT models.

Hamed Mahdavi

DetectGPT

Results: Supervised Machine Generated TextDetection
XSum GPT-2 Detection

0.981 0.997

1.00.889

0.90.991

WMT16-en mGPT Detection
0.888

0.957

0.946
0.838

0.800

0.80.80.7

Detection AUROC

0.915

0.795

0.863

0.6
0.5
0.6

0.4
1.0k
T
e g
d k
bas RoB-l elihoo Ran ogRan tectGsP)
L De Our
Lik(RoB

PubMedQA PubMedGPT Detection
Supervised
Unsupervised
0.768
0.713

0.9
0.8
0.2
0.7

0.604

0.6k
T
e g
d k
bas RoB-l elihoo Ran ogRan tectGsP)
L De Our
Lik(RoB

WMT16-de mGPT Detection
0.9620.773

0.836

0.795

0.838 0.861

0.664

0.5
0.00.0k
Te lg 0.2 d k
0.4-bas oB- lihoo Ran LogRan tectGrsP)RoB R Like
De (Ou0.537
0.394
ase B-lg
b0.6RoBRo

Detection Method

Tnk
P1.0
od 0.8nk
liho Ra LogRa etectGrs)
D (OuLike

Figure: Supervised models work well on in-distribution text,
DetectGPT performs better in out-of-distribution data.
Hamed Mahdavi

DetectGPT

Results: DetectGPT vs Supervised

Figure: DetectGPT and supervised models for detecting
machine-generated text have similar AUROC for GPT-3 generations.
Supervised models outperform DetectGPT in detecting typical text
like news articles.

Hamed Mahdavi

DetectGPTDetection AUROC

Results: Capacity of Mask Filling and Detection
Quality
1.0
0.9
0.8
0.7
0.6
0.5

5 perturbations

25 perturbations

Random
GPT2-sm
GPT2-md
GPT2-lg
GPT2-xl

60M

220M

770M

2.7B

60M

220M

770M

Mask filling model size (# parameters)

2.7B

Figure: The AUROC scores on 200 SQuAD contexts are presented
as curves. The mask-filling model’s capacity is associated withdetection performance across different source model scales. Randommask filling with uniform sampling performs badly, indicating that theperturbation function must generate samples on the data manifold.

Hamed Mahdavi

DetectGPT

Results: The effect of the number of perturbations

Detection AUROC

1.0 GPT-2

GPT-J

0.9
0.8
XSum
SQuAD
WritingPrompts

0.7
0.6

110

100

1000

110

Number of perturbations

1001000

Figure: DetectGPT’s discrimination power for classification, asmeasured by auROC (left) and auPR (right), is significantly improvedby averaging up to 100 perturbations. The fills are sampled fromT5-large, and the impact of varying the number of perturbations isevaluated.

Hamed Mahdavi

DetectGPTResults: Scoring Quality

Scoring Model

Base Model

GPT-2 GPT-Neo GPT-J

GPT-J GPT-Neo GPT-2
0.92(0.02)

0.83(0.04)

0.79(0.02) 0.85

0.64(0.06)

0.97(0.01)

0.83(0.02) 0.81

0.60
(0.09)

0.85(0.05)

0.99(0.00) 0.81

0.72

0.88

0.87

Figure: DetectGPT works well in scoring samples generated by thesame model (diagonal), but column means suggest that some models(such as GPT-Neo and GPT-2) may be better scorers than others(such as GPT-J). Mean (standard error) AUROC over XSum, SQuAD,and WritingPrompts is represented by white values, while row/column
mean is shown by black value
Hamed Mahdavi

DetectGPT

Results: Varying Edit Fractions

Detection AUROC

1.0Rank
DetectGPT

LogRank
Likelihood

0.08

0.16

Entropy

0.9
0.8
0.7
0.6
0.00

0.04

0.12

0.20

0.24

Fraction of GPT-J-generated news article re-writtenFigure: The performance of four top-performing methods degrades asrevisions become heavier when simulating human edits tomachine-generated text by replacing different fractions of model
samples with T5-3B generated text. DetectGPT consistently performs
the best in this experiment conducted on the XSum dataset. The
simulation is achieved by masking out random five-word spans untilr % of text is masked.
Hamed Mahdavi

DetectGPT

Discussion

Accessing to probability outputs of a language model canbe expensive (e.g. GPT3 API).
The paper does not discuss the social impacts of their
method. Does this work well equally for native andnon-native English speakers?
What is the effect of adding a prompt before generating
text?

Hamed Mahdavi

DetectGPT

Questions

Any Questions?

Hamed Mahdavi

DetectGPT