SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Table 7: Our retrieval defense significantly improves AI-generated text detection accuracy (at  $1\%$  FPR) over baselines on all settings, including our most diverse paraphrase attacks (+60L and +60L,60O).
Open-ended text generation with Wikipedia prompts (300 generated tokens)
GPT2-XL	OPT-13B	GPT-3.5 (davinci-003)
Original	+ 60L	+ 60L,60O	Original	+ 60L	+ 60L,60O	Original	+ 60L	+ 60L,60O
Baseline methods:
Watermark	100.0	68.9	57.2	99.9	63.7	52.8	-	-	-
DetectGPT	70.3	8.7	4.6	14.3	0.8	0.3	2.0	0.5	0.0
OpenAI	21.6	13.3	14.8	11.3	9.1	10.0	30.0	15.6	15.6
(Ours) Retrieval over corpus of 3K generations from model itself, with retriever:
SP	100.0	86.4	81.5	100.0	84.4	77.7	100.0	65.9	49.5
BM25	100.0	99.0	98.0	100.0	97.2	95.3	100.0	58.8	37.4
(Ours) Retrieval over corpus of 9K generations pooled from all three models, with retriever:
SP	100.0	72.1	63.2	100.0	74.6	65.6	100.0	63.1	45.6
BM25	100.0	85.0	78.7	100.0	87.2	79.1	100.0	58.8	37.4
Figure 8: ROC curves for text generated by GPT2-XL, before paraphrasing (solid lines) and after paraphrasing (dashed lines, pp). Different plots represent different clipping thresholds on the X-axis.