Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Authors: Kalpesh Krishna
Yixiao Song
Marzena Karpinska
John Frederick Wieting
Mohit Iyyer
Venue: NeurIPS 2023 poster (2023)
Keywords: AI-generated text detection
text detection
paraphrasing
attacks
retrieval
defenses
large language models
LLMs
TL;DR: We show that current AI-generated text detectors are brittle against paraphrasing attacks, and present a retrieval-based detector as a defense.

Abstract: The rise in malicious usage of large language models, such as fake content creation and academic plagiarism, has motivated the development of approaches that identify AI-generated text, including those based on watermarking or outlier detection. However, the robustness of these detection algorithms to paraphrases of AI-generated text remains unclear. To stress test these detectors, we build a 11B parameter paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition on surrounding context, and control lexical diversity and content reordering. Paraphrasing text generated by three large language models (including GPT3.5-davinci-003) with DIPPER successfully evades several detectors, including watermarking, GPTZero, DetectGPT, and OpenAI's text classifier. For example, DIPPER drops detection accuracy of DetectGPT from 70.3% to 4.6% (at a constant false positive rate of 1%), without appreciably modifying the input semantics.
To increase the robustness of AI-generated text detection to paraphrase attacks, we introduce a simple defense that relies on retrieving semantically-similar generations and must be maintained by a language model API provider. Given a candidate text, our algorithm searches a database of sequences previously generated by the API, looking for sequences that match the candidate text within a certain threshold. We empirically verify our defense using a database of 15M generations from a fine-tuned T5-XXL model and find that it can detect 80% to 97% of paraphrased generations across different settings while only classifying 1% of human-written sequences as AI-generated. We open-source our models, code and data.

Paraphrasing evades detectors of Al-generated text, but retrieval is an effective defense
Kalpesh Krishna
Yixiao Song*
Marzena Karpinska* Mohit Iyyer*
John Wieting
University Massachusetts Amherst, Google, Google DeepMind
(kal keakk,, jwietiiiig ing) @google com
y rixiaosong@umar umass eddu Imkarpinska, miyyer fécs. umass edu
Abstract
The rise in malicious usage of large language models, such as fake content ation and academic plagiarism, has motivated the development of approaches that identify Al Al-generated text, including those based watermarking outlier de- tection. However, the robustness of these detection algorithms to paraphrases Al-generated text remains unclear. To stress test these detectors, build IIB parameter paraphrase generation model (DIPPER) that can paraphrase paragraphs, condition surrounding context, and control lexical diversity and content reorder- ing. Using DIPPER to paraphrase text generated by three large language models (including -r13.5-davinci-003 successfully evades several detectors, including watermarking, GPTZero, DetectGPT, and OpenAl's text classifier, For example, DIPPER drops detection accuracy of DetectGPT from 70.3% 4.6% (at constant false positive rate of 1%), without appreciably modifying the input semantics.
To increase the robustmess of Al Al-generated text detection to paraphrase attacks, we introduce simple defense that relies on retrieving semantically-similar generations and must be maintained by language model API provider. Given candidate text, our algorithm searches a database of sequences previously generated by the API, looking for sequences that match the candidate text within certain threshold. We empirically verify our defense using a database fine-tuned T5-XXL model and find that it can detect 80% to 97% of paraphrased generations across different settings while only classifying 1% human-written sequences Al Al-generated. We open-source our models, code and data.'
15M generations from a
1 Introduction
Large language models (LLMs) such as ChatGPT [Schulman et al., 2022] exhibit an unprecedented ability write coherent and relevant long-form text in response user-specified prompts. These abil- ities have sparked fears of malicious applications such as automatically generating fake news articles or homework answers [Stokel- -alker, 2022]. To defend against these cases, several algorithms have recently been proposed to detect Al-generated text, including watermarking [Kirchenbauer al., 2023a), GPTZero [Tiaa,, 2023], DetectGPT [Mitchell et al., 2023], and OpenAl's text classi- fier [OpenAI, 2023a]. However, it remains unclear how robust these algorithms are to paraphrase attacks, in which Al-generated text from an LLM is rewritten by another (smaller) model convey approximately the same meaning but using different word choice and syntax.
Work done PhD student UMass, and partially student researcher in Google Research. I John Wieting and Mohit lyyer contributed equally as advisors. httpps: / gi thub. com ians deeofheeoon aaaaaadddeeeeeeee wwwedeeeetteppppaeeeeeeeeee won-paraphheaaa asee We use the mmasi-paraphrase definition of semantic equivalence [Bhagat and Hovy, 2013] in this paper.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

shocking findlngg scientist discovered herd of unicorns living in a remote valley.
these unioneo protccted by miki wants of worts towes glvng in mameeeti approracce A brg with their covcelled in mecccle ty licoboeed soud, feer gateercoaaodd bouter in dlaa the look lio mi ninuges. Some them are numoned be capeble speakingg ng large amount of caftorrt linuueeee selected those eees possiss bong
five
urspoli envirmment iohich aamouaded mouurato
humone
OpenAI
Watermark detector
prodictione Al-written (zve.S)
CGatGPT/GPT3
they tajs on andigarrs
(watermarked tokens in green, benign tokens in red)
There eerr rave any reporta of tham micng with pees a his trayllng in en unep endiomeme surmundeecty mounasing prrtecedd my thick duu wortte. Thatterrlia negul bok thermpgi doon cobeed cost und golden fouthrrs speaklin many longungne andd enough 'eal andmais.
DIPPER (our paraphraser)
Watermark detecctrr
predictionn
gonte. seruss
Unclear (r:110
Figure 1: An overview of paraphrasing attacks with DIPPER on watermarked text (Kirchenbauer et al., 2023a]. The original model generation (top) contains several "green" watermarked tokens that are counted by detector to judge whether the text was Al-generated. After paraphrasing, several green tokens are replaced with approximately semantically-cquiaalene red tokens, thereby fooling the detector. Actual outputs from a watermarked version of GPT2-XL and our paraphraser DIPPER.
In this paper, we first demonstrate the vulnerability of these existing detectors to paraphrase attacks (Section 3, 4). Such attacks require an external paraphraser model, since paraphrases generated by the base LLM are still susceptible to detection techniques such as watermarking. We train 1 11B parameter paraphrase generation model called DIPPER (or Discourse Paraphraser) to execute these attacks. DIPPER possesses two unique features that help its outputs evade Al-generated text detectors: (I) Paraphrasing long-form text in context: Most modern paraphrasers are exclusively trained on sentence-level data, ignoring discourse-level information. However, many critical use cases of LLMs involve generating long-form text as responses to detailed user-specified prompts. Thus, we train DIPPER to paraphrase paragraph-Ienge
texts, re-order content, and optionally leverage context such as user prompts; (2) Controlling output diversity: Another weakness of existing paraphrasers that they lack an easy way to control output diversity. An attacker may want to apply just the minimum amount of paraphrasing to evade a detector. DIPPER provides users with two intuitive scalar control knobs at inference time (lexical diversity, content reordering) that are trained end-to-end.
We use DIPPER to attack several recently proposed AI Al-generated text detectors (see Figure I for an attack overview). Experiments on multiple tasks and LLMs (including SPP3.5-davinei-000 show that after paraphrasing with DIPPER, a substantial fraction of Al-generated texts are misclassified as wwitten texts by all detectors. For example, DetectGPT [Mitchell al., 2023] correctly detects 70.3% of Al-generated sequences from GPT2-XL, but after paraphrasing. its detection rate drops to only 4.6% despite minimal semantic modification. We confirm the validity of DIPPER's paraphrases through several automatic evaluations and a human evaluation of semantic similarity.
Given the vulnerability of Al-generated text detectors paraphrasing, how can we defend against such attacks? In the second part of our paper (Section 5), we propose to use retrieval methods to detect Al-generated text instead of relying on statistical properties of the text watermarking. First, an LLM API provider stores every output generated by their model in a database. The API provider then offers service in which a semantic representation of a candidate text is compared to representations of every generation stored in the database. The search focuses on the semantics of the input and can leverage both standard IR methods such as BM-25 [Robertson et al., 1995] as well as semantic vector representations such as P-SP from Wieting et al. [2022]. Since paraphrasing does not modify the semantics of the input, this algorithm robust to paraphrasing attacks. Specifically, we find that 97.3% of PG19 paraphrases and 80.4% of Wikipedia paraphrases are successfully detected in large database of over 15M generations, at 1.0% false positive rate. We extensively discuss the limitations and scalability of retrieval-based detection in Section 5.4.
In contrast to concurrent work that also uses paraphrasing to attack AI Al-generated text detectors [Sada- sivan et al., 2023], our work offers more comprchensive attack experiments, a new and more powerful paraphraser, human evaluations of paraphrase quality, and finally novel defense mechanism based on retrieval to combat such attacks. To spur future research in this area, we will release our DIPPER model, data, and codebase for evaluating both existing detectors and our retrieval-basee method.
2 'These detection rates were computed at a constant false positive rate (FPR) of 1%. Due to the importance of low FPR in this task, we recommend using fixed low FPR rather than AUC-ROC values; Section 1.

---

2 Background on detectors of Al-generated text
In this section, we provide a brief overview of existing algorithms for detecting Al Al-generated text detection (see Appendix E for a detailed version). We also contrast our work to Sadasivan et al. [2023], concurrent effort which notes the efficacy of paraphrasing attacks but does not consider a retrieval-based defense in its pessimistic conclusion about the fate of Al-generated text detection.
watermark is modification the generated text that can be detected post-hoe by an algorithm while remaining imperceptible to human readers. Effective watermarks are difficult to remove and have little effect on the quality of generated text. Prior work has watermarked natural language using syntax tree manipulations [Topkara et al., 2005, Meral et al., 2009], and this area has received renewed interest with the advent of LLMs [Abdelnabi and Fritz, 2021, Grinbaum and Adomaitis, 2022]. Most recently, Kirchenbauer et al. [2023a] proposed a simple algorithm that watermarks LLMs by slightly perturbing probability distribution while generating text.
Statistical outlier detection methods detect Al-generated text based on its artifacts [See et al., 2019, Holtzman et al., 2020] instead of modifying the generative algorithm. Early methods detect statistical irregularities in entropy [Lavergne et al., 2008] and perplexity [Beresneva, 2016], while Gehrmann al. [2019] introduced the GLTR visualizer to assist humans in detecting Al- Al-generated text. The release of ChatGPT prompted the development of two new tools: closed-source GPTZero [Tian, 2023] and open-source DetectGPT [Mitchell et al., 2023]. The latter uses the observation that Al-generated text tends to have significantly higher LLM likelihood than meaningful perturbations of it.
Classifier methods train models distinguish human- written text from Al- Al-generated text. Early efforts used classifiers to detect fake reviews [Hovy, 2016] and news [Zellers et al., 2019], while others examined classifier performance across domains [Bakhtin et al., 2019] and decoding strate- gies [ppollto et al., 2020]. Most recently, OpenAl fine-tuned a GPT model to perform this task and released it as web interface [OpenAI, 2023a]. Their model uses generations from 34 LLMs, with the human-writted text from Wikipedia, WebText, and their internal human demonstration data.
Comparison to Sadasivan et al. (2023): In recent concurrent work, Sadasivan et al. [2023] also demonstrate the utility of paraphrasing attacks against Al Al-generated text detectors. Our experiments encompass more tasks, detection algorithms, and larger LMs like GPT3.5. Additionally, a discourse-level paraphrase model (DIPPER) that is much more suited to long-form text than the off- -hettheeehllf sentence-level paraphrasers used in their paper. More importantly, our retrieval- based defense directly contradicts the "impossibility result" of Sadasivan et al. [2023] and its associated proof, which states that optimal detector will perform random as the quality of LLM--genrrted text approaches that of human wwiiten text. The quality of generated text is irrelevant to our detector's accuracy because it relies only on a corpus search, and thus the proof is inapplicable. Other concurrent work [Chakraborty et al., 2023] has also shown the proof's invalidity in practical settings.
propose
3 Building a controllable discourse paraphraser
Having outlined existing methods to detect Al-generated text, we now focus a simple attack against all detection techniques: paraphrasing the generated text. Intuitively, paraphrasing alters the statistical properties of Al-generated text, which can fool outlier detection or classifiers while also reducing the number of watermarked tokens (Figure 1). To evade such detectors, a paraphraser must be able to handle context in the form of prompts or multi-sentence inputs. Its behavior should also be controllable in order to make as many/few changes as needed to evade a given detector. In all cases, it should not appreciably change the input semanties. Finally, to evade watermarking, the paraphraser must be different from the watermarked model, as otherwise the paraphrases will also be watermarked. Below, we detail how we construct a paraphraser (DIPPER) with all these properties."
Constructing paraphrase data: Our process involves fine-tuning a LLM on a parallel dataset of paragraph-level paraphrases, which we modify model control, external context and content reordering, We leverage the PAR3 dataset [Thai et al., 2022], which contains multiple translations of non-Er -nnglinls novels into English aligned at paragraph level. which we treat as paraphrases. More formally, let and g be aligned paragraphs, where Pl, PN denote sentences of p and qı. 92. .M.
'To better ground DIPPER's abilities in prior work, survey existing paraphrase models in Appendix D.I.
3

---

Step Align sentences between translation and transiation using semantic similarity. alignments- - (pt, qt). (p2, q2). (p3, q3q44. (p4, q(6)
Step Choose subset alighments (l22, q2), (p3, q3q4)) Step Shuffe sentences q. compute control codes
Stop 4: Input/ output mapping fine- tune XXLL discourse paraphrasing
coutase nomeuter velevedd
order
powmtorked
My companicn proved iraaling
anpravdd anuiiight poakiss, fortheed
pooto shufited onder)
Dyste or ginnall or deer)
My sou
bust bover inpaa down my
vesadd the butoon thip of albar.
Q,2 ioenet rin have over
p.2 Mry compan priyeved inadingg feowsdreyee
forehee tachingttabbbe casp
my sirscceed outer rapp
inportenee comsoriion
onter
delertt antraatt..
q4 heant was pound ng
motiio poppdl
pa don't plenl howe
addressd Cod with more fenvor aad
conaciation servorr nen praving to
be compast prppeeed
bencee and roseeeedooo
pk: ldvet how ongi itayydd
receive more conicattr heirt
quicrky rass was pound ng videents
contusation cny hear...
lira consuii comer of any
rounc
and loupant was obivious to
prourd
TS-XXL
p-ltt dant know long itaydd
thet por fon how long reight have
All fhor, (neat have vary toucting specticc for come to rediver
Lexical diversity: 40 100
bauciig dight my compai and foomem non they come.
Order diversity: 60/ 100
(ungraa Milierree
pC byy eoorey wrice lare ing
pa: don't think udd
(Hendbeix teo token map)
Figure 2: The method used to train DIPPER on English translations of the French novel The Nun. We first align sentences between the two translations to create parallel data. Next, a subset of the alignments are chosen; in this example, we use (p2, (2) and (p3, (3q4). We shuffle sentences, compute control codes, and fine-tune a TS-XXL LM to generate P2P3 given 939192 and the context P1 and p1.
denote sentences of g. Note that M may not be equal N when two translators disagree on when to merge and split sentences. We perform the following steps (overview in Figure 2):
1. Align sentences of p to sentences of q by using the semantic similarity scores from the paraphrase similarity metric in Wieting et al. [2019] to run the sequence alignment algorithm designed by Needleman and Wunsch [1970] which uses dynamic programming (metric details in Section 4. 4.1).
2. Choose subset of sentences P,...,, from the first paragraph. Let qy. be the corresponding alignment in the second paragraph. In Figure 2, i = 2. = 3, = 2. =
3. Re-order the sentences "y. randomly, and compute the diversity control codes between We shuffle the sentences to allow for the model to learn content
Pi...p, and shuffle(gr.... re-ordering. We compute lexical diversity (L) using unigram token overlap (FI score), and the order diversity (O) using the Kendall-Tau correlation of tokens of overlapping words between Pi...p and ((.ffee((g...(;))) also used in Krishna et al. [2020]. These scores are normalized to values (0, 20, 40, 60, 80, 100), where L = 20 roughly corresponds to 20% lexical modification.
4. Map the shuffled qy...qy' to Pi. leveraging context from the rest of and control codes using string concatenation. Let input shuffle((p....95))) We map.
lexical L, order- 0 + PL..PP-1 @ <p> input </p> + Pj+1..PPN Pi...p
where @ string concatenation. During inference, we can paraphrase any sequence of sentences by marking it with <p> tags, assigning the control codes (L, O) the desired diversity values.
Our final dataset contains 6.3M paraphrase pairs. We fine-tune a former [Vaswani al., 2017] on this data, initialized with the pretrained 1IB parameter T5-XXL checkpoint [Raffel et al., 2020]. See Appendix F.1 for details.
to-sequeecee Trans-
Experiments attacking detection algorithms with DIPPER
In this section, we describe our experimental setup in Section 4.1.4.2 and present our results in Section 4.3. Overall, we find that DIPPER evades all detectors across three LLMs (including GPT3.5).
4.1 Evaluation metrics
Our first metric measures how often the input text is correctly detected as Al-generated. Since detection rates are heavily dependent on the chosen detection threshold, the AUC-ROC metric is commonly used to measure detector performance [Mitchell et al., 2023], which considers the range of all possible thresholds. However, in this application, it critical that the false positive rate (FPR) is low; in other words, human- written text must almost never be classified as machine-generated [OpenAI, 2023a, Kirchenbauer et al., 2023a]. Hence, we fix the FPR to 1% for all detection algorithms (although even 1% is likely too high in practice), and adjust the detection threshold accordingly while reporting detection accuracies. Additionally, we also plot ROC curves focusing on the 0- 1 1% FPR region. Overall, expect detection rates to plummet on paraphrased text.
Detection accuracy:

---

Table 1: Performance of detection algorithms (at 1% FPR) before and after DIPPER paraphrasing on open-ended generation using Wikipedia prompts (300 generated tokens). As the diversity (L.O) increases, detection rates decrease across algorithms, with nearly perfect semantic similarity (Sim). *GPT3.5 DetectGPT scores computed using 200 samples at 20% FPR as it scores 0% at a 1% FPR.
Metric
Sim 1
Detection Accuracy
Detector +
Watermarks
DetectGPT
OpenAl
GPTZero
RankGen
GPT2-1.5B
100.0
70.3
21.6
13.9
13.5
DIPPER 20L
99.2
97.1
28.7
19.2
9.1
15.8
DIPPER 40L
98.4
85.8
15.4
17.8
7.3
18.0
DIPPER 60I.
96.9
68.9
8.7
13.3
7.1
19.8
DIPPER 60I., 600
94.3
57.2
4.6
14.8
1.2
28.5
OPT-13B
99.9
14.3
11.3
DIPPER 20I.
99.1
96.2
3.3
11.8
DIPPER 401.
98.6
84.8
1.2
11.6
3.8
6.6
DIPPER 60I.
97.1
63.7
0.8
9.1
6.3
9.3
DIPPER 60L, 600
94.6
52.8
0.3
10.0
1.0
13.5
GPT-3.5-175B, davinci-003
26.5*
30.0
7.1
1.2
DIPPER 20L
97.6
12.5
20.6
4.3
DIPPER 40L
96.7
8.0
22.4
DIPPER 60L.
94.2
7.0*
15.6
6.1
3.9
DIPPER 60L, 600
88.4
4.5
15.6
1.8
7.3
Human Text
1.0
1.0
1.0
1.0
1.0
Semantic similarity (Sim): Detection accuracy is an insufficient evaluation our attack's 's success. We also need to measure whether the original and paraphrased generations share approximately the same semantics. We measure semantic similarity using the state-of-the-aa semantic similarity model P-SP from Wieting al. [2022]. embedding averaging model trained large corpus of filtered paraphrase data [Wieting and Gimpel, 2018]. P-SP is a well-calibrated metric that performs well semantic calibration tests well plagiarism detection in STS benchmarks [Agirre et al., 2016]. P-SP is also robust against topically similar non-paraphrases. We found that P-SP it scores just 0.09 random pairs of paragraphs from the same book (topically similar paragraphs but not paraphrases) in the PAR3 dataset ('Thai et al., 2022]. In contrast, the average P-SP score of actual human paraphrase pairs PAR3 0.76. We consider semantics be approximately preserved if the P-SP score is greater than this average human paraphrase score of 0.76.
Besides semantic similarity, conduct several automatic evaluations, ablation studies, and human evaluations of intrinsic paraphrase quality in Appendix C.
4.2 Models, datasets & detection algorithms
Base language models: We conduct attacks on three language models of varying sizes that belong to different model families. We consider the GPT2-XL model (1.5B parameters) [Radford et al., 2019], the OPT-13B model [Zhang et al., 2022], and the meoxt-davinne family [Brown al., 2020], which has 175B parameters and has additionally been instruction tuned using reinforcement learning from human feedback [Ouyang et al., 2022], For all LMs, we sample generations that are 300 tokens long before passing them through DIPPER for the attack experiments.
0003 variant from the GPT-3.5
Natural language generation tasks: We experiment with two long-form text generation tasks, since most malicious upplications (e.g., fake article creation) associated with long-form outputs. First, we consider open-ended generation, where an LM generates a continuation to a two-sentence prompt. To obtain our prompts, sample 3K contiguous two-sentence chunks from the validation split of WikiText- 103 [Merity et al., 2017] 7] and use the next 300 tokens as the "human- written" continuation. Second, evaluate long-form question answering [Fan et al., 2019], in which an LM answers question with a 300-word answer (dataset details in Appendix F.2). For our main results, the human
Forr UPT2-XL and OPT- 13T-13B, we generate text using nucleus sampling [Holtzman et al., 2020] with 0.9. For mavinni--00 the default hyperparameters the API Playground (temperature 0.7).
5

---

reference answers continuations only used adjust detection thresholds of studied methods to maintain a FPR. Note that we are not removing human-wrinee text from our test set. Our metric equivalent having set with 50-50 split between machine/human wwitten for the same prompts, and observing the FPR-1% point in the ROC curve (also provided in Appendix H).
Detection algorithms: We attack five detec-
tors:? (1) watermarking [Kirchenbauer et al.,
Figure 3: Detector performance (at 1% FPR) on long-form QA before/after paraphrasing- As di- versity (L,O) increases, detection rates decrease with very high semantic preservation (Sim). WM: Watermark, D.GPT: DetectGPT, O.AI: OpenAl. *GPT3.5 D.GPT uses 100 samples at 20% FPR to show attack success, as it scores 0% at 1% FPR.
2023a]] (2) DetectGPT [Mitchell et al., 2023];
(3) GPTZero [Tian, 2023]; (4) OpenAl's text classifier [OpenAI, 2023a]:* and (5) RankGen- XL-all [Krishna et al., 2022a]. We use the de- fault hyperparameters for each detector. We also respect their minimum length specifications, dis- carding instances where any of the Al-generated text, human-writtee text, or paraphrased text is shorter than the minimum length.
Metric ->
Sim
Detection Accuracy +
W.M.
D.GPT
O.AI
Paraphrasing Al-generated text: We pass the prompts for each task and Al-generated responses to those prompts through DIPPER. Specifically, we feed the model input of the form,
1PT52-1.5B
100.0
74.9
59.2
DIPPER 201.
99.5
98.9
45.7
35.3
DIPPER 40I.
99.0
90.7
28.0
34.4
DIPPER 60I.
97.5
71.1
15.8
31.3
60L. 600
96.2
55.8
7.6
32.7
lexical LI, order O prompt <p> generated-text </p>,
OPT-13B
100.0
29.8
33.5
DIPPER 20L
99.6
98.3
15.0
24.5
DIPPER 40L
99.4
87.3
6.4
24.1
where L and O represent the paraphraser di- versity control codes and the <p> and </p> special tokens mark the boundaries of the text paraphrased. We use O = 0, 60 in our main attack experiments. After paraphrasing, we ensure that the Al-generated sequence, paraphrased sequence, and human- written sequence have an equal number words by truncating them to the length of the shortest among the three. To ensure higher semantic preservation, we iteratively paraphrase long quences three sentences at a time, keeping al- ready paraphrased text in the context of the generation. To show the effectiveness of attack, only paraphrase each generation once, rather than draw multiple samples until it evades detection.
DIPPER 60L
65.5
21.6
+ 60L, 600
92.9
51.4
1.5
21.6
GPT-3.5-175B
20, 40, 60 and
davinci-003
67.0*
40.5
+ DIPPER 20L
99.9
54.0*
43.1
DIPPER 40L
99.8
36.0*
43.1
DIPPER 60L
23.0*
40.1
60I,, 600
98.3
14.0
38.1
Human Text
1.0
1.0
4.3 Attacking A Al-generated text detectors
We present our results in Table and Figure 3. Overall we find that:
Paraphrasing significantly lowers detection accuracy while preserving input semanties. Across all LMs, detectors," and tasks, paraphrasing significantly lowers detection accuracy across all diversity control codes. For instance, paraphrasing GPT2-XI. open-ended generations reduces watermark detection accuracy from 100% to 57.2%, and DetectGPT accuracy from 70.3% to just 4.6%. Trends are similar even for large LMs like GPT-3.5, for which paraphrasing reduces OpenAl's text classifier accuracy from 30.0% to 15.6%. Additionally, DIPPER preserves semantics effectively,
Alternnttveey, if random half of the human-written data was used for threshold adjustment, we find the other half has FPR of 0.8%-1.25 1.2% across random splits, and this deviation will reduce with bigger dataset. model-agnostic detectors, as justified in Appendix F.3. 'This classifier was taken down in July 2023 by OpenAl due to its low accuracy.
We consider both -ppcifc
this task, was trained identify well- written continua- tions, so we hypothesize that could also act as reasonable Al-generatee text detector.
Whiile RankGen was not explicitly optimized
JeWe discuss this attack briefly in Section 4.4.
"Exeept RankGen, which scores paraphrases as Al-generated more often than non- paraphrased text. We
attribute this paraphrases being poorer continuations to the prompt compared to the original (Appendix C), an aspect RankGen bases its on. However, it has low overall performance since it is not trained for this task.

---

88%-99% paraphrases achieve P-SP score higher than the median score of human-written paraphrases. High semantic preservation is supported by careful human evaluations in Appendix C.2. Overall, we find that watermarking the most resilient detector paraphrasing.
Non-watermarking detectors are generally ineffee-
tive. We observe that all detectors apart from water- marking struggle with text generated by larger models
(nc)
opena
like OPT-13B and GPT-3.5,, achieving detection accu- racies < 50%. While DetectGPT is effective on the smaller GPT2-XI, model (74.9% on long-form QA), its accuracy drops to just 29.8% on OPT-13B. Further- more, GPTZero and RankGen perform the worst among the five detectors on all tested LMs (Table 1), as they are only able detect 15% of non-paraphrased Al- generated text. Thus, we recommend against using these detectors.
sporers
lopt
spczero
100
Thue Positive Rate (%6
60
40
ROC plots confirm the trends at different false pos- itive rates. In Figure 4, we plot the detection accuracy (true positive rate) at different values of FPR between 0% and 1% for GPT2-XI.. Overall, paraphrasing signif- icantly drops detection rates across all FPR thresholds (more plots in Appendix H).
1.0
False Positive Rate (%)
Figure 4: ROC plots (0-1% FPR) for GPT2- XL using different detectors, before (solid lines) and after paraphrasing (dashed). Para- phrasing reduces detection rate across FPRs, and our detector retrieval detects para- phrases best; full plots in Appendix H.
4.4 Alternative paraphrasing attacks
Paraphrasing multiple times:
Our presented attacks use just single paraphrase generated by DIPPER to evade detection. A simple way to further improve the effectiveness of a paraphrase attack is to sample multiple times! from DIPPER and choose a paraphrase that evades the detector while also preserving semantics. We do not perform this attack as it can only be done if an attacker has access to detector, which may be a strong assumption (see Appendix A.2). That being said, using multiple paraphrase samples can make the attacks even more potent against publicly available detectors.
Non-DIPPER paraphrasers: A second alternative is to non-DIP -IPPEIP paraphrasers that operate at the sentence level. These models can be deployed for long-form text inputs by paraphrasing the inputs sentence by sentence, ignoring prompt context. While the concurrent work of Sadasivan et al. [2023] shows that this method can also evade detection, our ablations in Appendix C show that non-contextual versions of DIPPER have lower quality and are less compatible with the prompt as DIPPER paraphrasers. Moreover, most existing paraphrasers lack fine-grained diversity control and multi-sentence input support (survey in Appendix D.I), two desired properties from an attacker's point of view: attackers want to modify long multi-sentence responses just enough to evade detection.
A more interesting option is to use LLM like ChatGPT to perform shot contextual paraphrasing. While this method is likely to provide accurate paraphrases, (
they may be detectable by strategies like watermarking (whether using the same API the original LLM different one). We thus expect sophisticated adversary to use their own private paraphraser (like DIPPER) to evade detection.
5 Defense against paraphrase attacks using retrieval
In Section 4.3, we showed that paraphrasing an effective attack against Al-generated text detectors. How can LLM API providers defend against these attacks? In this section, we propose retrieval over previously-generace sequences as defense against paraphrase attacks. At high level (Figure 5), an API provider first stores every sequence generated by their LLM in database. The API provider offers an interface that allows users to enter candidate Al-generated text as a query. The interface searches
Precisely compute (2) for different random seeds while sampling text. Alternatively, an attacker could also compute Lnpper(Jaigne (... [z))) but this will lead to excessive semantic drift from x. IIn initial experiments, we observed that DIPPER performs competitively with the much larger and more powerful GPT-3.5 davinci-OW0 model in terms of paraphrase quality, and significantly better at controlling diversity. This finding shows that specialized smaller models outperform LLMs in paraphrasing tasks.
7

---

Responas: Objjcta in unleas their trajectory.
Prompt:
Resporae: No. there upper link on how long sentence can be
upper limit on how ong sentiance
Databese of responses
Prompt: objocts around the Earth fall down?
Response: Objects wull full down unteess their tragectory.
DIPPER paraphraser
OpenAI
Prompt:
ChatGPT PeaGGPT/PPT
Response: Otoma Handolals He is the d4th...
Paraphrase: Things currently will not full undess their path...
detailed blogrephy of Barack Obaam,
Prompt: Why language modals things?
Resparee: for their abiitty generone realistic..
BM25 retriever
Generated by our API!
Figure 5: An illustration of Al-generated text detection with retrieval. Several users (including the attacker, shown as the purple emoji) feed prompts to the API which are collectively added to private API-side database. Candidate queries are compared against this database using a retriever like BM25.
over the entire database of sevviously-generatte text, trying to find a sequence that approximately matches the content of the imput query. This search can be done using semantic similarity scorer like P-SP [Wieting et al., 2022] or a retriever like BM25 [Robertson et al., 1995]. Since paraphrasing approximately preserves input semanties, we expect such defense to still be able to map paraphrased generations to their source. We formalize our detector in Section 5.1, and then conduct a controlled comparison with competing detectors in Section 5.2. We evaluate retrieval-baset detection at scale using a large retrieval corpus of 15M generations in Section 5.3. In Appendix A we extensively discuss limitations of retrieval-based detection and share ideas for enabling further scaling.
5.1 Formulating the retrieval defense
Let fiM be an L.M API (e.g., GPT-3.5) that takes a prompt i as input and returns continuation y. Let fiet be encoder (e.g.. TF-IDF, neural network) that embeds variable-length sequences into fixed-size vectors that represent the input semantics. Then, we do the following:
Building the database: Let .., N be the set of prompts that have been fed as input to the API in the past with y/ fLM (x;) being the LLM output. Here N can potentially be very large for popular APIs (we study up to N = 15M). We construct our database Y = yy,, ...Y.v by encoding every LLM API output with our retrieval encoder, or y; = fre( (y:). The database Y is dynamically updated and stored on the API side. It is inaccessible to clients except via the API described in the next step.
Querying the database: Let y be candidate text and y' (y'(y') its encoded vector. Suppose client wishes to know whether y' was generated by the API fLM- The API provider can check this by seeing whether the maximum similarity score of y an entry the database exceeds some detection threshold I' chosen by the API provider:
y' SE(1...N) Iy'l [Ya]
output score T, where score
We expect unperturbed machine- generrted text to always get score of 1.0, while paraphrasing the text may lower the detection score. Hence, lowering 7 will increase the detection rate of heavily- paraphrased text but also increase the false positive rate (i.e., human-written text that resembles sequences previously generated by the LLM API can be falsely flagged). Since N can be very large, the score can also be approximated using efficient nearest neighbor libraries like FAISS [Johnson et al., 2019]. However, in this work we only compute exact inner products.
As the retriever free, experiment with choices: P-SP [Wieting et al., 2022] and BM25 [Robert- son et al., 1995]. We implement BM25 using the retriv library from Bassani [2022]. In order to normalize and calibrate BM25 scores, compute the F1-score unigram token overlap [Rajpurkar et al., 2016] between the candidate y and the best retrieval y to get a detection score in [0, 1].
5.2 Controlled comparisons of retrieval with other Al-generated text detectors
First, we conduct a controlled comparison between the detection algorithms evaluated in Section 4.3 and our retrieval method on long-form question answering. 14 We construct three kinds databases,
1. 3K sequences generated by specific LM for one of the tasks;
"Corresponding results in open-ended text generation Wikipedia prompts are provided in Appendix G.I.

---

Table A comparison of retrieval against other detectors on long-form QA (300 generated tokens).
Our detector outperforms baselines (at 1% FPR) even with the most diverse paraphrases O+)
GPT2-XL
OPT-13B
GPT-3.5 (davinci-003)
Original
+ 60L.
+ +00
Original
60L.
60 L,O
Original
60L.
60 L,O
Watermark [2023a]
100.0
71.1
55.8
100.0
65.5
51.4
DetectGPT [2023]
74.9
15.8
7.6
29.8
3.2
1.5
0.0
0.0
OpenAl [2023a]
59.2
31.3
32.7
33.5
21.6
21.6
40.5
40.1
38.1
(Ours) Retrieval over corpus of 3K generations from model itself, with retriever:
SP
100.0
95.6
87.7
100.0
94.8
85.3
100.0
94.2
85.1
BM25
100.0
99.2
97.8
100.0
99.3
97.3
100.0
98.6
96.2
(Ours) Retrieval over corpus 9K generations pooled from all three models, with retriever:
SP
100.0
88.9
75.4
100.0
89.6
76.4
100.0
93.8
84.6
BM25
100.0
98.3
95.2
100.0
98.5
94.4
100.0
98.5
96.0
(Ours) Retrieval over 43K ShareGPT responses corpus of 3K generations from model itself, with retriever:
SP
100.0
94.0
84.8
100.0
94.2
84.7
100.0
94.1
84.9
BM25
100.0
98.9
97.5
100.0
99.0
97.3
100.0
98.4
95.5
2. 9K sequences formed by concatenating the generations from three LMS in this paper;
3. 46K sequences constructed by combining the 3K sequences from (1) with 43K LLM responses from ShareGPT--vicuna.
We expect (2) be a more difficult test for our method than (1), since the retriever needs distinguish between multiple generations from different models given the same prompt. On the other hand, (3) denotes more real-world setting with several diverse L.LM-generated outputs from ShareGPT, Next, we perform retrieval over this corpus using different types of queries: the original Al- Al-generated text, its DIPPER paraphrase, and numan-written text (each query with at least 50 tokens).
Table shows that across all LMs, retrieval is a much more effective detector than baseline detectors. On unperturbed Al-generated text, retrieval has 100% detection accuracy due exact match with the retrieval corpus. On paraphrased text, retrieval with BM25 is quite effective, detecting 97.8% of the highest-diversity paraphrases (L60, 060) on GPT2-XL, 97.3% on OPT-13B and 96.2% on GPT-3.5 in long-form question answering. This is significantly better than the next best alternative with competing detectors (55.8%, 51.4%. 18)1%). Even on our harder augmented databases, detection rates continue to be high: 95.2%, 94.4%, 96.0% for the 9K augmented database; 97.5%, 97.3% 95.5% for the ShareGPT augmented database. Finally, we observe that BM25 is a more effective retriever than P-SP, scoring 95.2% vs 75.4% on the augmented setting in GPT2-XL. These trends are consistent across different FPR thresholds, as shown in Figure 4.
In Appendix G.2, we additionally observe promising preliminary results that show the effectiveness of retrieval against text mixing attacks [Kirchenbauer al., 2023b].
5.3 Is retrieval an effective detector with large retrieval corpus?
In the previous section, we conducted experiments using the set of 9K sequences generated by all three models as the retrieval corpus. However, this is more of toy experiment: in practice, popular LLM API may serve millions of queries a day. As the corpus grows larger, the false positive rate (i.e., human-writter text falsely detected Al-generated) will grow. How well do retrieval-baste based detectors scale? To answer this question, we need access to a large corpus of Al-generated text. We utilize the training data used train RankGen [Krishna et al., 2022a], which contains over 70M Al-generated sequences. We use the Project Gutenberg and Wikipedia splits of the training data, each of which contain 15M sequences generated by T5-XXL model [Raffel et al., 2020] fine-tuned on the different documents in the same domain. We discard generations which are shorter than 50 tokens, and paraphrase subset of 2K generations to evaluate retrieval.
Retrieval is effective even with corpus size of 15M generations. Figure 6a, we plot the detection accuracy as function of retrieval database size. Overall, we observe that detection
15 thuugiingface co poo/dataeetee wawatttteet/nnnnn11eeee111
Vicuna, unf unfiltered

---

Retrieval over 2M generations
100
Detection rate at FPR 1%
100
Detection rate (%) at FPR - -1%
80
60
40
BM25, pg19 (paraphrased)
P.SP. pg19 (paraphrased)
BM25, wiki (paraphrased)
20
BM25, wiki
P-SP, wiki (paraphrased)
BM25. pg19
2
4
6
8
10
12
14
16
10
20
30
40
50
60
corpus size (milllons)
number words in query (paraphrase)
(a) Variation in strieval-based detection with re- trieval corpus size. Note consistently high detec- tion rates for paraphrases, which only slightly de- grades as the corpus is scaled to 15M generations.
(h) Variation in retrieval-based detection with dif- ferent query lengths. Overall. retrieval performs best with queries length 50+ tokens.
Figure 6: Detection rate using retrieval at 1% FPR w.r.t. corpus size (left) and query length (right).
accuracy remains consistently high across different corpus sizes (varying from 1M generations to 15M generations). We observe slight drops in performance as the corpus size increases: just 1% (98.3 to 97.3) on Project Gutenberg (PG19) and 9.6% (90.0 to 80.4) on Wikipedia. Consistent with the results in Section 5.2, BM25 continues to outperform P-SP across different corpus sizes.
Retrieval detection works best with 50 or more tokens of generated text. Another important factor for our retrieval-based detector is the query length: shorter queries are likely to have more matches (many of them spurious) compared to longer ones. In Figure 6b, we plot the detection accuracy of paraphrased sequences at various query lengths by truncating each sequence to its first X words before using it as query for BM25. We use a retrieval corpus of 2M generations for this experiment. We observe that BM25 struggles to detect paraphrased text with a query length of 20 (less than 25% accuracy), but the detection rate rapidly increases and begins to plateau at 50 tokens.
5.4 Scalability and limitations of retrieval-based detectors
In Appendix A we extensively discuss the scalability of retrieval (A.1), its limitations (A.2), ideas for improving retrieval-based detectors (A.3), and incentive structures for L.LM providers to implement retrieval (A.4). In summary, we believe that if OpenAl implemented with ChatGPT, they would need just 5TB of storage space per month (similar to modern portable hard disks). Furthermore, retrieval on ChatGPT scale takes 130 seconds per retrieval on a CPU-only Macbook Pro, which can certainly be further optimized. However, retrieval-basee detection has some important limitations: (1) potential privacy risk of exposing all LLM responses behind a binary classifier; (2) inability to use retrieval-basee based detection on open-source LLMs like LLAMA [Touvron et al., 2023]; and (3) the need implement and maintain retrieval infrastructure. We discuss mitigation strategies for the limitations in Appendix A.2.
detection is a scalable approach: we estimate
6 Conclusion
We present DIPPER, a controllable paraphraser that can rewrite paragraphs in context. We use DIPPER to stress test current Al-generated text detectors, and we find that DIPPER paraphrases casily evade these detectors while preserving input semantics. As a defense, we propose a simple retrieval- based detector which searches through a corpus previously-generated sequences from an LLM API for semantically-sinn outperforms baselines on paraphrased text, and scales effectively. We discuss the limitations and ethical considerations of our work in Appendix A, B. We have additionally open sourced our models, code and data to enable future research." I Since this paper's 's initial release, DIPPER has been extensively utilized in follow-up studies to measure the robustness of Al-generated text detection algorithms ([Lu et al., 2023, Koike al., 2023, Zhao et al., 2023, Yoo al., 2023, Patil al., 2023, Kirchenbauer et al., 2023b, Kumarage et al., 2023, Liu et al., 2023], name a few).
generations to a given query. We show that this defense significantly
10

---

Acknowledgements
We are very grateful to our anonymous reviewers for their detailed feedback which helped improve our paper. We also thank Kenton Lee, Katherine Thai, Slav Petrov, Fernando Pereira, Jon Clark, William Cohen, Tu Vu, Yapei Chang. Shufan Wang and the UMass NLP group for several useful discussions on earlier versions of this paper. Kalpesh Krishna was supported by a Google PhD Fellowship. Part of this work was done while Kalpesh was Student Researcher at Google Research, hosted by John Wieting.
References
Abdelnabi and M. Fritz. Adversarial watermarking transformer: Towards tracing text provenance with data hiding, In 2021 IEEE Symposium Security and Privacy (SP), pages 121-140, 2021.
E. Agirre, C. Banea, D. Cer, M. Diab, A. Gonvalez-Agirre, R. Mihalcea, G. Rigau, and J. Wiebe. SemEval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), 2016. doi: 10. 10.1853//1/S11-1 1081, URL https: /www. . aclweb. org/ anthology /S16 6 1081.
R. R. Agrawal, M. Turchi, and M. Negri. Contextual handling in neural machine translation: Look behind, ahead and on both sides. 2 annual conference of the European ussociation for muchine translation, pages 11 11-20, 2018.
S. Badaskar, S. Agarwal, and S. Arora. Identifying real or fake articles: Towards better language modeling- In Proceedings of the Third International Joint Conference Natural Language Processing:
Volume-11, 2008. URL httph: //////aaa aelalannoooooooo hogyooogy...oooo 10rg/00 2115.
A. Bakhtin, Gross, M. Ott, Y. Deng, M. Ranzato, and A. Szlam. Real or fake? learning to discriminate machine from human generated text. arXiv preprint 1906.09900.33333 2019.
E. Bandel, R. Aharonov, M. Shmueli-Scheuee Shnayderman, N. Slonim, and L. Ein-Dor. Quality controlled paraphrase generation. In Proceedings of the 60th Annual Meeting the Association for Computational Linguistics ( (Volume I: Long Papers), pages 596-609, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 21118665//11202222lloooogog https:
URL
/ttg//22200 acl- 1ong. .5..
E. Bassani, retriv: A user-friendly and efficient search engine in python., 2022. URL https:
/ //github. com/ AmenRa/retriv.
D. Beresneva. Computer-generated text detection using machine learning: A systematic review. In st International Conference Applications of Natural Language to Information Systems, NLDB, pages 421 126.426. Springer, 2016.
R. Bhagat and E. Hovy. Squibs: What paraphrase? Computational Linguistics, 39(3):433- 472, 2013. doi: 10.1162/COLL 00166. URL battps://wwet aclweb. org/ antho J13-3001.
J. Bradbury, R. Frostig, P. Hawkins, M. J. Johnson, C. Leary, D. Maclaurin, G. Necula, A. Paszke, J. VanderPlas, S. Loanderman-Milne and Q. Zhang. JAX: composable transformations of Nuthon+NumPy programs, 2018. URL http: //tg//gttub//tt////////////tttht com./ google/jax.
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, al. Language models few-shot learners. Advances in neural information processing systems, 33:1 13:1877-1901, 2020.
N. Carlini, F. Tramèr. E. Wallace, M. Jagielski, A. Herbert- Voss, K. Lee, A. Roberts, T. Brown, Song, U. Erlingsson, A. Oprea, and C. Raffel. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 27), pages 2633 2650. USENIX Association, Aug. 2021. ISBN 978-1-939913-22-33 URL https: www. usenix.org. conference/usen pow.reeece/eeeeeeeeeeoee/////eee http::////..aa.a.aaeeeattatattia
11

---

S. Chakraborty, A. S. Bedi, S. Zhu, B. An, D. Manocha, and F. Huang. On the possibilities of ai-generated text detection. arXiv preprint 3xXiv:2304044736 2023.
M. Chen, Q. Tang. S. Wiseman, and K. Gimpel. Controllable paraphrase generation with syntactic exemplar. In Proceedings of the Association for Computational Linguistics, 2019. doi: 10.18653/ vI/P19-1599, URL https:/ www. wwwweb.wwb...................... org/ wttgg///tthoollggeggggeeeee
M. Connor and D. Roth. Context sensitive paraphrasing with a global unsupervised classifier. In European Conference on Machine Learning, pages 105-115. Springer, 2007.
Damodaran. Parrot: Paraphrase generation for nlu., 2021. URL https:/ battpp:://githhie httggggtthhhhooo com/ hatphh/wiriiaammmmaaaaaaaaaeeeee Pat.arrrrooaaapaaas......
Y. Deng, A. Bakhtin, M. Ott, Szlam, and M. Ranzato. Residual energy-based models for text generation. In International Conference on Learning Representations, 2020.
A. Devaraj, I. Marshall, B. Wallace, and J. Li. Paragraph-level simplification of medical texts, In North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pages 4972-4984, 2021. doi: 10. 86633/v 1/2021 maacl-main.395. URL https:
a/acl poclanthooogy org/ / 2021 maacl -m 395.
T. Dopierre, C. Gravier, and W. Logerais. PROTAUGMENT,
Unsupervised diverse short-texts paraphrasing for intent detection meta-learning. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the I 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). pages 2454-2466, Online, Aug. 2021. Association for Computational Linguistics. doi: 10. https: http://aaaaanthccceo
URL
org/ /tg/22222 2021 aalaaclllnnnn.....ee 191.
A. Fan, Y. Jeriite, E. Perez, D. Grangier, J. Weston, and M. Auli. Eli5: Long form question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3558-3567, 2019.
S. Garg, S. Prabhu, H. Misra, and G. Srinivasaraghavan. Unsupervised contextual paraphrase generation using lexical control and reinforcement learning. arXiv preprint 1rXiv:2103.12777 2021.
S. Gehrmann, H. Strobelt, and A. Rush. GLTR: Statistical detection and visualization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111-116, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 1/P19-3019. URL https : / www.attooonooooooooooooooooooote wwgp:/ooooooro/199991 3019. 9.
D. Gomes, M. Costa, D. Cruz, J. Miranda, and S. Fontes. Creating billion-scale searchable web archive. Proceedings of the 22nd International Conference on World Wide Web, pages 1059-1066, 2013.
Google. How google search organizes information, 2023. URL URLhttpss//w...oggle.. com/ http://aaiiiiiigngooooooooeeeeaa http:///wwaaaaaaaoooooooo/ooaaaa
R. Goutham. High-quality sentence paraphraser using transformers in nlp, 2021. URL
https:://i com/ http://aasrrrtthann..eee Quwuuuee.e....... wtttr////eteeetteeeee////////// NewModels/ htw.:ide_lss..aaargeaapppepess
T. Goyal and G. Durrett. Neural syntactic preordering for controlled paraphrase generation. Pro- ceedings of the Association for Computational Linguistics, 2020. URL WtRLhtts:///axx /ttp:si////xxvvvor//// abs/2005. 02013.
E. Grechnikov, G. Gusev, A. Kustarev, and A. Raigorodsky. Detection artificial texts. RCDL2009 Proceedings. Petrozavodsk, pages 306-308, 2009.
A. Grinbaum and I., Adomaitis. The ethical need for watermarks in machine-generated language. arXiv preprint arXiv:2209.0311 2022.
12

---

A. Holtzman, J. Buys, I.. Du, M. Forbes, and Y. Choi. The curious case of neural text degeneration. In Proceedings of the International Conference on Learning Representations, 2020. URL https: / /ppeneeview net/ forum? id- ZYgGQyIVVH.
T. Hosking, H. Tang, and M. Lapata. Hierarchical sketch induction for paraphrase generation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume
I: Long Papers), pages 2489-250) Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. (0.18653/1122222aaclle
178. URL https: http:///aaaaaaaltaacceeec org/
2022.ac1 1.mm..177... 178.
J. House. Text and context in translation. Journal of pragmatics, 38(3):338-355 2006.
D. Hovy. The enemy your own camp: How well can we detect statistically-gennttt fake reviews adversarial study. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Aug. 2016. doi: 00.18653/11/P16-20577
URL htt.h:ttt////t.....t / htttp:///acanttoooory.oooo wtg./16-2050500
J. E. Hu, R. Rudinger, M. Post, and B. Van Durme. Parabank: Monolingual bitext generation and sentential paraphrasing via lexically-oonstrinned neural machine translation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 6521-6525 2019.
K.-H. Huang and K.-W. Chang. Generating syntactically controlled paraphrases without using annotated parallel pairs. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1022-1033, Online, Apr. 2021. Association for Computational Linguisties. doi: 10.1866331172001.cc--mmmaa https: / http//acaaatttt aww.aaaaaattooooyoocoooo oro. 2021 eeacleaaccc..ee main.88.
URL
D. Ippolito, D. Duckworth, C. KallissonBurcch and D. Eck. Automatic detection of generated text is easiest when humans are fooled. In Proceedings of the 58th Anmual Meeting of the Association for July 2020. doi: 10.18653/v1/2000 m6in.164. Computational Linguistics, pages 2trg/200 2020 acl -main.ainn........mm. 164.
URL httpps://act https///locaaahooooggggoooo org/
M. lyyer, J. Wieting, K. Gimpel, and L. Zettlemoyer. Adversarial example generation with syntactically controlled paraphrase networks. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan- guage Technologies. Volume 1 (Long Papers), pages 1875-1885, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/ URL https: / www. acl web. org an/thol ogy/ /N1 8-1170
S. Jean, S. Lauly, O. Firat, and K. Cho. Does neural machine translation benefit from larger context? arXiv preprint arXiv: 1704.05 0335, 2017.
S. Jean, A. Bapna, and O. Firat. Fill in the blanks: Imputing missing sentences for larger-context neural machine translation. arXiv preprint 1xXiv:190..1007. 2019.
J. Johnson, M. Douze, and H. Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535-547, 2019.
M. Bunezys-Dowmunt. Microsoft translator at WMT 2019: Towards large-scale document-level neural machine translation. In Proceedings of the Fourth Conference Machine Translation (Volume 2: Shared Task Papers. Day 1), pages 225-233 Florence, Italy, Aug. 2019. Association for Computational Linguistics. doi: 10. 1/1555//1/11995500 URL https: //act accl antho ogy org/W1 9- 5321.
V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781. 2020.
A. Kazemnejad, M. Salehi, and M. Soleymani Baghshah. Paraphrase generation by learning how to edit from samples. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online, July 2020. Association for Computational Linguistics. doi: 10. 1/.1866//1/2200 acl- acl.main.535. URL https: : hwlanntoooggyyyooooooooooooolloe org//020 acl wal-mmann... 535.
13

---

O. Khattab and M. Zaharia. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pages 39-48, 2020.
Kirchenbauer, J. Geiping, Y. Wen, J. Katz, Miers, and T. Goldstein. A watermark for large language models. arXiv preprint arXiv:2301. 2023a.
J. Kirchenbauer, J. Geiping, Y. Wen, M. Shu, K. Saifullah, K. Kong, K. Fernando, A. Saha, M. Gold- blum, and Goldstein. On the reliability of watermarks for large language models. arXiv preprins WwXiv:2306.0462: 2023b.
R. Koike, M. Kaneko, and N. Okazaki. Outfox: Lim-generated essay detection through in-context learning with adversarially generated examples. arXiv preprint arXiv:2307. 2023.
K. Krishna, J. Wieting, and M. lyyer. Reformulating unsupervised style transfer as paraphrase generation. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP), pages 737-762, 2020.
K. Krishna, Y. Chang, J. Wieting, and M. Iyyer. RankGen: Improving text generation with large ranking models. arXiv preprint o7X1v:2205.09726, 2022a.
K. Krishna, D. Nathani, X. Garcia, B. Samanta, and P. Talukdar. Few-shot controllable style transfer for low-resource multilingual settings. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7439-7468, 2022b.
S. Kuang, D. Xiong, W. Luo, and G. Zhou. Modeling coherence for neural machine transla- tion with dynamic and topic caches. In Proceedings of the 27th International Conference Computational Linguistics. Association for Computational Linguistics, Aug. 2018. URL org/C18 1050.
hatp:://acaatthllloo.
A. Kumar, S. Bhattamishra, M. Bhandari, and P. Talukdar. Submodular optimization based diverse paraphrasing and its effectiveness in data augmentation. In Conference of the North American Chapter of the Association for Computational Linguistics, 2019. doi: 10.18853/v//119-11 URL https:tpp:..............t...tt.. www.. acl web. org/ ant.aoooooooyooooo ogy/N1 933633.
A. Kumar, K. Ahuja, Vadapalli, and P. Talukdar. Syntax-guidedd controlled generation of para- phrases. Transactions of the Association for Computational Linguistics, 2020.
T. Kumarage, P. Sheth, R. Moraffah, J. Garland, and H. Liu. How reliable are ai-generated-text detectors? an assessment framework using evasive soft prompts. arXiv preprint arXiv:2310.05095, 2023.
P. Laban, T. Schnabel, P. Bennett, and M. A. Hearst. Keep it simple: Unsupervised simplification of multi-paragraph text. In Association for Computational Linguistics, 2021. doi: 116653/v/202/// acI-long 498.. URL https: https////aaaaaatlaaaaaaeaaa org 2021 acl- long. 498.
V. Lakshman, C. H. Teo, X. Chu, Nigam, A. Patni, P. Maknikar, and Vishwanathan. Embracing structure in data for billion-scale semantic product search. arXiv preprint arXiv:21 2021.
T. Lavergne, T. Urvoy, and F. Yvon. Detecting fake content with relative entropy scoring. PAN, 8: 2008
M. Lewis, M. Ghazvininejad, G. Ghosh, A. Aghajanyan, S. Wang, and L. Zettlemoyer. Pre-training via paraphrasing. Advances in Neural Information Processing Systems, 33: 13:18470-18481, 2020.
Li, X. Jiang, L. Shang, and H. Li. Paraphrase generation with deep reinforcement learning. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3865-3878, Brussels, Belgium, Oct.-Nov. 2018. Association for Computational Linguistics. doi: 10.18653/v 1/D18- 1/D18-1421 URL https::/aclanthollll
wwg//o8-110 1421.
14

---

7. Li, X. Jiang, I.. Shang, and Q. Liu. Decomposable neural paraphrase generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3403-3414, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.18655/v1/P19-11133 URL attpps:/ae acc http:p///aaaaaatoooooooooo//////
1332.
Z. Lin, Y. Cai, and X. Wan. Towards document-level paraphrase generation with sentence rewriting and reordering. In Findings of the Association for Computational Linguistics: EMNLP 2021, Nov. 2021. doi: 10. 18653/v1/202/ http:2222...nnnneeeeeeee findings-emnlp.89 URL https: http::/:s///alaahhooooooooo org/ 2021 findinggii ngs emnlp........... 8...
A. Liu, L. Pan, X. Hu, S. Meng, and L.. Wen. A semantic invariant robust watermark for large language models. arXiv preprint wr.iv:2310.063... 2023.
N. Lu, S. Liu, R. Hc, and K. Tang. Large language models can guided to evade ai-generated text detection. arXiv preprint arXiv:2305. 10847, 2023.
E. Mansimov, G. Melis, and L. Yu. Capturing document context inside sentence-level neural machine translation models with self-training. In Proceedings of the 2nd Workshop on Computational Approaches to Discourse, pages 143-153, Nov. 2021. doi: 10.18653/11/2211.comm main. URL org / 2021 codi masi--maain mmiin.44
attps://aclanthol.l
Maruf, F. Salch, and G. Haffari. survey on document-level neural machine translation: Methods and evaluation. ACM Computing Surveys (CSUR), 54(2):1-36 2021.
A. Max. Sub-sentencial paraphrasing by contextual pivot translation. In Proceedings of the 2009 Workshop on Applied Textual Inference (TextInfer), pages 18-26, 2009.
Y. Meng, X. Ao, Q He, Sun, Q. Han, Wu, C. Fan, and Li. ConRPG: Paraphrase generation using contexts as regularizer. In Proceedings of the 202 Conference on Empirical Methods in Natural Language Processing, pages 2551-2562, Nov. 2021. doi: 10. (0.18633/1/2222..com .enn222..mmlpmmai... 199. URL htwpt:tt///cmm http:///caaaahhhhaaee
org/ 2221. emn mimpp ma in. 199.
H. M. Meral, B. Sankur, A. S. Özsoy, T. Güngör, and E. Sevinç. Natural language watermarking via morphosyntactic alterations. Computer Speech Language, 2371):107-125, 2009.
Merity, Xiong, Bradbury, and R. Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2017.
L. Miculicich, D. Ram, N. Pappas, and J. Henderson. Document-level neural machine translation with hierarchical attention networks. In Empirical Methods in Natural Language Processing, 2018. doi: 10. 10.18653/v//DD 1325. URL https: ac a/l:://ccllttii anthology org/ D1 1-1---1333 11225.
E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, and C. Finn. Detectgpt: Zero-shot machine- generated text detection using probability curvature. arXiv preprint arXiv:2301. 2023.
S. B. Needleman and C. D. Wunsch. A general method applicable the search for similarities in the amino acid sequence of two proteins. Journal of molecular biology, 48(3):44-453, 1970.
New YorkPost. for
"growing URL https::/ Tha.gg.-waste--raafi--doo-------
Chatgpt 2023.
faces
pains' as
website traffic drops com/2023/070m555
first
time,
OpenAl. OpenAl Models GPT3.5, 2022. URL models/gpt-3-5.
OpenAl. AI text classifier, Jan 2023a. ni-text-classife
URL https://bata..opeaai.com
OpenAl. New ways to manage your data ChatGPT, 2023b. URL wwt.h:ttp///onet/tt com/
blog/new hetp//ewwwwer nww-ways----eeww to-tto-mm-t to-manage- your- ieata-in--i aataaainnhhhatenn chatgpt.
15

---

I. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155. 2022.
V. Patil, P. Hase, and M. Bansal. Can sensitive information be deleted from Ilms? objectives for defending against extraction attacks. arXiv preprint arXiv:2309. 2023.
A. Popescu-Beliss S. Loaiciga, C. Hardmeier, and D. Xiong, editors. Proceedings of the Fourth Work- shop on Discourse in Machine Translation (DiscoMT 2019), Hong Kong, China, Nov. 2019. Asso- ciation for Computational Linguistics. URL https: : / /aaccaaaaaa hol.anooooogy.........o... org/b1 9- 6500.
A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsu- pervised multitask learners. OpenAl Blog, 188), 2019. URL https: / openaiinn..... cwm/bbolmcccccccc og/ better-languee
models/.
C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21:1-67, 2020.
A. Rajauria. Pegasus fine-tuned for paraphrasing. 2020. URL http:: htttpss//hugginngaacc.co Sateer000//egasss_parree
Rajpurkar, Zhang, K. Lopyrev, and P. Liang. SQuAD: 100,000+ questions for machine compre- hension of text. In Proceedings of Empirical Methods in Natural Language Processing, 2016. doi: 10.18653/v1/D16- 1264. URL https: //www ac belweb. org/ anthology/DD6- 54.
Reuters. OpenAl, Google, others pledge to watermark Al content for safety, White House says, 2023a.
Reuters. EU lawmakers vote for tougher Al rules as draft moves final stage, 2023b.
A. Roberts, H. W. Chung, A. Levskaya, G. Mishra, J. Bradbury, D. Andor, S. Narang, B. Lester, C. Gaffney, A. Mohiuddin, al. Scaling up models and data with T5X and seqio. arXiv preprint arXiv:2203.1 1718,9, 2022.
S. E. Robertson, S. Walker, S. Jones, M. M. Hancock-Beaulieu, M. Gatford, et al. Okapi trec-3. Nist Special Publication Sp, 109:109, 1995.
A. Roy and D. Grangier. Unsupervised paraphrasing without translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguisties, pages 6033 6039, Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 10.186653/1//11--1600 URL https: http:///aaaaataho
otr/P19 1105.
V. S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, and S. Feizi. Can ai-generated text be reliably detected? arXiv preprint arXiv:2303.11156 2023.
J. Schulman, B. Zoph, C. Kim, J. Hilton, J. Mcnick, J. Weng, J. Uribe, L. Fedus, L. Metz, et al. ChatGPT: Optimizing language models for dialogue, 2022. URL https:/ //tpps:://///aaai/// com/ blog/ chatgpt.
A. See, A. Pappu, R. Saxena, A. Yerukola, and C. D. Manning. Do massively pretrained language models make better storytellers? In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 843-861 Nov. 2019. doi: 10. 10.1185//11/11- 1079. URL https: https://aaaaattthoe
org/ /K9 911079
R. Shokri, M. Stronati, C. Song, and V. Shmatikov. Membership inference attacks against machine learning models. 20/7 IEEE symposium on security and privacy (SP), pages 3-18. IEEE, 2017.
C. Staker-Walker. Ai bot chatgpt writes smart essays-should academics worry? Nature, 2022.
K. Thai, M. Karpinska, K. Krishna, W. Ray, M. Inghilleri, J. Wieting, and M. Tyyer. Exploring document-level literary machine translation with parallel paragraphs from world literature. In Empirical Methods in Natural Language Processing, 2022.
16

---

N. Thakur, N. Reimers, A. Rücklé, A. Srivastava, and I. Gurevych. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint 1xXiv:200.00666: 2021.
B. Thompson and M. Post. Paraphrase generation zero-shot multilingual translation: Disentangling semantic similarity from lexical and syntactic diversity. In Proceedings of the Fifth Conference Machine Translation, pages 561-570, Online, Nov. 2020. Association for Computational Linguistics. URL https: ac anthologhho wrg/2020/0000..oooo wmt-1.61
E. Tian. Gptzero: An ai text detector, 2023. URL http::ttt///oocm http://gttttttt../////
Tiedemann and Y. Scherrer. Neural machine translation with extended context. In Proceedings of the Third Workshop on Discourse in Machine Translation, pages 82-92, Copenhagen, Denmark, Sept. 2017. doi: 10. 10.1865/v1/W11748111 URL atttp:://aclanthollee
org/ //1g//W11: 48111.
M. Tkachenko, M. Malyuk, A. Holmanyuk, and N. Liubimov. Label Studio: Data labeling software, 2020-2022. URL https:ttt... //g: thub. com http://atteeeeeaaaeee abs albelllll--e studio. Open source software available from http:///gghbbb..omo////eeraa//bl
M. Topkara, C. M. Taskiran, and E. J. Delp III. Natural language watermarking. In Security, Steganography, and Watermarking of Multimedia Contents VII, volume 5681, pages 441-452 2005.
H. Touvron, Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.139711 2023.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones. A. N. Gomez, E. Kaiser, and I. Polosukhin. Attention is all you need. In Proceedings of Advances in Neural Information P Processing Systems, pages 5998 6008, 2017. URL http:/ ppapers nips.cce cc/ /aper/ 7181-attention alliilllllllll wlwalllloooeeeeeee pdd.
E. Voita, P. Serdyukov, R. Semnrich, and I. Titov. Context-aware neural machine translation learns anaphora resolution. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1264-1274, Melbourne, Aus- tralia, July 2018. Association for Computational Linguistics. doi: 10. 10.18653/v1/188-111 URL https: /arps://acai http:///aaaaattttacce/e wttp:/oooo..gg/1111111111
E. Voita, Sennrich, and I. Titov. Context-aware monolingual repair for neural machine translation. In Empirical Methods in Natural Language Processing, pages 877-886, 2019a. doi: 10.18653/v1/
D19- 1081 URL https: / /aclanttooooooo otgg/D1 1081.
E. Voita, R. Sennrich, and I. Titov. When good translation wrong in context: Context-aware machine translation improves on deixis, ellipsis, and lexical cohesion. In Proceedings of the Association for Computational Linguistics, 2019b. doi: 10.18653/v1/119-111 htts::
/ //aclanthology org/Pl1 /P15 1116.
L. Wang. Z. Tu, A. Way, and Q. Liu. Exploiting cross-sentence context for neural machine translation. In Proceedings of the 2017 Conference Empirical Methods in Natural Language Processing, pages 2831, 2017. doi: 10. 10.18653/11/D17-133.. D17-1301.
URL https: http///aaaaannoooaoo org/
Wieting and K. Gimpel. ParaNMT-50M: Pushing the limits of paraphrastic sentence embeddings with millions of machine translations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 451 1-42, Melbourne, Australia, July 2018. Association for Computational Linguistics. doi: 10.18653/v1/P1s 1/P18 1042. URL https: anthhooooggy /P18 1042.
//www. ac web. wwtp//tggooooooooooooooooo
J. Wieting. T. Berg- Kirkpatrick, K. Gimpel, and G. Neubig. Beyond BLEU.training neural machine translation with semantic similarity. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4344-4355 Florence, Italy, July 2019. Association for Computational Linguistics. doi: 10. 1/539 URL https: / / acc anthholooo loy.. org/P19-142 1427.
17

---

J. Wieting, K. Gimpel, G. Neubig, and T. Berg-kirkpatrric. Paraphrastic representations scale. In Proceedings of the 2022 Conference on Empirical Methods Natural Language Processing: Sys- tem Demonstrations, pages 379-388, Abu Dhabi, UAE, Dec. 2022. Association for Computational Linguistics. URL https: http///aaaaannnccnooooo wwg.:y....g///g/ emnip-demos.
S. Witteveen and M. Andrews. Paraphrasing with large language models. In Proceedings of the 3rd Workshop on Neural Generation and Translation, pages 215-220, Hong Kong, Nov. 2019, Association for Computational Linguistics. doi: 10.18653/11/D19-56222 URL https: / //aa //acaanthhlogyy
X. Xie, X. Lu, and B. Chen. Multi-task learning for paraphrase generation with keyword and part- of-speech reconstruction. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1234-1243, Dublin, Ireland, May 2022. Association for Computational Linguis- tics. doi: 10. 10.1663/1120022.mmiiinn. findings acl. 97.
URL https: /ttpss///caaaaaaa /awlaattoooogyyyoooaaaaaa org/2022.
H. Xiong, Z. He, H. Wu, and H. Wang. Modeling coherence for discourse neural machine translation. In Proceedings of the AAAI conference artificial intelligence, volume 33, pages 7338-7345, 2019.
W. Xu, C. Callison-Burch, and C. Napoles. Problems in current text simplification research: New data can help. Transactions of the Association for Computational Linguistics, 3:283-297, 2015.
K. Yang, D. Liu, W. Lei, B. Yang, H. Zhang, X. Zhao, W. Yao, and B. Chen. Gepg: A general frame- work for controllable paraphrase generation. In Findings of the Association for Computational Linguistics; ACL 2022, pages 4035-4047, 2022.
K. Yin, Fernandes, A. F. Martins, and G. Neubig. When does translation require context? a data-driven, multilingual exploration. arXiv preprint 2:Xiv:2009.07446. 2021.
K. Yoo, W. Ahn, and Kwak. Advancing beyond identification: Multi-bit watermark for language models. arXiv preprint 23080:2308.00220 2023.
R. Zellers, A. Holtzman, H. Rashkin, Bisk, A. Farhadi, F. Roesner, and Choi. Defending against neural fake news. Advances in neural information processing systems, 32, 2019.
Zhang, H. Luan, M. Sun, Zhai, J. Xu, M. Zhang, and Y. Liu. Improving the transformer translation model with document-level context. In Proceedings of the 2018 Conference Empirical Methods in Natural Language Processing, 2018. doi: 10.18653/1/D11--I10 1049. URL attps://ack: aalaccannhlloggoo wrg//orgg//........o..ooo 1049.
Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, V. Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.011 20002000.. 2022.
X. Zhao, P. Ananth, L. Li, and Y.-X. Wang. Provable robust watermarking for ai- si-generated text. arXiv preprint arXiv:2306. /7439, 2023.
18

---

Appendix
A Limitations of retrieval-based detection and ideas for scaling it further
This section extensively discusses the scalability, limitations, future work and and an LLM API provider's s incentive structures for retrieval-based detection. First, Appendix A.I, we discuss the scalability of retrieval-based detection in terms of compute requirements, storage space, and accuracy on larger databases. Next, in Appendix A.2, we first point out some limitations of using retrieval for Al-generated text detection (Section 5), some of which potentially apply to all existing detectors. Along with limitations, we provide several possible workarounds. ideas that can make the proposed retrieval detection work well at even larger scale than the one discussed in Section 5. Finally, in Appendix A.4 we briefly discuss why an LLM provider may be incentivized to implement retrieval-based detection, and the relationship of this detector with GDPR's right to be forgotten.
Appendix A.3, we then discuss
A.1 Scalability of retrieval-based detection
Retrieval-based detection requires the storage of a large database of LLM-generated responses, and querying this database to find matches for previously generated responses. How scalable this approach in terms of storage space, compute, and accuracy? In this section we perform approximate calculations of these requirements on OpenAl's ChatGPT [Schulman et al., 2022].
Storage space requirements: We estimate ChatGPT's outputs to take 5TB space monthly (similar to personal portable hard-disk) via the following calculations. ChatGPT currently gets about 2B monthly visits [New YorkPost, 2023]. Assuiig an average response length of 500 tokens per session, this corresponds to trillion tokens. Similar in size to LLaMA's training data [Touvron et al., 2023], this needs 5TB space. However, 5TB is small amount of storage compared to the industrial scale of information retrieval. For example, the Google Search index is over 100,000TB and has 100B+ pages [Google, 2023]. Major LLM service providers already have complex storage infrastructure to facilitate this defense. Additionally, OpenAl already stores conversations for at least 30 days (to monitor abuse, and potentially RLHF), even after user chooses opt-out [OpenAI, 2023b].
Compute requirements: Our retrieval experiments, conducted on 14-core CPU (similar a Macbook Pro), took 1 second per retrieval on a 15M sized corpus. Extrapolating to a corpus of ChatGPT's monthly usage (2B visits) would need 130 seconds/retrieval on Macbook. However, this is fully parallelizable, and can make use of GPUs (Google searches 100B+ entries in < 1 sec). Moreover, efficient similarity search has powerful libraries like FAISS available [Johnson al., 2019]. For comparison, ChatGPT itself takes 10 seconds/response, possibly using powerful 8-GPU A100 server. 16 Major LLM providers have massive compute clusters, and we believe the computational requirement of retrieval is much lower than hosting LLMs in the first place, which these providers are already adept at. Moreover, our proposed ideas in Appendix A.3 can further reduce compute costs.
Accuracy on larger databases: Our experiments were conducted on the RankGen training set [Kr- ishna al., 2022a], which the largest publicly available database of Al-generated text that we are aware of (15M generations each in four domains). Besides this, in Section 5 we also conducted experiments on a the ShareGPT corpus with 47K ChatGiP-ggeneatted responses. We note that extremely expensive and time consuming to create corpus of Al-generated text from scratch: at cost of $0.001 per 500-word response, collecting a billion ChatGPT outputs would cost $ SIM and take long time to collect due to rate limits. Hence, billion-scale experiment is likely only possible with an LLM provider's private database.
One of the concerns with larger database is that of semantic collisions: database will saturate with entries having similar semantics, especially for popular topics, and subsequently harm detection at scale. However, we note that:
1. Like other detectors, retrieval works best on longer sequences (Figure 6b). Long generations exponentially increase the likelihood of semantic divergences between pairs of entries.
16 hhtpps: wttttteerrrcccttteeeeeeeeeeeeeee http:///ww...ooooeeeeee//////tte
19

---

2. Retrieval compares the candidate response against the top-1 entry in the database, not the top-k. For false-positive candidates on popular topics, the top-k entries together are more likely to cover imput semanties (recall) rather than top-1 (precision).
3. The most effective retrievers use a combination of neural semantic encoders and token overlap scores [Thakur et al., 2021]. We also see this our experiments (Section 5), BM25 beats P-SP at detection. BM25 is not purely semantic driven: it uses TF-IDF token overlap.
4. The retrieval accuracy for unperturbed Al-generated text is always 100%, just like exact match searches in a modern search engine. Retrieval -based detection is also effective on substrings of unperturbed text (as shown Appendix G.2).
Overall, we are optimistic about our scaling plots (Figure 6a), and see just a 0.8% drop moving from IM to 10M database (PG19-BM25). We emphasize that BM25 is basic retriever, and is not optimized on our task. Information retrieval literature has many powerful retrievers and has shown success billion-scale corpus sizes [Gomes et al., 2013, Lakshman et al., 2021]. Google Search currently effectively indexes over 100B webpages [Google, 2023]. We have also suggested a dense retrieval mechanism Appendix A.3 which can be optimized on the underlying retrieval corpus.
Retrieval can easily be used in tandem with other detectors like watermarking. Finally, we note that our paper is the first proof-of-concep that shows retrieval-based detector could work, and anticipate future work to build upon it.
A.2 Limitations of retrieval for detection
While retrieval over previously-generate sequences is an effective defense against paraphrase attacks, also suffers from key limitations, some of which apply broadly to all existing detectors. We discuss these limitations below and discuss possible solutions:
Detection specific an API. Unlike other ceneral-purposs Al detection algorithms e.g. Ope- nAl's classifier [OpenAI, 2023a], retrieval can only detect generations from the API over which the database built. API #1 has no access the database of generations from API #2, and thus will not be able to detect generations produced by API #2.
2. Retrieval is limited to closed-source LLMs. Users of open- LLMs like LLAMA [Touvron et al., 2023] can freely generate outputs without the outputs being stored a central database. However, currently most major LLM providers operate their LLMs behind closed APIs. It is also important to note that watermarking [Kirchenbauer et al., 2023a]. the most promising alternative to retrieval, also has this limitation. Since watermarks are added during decoding rather than into the model weights, users of open L.LMs are free to generate text without watermarks. While other alternatives like DetectGPT [Mitchell et al., 2023] classifiers do not suffer from this issue, show that they either have low accuracy, or are extremely vulnerable to paraphrasing (Section 4.3).
3. The API provider needs to provide retrieval infrastructure. After the release of Chat- GPT [Schulman et al., 2022], Al chatbots are getting widespread adoption. At conservative rate of 5M queries a day, the database will have almost two billion entries in a year. Complex retrieval infrastructure (like modern search engines) will be necessary to retrieve over these large databases with low latency.
4. False positives due to training data memorization. Language models have been shown to memorize sequences verbatim from their training data [Carlini et al., 2021], such as the Gettysburg Address [Radford al., 2019]. Despite being originally written by humans, these sequences will be classified as moner--gneerate by our detector. To tackle this issue, we suggest API providers additionally perform retrieval over the training data used to train the model. If sequence is found in the training set as well as the generation database, it is likely to be an instance of training set memorization.
Privacy concerns. Providing a retrieval detection service partially exposes the database of previ- ously generated text for all users. This raises concerns of membership inference attacks [Shokri et al., 2017] on private user data which may appear the generated text (if present in user prompt). To mitigate this, we suggest: (1) the detection service should be provided only to trusted users under an agreement to not misuse the system, such as college teachers trying to detect cheating: (2) users should be encouraged not to enter any sensitive private data in their prompts to APIs,
20

---

practice already followed by ChatGPTI; (3) API providers only provide a binary output from this detector (Al-generated or not), rather than actual search results; (4) API providers rate-limit queries from IP addresses; and (5) differential privacy mechanisms or scrubbing private attributes to make it difficult/ / impossible to reconstruct the user prompt from just detector access.
Slight reduction in accuracy with large databases. As we observed in Section 5.3, the accuracy of detecting paraphrased text slightly degrades the database of retrievals gets larger. However, we found this decrease to be quite small (only 1 % on PG19 scaling 1M generations to 15M), despite using fairly primitive retrievers like BM25. Moreover, unperturbed Al-generated text will always be detected with 100% accuracy using our method, irrespective of corpus size. We discuss this topic more in Appendix A.1 under "Accuracy on larger databases".
Tasks with constrained output space or short outputs. Similar all other detection algorithms, it may be hard or even impossible to distinguish Al-generated outputs for tasks with a constrained output space (like sentence-level translation, classification) Section 5.3). Thus, we believe the main utility of AI Al-generated text detection is for longer-form generated text, and hence we focus on tasks like long-form QA and open-ended text generation with relatively lengthy outputs. Note that to avoid detection, sophisticated attacker may try to generate long-form text in smaller chunks using multiple API calls, where each newly-generatee chunk is incrementally concatenated to the prompt. This is not a concern for our method if retrieval is done over the corpus of prompts concatenated with generations.
very short outputs (as shown in
Iterative attacks with access to detector. Another concern is that attackers with access to detection algorithms will iteratively modify their perturbations until they avoid detection (as shown by Sadasivan et [2023]). While this a valid concern for all detectors, we believe retrieval has an important advantage over the alternatives. Since the corpus of previously- gennerate text is proprietary, only the API provider can provide access to this detection service- it impossible for attackers to locally reproduce this detector. This allows API providers to adopt several mitigation strategies such as (1) rate-limiting queries to avoid iterative attacks; (2) providing retrieval access only to verified users (e.g., teachers); and (3) detecting possible iterative attacks by analyzing previously queries to the retriever.
Lack of formal guarantees between threshold T, response length and detection rate. Unlike watermarking. we believe it is harder to establish formal relationships between the chosen threshold T, response length, and the corresponding TPR / FPR rates. Similar to DetectGPT / classifiers, we believe the threshold needs to be estimated empirically on the underlying corpus, retriever and candidate response distribution. A formal relationship between FPR and the threshold (like in watermarking) may be possible using information about the density of the retrieval database in the semantic vector space. We leave this exploration for future work.
A.3 Ideas to make retrieval detection work well at an even larger scale
In Section 5.3, we observed that our proposed retrieval detector effective even with large corpus of 15M previously-generated sequences. While we do not have access to a larger corpus of generations (billion-scale), in this section we describe some ideas to improve retrieval detection such a scale.
1. Timestamp filtering in retrieval corpus. To reduce the large search space, the detector interface could provide users with an option to restrict retrieval to only fixed time period during which the text was likely be generated. For instance, a common use-case of Al-generated text detection might be when teachers attempt to catch plagiarism in college essays. Teachers could restrict retrieval to only those generations created during the assignment window.
More sophisticated retrieval strategies. In our work, we only explore simple retrieval strategies like BM25. However, several more sophisticated retrieval strategies exist, which are known to boost performance [Thakur et al., 2021] and could be useful here. These include methods like re-ranking of top-k retrievals [Khattab and Zaharia, 2020] or dense retrieval (Karpukhin et al., 2020]. We note that these more complex methods are also slower, and latency is likely to be pressing concern for API providers.
3. Fine-tuning dense retrievers for the detection task. The retrievers in our work are not fine- tuned for the task of Al-generated text detection. However, we hypothesize that fine-tuning
17 "nttps: tetttas//chat..ppnate www.:hhaat.ppnni.cooo
21

---

retrievers this task can help retrievers adapt better to the retrieval corpus and detection task. Specifically, a contrastive learning approach could be adopted here: positive pairs are paraphrased or otherwise noised sequences paired with their generations, while negative pairs are human- written continuations paired with the machine-generated text.
A.4 Incentives for LLM providers to implement retrieval-based detection
Finally, in this section we discuss the incentive structures for LLM API providers, and why they might want to implement retrieval-based detection.
1.
There is substantial push from both the US and European governments to regulate companies to make their Al-generated text/images detectable. For instance, several major LLM providers made voluntary commitments to watermark their Al-generated content [Reuters, 2023a]. Sim- ilarly, the European government recently pushed for rulings to make Al-generated content de- tectable [Reuters, 2023b].
2.
In hypothetical scenario where there is lawsuit about the origin of some malicious Al-generated content, maintaining a database of previously generated responses could be a reliable method to prove innocence, given its strong performance competing Al A--genrrated text detectors. Keeping this dataset private, also protects LLM providers from the privacy risks of retrieval- based detection (Appendix A.2).
3. Major LLM providers are already storing their model-generated outputs to monitor abuse, as well as possibly help improve their products with RLHF preference-based training. For instance, OpenAl's ChatGPT stores the history for month even if users choose to opt-out [OpenAI, 2023b]. Hence, implementing a retrieval- based detection service on top of this database does not entail a resouree overhead in terms of storage, and reduces the engineering effort needed to implement this detector.
Is retrieval-based detection at odds with GDPR's right-to-bb--fogotteen One of the hurdles for LLM providers to implement retrieval-based detection is GDPR's right to be forgotten," 18 which allows attackers to request their data to be deleted by the L.M provider in order to avoid detection. However, we believe that Al-generated text detection is a sufficient cause to temporarily override GDPR deletion requests. We believe that Al-generated text detection could fall under the following GDPR guidelines for overrides: (1) "freedom of expression and information".
(2) "establishment of legal defense or in the exercise of other legal claims", and possibly (3) "comply with a legal ruling or obligation" in the future. As example of this, while OpenAl allows users to delete their chat history [OpenAl, 2023b], they retain for 30 days, and can review required to monitor for abuse.
B Ethical Considerations
Our goal in this paper is not to provide recipe for potential attackers (e.g., college students wishing to use ChatGPT their essays) to evade Al text detection systems. Rather, we wish to bring awareness to the wider community about the vulnerabilities of current Al-generated text detectors simple paraphrase attacks. These detectors are not useful in their current state given how easy they are to evade. We encourage the research community stress test their detectors against paraphrases, and to develop new detectors which are robust against these attacks. To facilitate such research, we open source our paraphraser and associated data code.
Furthermore, we propose not just an attack but also a potentially strong defense against this attack. Our detection strategy is simple, relying retrieval over corpus of previously genneraed sequences. We empirically show that such a detection algorithm could work at scale and provide extensive discussion on possible methods to improve performance (Appendix A.3), well as discussing possible limitations and approaches to tackling them (Appendix A.2). We hope that retrieval- based Al-generated text detectors rapidly improve and are eventually deployed in conjunction with other detection methods like watermarking classifiers.
rigght-to--e--ooootte
22

---

C Experiments measuring intrinsic paraphrase generation quality
Our experiments in Section 4 and Section 5 focused on attacking Al-generated text detectors with paraphrases and defending against these paraphrase attacks. We used DIPPER as the underlying paraphrase generation model for all of these experiments. Are DIPPER's s paraphrases actually good enough to make the attack worthwhile, and can simpler paraphrasers be just as effective as DIPPER? In this section, we conduct careful ablation experiments (Appendix C.1) and human evaluations (Appendix C.2) to validate the effectiveness of DIPPER at preserving the semantics of the input generation. Our results show that DIPPER effectively leverages surrounding context to paraphrase multiple sentences while preserving input semantics.
C.1 Ablation studies on DIPPER
In this section, we perform automatic evaluations confirm the efficacy of DIPPER a paraphraser. From a survey of existing paraphrasers that we carry out in Appendix D.I, DIPPER possess two unique features that differentiate it from other paraphrasers: (1) its ability to leverage context from outside of the text to be paraphrased (such as the prompt); and (2) its ability to paraphrase multiple sentences at once. How useful are these features while paraphrasing long sequences of text?
To answer this question, we first train an ablated version of DIPPER by constructing a training dataset (Section 3) without any left or right context, and then fine-tuning T5-XXL using the same hyperparameters as in Section 3. We call this model DIOPER-nO-Cix. We paraphrase IK open- ended generations from GPT2-XL using both DIPPER and DIPPER-nO-ctx, using each of the four configurations diversity control codes studied in this paper. We then evaluate the quality of the perplexity [Brown et al., 2020] of the prompt concatenated with the paraphrased continuation; (2) RANKGEN compatibility between the prompt and the paraphrased continuation [Krishna et al., 2022a]; and (3) unigram token overlap between the paraphrased continuation and the prompt.
paraphrased text using three metrics: (1)
Contextual paraphrasing leads to higher quality paraphrases. In Table 3 (Experiment 1), we observe that across all four control code configurations and all three metrics, paraphrases from DIPPER are preferred over paraphrases from DIPPER-nO-Cte Specifically, with the lexical and order control codes set to 60% (most diverse), DIPPER paraphrases are preferred by GPT3.5 perplexity 71% of the time compared to non-contextual paraphrases (average perplexity drop of 12.9 vs 14.2).
Paraphrasing multiple sentences at time is better than paraphrasing individual sentences. Next, we use our DIPPER-NO-ee no-ctx model to compare two settings: paraphrasing 3 sentences at a time vs paraphrasing sentence at time before concatenating. We hypothesize that the former will produce higher quality paraphrases since we expect it to better connect discourse elements across the text. Indeed, in Table (Experiment 2) across all control codes, GPT3.5 and RANKGEN usually prefer multi-sentence paraphrases over the single-sentence baseline. This preference is 71 % or higher for all control codes when evaluating with GPT-3.5 perplexity, reaching 83% for L60,060.
DIPPER paraphrases are close to the unperturbed GPT-2 XL generations. Finally, we compare DIPPER with the original GPT2-XL -XL generations (without paraphrasing) on the same three metrics. While we expect metrics to prefer non- pana-paaaaaaed text, strong paraphraser will produce text that close to the original in terms of these metrics. Table 3 (Experiment 3) confirms our hypothesis: at L20, RANKGEN has 50-50 preference between the two outputs, while GPT3.5 prefers the non-paraphrased generations just 61% of the time, with an average perplexity gain of just 0.4 (11.1 to 11.5). At more diverse control codes, preference for GPT2-XI. generations does go up (58% RANKGEN, 73% GPT3.5 for L60), but absolute scores continue to be close (11.1 vs 12.3 GPT-3.5 perplexity). Note that while all of these ablations use just single paraphrase sample, it is easy for an attacker to obtain multiple samples from DIPPER and choose the sample that maximizes these metrics (as discussed in Section 4.3).
C.2 Human evaluation of semantic preservation using DIPPER
The automati semantic similarity scores in Table and indicate that DIPPER generates paraphrases that are faithful to the original input paragraphs. To confirm this result with human evaluation,
23

---

Table 3: Ablation experiments demonstrate the high quality of DIPPER's paraphrases compared to alternatives. Displayed scores are the percentage of cases in which rewrite A is preferred over B by one of the three metrics, with subscripts showing absolute average scores on each metric across the dataset. Overall, DIPPER benefits from context outside the input (Experiment I), multi-sentence paraphrasing (Experiment 2), and not too far behind an-pparaphraed text in terms of quality (Experiment 3).
Open-ended generation with GPT2-XI. on Wikipedia prompts
RANKGEN-XL XL
GPT3.5 403 perplexity
unigram overlap with prompt
Control
rewrite
rewrite
rewrite
rewrite
rewrite A
rewrite B
Experiment 1: Is context helpful for paraphrasing?
rewrite = DIPPER with context
rewrite B = DIPPER no context
20I.
62% 10.2
38%
58%
42%
55% 41.3
45% 40.7
40L
62%
38% 8.7
64% 11.9
36%
57% 40.7
43% 39.8
60L.
64% 9.6
36% 7.9
66% 12.3
34% 13.3
55% 39.9
45% 39.2
60L,600
66% 8.3
34% 6.3
71% 12.9
29% 14.2
56% 39.4
44% 38.4
Experiment 2: Is it helpful to paraphrase multiple sentences at a time?
rewrite DIPPER 3 sentenees at time
rewrite DIPPER sentence at time
20L.
55% 9.4
45% 8.9
73% 11.7
27% 12.5
51% 40.7
49% 40.7
40L
55% 8.7
45% 8.5
71% 12.5
29% 13.3
46% 39.8
54% 40.3
60L
53%
47% 7.6
71%
29%
46% 39.2
54% 39.8
60L,600
57%
43% 5.3
83%
17%
47%
53%
Experiment 3: Doey paraphrasing preserve the quality of the original text?
rewrite no paraphrasing rewrite = DIPPER
20L
50% 10.4
50%
61%
39%
51% 41.6
49%
40L
57% 10.4
43%
9.8
67%
11.1
33% 11.9
55% 41.6
45%
40.7
60L.
58% 10.4
42%
9.6
73%
11.1
27%
12.3
58% 41.6
42% 39.9
60L,600
68%
10.4
32%
8.3
79%
11.1
21% 12.9
61%
41.6
39% 39.4
hire three native English teachers and/or editors on Upwork!! to evaluate the semantic fidelity of the paraphrases. As human evaluation is expensive, we fix the order diversity (0) to be 0 and focus on the impact of the lexical diversity. We evaluate paraphrases with the lexical codes L200, L40, and L60, corresponding to moderate, medium, and high lexical diversity. Twenty paraphrases are sampled randomly each lexical code, resulting in 60 original text and paraphrase pairs.
The evaluation is conducted on the platform Label Studio [Tkachenko et al., 2020-2022). As shown in the interface our annotation platform Figure the text to be paraphrased (highlighted in yellow) are preceded by its context. The annotators see the same amount of text as DIPPER. They need to first read the texts, select one point on the Likert scale, then provide free-form comments justifying their ratings. We estimated that the evaluation of each paraphrase takes 1.5 to 2 minutes. As such, we pay $15 as base rate with a bonus for the reasonable extra time that the annotators spend on the tasks.
Among the 60 original text and paraphrase pairs, the three annotators agreed on their choice 28.3% of the time, and 60% of the time the point they chose the scale differs by 1. Table reports how often each point on the Likert scale is chosen. Over 80% of the time, our annotators rate DIPPER's paraphrases nearly equivalent (4 out of 5) approximately equivalent out of 5).
qualitative analysis of the free-form annotator comments reveals systemic strengths and shortcom- ings of DIPPER. Table 8 provides two representative examples for each lexical code that is evaluated in our human study.
19 Phttps://wer wwwork com 20 Phttps:///ii hhttpss//Labelstud..t
24

---

Given the source text:
She was only hit by single 12-inch shell that wounded two crewmen. Both guns in her aft 12-inch pun turret, however, were disabled by shells that desonated prematurely in their barrels. Most of the other damage the HIMS New Zealand sustained
from shrupnel splinters. All all, the ship estimated have been strack up to twenty-five five shells, most which were smaller than 12-inch. She also sustained damage to her superstructure, masts, and rigging.
Here is paraphrase of the highlighted text:
The majority the damage HMS New Zealand was caused by shrapnel and fragments. all, the ship was thought to have been hit by as many as twenty-fit five shells, most of them smaller than 12 inches. She was also hit in her superstructure,
rigging.
Which of the following best describes the quality of the paraphrase?
the meaning the but differs words and/or structure. (t) 4 Nearly equivalent: the paraphrase preserves most information the source but differs in some minor factual details, PI
Approximately equivalent paraphrase
3 D--omewhat equivalent the paraphrase preserves some information in the source but differs in certain significant ways.
2- Topicallyy related: the paraphrase is topicalyy related to the source but most information the source not preserved
1-Not topically related: the paraphrase is not topically related to the source and preserves no information.
Please motivate your choice 2 3 sentences.
Add
Figure The interface of the annotation platform used in our human study
Table 4: This table shows how often each point in the Likert scale was chosen by annotators for the pairs of original and paraphrased texts. Twenty text pairs are randomly selected for each lexical code (L). 81.8% of the time, our model DIPPER provides a paraphrase which is nearly equivalent to the input in terms of semantic meaning.
4
3
2
L
Sum of 4 and
Approx. equivalent
Nearly equivalent
Somewhat equivalent
Topically related
20
95.0%
63.3%
31.7%
5.0%
0.0%
40
78.3%
45.0%
33.3%
21.7%
0.0%
60
70.0%
28.3%
41.7%
28.3%
1.7%
Total
81.1%
45.6%
35.6%
18.3%
0.6%
Strengths First, the third example in Table 8 exemplifies DIPPER's ability to leverage information from context to increase diversity while maintaining coherence (i.c., from line... reference the song 's title to reference to "I'm the Greatest"). The same is observed in row 2 where DIPPER uses the context to interchange he and Churchill. A paraphrase model without looking into context will have great difficulty in doing this and no prior paraphraser (see Table 5 for a list) is capable of that. Second, the example in the fifth row highlights DIPPER's 's ability to make significant changes to original texts with a high lexical diversity code (L60) (see the color coding) while preserving their semantie meaning as rated by the annotators.
Qualitative shortcomings: The first shortcoming that, when the original text contains new created proper names (unlike common people and country names), such as the ones in row 6 (Homing Attack and Slide Attack), high lexical code has tendency change such nouns, leading to the result that one of our annotators deems it to be only topically related to the original. However, this shortcoming can be overcome by decreasing the lexical code, which user can choose from continuous range (from 0 to 100). For instance, in row 1 with lex=20, the songs' names M's 's Confession and Gone Fishing are kept intact. Another shortcoming that DIPPER occasionally omits content from an original text. While in some cases such removal is acceptable (see row 6). in other cases it causes significant change the meaning the text (see row 4). However, the former case can be overcome by paraphrasing shorter paragraph at time.
25

---

Overall, the human study shows that DIPPER performs well at preserving the semantic meaning of original texts while introducing both semantic and syntactic diversity. Because DIPPER provides user-friendly controllabilty of output diversity, user can adjust the control code to find the most suitable paraphrase for their need.
D
Related work for discourse paraphrasing
D.1 Survey of paraphrase generation papers
As an important NLP task, paraphrasing has attracted much attention. Many models have been proposed to improve the quality of paraphrases. To position our model DIPPER and highlight its strengths, we conduct a survey of paraphrase generation papers from 2018 to 2022 (Table 5) and focus on the following four aspects:
1. Whether model can paraphrase paragraph at once,
2. whether model can merge split consecutive sentences when appropriate,
3. whether model leverages context surrounding input sentence when paraphrasing,
4. whether model provides control knobs for users to customize the output diversity.
The survey shows that only three out of 25 papers mentioned that their model can paraphrase more than one sentence (but not necessarily at once). None of them enables their model to merge or split sentences when paraphrasing. No model uses information from context surrounding an input sentence during inference time. Finally, 14 papers offer ways for users to customize the diversity of paraphrases. However, most diversity control methods such as constituency parses or exemplars may not be straightforward and intuitive to end-users users the scalar control knobs DIPPER.
In contrast to the papers in the survey, DIPPER nicely combines all desiderata into one model and offers intuitive control knobs for lexical and syntactic diversity. Automatic and human evaluation show that DIPPER can efficiently leverage context information and reorganize sentences while having high fidelity in meaning (Appendix C).
D.2 Other related work
In this section we discuss a few additional less related papers which were not included in our survey in Appendix D.1. Our discourse paraphraser is closely related work contextual machine translation, where source/target context is used to improve sentence-level machine translation [House, 2006, Jean al., 2017, Wang al., 2017, Tiedemann and Scherrer, 2017, Kuang et al., 2018, Agrawal et al., 2018, 8, Miculicich et al., 2018, Zhang et al., 2018, Xiong et al., 2019, Jean et al., 2019, Voita al., 2019a, Yin al., 2021, Mansimov al., 2021]. Prior work has shown that context helps with anaphora resolution [Voita et al., 2018], deixis, ellipsis, and lexical cohesion [Voita et al., 2019b). Efforts make paraphrase generation more contextual have been quite limited. A few efforts have attempted to use sentence level context to paraphrase phrases [Connor and Roth, 2007, Max, 2009], and dialogue context to paraphrase individual dialogues in a chat [Garg et al., 2021].
Our work also related to efforts in text simplification to go beyond a sentence, by collecting relevant datasets [Xu et al., 2015, Devaraj al., 2021] and building unsupervised algorithms [Laban et al., 2021]. Note that our work focuses on a general-purpose paraphrasing algorithm and is not tied to any particular style, but could be utilized for document-level style transfer using techniques like Krishna et al. [2020, 2022b]. Similar efforts have also been undertaken in machine transla- tion, [Popescu-Belis Belis et al., 2019, Junezys-Dowmunt, 2019, Maruf et al., 2021], attempting translate paraggaphssdoocumennt at once.
E More background on detectors of A Al-generated text
In this section, we provide an overview of existing algorithms that have been developed for the purpose of detecting machine-generated text. Such algorithms fall into three main categories: (1) watermarking algorithms, which modify the generative algorithm to encode hidden information unique the API (Appendix E.1); (2) statistical outlier detection methods, which do not modify
26

---

Table 5: The table shows the result of our survey of paraphrase generation papers from 2018 to 2022. We focus on four aspects: (1) whether model can paraphrase multiple sentences at once, (2) whether model is able merge or split an input sentence when appropriate, (3) whether model takes context surrounding the input sentence into consideration when paraphrasing, and (4) whether a model enables users to control the semantic and syntactic diversity of paraphrases. !Granularity levels word, phrase, and sentence. "Meng et al. [2021] use context for their dataset construction, but do not leverage during training/inference. 'The diversity score is combination of the unigram Jaccard distance and the relative position change for unigrams. "The code is represented by three dimensional vector corresponding to semantic similarity as well as syntactic and lexical distances between the input and output sentences.
Paper
Multi-senience
Merge Splits
Contextual
Diversity Control
Tyyer al. [2018]
X
Constituency parse
Li et al. [2018]
X
Roy and Grangier [2019]
Witteveen and Andrews [2019]
?
Kumar et al. [2019]
X
X
X
Hu et al. [2019]
X
Decoding constraints
Chen al. [2019]
X
Exemplar
Li et al. [2019]
X
X
Granularity control
Goyal and Durrett [2020]
X
X
Exemplar
Lewis al. [2020]
?
X
Thompson and Post [2020]
7--gram overlup Exemplar
Kumar ct al. [2020]
X
Kazemnejad ct al. [2020]
?
X
Krishna et al. [2020]
X
Rajauria [2020]
Meng et al. [2021]
X
X
Diversity score
Huang and Chang [2021]
Constituency parse
Lin al. [2021]
X
X
Goutham [2021]
Damodaran [2021]
Binary T--gram
Dopierre et al. [2021]
Bandel al. [2022]
Control code
Hosking et [2022]
Syntactic sketch
Yang et al. [2022]
X
Examplar + Keywords
Xie et al. [2022]
X
X
X
DIPPER (ours)
the generative algorithm but look for inherent artifacts in generated text (Appendix F.2); and (3) classifiers trained to discriminate machine- machine-generated text from human-written text (Appendix E.3). Finally, in Appendix E.4, compare and contrast our work to Sadasivan et al. [2023]. who also note the efficacy of paraphrasing attacks but do not consider retrieval-based defense their pessimistic conclusion about the fate of Al-generated text detection.
E.1 Watermarking language model outputs
A "watermark" is modification to the generated text that can be detected by statistical algorithm while remaining imperceptible human readers. Effective watermarks are difficult to remove and have little effect on the quality of generated text. Prior work attempted to watermark natural language using syntax tree manipulations [TTppaara al., 2005, Meral al., 2009], and this area has gotten renewed interest with large language models generating human-like text [Abdelnabi and Fritz, 2021, Grinbaum and Adomaitis, 2022]. Most recently, Kirchenbauer et al. [2023a] propose simple algorithm that only requires access to the LLM's logits at each time step to add watermarks. The watermark can then be verified with only blackbox access to the I.M and knowledge of specific hash function. This algorithm operates in three steps:
27

---

1. Mark random subset of the vocabulary "green tokens" (or tokens representing the water- mark, as shown in Figure 1) using the hash of the previously generated token as a random seed. A total of >/V| tokens are marked green where y the fraction of the tokens that are watermarked with default = 0.5.
2. Increase the logit value for every green token by a constant ō (= 2 by default), which denotes the watermark strength. This raises the probability of sampling green watermarked tokens, especially for high-entropy distributions.
3. Sample sequences using decoding algorithms such as nucleus sampling [Holtzman et al., 2020], leveraging the modified probability distribution each timestep before truncation.
Detecting the watermark: Verifying whether text is generated by a watermarked LM is possible with just knowledge of the hash function and tokemizer. Specifically. the verifier tokenizes the text and counts the number of green tokens it contains. This is used to calculate the standard normal score (z-score) for the hypothesis test. If the sequence with T tokens contains a certain number of the green token (denoted as | ) the 2-score can be computed by:
= = (Islc /T) (TT//T>>11 (T
Intuitively, higher z-score implies it is less likely for a human to have written the text (null hypothesis) since contains a higher than expected number of green tokens. Kirchenbauer al. [2023a] recommend using a high- value (z > 4, or p < 3 x 10-5) to reduce the risk of false positives Houman-writted text classified as Al-generated). Low false positive rates are critical in Al-generated text detection algorithms [OpenAI, 2023a] we discuss this in Section 4.1.
E.2 Statistical outlier detection methods
Unlike the watermarking algorithms, outlier detection algorithms make no modification to the generative algorithm. Instead, they attempt to distinguish between human-writter and machine- generated text based on the presence of artifacts in generated text [See et al., 2019, Holtzman et al., 2020]. Early methods detect statistical irregularities in measures such as entropy [Lavergne et al., 2008), perplexity [Beresneva, 2016), and n-gram frequencies [Grechnikov et al., 2009, Badaskar et al., 2008]. After the release of GPT-2, Gehrmann al. [2019] introduced the GLTR visualization tool to assist human verifiers in detecting machine-genen generated text. Most recently, the release of ChatGPT has prompted the development of two new tools, namely a closed-source tool called GPTZero [TTan, 2023], and open-source DetectGPT [Mitchell et al., 2023]. DetectGPT uses an observation that model godel-generated text lies the negative curvature regions of the model's 's log probability function. constructs multiple perturbations of the model generated text (using mask-and-fill strategy), and compares the log probability of the perturbations with the unperturbed generation. Text is considered model generated if the log probability of the unperturbed text is significantly higher than the log probability of perturbations.
E.3
Classifiers
The third class of detection methods relies on classifiers that are fine-tuned distinguish human- written text from machine-gent -eeiereeee text. Early efforts in this vein use classifiers to detect fake reviews [Hovy, 2016] and fake news [Zellers al., 2019]. Other related studies examine classification performance across domains [Bakhtin et al., 2019] and decoding strategies [lppolito et al., 2020]. Such studies inspired others use their insights improve generative performance [Deng al., 2020, Krishna et al., 2022a]. Most recently, OpenAl fine-tuned a GPT model to perform this discrimination task and released it web interface [OpenAI, 2023a]. They fine-tuned this classifier using generations from 34 language models, with text sourced from Wikipedia, WebText [Radford al., 2019], and their internal human demonstration data.
E.4 Comparison to Sadasivan al. (2023)
In very recent concurrent work, Sadasivan et al. [2023] also demonstrate the utility of paraphrasing attacks against AI Al-generated text detectors. While their work makes use of off-the-shelf sentence- level paraphrase models, DIPPER possesses advanced discourse-level rewriting capabilities well
28

---

fine-grained diversity control, which allows thoroughly analyze the effectiveness of various paraphrasing strategies. Our experiments also encompass more tasks, datasets, and detection algo- rithms. Moreover, we evaluate larger language models like GPT3.5-davinci-005 Finally and most importantly, our setrieval-based defense directly contradicts the "impossibility result" of Sadasivan al. [2023] and its associated proof, which states that even an optimal detector will approach the performance of a random classifier as the distance between the distributions of LLM-generated text and human generated text goes to zero. Since our detector does not rely on properties of the text but rather a corpus search, the quality of the generated text is irrelevant to the effectiveness of our detector, and thus their proof does not apply to our method.
F
More experimental details of our attack experiments
E1
Details for training our paraphraser DIPPER
Our paraphraser DIPPER is a eqquenceto-sequenec Transformer neural network [Vaswani et al., 2017]. initialized with the T5-XXL. 1.1 checkpoint [Raffel et al., 2020] and fine-tuned our paraphrase generation data, using early stopping on validation loss for held-out novels. During training, we find it helpful paraphrase maximum of 3 consecutive sentences time, which leads to better adherence to control codes. Our models are implemented in JAX [Bradbury et al., 2018] using the T5X library [Roberts et al., 2022] with the default fine-tuning hyperparameters. Our final dataset contains 6.3M paraphrase pairs. Training was done on 64 cloud TPUv3 chips, and took 6-12 hours to complete. At inference time, we use nucleus sampling [Holtzman al., 2020] with p = 0.75 and variety of control codes.
To make paper more intuitive, we have slightly modified the notation that our actual pretrained
model uses. Our pretrained model uses control codes 100- - L. and 100 - O, denoting lexical/order similarity rather than diversity. Also, <sent> is used instead of <p>. We will clearly document this in the code release.
F.2 Long-form question answering data processing
In Section 4 evaluate long-form question answering [Fan et al., 2019], in which an LM must answer how/why question (e.g., Why are almost all boats painted white?) with a 250-350 word answer. To build a long-form question answering dataset, we scrape questions from the r/explainlikeimfive subreddit posted between July December 2021.21 21 We randomly sample 500 questions from each of six popular domains on the subreddit (biology. physics, chemistry, economics, law, and technology) and pair each question with its longest human-written answer, which yields 3K long-form QA pairs.
F.3 Are successful attacks against model-specific detectors really an attack success?
In our experiments, attack two model-specific detectors (watermarking, DetectGPT) in addition to model-agnostic detectors (OpenAl classifier, GPTZero, RankGen). Model-specific detectors have been specifically designed judge whether response was generated by particular model. After paraphrasing, a response is not truly generated by that specific model anymore-it it has heen generated by the paraphrasing model. Hence, in sense, the inability of these detectors to detect paraphrased text denotes their robustness in model-specific detection.
While the argument above has merit, we argue that it is critical for model-specific detectors to be robust against paraphrasing attacks. Given their strong performance over model-agnostic methods (Section 4.3), and the large risk of perturbation attacks thuman-edited or nutomatically-edited we think it is important for model-specific detectors to bake perturbation robustness in their design for better downstream usability. Currently, the low performance of model-agnostic detectors (Sec- tion 4.3) makes them quite unusable, and OpenAl even took down their text classifier due to low performance [OpenAI, 2023a]. Currently, model-specific detectors (including watermarking and retrieval) seem to be the only reliable path towards robust Al-generated text detection.
21 We choose this period since current language models have been trained on internet data available before June 2021 [OpenAI, 2022], this prevents verbatim copying from training data.
29

---

G More retrieval-based detection experiments
G.1 Controlled comparisons of retrieval with other Al-generated text detectors on open-ended text generation
We conduct a controlled comparisons of retrieval on the open-ended text generation task with Wikipedia prompts (see Section 5.2). The result of the experiment is presented in Table 7.
G.2 Robustness against text shortening and mixing attacks
Our experiments in Section assumed that an attacker would paraphrase the entire LLM output to evade detection. However, alternative attack strategy that might be adopted in practice text shortening, or text mixing [Kirchenbauer et al., 2023b]. Here, attackers only use a substring of an LLM generated output (instead of the full output), and optionally mix it with text from other sources (like other LLM-ggenraaed outputs or human- human-writer written text).
In this section, we conduct some preliminary experiments to show the robustness of retrieval- -based detection to text shortening and mixing attacks. We adopt the same experimental setup as Section 5.3, utilizing BM25 retrieval on PG19 machine-generated corpus of 2M responses.
In Table see that retrieval is a promising defense strategy even against text shortening and mixing attacks. Overall, we see that unperturbed random substrings (from single or multiple generations) can still be detected quite easily (86.2% 94.7% detection rate 1% FPR). However, adding DIPPER paraphrasing on top of that reduces accuracy (56.15 % to 38.8% detection rate).
Table 6: Preliminary experiments measuring the robustness of retrieval-based detection to LLM response shortening and mixing attacks. Overall, we find that retrieval strong detector for truncated well mixed responses. Experiments conducted with BM25 retrieval in the PG19 data setup with a retrieval corpus of size 2M responses (from Section 5.3). Results shown are true positive rates the 1 % FPR threshold.
Candidate Text
Retrieval detection rate (1% FPR)
unperturhed I.I.M response
100.0
DIPPER SoPPER-parpphraeee response
98.2
50% truncated DIPPER- SAPPER-aaraphraee response
72.6
50% truncated LLM response
86.2
50% LLM response 1 + 50% LLM response
94.7
50% DIPPER- paraphhasd response
68.8
50% pirpee-parappraseed response DIPPER-paraphrased response
56.1
H ROC curves at different FPR
See Figure 8.
30

---

Table 7: Our retrieval defense significantly improves Al- Al-generated text detection accuracy (at 1% FPR) over baselines on all settings, including our most diverse paraphrase attacks (+60L and +60L,600).
Open-ended generation with Wikipedia prompts (300 generated tokens)
GPT2-XI.
OPT-13B
GPT-3.5 (davinci-003)
Original
60I.
+ 601,,000
Original
601.
601,600
Original
601.
601,,600
Baseline methods:
Watermark
100.0
68.9
57.2
99.9
63.7
52.8
DetectGPT
70.3
8.7
4.6
14.3
0.8
0.3
2.0
0.5
0.0
OpenAl
21.6
13.3
14.8
11.3
9.1
10.0
30.0
15.6
15.6
(Ours) Retrieval over corpus generations from model itself, with retriever:
SP
100.0
86.4
81,5
100.0
84.4
77.7
100.0
65.9
49.5
BM25
100.0
99.0
98.0
100.0
97.2
95.3
100.0
58.8
37.4
(Ours) Retrieval over corpus generations pooled from three models, retriever:
SP
100.0
72.1
63.2
100.0
74.6
65.6
100.0
63.1
45.6
BM25
100.0
85.0
78.7
100.0
87.2
79.1
100.0
58.8
37.4
detectgpt: detectgtt
openai watemmark
random bm255 retrieval (op)
detectigs detectgot lpp)
random rutrieval Ipp)
opizero
optzero
watermark
optzaro (po)
watermark (sp)
optzero (po)
watermark (pp)
100
100
80
True Positive Rate
True Positive Rate
60
60
40
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.5
1.0
1.5
2.0
2.5
False Positive Rate (%)
False Positive Rate (%)
detecigpt
openai openal (op)
random sim retrieval loo) retrieval
detectgpt detectipt (po)
detectapt(ppp
openai (pol watemmak
retrieval (po) bmi25 rutrieval (po)
gptzero
gotzero
optzero lpp)
watermark (pp)
(pp)
watermark (pp)
100
80
True Positive Rate
True Positive Rate
60
40
20
0
20
40
60
100
False Positive Rate (%)
False Positive Rate (%)
Figure 8: ROC curves for text generated by GPT2-XL, before paraphrasing (solid lines) and after paraphrasing (dashed lines, pp). Different plots represent different clipping thresholds on the X-axis.
31

---

Table 8: Representative model outputs of each lexical code with Likert ratings and comments from the annotators. The texts in bold in the original texts are the context. Red words are the content being changed in the original text and green words are the changed content in the paraphrases.
Original
Paruphrase
Annofator Comment
20
The first, originally used as Renard's theme, is heard during the manmmoth "Antonov" cue the recording, and is written for plano. The second theme, used in the "Christmas in Turkey" track of The World Is Not Enough, is reused in the "Going Down Together" track. Both themes were later adapted by David Arnold and leatured in the film's is score. The first theme up- pears cue "M's Confession" and the second theme appears in the cue Gonn Fishing". "
Both themes were subsequenily udapted film
5 5-Approx. equivalent While the paraphrase worded in a more simplistic way than the original and has shontemed clauses, the infor- mation presented in hoth is the same.
David Amold and used score. The first theme is used in the "M's Confession" cue, and the second in the "Gome Fishing" cue.
20 Churchill argues that the United States insti- tuted blood quantum laws bused upon rules of descendancy I... I He has argued that the United States has used them strip Indigenous people of their rights, land and culture in order to benefit those in power. designed to create a divide among Indigenous pco- ple, making them easier to contml and manipulate.
He has argued that the United States has used them to strip indigenous people of their rights, land, and culture in order benefit those in power. Churchill also argues that these laws were designed to create divisions among indigenous people in order to make them casier to control and manipulate. He has also gued that this manipulation continues today, in that indigenous people are of- ten foreed dony their heritage in order to have access tn resources and oppor- tunities that are denied to because of their heritage.
A-Apprx. equivalent
The paraphrase retains all the information of the orig- inal. but uses synonymous language in several places. Alsa, the paraphrase re- places the pronoan "he" with "Churchill" and vice
contends
these laws were
Churchill has further argued that this munipulation continues today, Indigenous people are oflen forced to dony their heritage in order to have ac- cess to resources and opportunities that are denied them as a result their heritage.
40
tailoring his composition "I'm Greatest" for Starr to record on the 1973 al- bum Ringo, Lennon referenced the song title with the lines [...] Lennon's original line was changed to "Back off Boogaloo" to reference the song's is title, creating line that Starr sang finished version.
Lennon's is original reference "I'm Greatest" was changed to "Back off Boogaloo" create the line Starr sang on the finished recording.
4-Nearly equivalent
The original does not specify that the line in question was "I'm the Greatest." However. this can be inferred from the rest of the passage. Other- wise, the paraphrase and the original quite similar.
40
Some monuments were vandalized and some were moved to new locations. [...] In cases, the monuments were moved to new loca- tions. For example, the stairway of the Great Pyru- mid Cholala in Mexico was relocated in the 19th century and then returned its original location 20th century. Other monuments were stroyed during wars or religious conflicts, such as the Ruddhas of Bamiyan Afghanistan, which were destroyed by the Talihan in 2001.
In some cases, mecuments were moved new locations. The Great Pyramid of Cholula, in Mexico, was moved in the 19th century and then relured to its original place in the 20th. Others were destroyed during wars or religious controversies, such
3- 3-Somewhat equivalent Only one major change in this shart text, but I feel it's significant. The origi- nal text speaks of the mov- ing of pyramid's 'ss stairease, paraphrased text im- plics that the entire pyramid moved and then replaced. This massively changes the meaning of this text.
Buddiius Bamiyan in Afghanistan, which were destroyed by the Taliban 2001.
60 Heaving wildly, the Hunyani began to descend rapidly. I...] The crew was unable regain con- trol and the Honyani crashed into the ground at 17:14, killing all onboard. The crash was investi- gated by the Civil Aviation Authority of Zimbabwe and the investigation concluded that the crash was caused by combination pilot weather conditions. The pilot had failed to prop- crly assess the weather conditions before takcoff, instead relying on a forecast from private weather service. The weather conditions worsened as the plane and the pilots already latigued, they were unable to appropriately react to the changing conditions. The combination of these factors to the plane losing control and crashing.
The crew could not regain control, and 17:14 the Hunyani plunged into the ground and all those on board were killed. The accident was investigated by Civil Aviation Authority Zim- babwe, which found that the crash was caused by combination pilot error, fatigue and the weather. The pilots had not properly examined the weather be- fore take-off, and instead had relied on a forecast from a private meteorologi- service. The weather deteriorated during the flight, and the exhausted pi- lots unable deteriorating conditions.
5 S-Approx. equivalent Well chosen words make the altemate version different hut retain the original mean- ing well. only difference is that the paraphrased ver- sian does have the cluding sentence, but it does not alter the meaning in way as facts have already been included previously.
fatigue, and
properly to the
60
Sonie's 's main means attack is the Spin Attuck, in which he curls into a bull and rapidly spins his bodl,, damaging enemies and certain obsta- cles upon collision. jumping or by rolling on the ground. Sonic can also use variety of other moves, such the Hom- ing Artnck, Slide Attack, and Wall Jump. He also has access power ups like the Speed Shoes, In- vincibility, and the Super Poel Out, In some games. Sonic can also use special items or vehicles to tra- verse levels.
Also, the character other atlacks, such as the homing ut- tack, the slide attack, and the wall jump. addition, can acquire power ups such as the Speed Boost, Super Speed, and the invincibility meter, and cor- tain games, he may make use of special items vehicles to traverse the world.
variety of
2 Topically related In the second part the pura- phrase, the writer ignores the actual maves and skills of Sonic and invents some that
may be performed by
entirely different not mentioned in the original text. The method of perform- ing the atlack moves has also heen missed out.
32


Reviews:
[{"review_id": "qelcW6LXWM", "reviewer": "Reviewer_qaDF", "summary": "This paper first demonstrates the vulnerability of existing\nAI-generated text detectors to paraphrases, and then proposes a\nretrieval-based method to alleviate the issue.  For the first\nexperiment, the authors trained a paragraph-level paraphrase\ngeneration system, called DIPPER, by fine-tuning an existing\ntext-to-text model, i.e., T5-XXL.  To this end, the authors compiled a\ndataset comprising pairs of paragraphs with two indication parameters\nto control lexical diversity and content reordering.  The proposed\nretrieval-based detector assumes the LLM API providers to save all\ntheir generated contents and to offer an interface for access the\ndatabase.  The authors experimented with a corpus of 15 million such\nexamples, and shows that this approach is more robust to paraphrased\ntexts compared to existing detectors.\n", "questions": "- In l.127, $L$ is defined as \"unigram token overlap\" but which of $p$\n  and $q$ is used to determine the denominator? [harmonic mean]\n\n- Input sequences in l.132 and l.180 contains ordinary tokens:\n  \"lexical\", \"order\", \"=\", and \",\".  Did the authors use them without\n  escaping as \"<p>\" and \"</p>\" ?  If so, wouldn't it distort the\n  embeddings for these tokens? [not escaped, following convention.]\n\n- How many training instances were used for obtaining DIPPER?\n  [6.3 million pairs, which should be included in the main paper.]\n\n- I'd like to suggest to tidy the layout.  For instance, Tables 1, 2\n  and Figure 5 appear in the previous page of their first mention in\n  the main text; Figure 3 shows a table; the mention of Figure 6\n  appears before Figures 4 and 5; Figure 6 is embedded in an\n  irrelevant paragraph. [the authors promised to do so.]\n\n- l.264: $L$ has already been used in l.127.\n", "limitations": "Yes. The authors describe the limitations in Appendix B and ethical considerations in Appendix A.\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- A paragraph-level paraphraser with the recipe to build it.\n\n- Demonstration of the vulnerability of existing AI-generated text\n  detectors to paraphrases.\n\n- A simple but effective retrieval-based method for robust\n  AI-generated text detection.\n", "weaknesses": "- The underlying assumption of the proposed method is not likely to be\n  realistic.  We can call LLM API providers for their ethics, but\n  there is no means to enforce them to conform to this approach.\n  Beside the providers, nowadays there are also increasing number of\n  publicly available strong LLMs, and we cannot regulate personal uses\n  of LLMs and AI-generated texts.  I acknowledge the discussion in\n  Appendix B.1, but nothing is discussed in the main paper.\n  [Through rebuttal, the authors explained some obligations of LLM\n  providers and promised to include its summary in the main paper.]\n\n- Scalability of the proposed approach is not sufficiently evaluated.\n  ll.297-298 states \"a popular LLM API may serve millions of queries a\n  day\" and it is likely to be true, given ChatGPT acquired more than\n  100 million users in the first two months and more than 13 million\n  unique users a day.  Compared to this, the experiment in this paper\n  uses up to only 15 million AI-generated texts, it is unclear whether\n  the proposed method is feasible (memory and time in addition to\n  accuracy) with billions or trillions of such text.  I acknowledge\n  the discussion in Appendix B.2, but nothing is discussed in the main\n  paper.\n  [The response reports on 100 seconds per retrieval at the client side\n  but this is misleading since the data store and retrieval operations\n  must locate on the server (LLM provider) side.  However, I\n  understand that the speed would not matter given the strong facility\n  at LLM providers and high parallelizability of the retrieval task.\n  The experiment with 15 million instance sounds not realistic but I\n  understand that we should accepted this if the authors are not in\n  giant tech.]\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed and thoughtful feedback! We thank the reviewer for supporting our contributions on paragraph-level paraphrasing, attacking AI-generated text detectors, and introducing a retrieval-based defense mechanism.\n\nThe reviewer voiced concerns about the practicality of the proposed retrieval-based detection algorithm, some of which the reviewer noted were discussed in our Appendix (and which we will summarize in the main body of the next version). While we address these concerns below, we want to emphasize that retrieval-based detection is only one of our three contributions. Besides retrieval, our contributions include: 1) a novel paragraph-level paraphraser which we will open-source (data + models) for the community to use; 2) a thorough experimental study attacking five AI-generated detectors on outputs from three diverse large language models on two real-world tasks. We will fully open-source the code and datasets for reproducibility.\n\n>  We can call LLM API providers for their ethics, but there is no means to enforce them to conform to this approach (retrieval-based detection)\n\nWe believe that LLM API providers may actually be enforced or even incentivized to store their model-generated outputs:\n\n* There is a substantial push from both the US and European governments to regulate companies to make their AI-generated text/images detectable. For instance, two weeks ago, Google, OpenAI and Meta made voluntary commitments to watermark their AI-generated content [1]. Similarly, the European government recently pushed for rulings to make AI-generated content detectable [2].\n\n* Companies are already storing their model-generated outputs. For instance, both ChatGPT and Bard are storing chat histories to help improve their products with RLHF training. Bard stores chat history for 18 months by default [6], and OpenAI stores the history for 1 month even if users choose to opt-out [5].\n\n* In a hypothetical scenario where there is a lawsuit about the origin of some malicious AI-generated content, maintaining a database of previously generated responses could be a reliable method to prove innocence, given its strong performance over competing AI-generated text detectors.\n\n> it is unclear whether the proposed method is feasible (memory and time in addition to accuracy) with billions or trillions of such text\n\nIn the “Global Rebuttal”, we provide a detailed analysis estimating these requirements for a ChatGPT-scale database. In summary, we estimate them to be relatively small. Specifically, we estimate that a month’s database of ChatGPT outputs needs 5TB of storage space per month and requires just 100 seconds per retrieval on a CPU-only Macbook Pro. This is trivial compared to the scale at which Google Search (100,000+ TB index) and ChatGPT (10-15 seconds on a powerful 8x A100 GPU server) are currently operating. Moreover, major LLM API providers (like OpenAI, Google) already have the infrastructure in-place to support services like Google Search and ChatGPT at scale.\n\nIn terms of accuracy, we are optimistic looking at our scaling curves (Figure 5a shows just 0.8% drop from 1M to 10M in BM25). Our experiments already use the largest publicly available AI-generated dataset (to the best of knowledge). Collecting a billion-scale dataset with ChatGPT will cost $1M, making the experiment only possible using OpenAI or Google’s private databases.\n\n> nowadays there are also an increasing number of publicly available strong LLMs, and we cannot regulate personal uses of LLMs and AI-generated texts\n\nThis is a valid concern, and we address this in the “Global Rebuttal”. Overall, we agree with the reviewer that our detector is restricted to closed-source LLMs. However, most major LLM providers are hosting their LLMs behind closed-source APIs. Additionally, watermarking, the most promising alternative to retrieval, also suffers from the same limitation and cannot be done on open LLMs. Other detectors either perform very poorly or are brittle against paraphrasing attacks. \n\n> I acknowledge the discussion in Appendix B.1, but nothing is discussed in the main paper…. I acknowledge the discussion in Appendix B.2, but nothing is discussed in the main paper.\n\nWe will add a summary of limitations of retrieval-based detection to the main body of the paper.\n\n> In L127, L is defined as \"unigram token overlap\" but which of p/q and is used to determine the denominator?\n\nIn L127 we use an F1 score to determine unigram token overlap between p and q. So this considers both p and q in the denominator before the harmonic mean.\n\n> L132, L180 contain ordinary tokens: \"lexical\", \"order\", \"=\", and \",\". Did the authors use them without escaping as \"<p>\" and \"</p>\"?\n\nWe did not escape “lexical” / “order” in order to leverage the pre-trained embeddings for these words. We don’t think this should have any major effect on training, because we are using pre-trained 11B-sized deep transformers models which are extremely compositional in nature. While embeddings may get slightly distorted, we expect higher layers to compose information from consecutive tokens to infer the differences between control tokens and content tokens. We also note that unescaped control tokens are standard practice in T5 fine-tuning (Fig 1 in [1]).\n\n> How many training instances were used for obtaining DIPPER?\n\nWe used a dataset of 6.3M pairs (which we will open source). We noticed convergence on held-out novels within 12 hours on 64 cloud TPUv3 chips.\n\n> I'd like to suggest to tidy the layout\n\nWe agree with these presentation issues, and will clean them up in the next version.\n\n[1] https://tinyurl.com/usgovt-ai  \n[2] https://tinyurl.com/eugovt-ai  \n[3] https://arxiv.org/abs/1910.10683  \n[4] https://openai.com/blog/new-ways-to-manage-your-data-in-chatgpt  \n[5] https://support.google.com/bard/answer/13594961  \n[6] https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text"}, {"review_id": "LTM6CEjUqd", "reviewer": "Reviewer_KFgu", "summary": "The submission \"Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense\" investigates detection methods for   text output generated by modern large language models. The contribution of this submission consist of two parts. First, the submission describes, trains and provides a state-of-the-art language paraphrasing model. This paraphrasing model is then used to critically evaluate a number of detection schemes for their robustness against paraphrasing, finding that many modern detection approaches are not robust to paraphrasing attacks that leave the semantics of the original text unchanged, but modify its wording.\n\nIn a second part, the authors use this finding as motivation to describe a detection strategy based on semantic retrieval. ", "questions": "A few smaller questions:\n\n* What about random substring detection? To my understanding, the submission only tests queries where the back-end of the query is removed. Would a detection of a random substring from the query perfom equally well? Bonus question: What about retrieval for several substrings from different corpus entries, i.e. what if a document is constructed based on text generated by mixing and matching multiple queries to the language model API?\n\n* How large is the Par3 dataset? Could the authors briefly comment on dataset size and finetuning compute required for the paraphraser in the main body?\n\n* I found the different FPR for DetectGPT in  a part of Table 1 somewhat confusing. I see that the authors want to be polite here, but to me it would be clearer to indicate that this detection method really scores 0% at the given FPR. A 20% table could be included in the appendix, although I agree with the authors that there is no practical value to detection schemes with this FPR.", "limitations": "There are a number of ethical implications to storing all user-generated text on the server for retrieval, concerning data privacy. It would be great if the authors could briefly  comment on this also in the main body, as this question is currently only discussed in Appendix B.1.4.\n\nOne question I have, related to this discussion of ethics, (this is a bit of an aside, the paper is great without answering this), is how retrieval would interact with regulations that include the \"right-to-be-forgotten\", like GDPR. Could a user ask the company to delete their generations, to prevent detection? Or should it be argued that detection is \"sufficient cause\" to stop deletion of user data?\n\n\nI also think there are soom possible solutions to this question, where reconstruction of x_i from y_i can be ruled out, possibly via bloom filters, or (minimal) differential privacy?", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This is a great submission. It that starts out with a clear hypothesis about the detection of LLM-generated text and executes its two mechanical parts - First generating the paraphraser, and then constructing the retriever, very well. \n\nIn more detail, the construction of a paraphrasing data through re-alignment of paragraph-level parallel translations is novel to me and quite interesting. The authors make the reasonable choice of finetuning existing encoder-decoder models for paraphrasing based on this corpus, a choice that they experimentally validate to work well.\n\nThen, they rightfully point out that the detection of generated can be cast as a retrieval problem, which they show indeed simplifies the problem, and allows for the leverage of existing similarity search tools. Just in case, I explicitely want to point out, that I don't consider the use of existing similarity search tools a weakness, but instead a correct conclusion based on the insight of the authors that the problem can understood in this manner. In a detailed evaluation, the authors compare LLM detection based on retrieval with other approaches, finding this to be a robust choice.\n\n\nFinally, I especially want to highlight the immediate practical value that the paraphrasing model would have as a tool for the community. Previous papers have often only hinted at the possibility of paraphrasing, or used general-purpose APIs to attempt to provide accurate paraphrases, and this has limited evaluations of the very practical threat model of paraphrasing in the literature. With the release of this paper and the paraphrasing model provided here, the authors would provide a practical tool to the community that will suddenly makes investigation of this threat model much more feasible.", "weaknesses": "I only see a few minor weaknesses, which I will point out below. These mainly orbit around questions of \"why\", which this submission does not always contain. \n\n* Why would these detectors break under paraphrasing? While we do observe that the detectors based on outlier detection and classifier methods break after paraphrasing, aren't some of these detectors kind of correct in their assessment that the text is not written by the original model anymore? This does not apply to all detectors, but those that claim to only detect a specific model (or model family) are not entirely incorrect? To phrase this question differently, would paraphrasing also break detectors that classify \"generic\" machine text, and are not specialized to detect particular models or model families? \n\n\n* Why is FPR low for detection?There seems currently no way to estimate the reliability of the retrieval-based detector formally and to guarantee a certain FP rate to my understanding? Is this a principal limitation, or could the method be modified to use a threshold not calibrated from existing data? Ideally, in a way that includes the size of the corpus, allowing for estimates of performance of this approach at larger scales than can be tested empirically in this work?\n\n* Is there a corpus size at which retrieval stops being meaningful, if it is based only on semantics? I could imagine, for the sake of the argument, that many school essays that summarize existing arguments about some fact or historical event on per-paragraph level, might be semantically the same. These would also be semantically similar to some output when the model was at one point queried with this topic separately?\n\n\nIn any case, the submission stands strong based on the empirical evidence it provides, and some of these questions might be left to future work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed and thoughtful feedback, and for strongly supporting our paper! In particular, we are grateful to the reviewer for supporting our 1) novel paraphraser and its open-sourcing; 2) our experimental analysis testing the robustness of AI-generated text detectors; 3) our casting of AI-generated text detection as a retrieval problem and its downstream robustness.\n\nThe reviewer further asks a number of interesting questions related to paraphrasing as an attack and retrieval as a defense. We address them below.\n\n> would paraphrasing also break detectors that classify \"generic\" machine text, and are not specialized to detect particular models\n\nOur experiments do show that paraphrasing drops performance in all model-specific detectors we tried (watermarking, DetectGPT) and two out of three model-agnostic detectors (OpenAI, GPTZero). However, it’s hard to fairly compare the relative effect of paraphrasing on these two classes due to a large performance gap between them in the first place. In Table 1, we see that even without paraphrasing, model-specific detectors significantly outperform model-agnostic detectors. OpenAI has even gone so far to remove their model-agnostic classifier from their website due to low accuracy [1]. Overall, we observe that model-specific watermarking is the most robust to paraphrasing attacks despite the large drop. It’s also important to note that paraphrased text is more “AI-generated” in some sense. One of our model-agnostic detectors (RankGen) thinks it is indeed so (7% vs 1% TPR for GPT3.5), but its low overall TPR makes it unusable as a detector.\n\nIt is technically true that model-specific detectors are not designed to detect outputs not entirely generated by the model itself. However, given their strong performance over model-agnostic methods, and the large risk of perturbation attacks (human-edited or automatically-edited), we think it’s important for model-specific detectors to bake perturbation robustness in their design for better downstream usability.\n\n> There seems currently no way to estimate the reliability of the retrieval-based detector formally and to guarantee a certain FP rate to my understanding?\n\nSimilar to DetectGPT / classifiers, we believe the threshold needs to be estimated empirically on the underlying data distribution. We will add some analysis on this in the next version, and analyze the out-of-distribution robustness of thresholds chosen on a subset of human data. A formal relationship between FPR and the threshold (like in watermarking) may be possible using information about the density of the retrieval database in the semantic vector space. We leave this exploration for future work.\n\n> Is there a corpus size at which retrieval stops being meaningful, if it is based only on semantics?…school essays that summarize existing arguments about some fact or historical event…might be semantically the same.\n\nThis is a great point about semantic overlap within the database on popular topics. We address this in the “Global Rebuttal” under “semantic collisions”. In summary, we believe that the chance for semantic collisions is low because,\n* retrieval-based detection uses pairwise comparisons, and the likelihood of semantic divergences between pairs exponentially increases with length\n* the most effective retrievers do not solely rely on semantics\n* unperturbed text will always have a 100% detection rate.\n\n> What about random substring detection? What about retrieval for several substrings from different corpus entries\n\nOur experiments in Fig 5b experimented with truncated paraphrases as queries. Below, we present results for other query types that the reviewer suggested. We adopt the same setup as 5b: PG19-BM25, 1% FPR:\n\nResults in paper:  \nfull length unperturbed query: 100%  \nfull length DIPPER query: 98.2%  \n50% trunc DIPPER query: 72.6%  \n\nNew results:  \n50% random unperturbed substring: 86.2%  \n50% of two different unperturbed queries concat: 94.7%  \n50% random DIPPER substring: 68.8%  \n50% of two different DIPPER queries concat: 56.1%\n\nOverall, we see that unperturbed random substrings (from single or multiple generations) can still be detected quite easily. However, adding DIPPER paraphrasing on top of that reduces accuracy.\n\n> How large is the Par3 dataset?… fine tuning compute?\n\nOur processed Par3 dataset has 6.3M pairs (which we will open source). We noticed convergence on held-out novels within 12 hours on 64 cloud TPUv3 chips.\n\n> … how retrieval would interact with regulations that include the \"right-to-be-forgotten\", like GDPR. Could a user ask the company to delete their generations, to prevent detection? Or should it be argued that detection is sufficient cause to stop deletion?\n\nWe agree with the reviewer that AI-generated text detection is a sufficient cause to temporarily override deletion requests [2]. We believe that AI-generated text detection could fall under the following GDPR guidelines for overrides: (1) “freedom of expression and information”,(2) “establishment of a legal defense or in the exercise of other legal claims”, and possibly (3) “comply with a legal ruling or obligation” in the future. As an example of this, while OpenAI allows users to delete their chat history [3], they retain them for 30 days and can review it if required to monitor for abuse. We also agree with the reviewer that differential privacy or scrubbing sensitive attributes can mitigate the privacy issue to some extent. We will mention this in the next version, and also move our discussion on data privacy from Appendix B.1.4 to the main body as requested by the reviewer.\n\n> I found the different FPR for DetectGPT in Table 1 confusing\n\nWe agree with this concern about our presentation. We will clarify this in the next version.\n\n[1] https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text  \n[2] https://gdpr.eu/right-to-be-forgotten  \n[3] https://openai.com/blog/new-ways-to-manage-your-data-in-chatgpt"}, {"review_id": "tDhUWrFz39", "reviewer": "Reviewer_VDA7", "summary": "This paper investigates the robustness of AI-generated text detection algorithms to paraphrasing. The authors train a language model to paraphrase text in an attempt to evade detection algorithms. The proposed model leverages longer contexts than existing sentence-level paraphrasers and offers users control over content diversity and reordering. After demonstrating the vulnerability of current detection algorithms to the paraphrasing model, the authors propose a retrieval-based defense in which the outputs of a language model are stored and matched against possible AI-generated text. The authors find their retrieval-based defense achieves superior robustness against paraphrasing when compared to existing watermarking and classification defenses.", "questions": "If the underlying language model is itself capable of diverse generations, is it possible that we might see a \"saturation\" problem in certain topic areas? E.g. if hundreds of students use the language model to generate one-paragraph summaries of the Gettysburg address, all of which are stored in the database, will this provide a kind of \"semantic coverage\" of the topic such that even human-generated summaries of the Gettysburg address would be flagged as AI-generated?\n\nI think the authors' proposed retrieval defense is promising, and their proposed improvements in Appendix B.2 are interesting, but I'm not sure whether the paper effectively argues for the feasibility of retrieval-based defenses at scale. In Appendix B.1, line 680, the authors state that \"At a conservative rate of 5M queries a day, [an AI text generation] database will have almost two billion entries in a year.\" While the authors perform experiments to validate their retrieval-based defense at a scale of 15 million entries, it is not immediately clear how this performance would extrapolate to a database three or more orders of magnitude larger (as in their hypothetical). Along these lines, it would be helpful if the authors could point to one or more analogous text-retrieval systems capable of operating at the aforementioned hypothetical scale with the precision required for AI text detection.", "limitations": "Yes", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The training scheme for DIPPER is clever and well motivated (e.g. control over content re-ordering and diversity). I think the paper benefits from including both an improved attack on AI text-detection and a simple but interesting defense.\n\nVery thorough experiments.\n\nI appreciate the discussions of the limitations of retrieval-based defenses in the appendix.", "weaknesses": "The authors do not appear to evaluate any non-DIPPER paraphrasers in their experiments. The claim in line 232 that \"our ablations in Appendix C show that these paraphrasers have lower quality and are less compatible with the prompt as DIPPER paraphrasers\" seems misleading, as DIPPER ablations are not the same as evaluations of existing off-the-shelf paraphraser models. While the paper presents strong results for DIPPER, I think the authors should either re-word this claim to be more clear or provide direct experimental comparisons to other paraphrasers.\n\nParaphrasing attacks might not be considered effective if they degrade the original text in certain ways (e.g. introduce grammatical errors, change the \"tone\" or \"voice\" of the text) even if they preserve some notion of semantic similarity. The results in the main paper heavily emphasize the semantic similarity metric of Wieting et al. as a measure of paraphrased text quality, although paraphrasers are evaluated under other metrics (e.g. human evaluations, perplexity) in the appendix. As things stand, it is not immediately clear in the main body of the paper whether DIPPER strongly preserves the quality of the original text beyond some notion of semantic similarity. This should probably be clarified by summarizing the results of the additional quality evaluations from Appendix C.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback and support for our paper! In particular, we are grateful for reviewer’s appreciation for our 1) novel paraphraser training algorithm DIPPER; 2) attacks on AI-generated text detectors using DIPPER; 3) novel defense mechanism and a discussion of its limitations; 4) thorough experiments in all these fronts.\n\nThe reviewer voiced a few concerns about our paraphraser evaluations, and had a few questions about the scalability of our retrieval-based defense mechanism. We address them below.\n\n> claim in line 232 that \"our ablations in Appendix C…\" seems misleading, as DIPPER ablations are not the same as evaluations of existing off-the-shelf paraphraser models.\n\nWe agree with this concern and will reword L232 to better reflect our contribution. In particular, our stance is that we expect non-DIPPER paraphrasers to also evade detection. However, we expect DIPPER will produce higher quality paraphrases than a non-contextual sentence-level alternative (as shown in our Appendix). Another advantage of using DIPPER is its fine-grained control knobs to vary diversity to find a sweet spot of retaining semantics, while fooling detection systems.  We did not find these traits in other off-the-shelf paraphrasers, as discussed in Appendix D.1. We will move some text from the Appendix to the main body to further highlight this.\n\n> Paraphrasing attacks might not be considered effective if they degrade the original text in certain ways (tone, style).. As things stand, it is not immediately clear in the main body of the paper whether DIPPER strongly preserves the quality of the original text.. This should probably be clarified by summarizing the results of the additional quality evaluations from Appendix C\n\nAs the reviewer pointed out, we have included several additional automatic and human paraphrasing evaluations in Appendix C to showcase the strengths of DIPPER. We will summarize these results in the main body of the paper. We agree with the reviewer that paraphrases are likely to modify style as shown in [1]. However, as mentioned in the previous paragraph, an important property of DIPPER is to provide fine-grained diversity control. Our lexical and order diversity control knobs allow an attacker to modify a generation *just enough* to evade detection: lower the value of the knobs, lesser the stylistic modification. \n\n> If the underlying language model is itself capable of diverse generations, is it possible that we might see a \"saturation\" problem in certain topic areas (the Gettysburg Address)?\n\nThis is a great point about semantic overlap within the database on popular topics. We address this in detail in the “Global Rebuttal” under “semantic collisions”. In summary, we believe that the chance for semantic collisions is low because:\n* retrieval-based detection uses pairwise comparisons, and the likelihood of semantic divergences between pairs exponentially increases with length;\n* the most effective retrievers do not solely rely on semantics;\n* unperturbed text will always have a 100% detection rate.\n\n> I'm not sure whether the paper effectively argues for the feasibility of retrieval-based defenses at scale.\n\nWe extensively discuss this in the “Global Rebuttal” to all reviewers, on axes of storage, compute and accuracy. Overall, we are quite optimistic about the feasibility of retrieval in terms of storage requirements (5TB for a database with 1.5-2B generations) and compute requirements (just 100 seconds for retrieval against a 1.5B database on a Macbook Pro). Importantly, the major players providing LLM API services (Google, OpenAI, Microsoft) have vastly superior computational infrastructure already in place to power services like Google Search, ChatGPT and Bing at scale.\n\nIn terms of accuracy, we are optimistic looking at our scaling curves (Figure 5a shows just 0.8% drop from 1M to 10M in BM25). Our experiments already use the largest publicly available AI-generated dataset (to the best of knowledge), and collecting a billion-scale dataset with ChatGPT will cost $1M.\n\n> it would be helpful if the authors could point to one or more analogous text-retrieval systems capable of operating at the aforementioned hypothetical scale\n\nTraditional information retrieval has a slightly different setup compared to our retrieval-based detection. Queries tend to be more information seeking (rather than looking for exact matches or paraphrases), and recall@k is also important besides precision. Nevertheless,  we think Google Search is an excellent example of text-retrieval operating at scale, since Google’s search index is over 100B webpages [3]. In academic literature, we found a few examples of experimental setups operating at a billion-scale [4, 5]. [4] shows a precision@1 of 60% for time-aware retrieval.\n\n[1] https://arxiv.org/abs/2010.05700  \n[2] https://eval.ai/web/challenges/challenge-page/1897/leaderboard/4475  \n[3] https://www.google.com/search/howsearchworks/how-search-works/organizing-information  \n[4] https://sobre.arquivo.pt/wp-content/uploads/creating-a-billion-scale-searchable-web-archive.pdf  \n[5] https://arxiv.org/abs/2110.06125"}, {"review_id": "8un4wUqAnE", "reviewer": "Reviewer_k3DJ", "summary": "The authors developed a powerful paraphrase generation model called DIPPER to test the robustness of AI text detection algorithms. DIPPER successfully evaded several detectors by paraphrasing text generated by large language models. To improve detection, they proposed a defense mechanism based on retrieving similar generations from a database, which detected 80% to 97% of paraphrased text while misclassifying only 1% of human-written sequences as AI-generated.\n\n\n\n\n", "questions": "Is it practical for popular LLMs which many users and queries, like ChatGPT to store all their generated texts?", "limitations": "The authors have addressed the limitations.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. Testing the robustness of existing detectors of AI-generated text is very interesting and important.\n\n2. A defence method is proposed to handle the paraphrasing problem discovered in this paper.\n\n3. The experimental results are solid and promising.", "weaknesses": "1. The proposed method needs the store of all AI-generated texts. I wonder whether it is practical for popular LLMs which many users and queries, like ChatGPT.\n\n2. More detection methods should be incorporated in experiments.\n\n3. It would be better if more datasets and tasks are used.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their useful feedback and support for our paper! In particular, we appreciate the reviewer for highlighting that 1) we study important research questions on robustness of AI-generated text detection; 2) our defense mechanisms for our attacks; 3) our paper shows solid experimental results.\n\nThe reviewer voiced some concerns about storage requirements, and requested experiments on more detection methods and datasets. We address these below.\n\n> The proposed method needs the store of all AI-generated texts. I wonder whether it is practical for popular LLMs which many users and queries, like ChatGPT.\n> Is it practical for popular LLMs which many users and queries, like ChatGPT to store all their generated texts?\n\nWe think it is very much practical at ChatGPT’s current usage rate (1.5-2B monthly visits) to store AI-generated outputs. We estimate it will take about 5TB storage space per month (detailed calculations in the “Global Rebuttal”), similar to the size of a personal portable hard-disk. Moreover, ChatGPT and Google Bard already store user’s chat history for improving their models with RLHF [1, 2]. Moreover, the major LLM providers like Google and OpenAI already have the storage infrastructure in-place which they use to host services like Google Search (100,000TB+ index size) and ChatGPT / GPT3.5. In our “Global Rebuttal” we also extensively discuss scalability in terms of compute and accuracy.\n\n> More detection methods should be incorporated in experiments.\n> It would be better if more datasets and tasks are used.\n\nOur experiments already cover a comprehensive set of five detection algorithms (DetectGPT, watermarking, OpenAI classifier, GPTZero and RankGen). At that time our research was conducted, we included every state-of-the-art contemporary AI-generated text detector we could find. Moreover, our experiments cover two practically relevant tasks modern large language models are used for: long-form question answering (six domains) and open-ended generation (two domains). Finally, our experiments are performed on outputs from three large language models (GPT2-XL, OPT-13B, GPT3.5 davinci-003), to provide a wide diversity of LM sizes and properties.\n\nNevertheless, we acknowledge that our scaled retrieval experiments (Section 5.3) cover outputs from only one kind of language model in two domains. As promised in the “Global Rebuttal”, we will conduct similar experiments on other AI-generated text databases like GPT4All and ShareGPT, which while smaller in size, are more diverse in nature than the RankGen training data.\n\n[1] - https://openai.com/blog/new-ways-to-manage-your-data-in-chatgpt  \n[2] - https://support.google.com/bard/answer/13594961"}, {"review_id": "Ne8DLQALjx", "reviewer": "Reviewer_44uy", "summary": "The paper can be split into two parts. In the first part, the authors propose a paraphrasing-based attack that circumvents various AI-generated text detectors. The authors introduce DIPPER, an 11B parameters Transformer model obtained by fine-tuning T5-XXL. By using DIPPER to create paraphrases from texts generated by LLMs, the authors show that they can successfully evade various commonly-used detectors. In the second part, the authors propose a retrieval-based generated text detector that can successfully detect instances of paraphrased generated text. \n", "questions": "- I would like to see what happens if text generated by an LLM is kept in the database and some semantically similar entries generated by humans would be present in the test dataset.\n\n- Following the first point, what would happen if you used DIPPER to paraphrase the human-generated text? Would there be higher similarity scores for the paraphrased samples if you have some semantically similar generated text in the database?\n\n- Have the authors tried any similarity metric tailored for paraphrasing? Such a contribution would make the paper significantly stronger, in my opinion.", "limitations": "- The authors have extensively discussed potential limitations in the main paper and the Supplementary materials for both their DIPPER model and the retrieval-based detection systems. While the limitations regarding the computation necessary for a large-scale retrieval system can be somewhat alleviated, as discussed in section B.2 of the Supplementary, I believe the similarity-based scoring approach to be a significant limiting factor. \n", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-written and easy to follow.\n\n- The authors introduce DIPPER, a paraphrasing model based on T5-XXL. The authors show that DIPPER can generate high-quality paraphrases and have tested the model in a human trial included in the Supplementary materials. \n\n- The paraphrasing attacks are effective against the models tested.\n\n- The authors have an extensive discussion about the limitations of their retrieval-based system.\n", "weaknesses": "- The paper's biggest weakness is the generated text detection via retrieval methods since many assumptions must be made for it to work correctly. \n\n  - First, one needs access to all of the text generated by the model we want to test against. This is significantly limiting since the data would probably be available only to the entity that is maintaining the LLM API. Maintaining such a big database of generated text would also be expensive from a storage perspective, and computationally expensive for the retrieval phase.\n\n  - Second, using a similarity score to compare the paraphrased text with retrieved texts, one must assume that the similarity between the generated text and its paraphrased variant would be high. However, the similarity score could also be large if the two texts have similar semantics. This would be true if one used something similar to the cosine similarity between two text embeddings obtained by a neural encoder [[1]]. If one maintains a significantly large and diverse database of generated text, the chance of it containing an entry that is semantically similar to a candidate text grows, leading to FP predictions.\n\n  - Third, the method would only be viable for closed, proprietary models, and would not apply to open-source models, since it is impossible to collect a database of all the text generated by an open-source LLM that is hosted by multiple people and institutions.\n\n- As far as I understand, the authors have used human-generated continuations only to adjust the detection thresholds to maintain a 1% FPR (lines 171-172). If this is the case, the evaluation methodology is somewhat flawed due to the points I've previously made. In my opinion, a better evaluation methodology would be using some parts of the human-generated data to fix a low FPR, and also have human-generated data with similar semantics to the LLM-generated one in the test set.\n\nOverall, I believe that the paper is nice and explores a potential direction that could be valuable in some use-cases, but the flaws and limitations previously mentioned make me question the practical efficiency of such a detector deployed at scale in the real world. I believe the paper could be much stronger if:\n\na) The authors would carefully build a large and semantically diverse database of generated texts and would add human-generated texts to the test dataset. The detection method would be significantly more convincing if it did not fail in this scenario.\n\nb) The authors would design some similarity metric that would result in a large score only if the candidate text is some paraphrase of the target. This could potentially greatly reduce the number of false positives.\n\n[1]: https://arxiv.org/pdf/1301.3781.pdf", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful feedback! We are grateful that the reviewer appreciated our writing, paraphraser, the paraphrasing attacks on detectors, and discussion on the limitations of retrieval.\n\nThe reviewer voiced concerns about the practicality of retrieval as a detection algorithm and mentioned technical challenges in implementing it at scale. We address these concerns below.\n\n>similarity score could also be large if the two texts have similar semantics...if one used encoder like word2vec…paper would be stronger if...authors design some similarity metric that would result in a large score only if candidate text is some paraphrase\n\nTo the best of our understanding, the reviewer is referring to topical similarity between two texts, which could lead to high cosine similarity between non-paraphrases. This may be true for methods like word2vec, but our paper uses the P-SP embedding method which has been **explicitly trained for paraphrastic similarity using parallel data**. P-SP [4] achieves state-of-the-art performance on the STS benchmarks [1], which include plagiarism detection, measuring paraphrastic similarity on varying degrees of semantic overlap. To further illustrate the robustness of P-SP to non-paraphrases sharing topics, we conducted an experiment on the Par3 dataset. We found that the average P-SP score of actual human paraphrase pairs in Par3 is 0.76. In contrast, the P-SP of random pairs of paragraphs from the same book is just 0.09 (topically similar but not paraphrases).\n\nWe have also added a discussion on “semantic collisions” in the “Global Rebuttal”. Overall, the chance of collisions between topically similar non-paraphrases is low because\n* the likelihood of semantic divergences between pairs exponentially increases with length\n* the most effective retrievers do not solely rely on semantics\n* unperturbed text will always have a 100% detection rate.\n\n>The paper would be stronger if authors would add human-generated texts to test set…with similar semantics to LLM-generated data.\n\nWe have effectively done this in Sec. 5.2b, 5.3. The database in 5.2b contains three generations for the same prompt (hence strong topical overlap), and the test set includes the human-written output for the *same prompts*. Similarly, in 5.3, we have hundreds of database entries from the same PG19 book which closely share topical content. Our test set includes the human-written continuation for those prompts.\n\n>the authors have used human-generated continuations only to adjust the detection thresholds to maintain 1% FPR.. better methodology would be using parts of the human-generated data to fix a low FPR, and keep rest in test set.\n\nWe do not think using human text for threshold adjustment removes them from the test set. A high FPR will push up the threshold, which in-turn will lower the true positive rate. This is equivalent to taking the y-coordinate for x=1% in a classifier’s ROC plot (TPR vs FPR). Moreover, in Fig 8 (Appendix), we plot the full ROC plots which consider every possible detection threshold. We find retrieval to be vastly superior and robust to paraphrasing compared to other methods on different thresholds.\n\nWhile we could use a random fraction of human-written text for threshold adjustment and evaluate the other fraction’s FPR, we expect it to result in a 1% +/- delta FPR since it’s in-distribution data. To validate this hypothesis, we empirically evaluated it in one setup (PG19, BM25), using 50% of the human text to estimate the threshold. We found the FPR on the other 50% of human text to be 0.8-1.2% across runs, and we expect variance to reduce with a bigger human dataset.\n\nHowever, we acknowledge that our paper does not test the out-of-distribution generalization of threshold adjustment (do thresholds for human dataset 1 give low FPR on human dataset 2?). We will do this in the next version.\n\n>Maintaining such a big database of generated text would be expensive from a storage perspective and computationally expensive\n\nIn the “Global Rebuttal”, we provide a detailed analysis estimating these requirements for a ChatGPT-scale database. In summary, we estimate them to be relatively small. We estimate the ChatGPT database needs 5TB of storage space per month, and require just 100 seconds per retrieval on a CPU-only Macbook Pro. This is trivial compared to the scale at which Google Search (100,000+ TB index) and ChatGPT (10-15 seconds on a powerful 8x A100 GPU server) are currently operating. Moreover, major LLM providers (like OpenAI, Google) already have the infrastructure to host services like Google Search and ChatGPT at scale.\n\n>One needs access to all of the text generated by the model…limiting since the data would be available only to the entity that is maintaining the LLM API\n\nWe agree that our detector can only be implemented by the LLM API provider, and have acknowledged this in Appendix B.1. However, major LLM providers may be incentivized to implement this detector, since it can help claim innocence in potential lawsuits about the origin of malicious AI-generated content. Moreover, there are increasing government discussions on regulating AI-generated content, and both the US and EU government have recommended major LLM providers to make their AI generations detectable [2, 3].\n\n>the method would only be viable for closed, proprietary models, and would not apply to open-source models\n\nThis is a valid concern, and we address it in the “Global Rebuttal”. Overall, we agree with the reviewer that our detector is restricted to closed-source LLMs. However, most major LLM providers are hosting their LLMs behind closed APIs. Also, watermarking, the most promising alternative to retrieval, suffers from the same limitation. Other detectors either perform very poorly or are brittle against paraphrases.\n\n[1] https://aclanthology.org/S16-1081  \n[2] https://tinyurl.com/usgovt-ai  \n[3] https://tinyurl.com/eugovt-ai  \n[4] https://aclanthology.org/2022.emnlp-demos.38"}]

Author rebuttal:
We are very grateful to the reviewers for their detailed feedback. While we address each reviewer’s questions in the individual rebuttals, we use this “global rebuttal” to address concerns shared by multiple reviewers.

We thank the reviewers for supporting the three contributions in our paper:
* a novel discourse paraphrasing model and its open-sourcing
* a comprehensive set of effective paraphrasing attacks on modern AI-generated text detectors
* a novel defense using information retrieval and extensive discussion of its limitations.

Many of the concerns center around our third contribution of retrieval-based detection. These concerns can be categorized as:
* issues of scalability (storage, compute, accuracy)
* limitation to closed-source LLMs
* semantic collisions

Below, we address each in detail.

**Scalability of retrieval - storage**: We estimate ChatGPT’s outputs to take 5TB space monthly (similar to a personal portable hard-disk) via the following calculations. ChatGPT currently gets about 2B monthly visits [3]. Assuming an average response length of 500 tokens per session, this corresponds to 1 trillion tokens. Similar in size to LLaMA’s training data, this needs 5TB space [2]. However, 5TB is a small amount of storage compared to the industrial scale of information retrieval. For example, the Google Search index is over 100,000TB and has 100B+ pages [1]. Major LLM service providers (like Google, OpenAI) already have complex storage infrastructure to facilitate this defense. Additionally, it’s likely that they already store their model outputs for future RLHF purposes.

**Scalability of retrieval - compute**: Our retrieval experiments, conducted on a 14-core CPU (similar to a Macbook Pro), took 1 second per retrieval on a 15M sized corpus. Extrapolating to a corpus of ChatGPT’s monthly usage (100x) would need 100 seconds/retrieval on a Macbook. However, this is fully parallelizable, and can make use of GPUs (Google searches 100B+ entries in < 1 sec). Moreover, efficient similarity search has powerful libraries like FAISS available. For comparison, ChatGPT itself takes 10 seconds/response, possibly using a powerful 8-GPU A100 server [4]. Major LLM providers have massive compute clusters, and we believe the computational requirement of retrieval is much lower than hosting LLMs in the first place, which these providers are already adept at. Moreover, our proposed ideas in B.2 can further reduce compute costs.

**Scalability of retrieval - accuracy on larger databases**: Our experiments were conducted on the RankGen training set [7], which is the largest publicly available database of AI-generated text that we are aware of (15M generations each in four domains). Besides this, there are a few other datasets of AI-generated text such as GPT4All (809K) and ShareGPT (350K generations). While these datasets are much smaller than ours, we will experiment with them in the next version to add more diversity to our results. We note that it is extremely expensive and time consuming to create a corpus of AI-generated text from scratch: at a cost of `$`0.001 per 500-word response, collecting a billion ChatGPT outputs would cost $1M and take a long time to collect due to rate limits. Hence, a billion-scale experiment is likely only possible with Google/OpenAI’s private database.

Overall, we are optimistic about our scaling plots (Fig 5a), and see just a 0.8% drop moving from a 1M to 10M database (PG19-BM25). We emphasize that BM25 is a basic retriever, and is not optimized on our task. Information retrieval literature has many powerful retrievers, and we have suggested a dense retrieval mechanism in B.2 which can be optimized on the underlying retrieval corpus. Moreover, retrieval can easily be used in tandem with other detectors like watermarking. Finally we note that our paper is the first proof-of-concept that shows a retrieval-based detector could work, and we anticipate future work to build upon it.

**Retrieval is limited to closed-source LLMs**: We agree with this and will add it to our limitations. However, besides Meta, all major LLM providers (OpenAI, Google, Anthropic, Microsoft, Cohere) operate their LLMs behind closed APIs. It’s also important to note that watermarking, the most promising alternative to retrieval, also has this limitation. Since watermarks are added during decoding rather than into the model weights, users of open LLMs are free to generate text without watermarks. While other alternatives (DetectGPT, classifiers) don’t suffer from this issue, our paper shows that they either have low accuracy, or are extremely vulnerable to paraphrasing. In fact, OpenAI recently took down their classifier due to low accuracy [5].

**Semantic collisions**: Reviewers raised a concern about our retrieval database saturating with entries having similar semantics (especially for popular topics), which will harm detection at scale. In response, we note (and will update our paper to include) that:
* Like other detectors, retrieval works best on longer sequences (Fig 5b). Long generations exponentially increase the likelihood of semantic divergences between pairs of entries.
* Retrieval compares the input against the top-1 match, not top-k. For false-positive inputs on popular topics, the top-k entries *together* are more likely to cover input semantics (recall) rather than top-1 (precision).
* The most effective retrievers use a combination of neural semantic encoders and token overlap scores [6]. We also show this, BM25 beats P-SP at detection. BM25 is not fully semantic driven: it uses TF-IDF token overlap.
* The retrieval accuracy for unperturbed AI-generated text is always 100%, just like exact match searches in Google Search. Retrieval is also effective on substrings of unperturbed text (see rebuttal to KFgu).

[1] tinyurl.com/ggsdb  
[2] tinyurl.com/fbllama  
[3] tinyurl.com/chatnyp  
[4] tinyurl.com/chatgpu  
[5] tinyurl.com/oaicls  
[6] tinyurl.com/beirev  
[7] tinyurl.com/rrkgn

Meta-review: This is a solid paper with three key contributions: a novel paragraph-level paraphrase model, an “experimental study attacking five AI-generated detectors", and a new detection/defence method based on semantic retrieval. In the words of one of the reviewers, this paper will "serve as a solid foundation in the efforts against paraphrasing attacks and can be a good contribution to NeurIPS", while the approach will have good practical value for the community. However, the authors should ensure they fully address all of the reviewers’ concerns in the camera ready at least as extensively as they have done here.
Decision: Accept (poster)