SlowGuess/NeurIPS_2023_OCR huggingface dataset:
hire three native English teachers and/or editors on Upwork<sup>19</sup> to evaluate the semantic fidelity of the paraphrases. As human evaluation is expensive, we fix the order diversity  $(O)$  to be 0 and focus on the impact of the lexical diversity. We evaluate paraphrases with the lexical codes  $L20$ ,  $L40$ , and  $L60$ , corresponding to moderate, medium, and high lexical diversity. Twenty paraphrases are sampled randomly for each lexical code, resulting in 60 original text and paraphrase pairs.
The evaluation is conducted on the platform Label Studio [Tkachenko et al., 2020-2022].<sup>20</sup> As shown in the interface of our annotation platform Figure 7, the text to be paraphrased (highlighted in yellow) are preceded by its context. The annotators see the same amount of text as DIPPER. They need to first read the texts, select one point on the Likert scale, then provide free-form comments justifying their ratings. We estimated that the evaluation of each paraphrase takes 1.5 to 2 minutes. As such, we pay $15 as a base rate with a bonus for the reasonable extra time that the annotators spend on the tasks.
Among the 60 original text and paraphrase pairs, the three annotators agreed on their choice  $28.3\%$  of the time, and  $60\%$  of the time the point they chose on the scale differs by 1. Table 4 reports how often each point on the Likert scale is chosen. Over  $80\%$  of the time, our annotators rate DIPPER's paraphrases as nearly equivalent (4 out of 5) or approximately equivalent (5 out of 5).
A qualitative analysis of the free-form annotator comments reveals systemic strengths and shortcomings of DIPPER. Table 8 provides two representative examples for each lexical code that is evaluated in our human study.
19. https://www.upwork.com
20https://labelstud.io/
Given the source text:
She was only hit by a single 12-inch shell that wounded two crewmen. Both guns in her aft 12-inch gun turret, however, were disabled by shells that detonated prematurely in their barrels. Most of the other damage the HMS New Zealand sustained was from shrapnel and splinters. All in all, the ship was estimated to have been struck by up to twenty-five shells, most of which were smaller than 12-inch. She also sustained damage to her superstructure, masts, and rigging.
Here is a paraphrase of the highlighted text:
The majority of the damage to HMS New Zealand was caused by shrapnel and fragments. In all, the ship was thought to have been hit by as many as twenty-five shells, most of them smaller than 12 inches. She was also hit in her superstructure, masts, and rigging.
Which of the following best describes the quality of the paraphrase?
$0$  5-Approximately equivalent: the paraphrase preserves the meaning of the source but differs in words and/or structure.[1]
4-Nearly equivalent: the paraphrase preserves most information in the source but differs in some minor factual details.[2]
3-Somewhat equivalent: the paraphrase preserves some information in the source but differs in certain significant ways.[3]
2-Topically related: the paraphrase is topically related to the source but most information in the source is not preserved.[4]
1-Not topically related: the paraphrase is not topically related to the source and preserves no information.[5]
Please motivate your choice in 2 to 3 sentences.
Figure 7: The interface of the annotation platform used in our human study
Table 4: This table shows how often each point in the Likert scale was chosen by 3 annotators for the pairs of original and paraphrased texts. Twenty text pairs are randomly selected for each lexical code (L).  $81.8\%$  of the time, our model DIPPER provides a paraphrase which is nearly equivalent to the input in terms of semantic meaning.
L	Sum of 4 and 5	5 Approx. equivalent	4 Nearly equivalent	3 Somewhat equivalent	2 Topically related
20	95.0%	63.3%	31.7%	5.0%	0.0%
40	78.3%	45.0%	33.3%	21.7%	0.0%
60	70.0%	28.3%	41.7%	28.3%	1.7%
Total	81.1%	45.6%	35.6%	18.3%	0.6%
Strengths First, the third example in Table 8 exemplifies DIPPER's ability to leverage information from context to increase diversity while maintaining coherence (i.e., from line... reference the song's title to reference to "I'm the Greatest"). The same is observed in row 2 where DIPPER uses the context to interchange he and Churchill. A paraphrase model without looking into context will have great difficulty in doing this and no prior paraphraseer (see Table 5 for a list) is capable of that. Second, the example in the fifth row highlights DIPPER's ability to make significant changes to original texts with a high lexical diversity code (L60) (see the color coding) while preserving their semantic meaning as rated by the annotators.
Qualitative shortcomings: The first shortcoming is that, when the original text contains new created proper names (unlike common people and country names), such as the ones in row 6 (Homing Attack and Slide Attack), a high lexical code has a tendency to change such nouns, leading to the result that one of our annotators deems it to be only topically related to the original. However, this shortcoming can be overcome by decreasing the lexical code, which a user can choose from a continuous range (from 0 to 100). For instance, in row 1 with  $\text{lex} = 20$ , the songs' names M's Confession and Gone Fishing are kept intact. Another shortcoming is that DIPPER occasionally omits content from an original text. While in some cases such removal is acceptable (see row 6), in other cases it causes significant change in the meaning of the text (see row 4). However, the former case can be overcome by paraphrasing a shorter paragraph at a time.
Overall, the human study shows that DIPPER performs well at preserving the semantic meaning of original texts while introducing both semantic and syntactic diversity. Because DIPPER provides user-friendly controllability of output diversity, a user can adjust the control code to find the most suitable paraphrase for their need.
D Related work for discourse paraphrasing
D.1 Survey of paraphrase generation papers
As an important NLP task, paraphrasing has attracted much attention. Many models have been proposed to improve the quality of paraphrases. To position our model DIPPER and highlight its strengths, we conduct a survey of paraphrase generation papers from 2018 to 2022 (Table 5) and focus on the following four aspects:
1. Whether a model can paraphrase a paragraph at once,
2. whether a model can merge or split consecutive sentences when appropriate,
3. whether a model leverages context surrounding an input sentence when paraphrasing,
4. whether a model provides control knobs for users to customize the output diversity.
The survey shows that only three out of 25 papers mentioned that their model can paraphrase more than one sentence (but not necessarily at once). None of them enables their model to merge or split sentences when paraphrasing. No model uses information from context surrounding an input sentence during inference time. Finally, 14 papers offer ways for users to customize the diversity of paraphrases. However, most diversity control methods such as constituency parses or exemplars may not be straightforward and intuitive to end-users as the scalar control knobs in DIPPER.
In contrast to the papers in the survey, DIPPER nicely combines all desiderata into one model and offers intuitive control knobs for lexical and syntactic diversity. Automatic and human evaluation show that DIPPER can efficiently leverage context information and reorganize sentences while having high fidelity in meaning (Appendix C).
D.2 Other related work
In this section we discuss a few additional less related papers which were not included in our survey in Appendix D.1. Our discourse paraphrase is closely related to work on contextual machine translation, where source/target context is used to improve sentence-level machine translation [House, 2006, Jean et al., 2017, Wang et al., 2017, Tiedemann and Scherrer, 2017, Kuang et al., 2018, Agrawal et al., 2018, Miculicich et al., 2018, Zhang et al., 2018, Xiong et al., 2019, Jean et al., 2019, Voita et al., 2019a, Yin et al., 2021, Mansimov et al., 2021]. Prior work has shown that context helps with anaphora resolution [Voita et al., 2018], deixis, ellipsis, and lexical cohesion [Voita et al., 2019b]. Efforts to make paraphrase generation more contextual have been quite limited. A few efforts have attempted to use sentence level context to paraphrase phrases [Connor and Roth, 2007, Max, 2009], and dialogue context to paraphrase individual dialogues in a chat [Garg et al., 2021].
Our work is also related to efforts in text simplification to go beyond a sentence, by collecting relevant datasets [Xu et al., 2015, Devaraj et al., 2021] and building unsupervised algorithms [Laban et al., 2021]. Note that our work focuses on a general-purpose paraphrasing algorithm and is not tied to any particular style, but could be utilized for document-level style transfer using techniques like Krishna et al. [2020, 2022b]. Similar efforts have also been undertaken in machine translation, [Popescu-Belis et al., 2019, Junczys-Dowmunt, 2019, Maruf et al., 2021], attempting to translate paragraphs/documents at once.
E More background on detectors of AI-generated text
In this section, we provide an overview of existing algorithms that have been developed for the purpose of detecting machine-generated text. Such algorithms fall into three main categories: (1) watermarking algorithms, which modify the generative algorithm to encode hidden information unique to the API (Appendix E.1); (2) statistical outlier detection methods, which do not modify
Table 5: The table shows the result of our survey of paraphrase generation papers from 2018 to 2022. We focus on four aspects: (1) whether a model can paraphrase multiple sentences at once, (2) whether a model is able to merge or split an input sentence when appropriate, (3) whether a model takes context surrounding the input sentence into consideration when paraphrasing, and (4) whether a model enables users to control the semantic and syntactic diversity of paraphrases.  ${}^{1}$  Granularity levels are word,phrase,and sentence.  ${}^{2}$  Meng et al. [2021] use context for their dataset construction, but do not leverage it during training/inference.  ${}^{3}$  The diversity score is a combination of the unigram Jaccard distance and the relative position change for unigrams.  ${}^{4}$  The code is represented by a three dimensional vector corresponding to semantic similarity as well as syntactic and lexical distances between the input and output sentences.
Paper	Multi-sentence	Merge / Splits	Contextual	Diversity Control
Iyyer et al. [2018]	X	X	X	Constituency parse
Li et al. [2018]	X	X	X	X
Roy and Grangier [2019]	X	X	X	X
Witteveen and Andrews [2019]	✓	?	X	X
Kumar et al. [2019]	X	X	X	X
Hu et al. [2019]	X	X	X	Decoding constraints
Chen et al. [2019]	X	X	X	Exemplar
Li et al. [2019]	X	X	X	Granularity control1
Goyal and Durrett [2020]	X	X	X	Exemplar
Lewis et al. [2020]	✓	?	X	X
Thompson and Post [2020]	X	X	X	n-gram overlap
Kumar et al. [2020]	X	X	X	Exemplar
Kazemnejad et al. [2020]	X	?	X	X
Krishna et al. [2020]	X	X	X	X
Rajauria [2020]	X	X	X	X
Meng et al. [2021]	X	X	X2	Diversity score3
Huang and Chang [2021]	X	X	X	Constituency parse
Lin et al. [2021]	✓	X	X	X
Goutham [2021]	X	X	X	X
Damodaran [2021]	X	X	X	Binary
Dopierre et al. [2021]	X	X	X	n-gram
Bandel et al. [2022]	X	X	X	Control code4
Hosking et al. [2022]	X	X	X	Syntactic sketch
Yang et al. [2022]	X	X	X	Exemplar+Keywords
Xie et al. [2022]	X	X	X	X
DIPPER (ours)	✓	✓	✓	✓
the generative algorithm but look for inherent artifacts in generated text (Appendix E.2); and (3) classifiers trained to discriminate machine-generated text from human-written text (Appendix E.3). Finally, in Appendix E.4, we compare and contrast our work to Sadasivan et al. [2023], who also note the efficacy of paraphrasing attacks but do not consider a retrieval-based defense in their pessimistic conclusion about the fate of AI-generated text detection.
E.1 Watermarking language model outputs
A "watermark" is a modification to the generated text that can be detected by a statistical algorithm while remaining imperceptible to human readers. Effective watermarks are difficult to remove and have little effect on the quality of generated text. Prior work attempted to watermark natural language using syntax tree manipulations [Topkara et al., 2005, Meral et al., 2009], and this area has gotten renewed interest with large language models generating human-like text [Abdelnabi and Fritz, 2021, Grinbaum and Adomaitis, 2022]. Most recently, Kirchenbauer et al. [2023a] propose a simple algorithm that only requires access to the LLM's logits at each time step to add watermarks. The watermark can then be verified with only blackbox access to the LM and knowledge of a specific hash function. This algorithm operates in three steps:
1. Mark a random subset of the vocabulary as "green tokens" (or tokens representing the watermark, as shown in Figure 1) using the hash of the previously generated token as a random seed. A total of  $\gamma |V|$  tokens are marked green where  $\gamma$  is the fraction of the tokens that are watermarked with default  $\gamma = 0.5$ .
2. Increase the logit value for every green token by a constant  $\delta$  ( $= 2$  by default), which denotes the watermark strength. This raises the probability of sampling green watermarked tokens, especially for high-entropy distributions.
3. Sample sequences using decoding algorithms such as nucleus sampling [Holtzman et al., 2020], leveraging the modified probability distribution at each timestep before truncation.
Detecting the watermark: Verifying whether a text is generated by a watermarked LM is possible with just knowledge of the hash function and tokenizer. Specifically, the verifier tokenizes the text and counts the number of green tokens it contains. This is used to calculate the standard normal score  $(z$ -score) for the hypothesis test. If the sequence with  $T$  tokens contains a certain number of the green token (denoted as  $|s|_G$ ), the  $z$ -score can be computed by:
$$
z = (| s | _ {G} - \gamma T) / \sqrt {T \gamma (1 - \gamma)}
$$
Intuitively, a higher  $z$ -score implies it is less likely for a human to have written the text (null hypothesis) since it contains a higher than expected number of green tokens. Kirchenbauer et al. [2023a] recommend using a high  $z$  value ( $z > 4$ , or  $p < 3 \times 10^{-5}$ ) to reduce the risk of false positives (human-written text classified as AI-generated). Low false positive rates are critical in AI-generated text detection algorithms [OpenAI, 2023a]—we discuss this in Section 4.1.
E.2 Statistical outlier detection methods
Unlike the watermarking algorithms, outlier detection algorithms make no modification to the generative algorithm. Instead, they attempt to distinguish between human-written and machine-generated text based on the presence of artifacts in generated text [See et al., 2019, Holtzman et al., 2020]. Early methods detect statistical irregularities in measures such as entropy [Lavernne et al., 2008], perplexity [Beresneva, 2016], and  $n$ -gram frequencies [Grechnikov et al., 2009, Badaskar et al., 2008]. After the release of GPT-2, Gehrmann et al. [2019] introduced the GLTR visualization tool to assist human verifiers in detecting machine-generated text. Most recently, the release of ChatGPT has prompted the development of two new tools, namely a closed-source tool called GPTZero [Tian, 2023], and open-source DetectGPT [Mitchell et al., 2023]. DetectGPT uses an observation that model-generated text lies in the negative curvature regions of the model's log probability function. It constructs multiple perturbations of the model generated text (using a mask-and-fill strategy), and compares the log probability of the perturbations with the unperturbed generation. Text is considered model generated if the log probability of the unperturbed text is significantly higher than the log probability of perturbations.
E.3 Classifiers
The third class of detection methods relies on classifiers that are fine-tuned to distinguish human-written text from machine-generated text. Early efforts in this vein use classifiers to detect fake reviews [Hovy, 2016] and fake news [Zellers et al., 2019]. Other related studies examine classification performance across domains [Bakhtin et al., 2019] and decoding strategies [Ippolito et al., 2020]. Such studies inspired others to use their insights to improve generative performance [Deng et al., 2020, Krishna et al., 2022a]. Most recently, OpenAI fine-tuned a GPT model to perform this discrimination task and released it as a web interface [OpenAI, 2023a]. They fine-tuned this classifier using generations from 34 language models, with text sourced from Wikipedia, WebText [Radford et al., 2019], and their internal human demonstration data.
E.4 Comparison to Sadasivan et al. (2023)
In very recent concurrent work, Sadasivan et al. [2023] also demonstrate the utility of paraphrasing attacks against AI-generated text detectors. While their work makes use of off-the-shelf sentence-level paraphrase models, DIPPER possesses advanced discourse-level rewriting capabilities as well
as fine-grained diversity control, which allows us to thoroughly analyze the effectiveness of various paraphrasing strategies. Our experiments also encompass more tasks, datasets, and detection algorithms. Moreover, we evaluate larger language models like GPT3.5-davinci-003. Finally and most importantly, our retrieval-based defense directly contradicts the "impossibility result" of Sadasivan et al. [2023] and its associated proof, which states that even an optimal detector will approach the performance of a random classifier as the distance between the distributions of LLM-generated text and human generated text goes to zero. Since our detector does not rely on properties of the text but rather a corpus search, the quality of the generated text is irrelevant to the effectiveness of our detector, and thus their proof does not apply to our method.
F More experimental details of our attack experiments
F.1 Details for training our paraphraseer DIPPER
Our paraphrase DIPPER is a sequence-to-sequence Transformer neural network [Vaswani et al., 2017], initialized with the T5-XXL 1.1 checkpoint [Raffel et al., 2020] and fine-tuned on our paraphrase generation data, using early stopping on validation loss for held-out novels. During training, we find it helpful to paraphrase a maximum of 3 consecutive sentences at time, which leads to better adherence to control codes. Our models are implemented in JAX [Bradbury et al., 2018] using the T5X library [Roberts et al., 2022] with the default fine-tuning hyperparameters. Our final dataset contains 6.3M paraphrase pairs. Training was done on 64 cloud TPUv3 chips, and took 6-12 hours to complete. At inference time, we use nucleus sampling [Holtzman et al., 2020] with  $p = 0.75$  and a variety of control codes.
To make our paper more intuitive, we have slightly modified the notation that our actual pretrained model uses. Our pretrained model uses control codes  $100 - L$  and  $100 - O$ , denoting lexical/order similarity rather than diversity. Also,  $\langle \text{sent} \rangle$  is used instead of  $\langle \text{p} \rangle$ . We will clearly document this in the code release.
F.2 Long-form question answering data processing
In Section 4 evaluate long-form question answering [Fan et al., 2019], in which an LM must answer a how/why question (e.g., Why are almost all boats painted white?) with a 250-350 word answer. To build a long-form question answering dataset, we scrape questions from the r/explainlikeimfive subreddit posted between July to December 2021.[21] We randomly sample 500 questions from each of six popular domains on the subreddit (biology, physics, chemistry, economics, law, and technology) and pair each question with its longest human-written answer, which yields 3K long-form QA pairs.
F.3 Are successful attacks against model-specific detectors really an attack success?
In our experiments, we attack two model-specific detectors (watermarking, DetectGPT) in addition to model-agnostic detectors (OpenAI classifier, GPTZero, RankGen). Model-specific detectors have been specifically designed to judge whether a response was generated by a particular model. After paraphrasing, a response is not truly generated by that specific model anymore—it has been generated by the paraphrasing model. Hence, in a sense, the inability of these detectors to detect paraphrased text denotes their robustness in model-specific detection.
While the argument above has merit, we argue that it is critical for model-specific detectors to be robust against paraphrasing attacks. Given their strong performance over model-agnostic methods (Section 4.3), and the large risk of perturbation attacks (human-edited or automatically-edited), we think it is important for model-specific detectors to bake perturbation robustness in their design for better downstream usability. Currently, the low performance of model-agnostic detectors (Section 4.3) makes them quite unusable, and OpenAI even took down their text classifier due to low performance [OpenAI, 2023a]. Currently, model-specific detectors (including watermarking and retrieval) seem to be the only reliable path towards robust AI-generated text detection.
21We choose this period since current language models have been trained on internet data available before June 2021 [OpenAI, 2022], this prevents verbatim copying from training data.
G More retrieval-based detection experiments
G.1 Controlled comparisons of retrieval with other AI-generated text detectors on open-ended text generation
We conduct a controlled comparisons of retrieval on the open-ended text generation task with Wikipedia prompts (see Section 5.2). The result of the experiment is presented in Table 7.
G.2 Robustness against text shortening and mixing attacks
Our experiments in Section 5 assumed that an attacker would paraphrase the entire LLM output to evade detection. However, an alternative attack strategy that might be adopted in practice is text shortening, or text mixing [Kirchenbauer et al., 2023b]. Here, attackers only use a substring of an LLM generated output (instead of the full output), and optionally mix it with text from other sources (like other LLM-generated outputs or human-written text).
In this section, we conduct some preliminary experiments to show the robustness of retrieval-based detection to text shortening and mixing attacks. We adopt the same experimental setup as Section 5.3, utilizing a BM25 retrieval on a PG19 machine-generated corpus of 2M responses.
In Table 6 we see that retrieval is a promising defense strategy even against text shortening and mixing attacks. Overall, we see that unperturbed random substrings (from single or multiple generations) can still be detected quite easily (86.2% to 94.7% detection rate at 1% FPR). However, adding DIPPER paraphrasing on top of that reduces accuracy (56.1% to 68.8% detection rate).
Table 6: Preliminary experiments measuring the robustness of retrieval-based detection to LLM response shortening and mixing attacks. Overall, we find that retrieval is a strong detector for truncated as well as mixed responses. Experiments are conducted with BM25 retrieval in the PG19 data setup with a retrieval corpus of size 2M responses (from Section 5.3). Results shown are true positive rates at the  $1\%$  FPR threshold.
Candidate Text	Retrieval detection rate (1% FPR)
unperturbed LLM response	100.0
DIPPER-paraphrased response	98.2
50% truncated DIPPER-paraphrased response	72.6
50% truncated LLM response	86.2
50% LLM response 1 + 50% LLM response 2	94.7
50% DIPPER-paraphrased response	68.8
50% DIPPER-paraphrased response 1 + DIPPER-paraphrased response 2	56.1
H ROC curves at different FPR
See Figure 8.