Paper: Raidar: geneRative AI Detection viA Rewriting (ICLR-2024)

Reviewer comment (Experiments):
In the discussion of datasets and model baseline used, the definition and characteristics should definitely be expanded by including the size of the dataset plus other unique characteristics of the data that’s worth mentioning as well as the quantified performance of the baseline models just to give an idea to the readers of their relevance and why they are being used/compared in the study.

Author response:
**Characteristics of the dataset.**

We show the statistics of the dataset in Table 6,7 in the appendix.

Reviewer comment (Experiments):
The discussion in Section 4 seems to be too condensed and there are sudden insertions of models for comparison. Claude was tested but was not mentioned in the baseline models. The reasons for this and why Claude is a viable candidate for comparison are unclear.

Author response:
**Justification for Additional LLM generated text, such as from Claude, included for comparison.**

To evaluate the robustness of our model, we expanded our testing to include LLMs from companies beyond OpenAI. We incorporated Claude data from the Ghostbuster paper, as suggested by reviewer 3, because Claude, developed by Anthropic, differs from OpenAI's models, offering a diverse robustness check. However, due to Ghostbuster's data limitations, we could only obtain one dataset for Claude. Furthermore, we integrated LLaMA, an LLM from Meta, and conducted extensive tests across six datasets. The results demonstrate our method's ability to effectively identify content generated by LLaMA, despite not being trained on it. This highlights our approach's capability to reliably detect machine-generated content across various company products.

Reviewer comment (Evaluation):
There is no concrete or definitive discussion for the results presented in Tables 1 and 2, which are the core of the study. The table captions and Section 4.3 simply state the observed increments against the previous systems without further explanation. More discussion is needed on particular weaknesses in the texts by previous works that the current work addresses, with examples. Clarification is needed on why the proposed methods achieve large performance jumps in datasets such as code and Arxiv compared to natural language datasets. In the robustness experiments, the previous detector models are left out, but they should be included to enable better comparison of true performance.

Author response:
**Discussion in Table 1,2**

Thank you for your feedback. We will update in our revision.
Our investigation into the significant performance improvement on code and arXiv datasets revealed that these are highly specialized tasks. LLMs like GPT often fall short in matching the quality of human-generated content in these areas, leading to a more pronounced disparity between machine-generated and human-generated texts, especially in rewriting scenarios. Previous methods, which primarily used smaller models such as OPT and GPT-2 as their score functions, struggled more with complex tasks like coding and academic writing. In contrast, our method utilizes the advanced capabilities of GPT-3.5, which better handles these challenging tasks, resulting in superior outcomes than prior approach. Additionally, for a comprehensive robustness evaluation, we included the state-of-the-art DetectGPT in Table 4, allowing for a direct comparison of its effectiveness.

Reviewer comment (Experiments):
To strengthen the paper, the authors should also experiment with second-language or non-native speaker writing corpora such as TOEFL or ESL essay corpus to assess whether the proposed method has the known limitation of confusing AI-generated texts with non-native speaker-written texts. Sources include existing works (https://arxiv.org/abs/2304.02819) and online databases (https://www.clarin.eu/resource-families/L2-corpora, https://uclouvain.be/en/research-institutes/ilc/cecl/learner-corpora-around-the-world.html). Regardless of the result, this would add valuable insight to the paper.

Author response:
**Experiment with second-language and non-native speaker**

Thank you for your suggestion in strengthening our paper. We have cited the paper [1] and conducted the experiment in our paper. We utilized 200 examples from the ASAP dataset, featuring TOEFL essays authored by non-native English speakers. Our experiments, detailed in the subsequent table, involved training on the ASAP dataset and testing on it, as well as training on our original arXiv dataset and testing on ASAP in an out-of-domain context. Our method, not relying on word semantics, showed strong generalization to the previously unseen ASAP data. Our approach won't mistakenly identify non-native speaker-written texts as AI-generated, given that LLMs might also classify these as lower quality, prompting substantial rewrite edits. Our empirically results below validate this.

| Training Source | Testing Source | F-1 Detection Score |
|-----------------|----------------|---------------------|
| ASAP Dataset    | ASAP Dataset   | 98.76               |
| Arxiv Dataset   | ASAP Dataset   | 81.16               |
| Arxiv Dataset   | Arxiv Dataset  | 81.95               |

Reviewer comment (Experiments):
How large is the feature set or dimension for training the classifiers? Did the authors conduct hyperparameter optimization with the classifiers?

Author response:
**Q1: Feature size and hyperparameter for training classifier.**

The Feature size is 28 and we did model selection for the classifier.

For each input, our method generates one Levenshtein score and three sets of bag-of-words edit features—derived from one, two, and three bags of words—resulting in four distinct metrics per prompt. We apply seven different rewriting prompts to each input, leading to a total of 28 features. To process these features, we employed the Logistic Regression and XGBoost algorithms using their default configurations in the Sklearn package. After evaluating their performance, we selected and presented the results from the classifier that demonstrated better efficacy. Based on our results, we use logistic regression for all our experiments except for the essay dataset.

Reviewer comment (Experiments):
How is the setup for comparison done for the preliminary experiments shown in Tables 1 and 2? What classifier was used for these experiments? There were two algorithms mentioned (LR and XGBoost), but these were never specified in the tables. The same lack of clarity applies to Tables 4 and 5. This part should be revised and enriched with details to clarify which classifier was used for each result.

Author response:
**Q2: Classifier Setup for Table 1,2,3,4.**

We use logistic regresssion for all experiments except for Table 1's student essay, which we used XGBoost. We run both and use the one that performs higher. We find XGboost performs better on student essay tasks, so we used XGBoost for this task in Table 1 and logistic regression for all other experiments.

Reviewer comment (Experiments):
There are six datasets chosen for experimentation, but why are only subsets used for results presented in Tables 3, 4, and 5?

Author response:
**Q4: Dataset choice in Table 3-5.**

Thank you for your question. In our experiments for Tables 3 and 4, adjustments in the generated texts were necessary, involving changes to both the prompts and the generation models. However, since the datasets comprising News, Creative Writing, and Student Essays, as released by Ghostbuster, are fixed and do not permit such manipulations, we were unable to conduct analyses on these datasets for these tables. 

Following your suggestion, for Table 5, we have provided comprehensive results since we can vary the rewriting model, ensuring that the entire scope of our findings and analyses for this specific table is thoroughly detailed and accessible. We show the complete results below:

| LLM for Rewriting | News  | Creative Writing | Student Essay | Code  | Yelp  | Arxiv |
|-------------------|-------|------------------|---------------|-------|-------|-------|
| Ada               | 55.73 | 62.50            | 57.02         | 77.42 | 73.33 | 71.75 |
| Text-Davinci-002  | 55.47 | 60.59            | 58.96         | 82.19 | 75.15 | 59.25 |
| GPT-3.5 turbo     | 60.29 | 62.88            | 64.81         | 95.38 | 87.75 | 81.94 |

Reviewer comment (Evaluation):
In Section 4.4, the statement “In Table 5, while all models achieve significant detection performance” suggests that a statistical test was conducted. Details of this test should be provided. If no such test was conducted, the wording should be changed from “significant” to “substantial.”

Author response:
**Q5: Replace word ”significant” with “substantial”**

Thank you. We updated.

Reviewer comment (Experiments):
Why is only the Yelp data used for special test cases such as the effect of content length? Why are other datasets not included? This selectiveness lacks explanation.

Author response:
**Q6: Additional results for effect of content length**

Thank you for your suggestion. We've included supplementary results on the impact of content length in Appendix Figures 7, 8, and 9. Generally, longer content tends to enhance detection performance, with the notable exceptions of code and arXiv papers. We conjecture that this deviation may be attributed to LLMs struggling with rewriting longer pieces of code or academic papers, likely due to the intrinsic complexity associated with these specific tasks.

Reviewer comment (Reproducibility):
The paper is missing some important details. For example, the paper does not provide a complete explanation of how the human reviews from the Yelp dataset were selected, nor does it provide a full explanation of how GPT 3.5 was prompted to generate data in the code/Yelp/arXiv domains. The paper also does not describe which scoring model was used for the DetectGPT results.

Author response:
**1. How Human Reviews from the Yelp was selected**

We select the first 2000 reviews from the Yelp Review dataset.

**1. How GPT 3.5 was prompted to generated data**

**Code:** Human eval dataset offers code specification and the completed code for each data point. We first use GPT to generate a detailed description of the function of the code by prompting it with “Describe what this code does {code specification}{code}”. The result, termed {pseudo code}, is an interpretation of the code. Subsequently, we prompt GPT with "I want to do this {pseudo code}, help me write code starting with this {code specification}," to generate Python code that adheres to the given input-output format and specifications. This way, we create the AI-generated code data.

**Yelp:** When tasked with crafting a synthetic Yelp review, prompting GPT-3.5 with "Help me write a review based on this {original review}" resulted in verbose and lengthy text. However, we discovered that using the prompt "Write a very short and concise review based on this: {original review}" yielded the most effective and succinct AI-generated reviews.

**Arxiv:** In our experiment with Arxiv data, which includes titles and abstracts, we synthesized abstracts by using the title and the first 15 words of the original abstract. We employed the prompt “The title is {title}, start with {first 15 words}, write a short concise abstract based on this:”, which successfully generated realistic abstracts that align with the titles.

**1. Which scoring model was used for DetectGPT:**

We used the facebook/opt-2.7B model, which boasts a larger parameter count compared to the GPT-2 model with its 1.5B parameters.
We use the T5-3B model for the mask filling model, which is stated to be used for almost all experiments in the original paper.

Reviewer comment (Evaluation):
I believe the results on OOD generalization are slightly misleading. Because DetectGPT and GPTZero should produce the same scores in-domain and out-of-domain, GPTZero actually has the highest score for the OOD creative writing task.

Author response:
**2. Discussion on Results for OOD Generalization**

Thank you for bringing this out. Yes, DetectGPT and GPTZero display consistent performance in both in-domain and out-of-domain tests, with our model outperforming them in out-of-domain scenarios, except for GPTZero in Creative Writing. However, The reliability of GPTZero's zero-shot capabilities for Creative Writing is uncertain due to its blackbox nature and unknown training data. Table 1 shows GPTZero's performance variability across tasks, notably dropping to 25 in Yelp Reviews, suggesting weak out-of-domain generalization and large variation over tasks.
We will include those discussions over this in our revision.

Reviewer comment (Experiments):
Table 4 (robustness across generation models) is difficult to interpret without results from the baselines. Although the model seems to generalize well to generators it was not originally trained on (ada and davinci), these are also weaker models which (as stated in the table caption) results in an easier detection problem. Additionally, generalization results in the paper would benefit from the inclusion of models which are not from OpenAI, since these are likely more similar to one another than models like Claude, LLaMA, etc.

Author response:
**3. Baselines for Table 4 Results**

Following your suggestion, we add DetectGPT baseline and updated Table 4:

| LLM Model Used for Text Generation | In Distribution Code | In Distribution Yelp | In Distribution arXiv | Ours Out of Distribution Code | Ours Out of Distribution Yelp | Ours Out of Distribution arXiv | DetectGPT Code | DetectGPT Yelp | DetectGPT arXiv |
|------------------------------------|----------------------|----------------------|-----------------------|-------------------------------|--------------------------------|---------------------------------|----------------|----------------|-----------------|
| Ada                                | 96.88                | 96.15                | 97.10                 | 62.06                         | 72.72                          | 70.00                           | 67.39          | 70.59          | 69.74           |
| Text-Davinci-002                   | 84.85                | 65.80                | 76.51                 | 75.41                         | 51.06                          | 60.00                           | 66.82          | 68.70          | 66.67           |
| GPT-3.5-turbo                      | 95.38                | 87.75                | 81.94                 | 81.07                         | 71.43                          | 48.74                           | 70.97          | 65.45          | 66.67           |
| GPT-4-turbo                        | 80.00                | 83.42                | 84.21                 | 83.03                         | 79.73                          | 74.02                           | 70.97          | 69.64          | 66.99           |
| LLAMa 2                            | 98.46                | 89.31                | 97.87                 | 70.96                         | 89.30                          | 74.41                           | 68.42          | 67.24          | 66.67           |

Reviewer comment (Experiments):
Are all detection tasks in this paper at the paragraph level? How does the median length of documents vary across tasks? Did you try the method for any longer documents?

Author response:
**Q1: Detection tasks at paragraph level?**

Yes, our detection tasks are at the paragraph level. We calculate the median length of documents across tasks and shown in Table 6,7, and also below:
|                      | News | Creative Writing | Student Essay | Code | Yelp | Arxiv |
|----------------------|------|------------------|---------------|------|------|-------|
| Dataset Size         | 730  | 973              | 22172         | 164  | 2000 | 350   |
| Median Length        | 38   | 21               | 96            | 96   | 21   | 102   |
| Minimum Length       | 2    | 2                | 16            | 2    | 6    | 19    |
| Maximum Length       | 122  | 295              | 1186          | 78   | 1006 | 274   |

Reviewer comment (Experiments):
Did you experiment with combining the invariance, equivariance, and uncertainty features?

Author response:
**Q2: Experiment combining invariance, equivariance, and uncertainty**

Yes, following your suggestion, we tried combining them by concatenating the features over two datasets. Yet we did not achieve better results by combining.

| Methods   | News  | Creative Writing |
|-----------|-------|------------------|
| Single    | 60.29 | 62.88            |
| Combined  | 53.72 | 58.18            |

Reviewer comment (Evaluation):
It would be nice to actually check the quality of human/LM written ext., i.e. which between the human and LM text do another set of humans prefer? It could be that the LM-written text is actually better quality (and thus requires fewer rewrites).

Author response:
Thank you for your suggestion. Yes, we manually checked the quality of human/LM text. In general, machine generated text is similar or less preferred by human than human-written text, suggesting that LLM perceive text quality in a different way than human. Due to the time limit, we conducted a user study on Arxiv and Yelp datasets over 40 random examples over 3 users due to the time constraint, produced by GPT-3.5. We show the results below. We will conduct experiments over more users.

| | Yelp Reviews | Arxiv Abstract |
|---------------------------------------------------------------|--------------|----------------|
|                % that Machine Generated Text are Preferred Human Written Text                                                | 53.3%        | 26.7%          |

Reviewer comment (Experiments):
It would be nice to include a discussion on how this method fares when the LLM is instruction-tuned to do rewrites the way humans do rewrites. Given that text rewriting is a widely used use case for LLMs, it would not be a surprise if these models are instruction-tuned on a task mixture that includes text rewriting.

Author response:
Thank you for your suggestion. In Section 4.4 of our paper, we explored a related concept by prompting the model to emulate human-style writing. Our findings indicated a reduction in detection effectiveness under these conditions. This scenario can be viewed as a rudimentary form of LLM fine-tuning, where the prompt is adjusted rather than the model's parameters. Extrapolating from this trend, it's plausible that more extensive LLM fine-tuning would further diminish the efficacy of our detection method. Consequently, future research should focus on adapting detection models to effectively identify content from fine-tuned LLMs that closely mimic human rewrites. As our study is pioneering in examining rewriting for detection, and considering the high costs associated with fine-tuning LLMs, we suggest this as future research.

Reviewer comment (Experiments):
Proposed method is critically dependent on LLM. Any change to LLM due to continual fine-tuning with new data might have unknown consequences for the detection method. The method might not be robust to LLM fine-tuning.

Author response:
**1. Method Robustness to different LLM, such as finetuning**

Thank you for your question. We ran the experiment on GPT-3.5 and GPT-4. GPT-4 can be treated as an advanced, continual fine-tuned LLM with new real-world data. We show the results In Table 7 in the appendix, as well as follows. 

Our detector, trained only on GPT-3.5, can still work well on detecting GPT-4-Turbo generated data. Our method outperforms  established state-of-the-art DetectGPT. This means our method is robust to LLM finetuning.

| Test Data source | Data Detector | Code | Yelp | Arxiv |
|------------------|---------------|------|------|-------|
| GPT-3.5-Turbo    | Ours trained with GPT-3.5-Turbo | 95.38 | 87.75 | 81.94 |
| GPT-4-Turbo    | Ours trained with GPT-3.5-Turbo | 83.07 | 79.73 | 74.02 |
| GPT-4-Turbo      | Baseline DetectGPT              | 70.97 | 66.94 | 66.99 |

In Table 4 in our paper, we also evaluate our method over detecting text generated from 5 different LLMs, demonstrating the robustness of our detector to different LLMs.

Reviewer comment (Evaluation):
Though the proposed method is simple and computationally low cost, there is a cost associated with every call to the detection algorithm because of the calls made to LLM for rewriting. This might not be acceptable in some scenarios it is cheaper and desirable to have a model decoupled for LLM after training.

Author response:
**2. The cost associated with calling the LLM API**

Thank you for bringing up the cost. While there is no free algorithm, in Table 5, we have shown using cheaper OpenAI models, such as Ada which is 10 times cheaper for detection. Given the price reduction of OpenAI’s recent API cost, our approach will be more cost efficient.

In addition, to avoid the cost of OpenAI API, we also show that open-source LLaMA-7B, which can be run on a single RTX 3090 Ti GPU, can also be integrated into our framework, avoiding the fee to pay for OpenAI API calls. As shown below, it only leads to a performance drop of a few points, still achieving up to 85 points F-1 detection accuracy score.

| LLM for Rewriting | News  | Creative Writing | Student Essay | Code  | Yelp  | Arxiv |
|-------------------|-------|------------------|---------------|-------|-------|-------|
| Ada               | 55.73 | 62.50            | 57.02         | 77.42 | 73.33 | 71.75 |
| Text-Davinci-002  | 55.47 | 60.59            | 58.96         | 82.19 | 75.15 | 59.25 |
| GPT-3.5 turbo     | 60.29 | 62.88            | 64.81         | 95.38 | 87.75 | 81.94 |
| LLAMA 2           | 56.26 | 61.88            |       60.48      | 85.33 | 74.85 | 72.59 |

Reviewer comment (Writing):
In Table 1 you write "The results are in domain testing, where the model has been trained on the same domain." but you also use the term "in distribution" Are these equivalent terms? I assume you consider each of the datasets as a different domain (e.g. news domain vs creating writing). What is the difference between in-distribution and in-domain?

Author response:
**Q1: Term Usage, the difference between in-distribution and in-domain**

Sorry for the confusion. Yes, they are the same terms. We will use a consistent “in domain” word in our revision.

Reviewer comment (Experiments):
In Table 2, what is the domain for training and what is the domain for testing?

Author response:
**Q2: The training and testing domain for Table 2**

For all experiments in Table 2, we use logistic regression, and use the same source and target for invariance, equivariance, and uncertainty.
News: train on Creative Writing and test on News.
Creative Writing: train on News and test on Creative Writing.
Student Essay: train on News, and test on student Essay.

Reviewer comment (Experiments):
Would the effectiveness of detection improve further if multiple LLMs are asked to rewrite the input text and invariance scores for each LLM are used as features for training the logistic regression classifier? Also, would edit distance between the rewritten texts from different LLMs be useful as additional features for such a classifier?

Author response:
**Q3: The effectiveness of using multiple LLM to rewrite the input text**

Thank you for your suggestion. We conducted this study on Yelp and Arxiv, results show that the suggested method can improve the detection score by up to 10 points. We show the results in Table 9 and also follows:

| Methods               | Yelp Reviews | Arxiv Abstract |
|-----------------------|--------------|----------------|
| GPT-3.5 Only          | 87.75        | 81.94          |
| GPT-3.5 + Ada         | 85.71        | 92.85          |
| GPT-3.5 + Davinci-003 | 85.53        | 88.40          |
| GPT-3.5 + Davinci-003 + Ada | 81.76  | 90.00          |


**Q3: Edit distance between the rewritten texts from different LLMs as additional features.**

Yes, they can be useful features. Following your suggestion we run the experiment on Yelp and arxiv. We can achieve up to to 8 points gain on arxiv dataset. We also find the improvement is dataset dependent.

| Methods               | Yelp Reviews | Arxiv Abstract |
|-----------------------|--------------|----------------|
| GPT-3.5 Only          | 87.75        | 81.94          |
| GPT-3.5 / Ada         | 67.85        | 89.21         |
| GPT-3.5 / Davinci-003 | 78.41       | 81.94        |
| Ada / Davinci-003 + Ada | 66.25  | 90.51          |