On the Zero-Shot Generalization of Machine-Generated Text Detectors
Authors: Xiao Pu
Jingyu Zhang
Xiaochuang Han
Yulia Tsvetkov
Tianxing He
Venue: EMNLP 2023 Findings (2023)
Keywords: detection
NLG
zero-shot generalization

Abstract: The rampant proliferation of large language models, fluent enough to generate text indistinguishable from human-written language, gives unprecedented importance to the detection of machine-generated text. This work is motivated by an important research question: How will the detectors of machine-generated text perform on outputs of a new generator, that the detectors were not trained on? We begin by collecting generation data from a wide range of LLMs, and train neural detectors on data from each generator and test its performance on held-out generators. While none of the detectors can generalize to all generators, we observe a consistent and interesting pattern that the detectors trained on data from a medium-size LLM can zero-shot generalize to the larger version. As a concrete application, we demonstrate that robust detectors can be built on an ensemble of training data from medium-sized models.

On the Zero-Shot Generalization of Machine-Generated Text Detectors
Xiao Pu
Jingyu Zhang
Xiaochuang Han
Peking University
Johns Hopkins University jzhan237@jhu. edu
University of Washington xhan77@cs wwt.//ccw.wssinnttoncccc edu
puxiaoêstu. pku. edu.cn
Yulia Tsvetkov
Tianxing He University of Washington moosehe@cc.washhneee edu
University of Washington
Abstract
et al., 2023; Kirchenbauer et al., 2023; Zellers et al., 2019). However, in most studies, the de- tector is tested on the same generator model that it is trained/tuned on.
The rampant proliferation of large language models, fluent enough to generate text indistin- guishable from Human-written language, gives unprecedented importance to the detection of machine- generraee text. This work is moti- vated by important research question: How will the detectors of machine-generater text perform on outputs of new generator, detectors
This study is motivated by underexplored re- search question: How will the detector perform different generator that it is not trained on? This question is important due to multiple reasons: (I) LLMs are becoming increasingly large and expen- sive. Some of the most recent models are either too large to fit into a common GPU (e.g.. LLaMA- 65B) or require payment from the user (OpenAI, 2023), making the collection of training samples difficult. (2) The number of released LLMs is grow- ing rapidly. real application scenario. the detec- tor needs cover a wide range of LLMs (including the ones the detector is not trained on), instead of only one generator.
trained on? We begin by collecting generation data from a wide range of LLMs. and train neural detectors on data from each generator and test its performance held- out generators. While none the detectors can generalize all generators, sistent and interesting pattern that the detectors trained on data from a medium-size LLM can zero-shot generalize the larger version. As a concrete application, we demonstrate that ro- bust detectors can built on an ensemble of training data from medium-sized models.
observe con-
In this work, we collect generation data from a wide range of LLMs. We then train neural detec- tors data from each generator and test its perfor- mance on other generators. Our primary findings include: (1) In many cases, detectors can zero-shot generalize to a held-out generator (Figure 1). particular, we observe an interesting pattern that the detector for the medium version of an LLM can generalize to the larger version. (2) None of the detectors generalizes to all generators, implying that an ensemble of detectors/data necessary for wider coverage. (3) As a concrete application, we demonstrate that robust detectors can be built on an ensemble of training data from medium-sized models; Excluding large-versions only leads to a minor drop performance.
1 Introduction
Thanks to large-scale pretraining and tuning with human feedback (Ouyang et al., 2022), large lan- guage models (LLMs) (Chung et al., 2022: Zhang et al., 2022; Touvron et al., 2023) are now able to follow instructions and generate realistic and consistent texts. A prominent example is the re- cently developed ChatGPT GPT4 model (Ope- nAI, 2023), which when instructed, can write docu- ments, create executable code, or answer questions that require world knowledge. the erecirnegeenerated texts have high quality and cannot casily be distinguished from genuine human texts (Dugan et al., 2022; Gehrmann et al., 2019). These trends give an unprecedented importance the detection of machine- generatee text (Su et al., 2023; Jawahar al., 2020; Pagnoni et al., 2022a). A lot of work has been devoted to proposing ef-
lot of scenarios,
2 Methodology
We begin by giving an overview of our experiment structure and establish some notations. This study includes detection tailed in $3), and we construct train/dev/test sets for each generator. In $4.1, we train neural detectors on
ficient detection models or algorithms (Mitchell
range of popular LLMs (de-
Code and datasets will he available at https://github genesaggeeaaeleeeeeeeee lization. generalization.
sets

---

GPT-J
2.7
OPTU
2.78
GPt:nne 2.73
LLaMA-7
TB
GPT-10
GP T-nno
T-n00
1.38
1.38
LLaMA 1388
GPT-neo 138
GPT-nRo LA 13B
125M
125M
GPT-1
OPT-1
G
GPT-3
GPTS
GPT-2 md
GPT-2 md
GPT-2 md
GPT-21g lg
RealNews
GPT-21g
IMDBReview
GPT2 GPT22
Wiki
Acc-Gap
1%
M
Acc-Gap
2%
N Acc-Gapy < 4%
DM
Figure The generalization ability of detectors when applied differsent generators, measured by Acc-Gap (defined in $2). Detectors for a medium-size generator can zero-shot generalize to the larger-version model (highlighted by dotted green).
data from each generator and test its performance on other generators. In $4.2, we further consider an ensemble setting, where the detector is trained on data composed of multiple generators, and test its generalization ability on held-out generators.
RealNews
IMDBReview
M
Gap
Gapar D
Gap
Gap
GPT3
GPT4
3.64%
5.46%
1.47%
5.48%
LLa7B
LLa13B
-1.11%
1.18%
-1.50%
1.47%
Neo13B
Neo2.7B
0.04%
-2.11%
3.41%
GPT2gg
GPT2xl
-4.40%
4.94%
-0.02%
0.32%
We denote the detector model trained on data from generator model M as DM- Since we will test the accuracy of DM on data from different gen- erators, we use Acc (M)) to denote the accuracy DM on the test set of generator N. Finally, we define Acc-Gap Dar to measure the drop of perfor- mance when the detector is trained on generator M instead of N itself:
Table I: Acc-Gap from medium-version lerge-version models based on the ELECTRA detector, as well as in the opposite direction. The generalization from the medium-version model to the large-version is better than the opposite direction.
is subset of the c4 dataset (Raffel et al., 2019) named "realnewslike". For the reviews domain, utilize the IMDBreview dataset (Maas et al., 2011). As for the knowledge domain, we utilize the Wikipedia dataset (Foundation)
Acc-Gap DM ACcn (Dx) - (DM) (1)
We expect Acc-Gap to be larger than zero in gen- eral, and a large Ace-Gap means DM has poor generalization on generator N.
For each dataset, we first randomly sample 5000 real-world human-written samples, with a train/dev/test split ratio of 8:1:1. For all samples, we truncate the first 20 tokens serve as prompts and feed them into different generators for text tinuation, yielding 5000 machine- machine-generated sam- ples. For generation we apply nucleus sampling (Holtzman et al., 2020) with p = 0.96, following the setting in Pagnoni et al. (2022b). We truncate each sample so that its length is around 120 tokens. For all training or test sets in this work, we keep the ratio of human and machine text to be 1:1.
3 Experiment Setup
Generators We include the detection of a total of 13 popular LMs in our study, including GPT- 1 (Radford et al., 2018), GPT-2 models (small, medium, large, and xI) (Radford et al., 2019), GPT- 3 (text-davinci-003) (Ouyang et al., 2022), GPT-4 (OpenAI, 2023), three GPT-Neo models (125M, 1.3B and 2.7B) (Black et al., 2021; Gao et al., 2020), GPT-J (Wang. 2021), !), and LLaMA (7B and 13B) (Touvron et al., 2023).'
Detectors For data from each generator, we train a ELECTRA-large model (Clark et al., 2020) as binary classifier. The detectors were trained for epoch with learning rate of 5e-6 (training
Datasets We consider data from three do- mains: news, reviews and knowledge. For the news domain, we utilize the RealNews dataset, which
'The Wikipedia dataset we used is directly obtuined from Hugging Face, data subset "20220301 een" (Page link: https: / //huggingfar ace. co/datasets /wik wikipedia).
'hhree are larger versions of L.MMM, but we find it diffi- cult to fit it into our GPU.

---

Baseline
Pruned (Proposed)
Pruned (Comparison)
RealNews Acc(%)
Ensemble Vote
-GPT4
~GPT4-GNe02.7B
-GPT3
-L13B
Data-mix
Prob-avg
-L.13
-L13B-GPT2xl
-GPT4
-L7B
Average
81.1
81.1
88.0
88.6 (40.6)
88.2 (+0.2)
79.5 (-8.5)
91.6 (+3.6)
Worst-case
50.6
50.5
84.5
84.6 (+0.1)
83.6 (-0.9)
42.7 (-41.8)
80.8 (-3.7)
GPT4
50.6
50.5
88.2
(-3.6)
85.9 (-2.3)
42.7 (-45.5)
93.2 (+5.0)
GPT3
52.9
52.8
87.0
87.2 (40.2)
87.5 (+0.5)
52.3 (-34.7
91.4 (+4.4)
L13B
58.8
58.4
84.5
85.0 (+0.5)
83.6 (-0.9)
(-1.0)
80.8 (-3.7)
L7B
61.6
61.3
86.1 (+0.1)
(+0.1)
83.7 (-2.3)
82.6 (-3.4)
IMDBReview
Ensemble Vote
-GPT4
GGTT4GNee2.78
-GPT3
-L13B
Data-mix
Acc(%)
Prob-avg
-L13B
-L13B-GPT2xl
-GPT4
-L7B
Average Worst-case
85.2
85,1
94.3
93.2 (-1.))
94.1 (-0.2)
89.0 (-5.3)
93.7 (-0.6)
52.0
51.8
93.7
92.2 (-1.5)
92.7 (-1.0)
62.6 (-31.))
89.8 (-3.9)
GPT4
52.0
51.8
94.4
93.4 (-1.0)
94.4 (0)
(-31.61116))
94.4 (0)
GPT3
54.1
54.2
(-0.6)
93.5 (-0.2)
6-.6(-31.1)
(0)
L13B
70.2
70.1
94.0
92.2 (-1.8)
92.7 (-1.3)
93.5 (-0.5)
89.8 (-4.2)
L7B
72.1
71.9
94.1
(-1.)
93.9 (-0.2)
93.7 (-0.4)
91.7 (-2.4)
Table Accuracy of the baseline detectors and detectors trained on pruned data. "L13B/7B" refers to the LLaMA 13B/7B generator. We highlight the results for the data-mix model becuase it serves as the base for the pruned models. is shown that pruning out the large-version LLMs only induce minimal accuracy loss.
for more epochs only gives minimal improvement on the dev set). For the data-mix baseline and pruned models in $4.2, epochs of training is used. We use Adam optimizer (Kingma and Ba, 2014) with 31 = 0.9, 32 = 0.999. The average accuracy (when tested on the same generator it is trained on) of all detectors in news, review and knowledge domains are 94.1%, 96.2% and 94.9%, separately.
izes to GPT4.3 This somewhat surprising be- cause generations from the large-version generator is commonly considered to have higher quality. Interestingly, the generalization of the reverse di- rection weaker on RealNews and IMDBReview. As shown in Table 1, when attempting to general- ize from the large-version models to medium ones using ELECTRA detectors, the generalization per- formance is slightly worse, reflected by larger Acc-Gapp For the reason behind, we conjecture that comparing the larger model, the medium generator making similar but wider range of artifacts in generations, leading to a smooth gen- eralization to the detection the larger model. We also experiment with additional base detectors, e.g. ALBERT Large v2 (Lan et al., 2019) and find that the key observation the detectors trained for the medium-size models can generalize larger- size models-still holds. These results are omitted for brevity.
4 Experiment Results
4.1 On Generalization Ability of Detectors
As explained in $2, we compute Acc-Gap to reflect the generalization ability of detectors trained on each generator. Figure depicts the Acc-Gap of each detectorr/generator pair. We link from node M to node N Ace-Gap DM < T (good gener- alization), where the threshold T set to small number from {1%, 2%, 4%). On the other hand, in Figure 3 (Appendix B), node M is linked to node N when Acc-Gap PM > 20% (poor gener- alization). For statistical significance, we utilize bootstrapping (Koehn, 2004) and generate 100 vir- tual test sets by sampling with replacement from the original test set. We then conduct one-sided t-test and use a p-value of 0.05.
Sccond, Figure 3 (Appendix B) shows that none of the detectors, on its own, generalize to all generators. In particular, GPT3 and GPT4 seem "isolated" from other familics of generators. This result indicates that if we want an "universal" de- tector which can cover all generators, an ensemble of detectors/data is necessary. We explore this di- rection in the next section.
We observe across the three datasets. First, the detectors for the medium-version L.MSs can generalize to the large-version models. For example, DLLAMA7B generalizes to LLaMA13B, and DGPTS general-
interesting patterns shared
Strictly speaking, GPT3 is not a "small version" of GPT4, But they are from the same company, and our experiments consistently show they are strongly related.

---

4.2
Pruning Out Large-Version LLMs in Mixed Training Dataset
88
We now demonstrate concrete application of our findings, and the following realistic threat scenario is considered: The task is still binary classification but the machine text is composed of generations from a range of models (listed in $3). For simplic- ity, we use a uniform data ratio for the generators. Following results of the last section, an ensemble of detectors/data is necessary. We begin by com- paring two baselines: (1) Model ensemble, where we aggregate predictions from all detectors by ma- jority voting confidence (probability) average; (2) Data mixing, where train new ELECTRA- large detector by mixing up the training data from all generators.*
Aucuracy
86
84
GP71
GPT2 sm
GPT2 md
GPT2 lg
GPTZ d
GP13
GPT4
Gneo 125M
Gree 1.
Gree 2.7G
GPD)
UaMA 78
LAMA 13g
Gneo? -1138 -6P72a
Figure 2: Accuracy comparison on each generator be- fore and after pruning out four large-version LLMs the RealNews dataset.
parison experiments where both medium and large versions are pruned out. As expected, this results in worse performance on detection of the pruned generators, reflected by the worst-case accuracy. Especially, the comparison experiment of pruning out both GPT3 and GPT4 is quite alarming: The detector trained from combined data of all other generators only has accuracy around 42% (Real- News) or 62% (IMDBReview). This implies that if OpenAl did not give public access to generations of the two models, existing detectors would fail.
For each baseline detector D. we report the aver- age accuracy on all generators, and the worst-case accuracy which is min A (D). Accuracy on the four largest generators is also reported. We con- duct experiments RealNews and IMDBReview datasets, and the results for baselines are shown in the left part of Table 2. It shown that the data- mix model outperforms the ensemble approach by a large margin. Therefore, we base our pruning experiments on the data-mix model.
Related Work
Following insights from the last section, we then prune out data from the versinn language models (i.e., GPT4 GPT-Neo2.78 BB, LLaMA13B and GPT-2xl) and train a detector by mixing up training data from the remaining generators. The degree of drop on the worst-case accuracy reflects the zero-shot generalization ability of the proposed detector.
We now discuss the literature most related to our work, and defer more complete review to Ap- pendix A. Pagnoni et al. (2022a) demonstrate the degraded performance of trained detectors under different threat scenarios, while the range of gener- ator models is not wide up-to-date as work. Liang al. (2023) study the bias of detectors for LLMs in the case of non-native English writers. a very recent and concurrent work, Mireshghallah et al. (2023) study the generalization of detectors under the DetectGPT (Mitchell et al., 2023) algo- rithm, which is also shown to be far from perfect. trained detector, DetectGPT relies access to the generator LLM, which might be expensive.
Also shown in Table 2, the accuracy of the pro- posed detectors (both average and worst-case) re- mains similar to or only slightly decreases com- pared to the data-mix baseline. Figure 2 provides detailed information on the changes in accuracy after pruning out four large-version models. The accuracy of the proposed detector only experiences a slight decrease (<3%) for GPT4 and LLaMAI3B Our results show that in the case of limited budget or computing, data from the medium- version LM can decently approximate the large- version in ensembled data collection.
Comparing
6 Conclusion and Discussion
In this work, we observe a generalization relation- ship among detectors trained on different genera- tors in three domains, where detectors for medium- version models demonstrate the ability to effec- tively generalize the larger-version. Building upon this finding, we prune out data from large- version generators in an ensembed training dataset and demonstrate that the performance loss is min-
On the right part of Table 2, we conduct com-
*We have tried another baseline where we average parameters from detectors. However, the performance worse than the majority-voting baseline, and we omit this result.
"Our data collection for GPT4 costs around $450.

---

imal. Our results indicate that practitioners with limited budget or computing resourees can use data from medium-size LLMs as a good approximation for the large version.
direction.
Acknowledgements
We thank the reviewers, the area chair, Prof. Xiao- jun Wan, Hao Wang, Shangbin Feng, and Yichen Wang. We gratefully acknowledge support from NSF CAREER Grant No. I1S2142739. This ma- terial is funded in part by the DARPA Grant un- der Contract No. HR001 120C0114. This research is supported in part by the Office of the Director of National Intelligence (ODNI), Intelligence Ad- vanced Research Projects Activity (IARPA), via the HIATUS Program contract #2022-22072200004
With the rapid release of various LLMs and gen- eration APIs, detector needs to cover a wide range of generators. While our work makes some initial progress, experiments show that the detection of an unseen (or non-public) generator is still a dif- ficult and open question. We hope our work could motivate more research devoted to this important direction.
Limitations
The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, ei- ther expressed or implied, of ODNI, LARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copy- right annotation therein.
Our work focuses on supervised detector mod- els and there are other approaches for machine- generated text detection (Appendix A). In a very recent and concurrent work, Mireshghallah et al. (2023) studies the generalization of detectors under the DetectGPT (Mitchell al., 2023) algorithm, which is also shown be far from perfect. Com- paring to a trained detector, DetectGPT relies on access to the generator LLM, which might be ex- pensive. It is also interesting to base the detector on a larger LM than ELECTRA-large, but we surmise the observations should be similar.
References
Sahar Abdelnabi and Mario Fritz, 2021, Adversarial wa- termarking transformer: Towards tracing text prove- nance with data hiding. In 2021 IEEE Symposium Security and Privacy (SP). pages 121-140. IEEE.
The zero-shot generalization ability of detectors shown in this work implies that different genera- tors are making similar artifacts based on which the detectors make decisions. As future work, would be interesting to examine the salient features (Zeiler and Fergus, 2014) and compare between machine/human-geneeee Finally, our experiments show that the detec- tion of an unseen non-public generator is still difficult and open question. For example, the combination of data from all other generators can not generalize to GPT3 and GPT4. This important research direction deserves more research efforts.
Anton Bakhtin, Sam Gross, Myle Ott, Yuntian Deng. Marc' Aurelio Ranzato, and Arthur Szlam. 2019. Real fake? learning to discriminate machine from human generated text.
text.
Sid Black, Gao Leo, Phil Wang. Connor Leahy, and Stella Biderman. 2021. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh- Tensorflow. If you this software, please cite it using these metadata.
Hyung Won Chung, Le Hou, Shayne Longpre. Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Al- bert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdh- ery. Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu. Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Ja- cob Devlin. Adam Roberts, Denny Zhou, Quoe Le, and Jason Wei. 2022. Scaling instruction-finctunee language models.
Ethics Statement
The detection of machine- machine-gemerated text has impor- tant applications such as detecting fake news and fake reviews on the internet. However, it could also introduce new risks: Malicious parties can use re- leased detectors to develop text generation systems that evade existing detectors in an adversarial man- ner. Our experiments show that the detection of an unseen (or non-public) generator is still difficult and open question, and we hope our work could motivate more research devoted to this important
Kevin Clark, Minh-Thang Luong, Quoe V Le, and Christopher D Manning, 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv: 10555.

---

Liam Dugan, Daphne Ippolito, Arun Kirubarajan. Sherry Shi, and Chris Callison-Burre Burch. 2022. Real or fake text?: Investigating human ability to de- tect boundaries between human- written and machine- generated text. arviv.
machine- -echireegeneeaated text detection under data lim- itation with contrastive learning. arXiv preprint arXiv:2212. 10341.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 201 I. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142-150, Portland, Oregon, USA. Association for Computational Lin- guistics.
Wikimedia Foundation. Wikimedia downloads.
Leo Gao, Stella Biderman, Sid Black, Laurence Gold- ing. Travis Hoppe, Charles Foster, Jason Phang. Ho- race He, Anish Thite, Noa Nabeshima, et al. 2020. The pile: An 800gb dataset of diverse text for lan- guage modeling, arXiv preprint arXiv:2101.00022
Fatemehsadat Mireshghallah, Justus Mattern, Sicun Gao, Reza Shokri, and Taylor Karg-Kirkpatrick. 2023. Smaller language models are better black-box machine-generated text detectors.
Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. GLTR: Statistical detection and visual- ization of generated text. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 111-116, Florence, Italy. Association for Computational Lin- guistics.
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D Manning. and Chelsea Finn. 2023. Detecigpt: Zero-shot machine-generated text detec- tion using probability curvature. arXiv preprint arXiv:2301.11305
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text de- generation. In International Conference on Learning Representations.
OpenAL. 2023. Gpt-4 technical report.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744,
Daphne Ippolito, Daniel Duckworth, Chris Callison- Burch, and Douglas Eck. 2020. Automatic detection of generated text easiest when humans are fooled.
Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks Lakshmanan, V.S. 2020. Automatic detection of ma- chine generated text: A critical survey. In Proceed- ings of the 28th International Conference on Com- purational Linguistics, pages 2296-2309, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics.
Artidoro Pagnoni, Martin Graciarena, and Yulia Tsvetkov. 2022a. Threat scenarios and best prac- tices to detect neural fake news. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1233-1249, Gyeongju, Republic of Korca. International Committee on Computational Linguistics.
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6982
Artidoro Pagnoni, Martin Graciarena, and Yulia Tsvetkov. 2022b. Threat scenarios and best prac- tices to detect neural fake news. In Proceedings of the 29th International Conference on Computational Linguistics, pages 1233-1249.
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, lan Miers, and Tom Goldstein. 2023. A watermark for large language models.
Alec Radford, Karthik Narasimhan, Tim Salimans, llya Sutskever, et al. 2018. Improving language under- standing by generative pre-training.
Philipp Koehn. 2004. Statistical significance tests for
machine translation evaluation. Proceedings of the
2004 Conference on Empirical Methods in Natural Language Processing, pages 388-395, Barcelona, Spain. Association for Computational Linguistics.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAl blog. 188):
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learn- ing of language representations. arXiv preprint arXiv: 1192..
Colin Raffel, Noam Shazeer, Adam Roberts. Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019, Exploring the limits of transfer learning with a unified text-to-text trans- former. arXiv e-prints.
Weixin Liang. Men Yuksekgonul, Yining Mao, Eric Wu, and James Zou. 2023. Gpt detectors are biased against nan-na english writers.
Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert- Jeff Wu, Alec Radford, Gretchen Krueger, Jong Wook Kim, Sarah Kreps. Miles McCain, Alex Newhouse, Jason Blazakis, Kris
Xiaoming Liu, Zhaohan Zhang, Yichen Wang, Yu Lan, and Chao Shen. 2022. Coco: Coherence-enhancee

---

McGuffie, and Jasmine Wang. 2019. Release strate- gies and the social impacts of language models.
Jinyan Su, Terry Yue Zhuo, Di Wang, and Preslav Nakov. 2023. Detectllm: Leveraging log rank information for zero-shot detection of machine-generated text.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. 2023. Llama: Open and effi- cient foundation language models. arXiv preprint arXiv:2302.13971.
Ben Wang. 2021. Mesh-Transfen Teash-Trnnfoome-JAX. Model- Parallel Implementation of Transformer Lan- guage Model with JAX. https://github. com/ kingoflolz/meshh- transformer-
Matthew D. Zeiler and Rob Fergus. 2014. Visualizing understanding convolutional networks. In Com- puter Vision ECCV 2014, pages 818-833, Cham. Springer International Publishing.
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake H. Wallach, H. Larochelle, A. Beygelzimer, d'Alché-Bue, Fox, and R. Garnett, editors, Ad- vances in Neural Information Processing Systems 32, pages 9054 9065. Curran Associates, Inc.
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2020. Defending against neural fake
Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher De- wan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mi- haylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. Opt: Open pre- trained transformer language models.

---

Supplemental Materials
A
Related Work
GPT-J
GPT- neo 2.78
Research detecting machine- machine-generated text be roughly divided into two categories: supervised training and zero-shot detection (To clarify, in the literature "zero-shot" usually means that the ap- proach does not require training data, while our work focus on zero-shot generalization to the de- tection of a held-out generator).
LLaMA 7B
LLaMA 138
GPT-neo
125M
In the case of supervised methods, Bakhtin et al. (2019) train energy-based model to identify machine-generae text. Zellers al. (2020) trainn GROVER detector and finds that models exhibit- ing superior performance in generating neural dis- information are also highly effective in detecting their own generated content. Both Solaiman et al. (2019) and Ippolito et al. (2020) propose zero-shot approaches to detect machine machine-generated text and evaluate the capability of pretrained models. Liu et al. (2022) present coherence- baaerennce-based contrastive learning model detect the machine-generated text under low-resource scenario. Kirchenbauer al. (2023) propose a watermarking method (Ab- delnabi and Fritz, 2021) which introduces designed noise which is imperceptible to human readers. Mitchell et al. (2023) propose DetectGPT, zero- shot method that utilizes a novel curvature-basee criterion to determine whether text is generated by specific model. This approach has demonstrated superior detection capabilities compared other existing zero-shot methods. While DetectGPT does not require training a separate detector, relies on access to the generator LLM, which can be costly. Recently, Su et al. (2023) follow up the work of De- tectGPT and introduce two new zero-shot methods: DetectL.LM-LRR and DetectL.LM-NPR.
GPT-4
GPT-2
GPT-3
GPT-2 md
GPT-2 xl
GPT-2 lg
GPT-J
GPT- neo 2.78
LLaMA 7B
GPT- neo
LlaMA 138
GPT- neo
GPT-1
GPT-4
GPT-2 sm
GPT-2 md
GPT-2 xl
GPT-2 lg
IMDBReview
GPTJ
GPT- neo 2.7B
GPT- neo
LLaMA
GPT-neo
125M
GPT-1
B Auxiliary Results
GPT-2 sm
Figure 3, we plot detector-generatoo pairs with large (>20%) Acc-Gap on the three datasets. shows that none of detectors able to generalize to all generators. For example, all detectors except DGPT4 has large accuracy gap for GPT3.
GPT-3
GPT-2
GPT-2
Wiki
GPT-2 lg
In Figure 4, 5 and 6, we give detailed heatmaps of Acc-Gap for every detector/generator pair the three datasets. The reported numbers are cal- culated as the averages of Ace-Gap obtained by bootstrapping 100 times.
M
N
Acc-Gap DM
20%
Figure 3: Detector-generator pairs with large (>20%) accuracy gap.

---

OPT-
26
38
40
36
47
14
41
41
40
32
32
35
-00
GPT/2 md
18
1.5
225
3.9
47
3.1
17
27
27
GPT-2 9
3.1
29
14
-1.4
49
4.1
7.5
9
20
20
22
Detectors
GPT-
41
48
48
47
42
3.6
A2
44
42
43
33
32
35
50
32
GPT-060
38
20
GPT-NMO md
2.5
2.2
3.3
8.2
9.5
48
22
0.037
8.5
13
17
12
GPT-n00
GPT-
8.5
6.3
34
0.74
S1
3.7
0.7
-1.7
4
LLaMA 78
17
15
13
12
6.9
6G
16
BB
7.2
-1.1
16
LLaMA
GPT-1
GPT-2
GPT-4 m2
G9T23
GPT:2
GPT3
GPT4
OPT4400 md
GPT kg
GPTN
LLaMA
-0
LLab6
Generators
Figure 4: Heapmap of Ace-Gap for detector/generator pairs on the RealNews dataset.
50
GPT1
1.8
13
22
30
48
90
4.8
32
34
57
37
35
28
40
GPT2sm
0.5
0.42
5.2
48
1.7
21
28
38
3
21
2.7
GPT31g
4.1
30
GPT2xl
1.4
0:47
0.32
7.2
4.5
4.6
6.2
10
detectors
GPT3
48
48
48
49
20
1.5
43
44
43
40
35
35
38
GPT4
50
30
46
40
49
5.6
50
4B
45
42
38
95
38
GPTneeem
18
11
47
19
19
32
31
20
255
9.2
GPTnoolg
4.5
4.5
-3.1
8.9
GPTJ
7.0
7.9
78
8.2
8.8
43
7.5
8.4
3.3
-4.8
11
10
LLMATD
13
13
12
12
'2
43
32
12
11
7.2
3/8
-1.5
13
LLSMA13B
15
15
14
13
13
36
13
12
8.7
5.6
1.5
16
GPT1
GP7200 GPT22m
08770
197241
GP74
GPThooms
GPTJ
LLMM
LLSMA
aieage
generators
Figure 5: Heapmap Acc-Gap for detector/generrttor pairs on IMDBReview dataset.

---

GPT-1
GPT-2
0.9
1.5
24
-00
GPT.2 md
0.11
-3
30
40
47
0.4
15
20
30
35
35
17
GPT-2
1.1
0.6
1.3
3-4
43
40
1.5
9.9
14
22
29
27
15
xP
C.97
0.6
1.2
-0.13
48
1.2
0.55
21
21
11
50
Detectors
36
0.5
8.2
GPT-960 md
1.8
1.4
3.4
29
47
17
3:34
5.
9.
GPT-1e0
3.3
2.8
3.5
3.3
27
23
46
32
C:36
97
9:7
10
83
GPT-
8.8
8.1
8.4
7.8
7.4
25
47
B.E
6.1
4.8
8.5
5.8
11
8-1
877
10
47
Y.1
13
22
GMT-
GPT/2
GPF2
09722x
GPT3
GPY-4
TPT- ST
OPTJ
LLMMA
UoMA 150
G9T22
GPT-Kk
Generators
Figure 6: Heapmap of Ace-Gap for detector/generrtoo pairs the Wikipedia dataset.


Reviews:
[{"review_id": "L6McgH4Je6", "reviewer": "Reviewer_FNgs", "summary": "", "questions": "- l114; What wikipedia dataset?\n\nThank you authors for the rebuttal and answering the questions I had. I will raise my excitement score to a 4 based on some of the empirical results and additional experiments that shed light on the consistency of the method.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the insightful comments from the Reviewer. We would like to provide responses for the \"reasons to reject\" and \"Questions For The Authors\" sections:\n\n> - I imagine the choice of classifier has an impact here in that a different architecture could bias towards generations of a certain type or models or a certain type. You discuss scaling up the detector in the limitations section, but it'd be nice to see a larger array of things.\n\nPer the Reviewer's request, we experiment with an additional base detector, namely Albert [1]. While the absolute accuracies are slightly different, we find that the key observation---that the detectors trained for the medium-size models can generalize to larger-size models---still holds, as shown in Table 1. This experiment shows that the results are not Electra-specific, but are more general. We will add these experiments and discussion in the final revision of the paper.\n\n\n| M   | N   | **$\\text{Gap}^{D_M}_N$** | **$\\text{Gap}^{D_N}_M$** |\n| --- | --- | -------------------- | ---------------------- |\n|GPT3    | GPT4    | 1.22\\%  | 1.60\\% |\n|LLa7B   | LLa13B  | -0.22\\% | 0.17\\% |\n|Neo1.3B | Neo2.7B | -0.13\\%  | 0.09\\% |\n|GPT2lg  | GPT2xl  | -1.44\\% | 1.70\\% |\nTable 1. We train detectors of generated text based on the Albert LM in the news domain, and evaluate their capacity to detect machine-generated text from PLMs (listed in col. M and N). ${Gap}^{D_M}_N$ denotes the drop of accuracy rate when the detector is trained on generator M (medium-version model) instead of N(large-version model) itself, and ${Gap}^{D_N}_M$ denotes the accuracy drop in the opposite direction. When detector is based on ALBERT Large v2, we have an observation similar to when the detector is based on Electra, i.e., the generalization performance is slightly worse than in the other direction, reflected by a larger Acc-Gap.\n\n\n>  - I imagine the data limitation is a big one. A classifier with just a few examples is unlikely to be robust at all, whereas for such a large model like you're using a larger classifier could utilize much more data to get a better fit on the distribution for the generator at hand. \n\nPer the reviewer's request, we expand the training samples (from 4000 machine-generated samples per generator in the training set to 9000). While the average accuracy of all detectors (when tested on the same generator on which they were trained) has increased from 94.1\\% to 96.9\\%, we got a consistent observation as the main findings in our paper: the detector trained on a medium-size model demonstrates a noteworthy ability to generalize to the larger-size model. We present the additional experimental results in Table 2, and we will add this discussion in the next revision.\n\n  \n| M   | N   | $\\text{Gap}^{D_M}_N$ | $\\text{Gap}^{D_N}_M$ |\n| --- | --- | -------------------- | -------------------- |\n|GPT3    | GPT4    | 3.09\\%  | 6.23\\% |\n|LLa7B   | LLa13B  | -2.66\\% | 3.62\\% |\n|Neo1.3B | Neo2.7B | 0.47\\%  | 1.07\\% |\n|GPT2lg  | GPT2xl  | -2.48\\% | 3.49\\% |\nTable 2. We train detectors of generated text on the expanded dataset in the news domain, and evaluate their capacity to detect machine-generated text from PLMs (listed in col. M and N).${Gap}^{D_M}_N$ denotes the drop of accuracy rate when the detector is trained on generator M (medium-version model) instead of N(large-version model) itself, and ${Gap}^{D_N}_M$ denotes the accuracy drop in the opposite direction. We find that when attempting to generalize from the large-version models to medium ones, the generalization performance is slightly worse than in the other direction, reflected by a larger Acc-Gap.\n  \n\n> - The data types are a mystery to me, but its conceivable that datasets that have been studied and designed to be prompts could service you well here (e.g. RealToxicityPrompts) \n\nOur data collection pipeline follows common practices as the related work, including \"Threat Scenarios and Best Practices for Neural Fake News Detection\" [2], \"Can AI-Generated Text be Reliably Detected?\" [3] and \"Red Teaming Language Model Detectors with Language Models\" [4], using partial human samples as input prompts to the language model for text completion, thereby constructing the dataset.\n\n\n>  2. The Datasets section could be better described. I have no idea what aspect of wikipedia you took data from. I don't really know whether you sample text at a document level and then truncate to create your data for the discriminator etc.\n\nand\n\n>  l114; What wikipedia dataset?\n\nThe Wikipedia dataset we used was directly obtained from Hugging Face, data subset \"20220301.en\". This particular dataset is a popular resource available on the Hugging Face platform (page link: https://huggingface.co/datasets/wikipedia). We add the citation for this dataset in our paper (line 350) according to the \"Citation Information\". We will incorporate the Hugging Face webpage link into the citation for this dataset in the next revision.\n\n  \n[1] Lan, Zhenzhong, et al. \"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations.\" (2020). In Proc. ICLR\n  \n[2] Pagnoni A, Graciarena M, Tsvetkov Y. Threat scenarios and best practices to detect neural fake news[C]//Proceedings of the 29th International Conference on Computational Linguistics. 2022: 1233-1249.\n\n[3] Sadasivan V S, Kumar A, Balasubramanian S, et al. Can ai-generated text be reliably detected?[J]. arXiv preprint arXiv:2303.11156, 2023.\n\n[4] Shi Z, Wang Y, Yin F, et al. Red Teaming Language Model Detectors with Language Models[J]. arXiv preprint arXiv:2305.19713, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1) This problem is well-motivated and the research question is an important one\n\n2) The study is nicely scoped and doesn't try and do too much.\n\n3) The addition of bootstrapping here is good to establish variance and be able to conduct a t-test.", "reasons_to_reject": "1) I think the experiments could be expanded. The choice of classifier, the amount of training data (4000 machine-generated samples in train per generator), and the data types could be varied. \n\n- I imagine the choice of classifier has an impact here in that a different architecture could bias towards generations of a certain type or models or a certain type. You discuss scaling up the detector in the limitations section, but it'd be nice to see a larger array of things.\n- I imagine the data limitation is a big one. A classifier with just a few examples is unlikely to be robust at all, whereas for such a large model like you're using a larger classifier could utilize much more data to get a better fit on the distribution for the generator at hand.\n- The data types are a mystery to me, but its conceivable that datasets that have been studied and designed to be prompts could service you well here (e.g. RealToxicityPrompts)\n\n2) The Datasets section could be better described. I have no idea what aspect of wikipedia you took data from. I don't really know whether you sample text at a document level and then truncate to create your data for the discriminator etc. \n\nI understand that this is a short paper, so the scope of experiments and contribution is less than a long paper, but some additional clarity on these facets would go a long way to improving the paper.", "questions_to_authors": "- l114; What wikipedia dataset?\n\nThank you authors for the rebuttal and answering the questions I had. I will raise my excitement score to a 4 based on some of the empirical results and additional experiments that shed light on the consistency of the method.", "paper_topic_and_main_contributions": "This paper investigates whether the detectors for model/machine-generated text work across different text generators in a zero-shot manner (i.e. the different text generators' generations are not in the training set). They find that generalization is hard, but detectors trained on outputs of medium-sized models generalize reasonably well to larger ones. They use this insight to build detectors that are fairly robust by using an ensemble of these medium-sized models.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "- The results section is a little confusing to read through and I had to read through it a few times to understand what's going on, so a rewrite to be a little crisper and clearer here would be wonderful."}}, {"review_id": "wP7F2vZ6fZ", "reviewer": "Reviewer_wjLU", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate that the reviewer took the time to carefully review our paper and shared interesting suggestions.\n\n\n>  The authors of the article did not explain why the choice of the detector model fell on ELECTRA-large,\n\nBecause prior related work [1] shows ELECTRA has strong performance as a detector.\n\n\n\n> I would like to see a comparison with other models for detector in this task. \n\nPer the Reviewer's request, we experiment with an additional base detector, namely Albert [2]. While the absolute accuracies are slightly different, we find that the key observation---that the detectors trained for the medium-size models can generalize to larger-size models---still holds, as shown in Table 1. This experiment shows that the results are not Electra-specific, but are more general. We will add these experiments and discussion in the final revision of the paper.\n\n\n| M       | N       | **$\\text{Gap}^{D_M}_N$** | **$\\text{Gap}^{D_N}_M$** |\n| ------- | ------- | ------------------------ | ------------------------ |\n| GPT3    | GPT4    | 1.22\\%                   | 1.60\\%                   |\n| LLa7B   | LLa13B  | -0.22\\%                  | 0.17\\%                   |\n| Neo1.3B | Neo2.7B | -0.13\\%                  | 0.09\\%                   |\n| GPT2lg  | GPT2xl  | -1.44\\%                  | 1.70\\%                   |\nTable 1. We train detectors of generated text based on the Albert LM in the news domain, and evaluate their capacity to detect machine-generated text from PLMs (listed in col. M and N). ${Gap}^{D_M}_N$ denotes the drop of accuracy rate when the detector is trained on generator M (medium-version model) instead of N(large-version model) itself, and ${Gap}^{D_N}_M$ denotes the accuracy drop in the opposite direction. When detector is based on ALBERT Large v2, we have an observation similar to when the detector is based on Electra, i.e., the generalization performance is slightly worse than in the other direction, reflected by a larger Acc-Gap.\n\n\nThe Reviewer also suggests trying the distilled versions for both the LLM and the detector, which we agree is a good and interesting idea. Although this falls outside the scope of our current study which focuses on the range of existing LLMs, we will add this into our discussion of future work and we will acknowledge the Reviewer, thanks!\n\n\n[1] Pagnoni A, Graciarena M, Tsvetkov Y. Threat scenarios and best practices to detect neural fake news[C]//Proceedings of the 29th International Conference on Computational Linguistics. 2022: 1233-1249.\n\n[2] Lan, Zhenzhong, et al. \"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations.\" (2020). In Proc. ICLR", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "This study examines a very relevant topic at the moment related to the development of large language models (i.e., GPT 4, GPT-Neo2.7B, LLaMA13B and GPT-2xl) and using them for base NLP tasks. A positive result, namely, this study shows that in the case of limited budget or computing, data from the medium-version LM can decently approximate the large-version in an ensembled data collection, show the usefulness of the study in the development of approaches for identifying machine-generated texts. Report on the work of the two considered baselines (model ensemble, data mixing) can further help in the development and improvement of models for solving the current problem.", "reasons_to_reject": "The authors of the article did not explain why the choice of the detector model fell on ELECTRA-large, I would like to see a comparison with other models for detector in this task. It is necessary to provide for comparison any other approaches to solve this problem, the paper presents only the baselines implemented by the authors. As an important result, the researchers cite the possibility for the medium-version LLMs generalize to the large-version models. In this regard, it is interesting to look at the result using distilled model for both LLM and detector. It makes sense to conduct an experiment where the ensemble of detectors consists of distilled models corresponding to each of the LLMs selected for generating training data. ", "questions_to_authors": "", "paper_topic_and_main_contributions": "This study aims to solve the problem of identifying differences between machine-generated texts and genuine human texts. The main idea is to generate data from different models and then train neural detectors on data from each generator and test its performance on other generators. Two baselines are compared: model ensemble, where we aggregate predictions from all detectors by majority voting or conﬁdence (probability) average and data mixing, where we train a new ELECTRA-large detector by mixing up the training data from all generators. There are also experiments with prune out data from large version generators in an ensembed training dataset.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "YdBL6sMEFG", "reviewer": "Reviewer_GhLs", "summary": "", "questions": "NA", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the comments and we would like to provide explanation and counterarguments regarding the points the reviewer has raised. \n\nThe reviewer questions the way we prompt LLMs to collect samples from the instruction tuned models, i.e. by truncating the initial 20 tokens from the human samples as input prompts for LLMs. For our experiments involving the instruction-tuned GPT-4 and GPT-3, however, **we do employ a specific prompt**, \"Please continue this text in about 90 words:\", preceding the initial 20 tokens to guide the models in generating coherent text. This minor implementation detail is omitted in the current version because it is not critically relevant to the main research questions and their corresponding experiments in the paper, and due to the space limits. We will incorporate this detail into the main text or as an appendix. \n\nFurthermore, our pipeline to use prefix from human-written samples as input prompts for text completion in the data preparation process simply follows relevant prior work, e.g. \"Threat Scenarios and Best Practices to Detect Neural Fake News\" [1], \"Can AI-Generated Text be Reliably Detected?\" [2] and \"Red Teaming Language Model Detectors with Language Models\" [3].\n\nImportantly, we would like to clarify that **our intent is not to showcase the usage of prompts**; Rather than how to do prompting, our focus is on the detection of the generation of different models.\n\n\nTo the comment on **\"Human writing data \\& its distribution\"** and **\"Binary detector's ability\"**: \n\nThe reviewer argues that selecting only one or a few datasets may not adequately represent all human writing styles. While this point seems valid, please note that our data collection pipeline follows common practices as the related works (also discussed above). The distribution of human samples is not the focus of our work, instead, the focus and findings of our paper center around **the generalization patterns observed among detectors trained on different LLMs' generations**. We experiment on datasets from different domains and the consistency in generalization patterns across all three domains lead us to believe that these findings are consistent and not domain-specific. **While the concern raised by the reviewer regarding potential bias in human samples is valid, it does not impact our conclusions and is not the focus of this work.**\n\n  \n[1] Pagnoni A, Graciarena M, Tsvetkov Y. Threat scenarios and best practices to detect neural fake news[C]//Proceedings of the 29th International Conference on Computational Linguistics. 2022: 1233-1249.\n\n[2] Sadasivan V S, Kumar A, Balasubramanian S, et al. Can ai-generated text be reliably detected?[J]. arXiv preprint arXiv:2303.11156, 2023.\n\n[3] Shi Z, Wang Y, Yin F, et al. Red Teaming Language Model Detectors with Language Models[J]. arXiv preprint arXiv:2305.19713, 2023.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Conducted diverse and rigorous experiments using diverse language models with varying size.\n- Contrary to the intuitive belief that texts generated by larger language models would be more difficult to discriminate, this paper discovered a counter-intuitive observation. \n- Based on this finding, the paper exhibited simply removing text generated from large language model can help detecting AI generated text from human writing.", "reasons_to_reject": "Personally, the most concerning aspect of this paper, in my view, is the lack of concrete experimentation settings. I have the following negative opinions about this setup:\n\n- Human writing data & its distribution: The distribution of  human writing is highly diverse. While the authors categorized the dataset into domains such as news, knowledge, and reviews, even within each domain, there can be a wide variety of writing styles. Approximating this complex and challenging distribution with 5000 samples from a single dataset can be highly biased and is likely to represent only a small portion of the entire human writing distribution. In such cases, it is unclear whether the performance in the paper can be seen as the actual performance of AI-generated detection.\n- Language model text generation: When using Language Models (LLM) to generate text, it has become a common paradigm to provide various prompts. Specifically, the capability of large models like GPT-4 or ChatGPT comes from following such complex instructions. However, in this paper, the authors adopt a naïve method of purposelessly auto-generating from the initial 20 truncated tokens. So, I believe drawing conclusions that small models can sufficiently detect text generated by large models is quite misleading. \n- Binary detector's ability: In this paper, the two aforementioned datasets are used to train a binary classifier, which is then used as a AI generated text detector. However, I believe that such a trained detector does not possess the ability to reliably distinguish between human language and machine-generated language. Specifically, when distinguishing between the two types of text collected earlier, there could be various cues or biases that could assist in making the distinction. For instance, the human news data may be limited to news-related semantics, while the text generated by the machine could cover various semantics, making it easier to differentiate between the two. Neural models are known to utilize various unintended cues (i.e., shortcut cues) for learning in such scenarios.", "questions_to_authors": "NA", "paper_topic_and_main_contributions": "This paper discusses a task related to distinguishing text generated by a language model and text written by humans. Specifically, it examines whether a detector can effectively generalize when using text generated by different decoder-based language models during training and testing. The study discovers that a medium-sized language model can also detect text produced by a large language model effectively. Based on this finding, the paper proposes a method to train the detector by excluding text generated by the large language models (e.g., GPT4) from the set of texts generated by different language models. The pruning method (i.e., data exclusion) detects AI generated text better than other baseline ensemble techniques. However, personally, I find the experimental setup to be insufficiently concrete and practical to draw the same conclusions as the authors (see the weakness).", "excitement": 2, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers on the whole agree on the soundess of the work, and it's excitement. While the points raised by reviewer GhLs on `Human writing data & its distribution` are correct, the paper's focus is on whether or not detectors can identify the data from generators they have not seen data for previously.  Admitedly while this is a closely related question - a person with a new wrting style is akin to a new generator, the authors have followed the methodology of recent work in the area.  It would of been good the challenge some of these practices as the authors know their limitations.
Decision: Accept-Findings