Token Prediction as Implicit Classification to Identify LLM-Generated Text
Authors: Yutian Chen
Hao Kang
Vivian Zhai
Liangze Li
Rita Singh
Bhiksha Raj
Venue: EMNLP 2023 Main (2023)
Keywords: Machine-generated text detection
Text-to-Text Transfer Transformer (T5)
Large language model
Transfer learning
Large language model

Abstract: This paper introduces a novel approach for identifying the possible large language models (LLMs) involved in text generation. Instead of adding an additional classification layer to a base LM, we reframe the classification task as a next-token prediction task and directly fine-tune the base LM to perform it. We utilize the Text-to-Text Transfer Transformer (T5) model as the backbone for our experiments. We compared our approach to the more direct approach of utilizing hidden states for classification. Evaluation shows the exceptional performance of our method in the text classification task, highlighting its simplicity and efficiency. Furthermore, interpretability studies on the features extracted by our model reveal its ability to differentiate distinctive writing styles among various LLMs even in the absence of an explicit classifier. We also collected a dataset named OpenLLMText, containing approximately 340k text samples from human and LLMs, including GPT3.5, PaLM, LLaMA, and GPT2.

Token Prediction as Implicit Classification to Identify LLM-Generated Text
Yutian Chen', Hao Kang!, Yiyan Zhai, Liangze Li, Rita Singh, Bhiksha Raj
Carnegie Mellon University
Pittsbrgh,, PA 15213
(yutianch, haok, yiyanz, liangzel, rsingh, bhikshaj@andreww cmu. edu
Abstract
hundreds of billions has significantly improved the text generation quality, presenting an unprece- dented challenge to the detection task. To over- come this challenge, we propose using the inherent next-token prediction capability of the base LM for detection task, aiming not just to determine whether or not the text is generated but also to identify its source.
This paper introduces a novel approach for identifying the possible large language mod- els (LLMs) involved text generation. Instead of adding an additional classification layer to base LM, we reframe the classification task as a next-token prediction task and directly fine-tune the base LM to perform it. We uti- lize the Text-to- Text Transfer Transformer (TS) model as the backhone our experiments. We compared approach to the more direct ap- proach of utilizing hidden states for classifica- tion. Evaluation shows the exceptional perfor- mance of our method in the text classification task, highlighting its simplicity and efficiency. Furthermore, interpretability studies the fea- tures extracted by our model reveal its ability differentiate distinctive writing styles among various LLMs even in the absence of explicit classifier. We also collected a dataset named OpenLL,Text, containing approximately 340k text samples from human and LLMs, including GPT3.5, PaLM, LLaMA, and GPT2.
2 Related Work
2.1 Generated Text Detection
Bouriing-based approaches to machine- machine-generater generated text detection can be broadly classified into two categories: unsupervised learning and supervised learning. Unsupervised learning includes GLTR de- veloped by Gehrmann et al. (2019) that uses linguis- tic features like top-k words to identify generated text. Another unsupervised approach, DetectGPT by Mitchell et al. (2023), employs a perturbation- based method by generating a modifications of the text via pre-trained language model and then com- paring log probabilities of original and perturbed samples. Supervised Learning includes GROVER (Zellers et al., 2020) that extracts the final hidden state and uses a linear layer for prediction in a dis- criminative setting. Energy-based models (Bakhtin et al., 2019) have also been investigated for dis- criminating between text from different sources. Solaiman et al. (2019) fine-tuned RoBERTa model GPT-2 generated text, resulting in accuracy of 91 % GPT2-1B in GPT2-Outpun dataset.
1 Introduction
In recent years, generative LLMs have gained recognition for their impressive ability produce coherent language across different domains. Con- sequently, detecting machine generrted text has become increasingly vital, especially when ensur- ing the authenticity of information critical, such as legal proceedings.
Traditionally, techniques like logistic regression and support vector machines (SVM) have been used for detection tasks, as explained by Jawahar al. (2020). The analysis of textual features like perplexity is proven to be effective as well (Wu et al., 2023). Recent advancements have introduced the use of language model itself to detect generated text, such as the AI Text Classifier released by OpenAl (2023) and Solaiman et al. (2019).
2.2 Text-to-Text Transfer Transformer
Text-to-Text Transfer Transformer (T5) (Raffel et al., 2020) has gained recognition due to its sim- plicity by converting all text-based language prob- lems into a text-to-text format. Raffel et al. and Jiang et al. (2021) have shown that T5 out- nut-performs BE.T-- models on various natural language processing tasks.
However, the exponential growth in the num- ber of parameters from hundreds of millions to
However, prior approaches have not emphasized the use of T5 in the task of distinguishing the lan- guage model responsible for text generation. Fur- thermore, existing approaches have not directly leveraged the next-token prediction capability
Two authors contribute cqually to this work. 'The implementation of the classification model, training process, and datasct collection is publicly available on https: thu. www/ Www.aaaarhheeeannnnnmmma Sentinen meeelppuue -uullcc

---

the model for this particular task. Our approach advances the field by choosing T5 model as the base LM and using next-token prediction ca- pability to improve the accuracy and efficiency of distinguishing the origin of the text.
Next taken
Probability Distiibutin
T3-small Backbonn
Desoder Block x
Fond Forward
Enexder Block
3 Dataset
Feed: fred--oowand MLP
Masked Multi-head tiention
Seif-A Denton
Masked Mulli selli-bedd A ttentoon
3.1
Data Collection
E
Ecro
The dataset we collected, named OpenLLMText, consists of approximately 340,000 text samples from five sources: Human, GPT3.5 (Brown et al., 2020), PaLM (Chowdhery et al., 2022), LLaMA- 7B (Touvron al., 2023), and GPT2-1B (GPT2 ex- tra large) (Radford et al., 2019). The OpenL dataset, along with the response collected from OpenAl & GPTZero, is publicly available on Zen- odo.
Figure 1: TS-Sentinel model architecture
the distribution of length, punctuation, tokens and word classes in OpenLLMText across different sources. Results indicate that significant bias exists between different sources. For detailed anal- ysis and visualization, please refer the Appendix A.
Human text samples are obtained from the OpenWebText dataset collected by Gokaslan and Cohen (2019). GPT2-1B text samples stem from GPT2-Output dataset released by OpenAl (2019). As for GPT3.5 and PaLM, the text samples are col- lected with prompt "Rephrase the following para- graph by paragraph: [Human_Samplee structing LLaMA-7B to rephrase human text sam- ples is ineffective, due to the lack of fine-tuning for instruction following of LLaMA-7B. Hence, we provided the first 75 tokens from the human samples as context LLaMA-7B and obtained the text completion as the output. For further details, including the temperature and sampling method for each source, please refer to the table 3 in Appendix A.
4 Method
Our approach can be formulated as follows. Let Σ represent the set of all tokens. The base LM can be interpreted as a function LM: Σ* x E + R. Given Σ, LM(s o) esti-
But in-
string Σ' and token mates the probability of the next token being a. Let Y denote the set of labels in OpenLLMText, which contains "Human". "GPT-3.5". etc. We establish a bijection f Y + V, where y C Σ acts as a proxy for the labels. By doing so, we reformulate the multi-class classification task Σ" + Y into a next-token prediction task Σ. - V. Hence, the multi-clat classification task can be solved directly using LM:
We partitioned the OpenLLMText into train (76%), validation (12%) and test (12%) subsets. The detailed breakdown is listed in Table 4 in Ap- pendixA A.
1 - f-
arg max LM(s. vEY
(1)
4.1 T5-Sentinel
3.2 Data Preprocessing
T5-Sentinel is the implementation of our approach using T5 model. Unlike previous learning- based approaches where final hidden states are extracted and passed through separate classifier (Solaiman et al., 2019; Guo et al., 2023), T5-Sentinel directly relies on the capability of the T5 model to predict the conditional probability of next token. In other words, we train the weight and embedding of the T5 model and encode the classification problem into sequence-to-sequenee shown in Figure 1.
We noticed stylistic differences among different models. For instance, LLaMA generates V\n for newline character instead of An as in other sources. To address the inconsistency, we followed a similar approach as Guo et al. (2023) to remove direct indi- cator strings and transliterate indicator characters.
3.3 Dataset Analysis
To avoid potential bias and shortcuts that can be learned by model unexpectedly, we analyzed
completion task
We use reserved tokens that do not exist the text dataset as y. During fine-tuning, we use the
'Opent. can be downloaded at: https://zenodo. org/rrcoords/285522

---

KLP
Petumas
Su4
T5-small Backbose
Encader Block
Decader Block
Eess
ExZAD>
RAN
Figure 2: T5-Hidden model architecture
30
0.2
3.4
Figure 4: ROC curves for T5-Sentinel for each one-vs- rest classification task
0m0
10
PAP
112
42
G11
LUMO
Phoidedd
Figure Confusion matrix of T5-Sentinel
10
AdamW optimizer (Loshchilov and Hutter, 2017) with mini-batch size of 128. The learning rate is 10- with weight decay of 10-5 and train for 15 epochs.
Figure 5: DET curves for 15-Sentinel for each one-vs- rest classification task
0.833 of T5-Hidden, both under the probability threshold of 0.5. The confusion matrix for T5- Sentinel presented Figure To illustrate the performance under different probability threshold, we plot the ROC curves and Detection Error Trade- off (DET) curves on each one-vs-rest task in figure 4 and respectively.
4.2 TS-Hidden
To evaluate the effectiveness of not using ad- ditional classifier while accomplishing the same classification task, we also fine-tuned the T5 model with classifier attached, denoted T5-Hidden.
As illustrated with Figure 2, the classifier in T5- Hidden uses the final hidden state from the decoder block of the T5 model and computes the probabil- ity for each label after taking softmax over its output layer. T5-Hidden trained under identical configuration as T5-Sentinel.
5.2 Human-LLMs Binary Classification
For generated text detection task, we compare T5-Sentinel against T5-Hidden and two widely- adopted baseline classifiers, the Al text detector by OpenAl and ZeroGPT.
5 Evaluation
Figure 6 displays the ROC curves obtained from our experiments and the detailed performance met- rics such AUC, accuracy, and Fl score sum- marized in Table 1. Additionally, the performance of each classifier on the gener- ation text detection subtask for each LLM source in OpenLLMText, as shown Table Notably, T5- Sentinel outperforms the baseline across all sub- tasks in terms of AUC, accuracy, and F1 score.
T5-Sentinel and T5-Hidden are evaluated on the test subset of OpenLLMText dataset with receiver operating characteristic (ROC) curve, area under ROC curve (AUC) and F1 score.
compare
5.1 Multi-Class Classification
We breakdown the evaluation on multi-class classi- fication, i.e.. identify the specific LLM responsible for text generation, into one-vs-rest classification for each label.
Interpretability Study
As presented Table 5, TS-Sentinel achieves a superior weighted Fl score of 0.931 compared
Our interpretability studies, including a dataset ablation study and integrated gradient analysis,

---

AUC
Accuracy
F1
Recall
Precision
OpenAl
0.795
0.434
0.415
0.985
0.263
ZeroGPT
0.533
0.336
0.134
0.839
0.148
T5- 0.924
0.894
0.766
0.849
0.698
TS-Sentinel
0.965
0.956
0.886
0.832
0.946
Table 1: Evaluation result for T5-Sentinel and T5-Hidden on Human- binary classification problem comparing
to that of baselines OpenAl (2023); ZeroGPT (2023) test sushet of OpenLLMTex dataset.
Task
Human v. GPT3.5
Human v. PaLM
Human v. LLaMA
Human v. GPT2
Metric
AUC
Acc
Fl
AUC
Acc
F1
AUC
Acc
F1
AUC
Acc
F1
OpenAl
.761
.699
.694
.899
.659
.743
.676
.573
.709
.901
.768
.809
ZeroGPT
.576
.493
.555
.735
.662
.649
.367
.375
.519
.435
.382
.504
Solaiman al.
.501
.499
.005
5508
.501
.013
.524
.533
.027
.870
.748
.666
T5-Hidden
.971
.922
.916
964
.914
.908
.806
.746
.779
.965
.910
.903
std
.011
.022
.026
.020
.035
.033
.062
.084
.077
.019
.024
.017
TS-Sentinel
.970
.914
.906
.962
.906
.898
.964
.903
.901
.965
.922
.904
Table 2: Evaluation results for T5- Sentinel, T5- Hidden and baselines on each specific toman-to-LLMM binary classification task. For T5-Hidden model, also tested with 5 random initializations and report the standard deviation of metrics under each task in italic.
1.-
HAPEST
ILPA
-0-
06
1.2
-0-
Figure 6: ROC curves for OpenAl classifier, ZeroGPT, T5-Hidden and the proposed T5-Sentinel on test subset of LMenLLMText
-0-
-25
25
Figure t-SNE plot for T5-Sentinel on test subset of OpenLLMText dataset under perplexity of 100
show that T5-Sentinel does not rely on unexpected shortcuts. Additionally, we employ t-distributed Stochastic Neighbor Embedding (t-SNE) projec- tion (yan der Maaten and Hinton, 2008) on the hid- den states of the last decoder block of TS-Sentinel. The resulted t-SNE plot, shown in Figure 7, demon- strates the model's ability to distinguish textual contents from different sources, corroborating the evaluation results discussed carlier. For compari- son, also plotted the t-SNE plot of T5-Hidden on the test subset of OpenLLMTexx in figure 8, re- sults show that the T5-Sentinel cannot distinguish LLaMA with other source of sample correctly. This aligns with the evaluation reported in table 2.
3-
-0-
-0
Figure 8: I-SNE plot for T5-Hidden on test subset of OpenLLMText dataset under perplexity of 100

---

6.1 Dataset Ablation Study
Some samples for visualization can be found in appendix D.
Ablation study is conducted on the LMTeLt dataset to further investigate which feature is uti- lized by the T5-Sentinel to make classification. We design 4 different cleaning configurations OpenLLMText: i) compress consecutive newline characters to one; ii) transliterate Unicode charac- ters to ASCII characters iii) remove all punctu- ation: iv) cast all characters to lower case. Eval- uation results for each one-vs-rest binary classifi- cation task in Table 6 shows that T5-Sentinel is quite robust to perturbations in the input text. For ablation configuration i), ii) and iv), the AUC and F1 score are almost identical. However, the perfor- mance drop significantly under condition iii) (with AAUC = 0.3)..
We notice the existence substantial gradients on non-uunctuation tokens, especially on syntax structures like clauses (Sample 2, Appendix D) and semantic structures like repetitive verbs (Sample 4, Appendix D), indicating that the gradients are not exclusively overfitting punctuation tokens, Rather, the drop in performance of the model with- out punctuation uppears stem from the fact that the removal of punctuation disrupts the overall se- mantic structure within the text that the model has correctly learned.
Conclusion
In conclusion, this paper demonstrates the effee- tiveness of involving next-token prediction in iden- tiyingg possible LLMs that generate the text. We make contributions by collecting and releasing the OpenLLMText dataset, transferring the classifica- tion task into the next-token prediction task, con- ducting experiments with T5 model to create the T5-Sentinel, and providing insight on the differ- ences of writing styles between LLMs through interpretability studies. In addition, provide compelling evidence that approach surpasses T5 T5-Hidden and other existing detectors. As it elim- inates the requirement for an explicit classifier, our approach stands out for efficiency, simplicity, and practicality.
To prove that T5-Sentinel is not overfitting to specific punctuation, we independently remove each punctuation in ASCII from input text and evaluated the performance of model on each one- vs-rest classification task. Results show that only the removal of period and comma cause significant performance degradation (shown in Table 6). This can be due the fact that T5-Sentinel is utilizing syntax structure of input sample to distinguish text from human, GPT3.5 and PaL.M instead of overfit- ting on these two punctuation. In section 6.2, we confirm this hypothesis with integrated gradient analysis.
6.2 Integrated Gradient Analysis
Limitations
The integrated gradient method, proposed by Sun- dararajan et al. (2017), is a robust tool for attribut- ing the prediction of a neural network to the input features. Here, we apply the integrated gradient method the word embedding of input text sam- ple and calculated the integrated gradient of each token using the following formula:
The OpenLLMText dataset we collected is based the OpenWebText dataset. The original OpenWebText dataset collects human written En- glish content from Reddit, an online discussion website mainly used in North America. Hence, the entries from human in dataset may bias towards native English speakers' wording and tone. This might lead to a degraded performance when the detector trained on LpenLLMText dataset is given numan-written text from non-native English speak- ers. This tendency misclassify non-native En- glish writing as machine- -eeeeneee also men- tioned by Liang al. (2023).
m
@T((0
+ f zo)),
20 m
IG(x) =
dr
(2)
where zg denotes the word embedding of the input text same length as x but filled with <pad> token, which is considered as bascline input.
The visualization tool we developed uses equa- tion2 2 with m 100 to calculate the integrated gra- dient of each token and show the attribution of each token in the prediction made by T5-Sentinel model.
References
Anton Bakhtin, Sam Gross, Myle Out, Yuntian Deng, Mare' Aurelio Ranzato, and Arthur Szlam. 2019. Real fake? learning discriminate machine from human generated text.
'Implemented with Python package Unideod https: / //pypi www.orrggg///oiiiieeoo Unwde//eeeettndeeddde/eeeeeeee

---

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler. Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Firie Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark. Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language models are few-shot learners.
Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. Detectgpt: Zero-shot machine-generated text detec- tion using probability curvature.
OpenAL. 2019, Gpt2-output dataset. https://githeee thub com/ comm/openai//ee pepenai/gpt-ii 2-output-
OpenAL. 2023.
https://beta: http://btaa.ppnii..cc al ai-text-classifier
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners.
Aakanksha Chowdhery. Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsyyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vin- odkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin. Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontack Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, An- drew M. Dai, Thanumalayan Sankaranarayana Pil- lai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lec, Zongwei Zhou, Xuezhi Wang. Brennan Sacta, Mark Diaz. Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern Douglas Eck, Jeff Dean, Slav Petrov, and Noah Fiedel. 2022. Palm: Scaling language mod- eling with pathways.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former.
Irene Solaiman, Miles Brundage, Jack Clark, Amanda Askell, Ariel Herbert- Voss, Jeff Wu, Alec Radford. Gretchen Krueger, Jong Wook Kim. Sarah Kreps. Miles McCain, Alex Newhouse, Jason Blazakis, Kris McGuffie, and Jasmine Wang. 2019, Release strate- gies and the social impacts of language models.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. A Axiomatic attribution for deep networks.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models.
Sehastian Gehrmann, Hendrik Strobelt, and Alexan- der M. Rush. 2019. Gltr: Statistical detection and visualization of generated text.
Laurens van der Maaten and Geoffrey Hinton, 2008 Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579-2605.
Aaron Gokaslan and Vanya Cohen. 2019. Open- webtext corpus. http://Skkliionee7.. gi gitb... io/ OpenWebTextCorpu..
Kangxi Wu, Liang Pang. Huawei Shen, Xueqi Cheng, and Tat-Seng Chua. 2023. Limdet: A large language models detection tool.
Biyang Guo, Xin Zhang, Ziyuan Wang, Minqi Jiang, Jinran Nie, Yuxuan Ding, Jianwei Yue, and Yupeng Wu. 2023. How close is chatgpt to human experts? comparison corpus, evaluation, and detection.
Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2020. Defending against neural fake news.
Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks S. Lakshmanan. 2020. Automatic detection of machine generated text: A critical survey. CoRR, abs/2011.01314.
ZeroGPT. 2023. AI Detector. https://ww..zrreggg com.
A Dataset
Kelvin Jiang, Ronak Pradeep. and Jimmy Lin. 2021. Ex- ploring listwise evidence reasoning with 15 for fact verification. https://aclanthoooggy wacooog..o.ggggg org/2021. acl-short.55 51.
A.1 Length Distribution
The length distribution of text sample from each source is presented in Figure 9. Since trun- cated the text first 512 tokens during training and evaluation, the actual length distribution for each source received by the classifier is shown in Figure 10, which is approximately the same across various sources.
Weixin Liang. Mert Yuksekgonul, Yining Mao, Eric Wu, and James Zou. 2023. Gpt detectors are biased against non-native english writers.
Ilya Loshchilov and Frank Hutter. 2017. Fixing weight decay regularization in adam. CoRR,
155/07

---

0.995
Human
PMNM
0.0%%
LLAAA
10.0%
Humon
GPT2
GP13.5
PAUM
0.2%%
LLAMA
930%
25%
0.015
5.0.0%
0.50%
0.25%
0.0%%
0.00
10.0%
0.25%
200
1000
15D0
2003
2500
Figure 9: Distribution of sample length measured by the number tokens in the OpenLLMText dataset.
Semsle Length
Figure 10: Distribution of sample length measured by the number tokens actually received by T5-Sentinel during training, validation and test.
A.2 Punctuation Distribution
Figure 11 shows the distribution of top-40 ASCIT punctuation in OpenLLMText dataset. For most of the punctuation, all LLMs tend to generate them with similar frequency. However, PaLM does tend to generate "*" more frequently than other sources. However, further experiments on dataset clean- ing (indicated in 6 in Appendix C) show that T5- Sentinel is not relying on this feature to identify PaLM generated text.
3.01%
100%
1.00%
20000%
A.3 Token Distribution
2.0%%
The distribution most commonly seen tokens from each source is presented in Figure 12. It is worth noting that while the GPT2 source lacks sin- gle quotation marks and double quotation marks, the overall token distributions from all sources hibit consistent pattern.
102%
5.00%
Figure 11: Distribution of ASCII Punctuations in OpenLLMText
A.4 Word-Class Distribution
Figure 13 displays the word-class distribution like noun, adjective and others for each source in OpenLLMText dataset. The distribution is almost identical across all sources.
Human
GPT3.5
PalM
LLaMA
GPT2
West commin opertt Litho 4ntoot
B
Evaluation
The detailed evaluation results for human-to-LLA binary classification tasks and one-to-rest binary classification tasks are separately listed Table and 5.
Dataset Ablation Study
Figure 12: Distribution of tokens in LpenLLMText
Table 6 has shown the performance of TS-Sentinel under different dataset cleaning methods.

---

Source
Dataset/Tool
Generation Method
Temp
Top-p
Human
OpenWebText dataset
GPT3.5
OpenAl's 3 5-trbo API
Rephrase human samples
1
PaLM
text-bison-001 API
Rephrase human samples
0.4
0.95
LLaMA
LLaMA-7B model
Text completion
0.95
0.95
GPT2
GPT2-Output dataset
Random hidden state
1
1
Table 3: Sources and details of text samples in OpenLLMText
Subset train 51205
Human
GPT3.5
PaLM
LLaMA-7B
GPT2-1B
Total
51360
46525
50099
65079
2641668
valid
10412
10468
3485
9305
10468
44138
test
7367
7385
7400
6587
7385
36124
Total
68984
69213
57410
65991
82932
344530
Table 4:
Number of entries from each source in each subset of LMText.
Task
AUC
Accuracy
F1
Sentinel
Hidden
Sentinel
Hidden
Sentinel
Hidden
Human v. Rest
0.965
0.965
0.956
0.894
0.886
0.766
GPT3.5 Rest
0.989
0.989
0.979
0.980
0.949
0.950
PaLM v. Rest
0.984
0.984
0.957
0.947
0.901
0.881
LLaMA-7B v. Rest
0.989
0.989
0.989
0.899
0.969
0.616
GPT2-1B v. Rest
0.995
0.995
0.981
0.969
0.955
0.929
Average (weighted)
0.984
0.984
0.972
0.939
0.931
0.833
Table 5: Evaluation result for each one-vs-rest classification task for T5- 15-Sentinel and Ti-Hidden on OpenLLMText test subset. Accuracy, and Fi-score are calculated under probability threshold of 0.5.
One-vs-rest
Human
GPT3.5
PaLM
LLaMA
GPT2
Metric
AUC
AUC
AUC
Fl
AUC
F1
AUC
F1
Original
965
886
.989
.949
.944
901
.989
.969
.995
.955
Newline
965
.86
.989
.949
.984
.901
.989
.969
.995
.955
Unicode
.947
.832
.987
.941
.983
.895
.987
.946
.988
.907
Punc
.775
.493
.590
.096
.679
.120
.974
.880
.922
.729
.918
.661
.877
.645
.886
.619
.993
.954
.986
.882
.946
.784
.954
.861
.931
.794
.993
.974
.991
.922
967
.90
.989
.949
984
.904
.989
.968
.995
.955
966
.888
.989
.949
984
.03
.988
.968
.995
.954
966
.89
.899
.949
.84
.055
.988
.965
.955
.952
969
.84
.988
.948
.833
.903
.989
.968
.94
.946
966
.811
.888
.947
.933
.901
.985
.961
.995
.951
.964
.881
988
.946
.978
.891
.989
.968
.955
.953
Lower
.966
.863
.984
.928
.973
.889
.987
.962
.989
.914
Table Evaluation results of Sentinel on OpenLLMText dataset under different ablation configurations. "New- line", "Unicode", "Lower" and "Punc" stands for the cleaning configuration i) to iv) respectively. Each nested row under "Punc" represents removing that specific punctuation. means the accuracy drop by a considerable amount. Bold and dtaic represents the worst and second-worst entries in that column.

---

52%
Sample Label: PaLM, Predicted: PaLM
GPT3.
0%
LLM
PhLM
HTC has had a tough year Revenue down
lost a patent suit to Apple and it re w criticism for pulling the plug on Je lly Bean
updates for some of its phones The company
needs win and it hoping that the D roid
DNA will be it The DNA is high end phone
with a 5- inch 10 80 p display the same quad
09
62%
core Snapdragon chip as the Opti mus G and Nex 4, and GB of RAM It powerful phone with a beautiful screen but there are
0%
A0
Word chasse didrrbutton Wiet dstastt
some trade off s The first trade off is battery
Figure 13: Distribution of word classes in OpenLLMTex:
life The DNA battery is smaller than the
batteries in the Opti mus G and Galaxy S III and it doesn tt last as long
D Integrated Graident Samples
(... (..Truncated)
Some samples of integrated gradient results are presented below. Brighter background indicates a higher integrated gradient value on the token and meaning that specific token contributes more on final prediction result. Sample - 5 are randomly chosen from the test set of OpenLLMText dataset.
Sample 4. Label: LLaMA, Predicted: LLaMA
... Barack Obama s aid he did not expect resident e lect Donald Trump to follow his
administration s bluepring s in dealing with
Russia yet hoped that Tru mp would stand up to Moscow speaking during a joiitt press ap- pearance in White House after meeting Trump
Sample 1. Label: Human, Predicted: Human
Out going US President Barack Obama said
The president also said that while Americans are concerned about Russian interference in last year election campaign and what might mean for the ture democracy there was
that he did not expect President lect Donald Trump to follow his administration s blueprint s dealing with Russia, yet hoped that Trump would stand up to Moscow My hope is that the president leet coming in takes similarly constructive approach he repeated allegations that Russia had engaged
no evidence that votes were rig ged by outside
acttors at any point said Obama during a joint
However
press appearance in White House after meeting
Trump Out going US President Barack O b
in cyber attack is against the US Although US intelligence officials blame Russia for øyber
mas aid that while Americans are concerned
about Russian interference in to last year
attack the Democratic National Committee
Truncated)
they have not provided any substantial proof
the (... Truncated)
Sample Label: GPT2, Predicted: GPT2
Gene ric and other online electric curb side me- ter sizes There have been 11 50 comparison between generic sets of in line -3 gau ge sin- gle phase and Can ten a type electric curb side meters in Ontario over the past 20 years (19 75 28 total ling 2.3 M km Here are samples of current (10 year average home electric curb side meters from selected suppli-
Sample 2. Label: GPT3.5, Predicted: GPT3.5
Barack Obama has stated that he hopes Presi-
Jent e lect Donald Trump will confront Russia
despite not expecting him to follow the cur-
rent administration
policies The wished Russia well and acknowledged it as an important partner to the world Obama ex-
W hilst
ers
All currently available meters have a restriction and are marked with the size in deci mal format and a code of or PF
pressed hope for Trump success not just by its own people but by people around the world
Obama commented that not everything that
(... TTuncated)
had worked for Trump
Truncated)


Reviews:
[{"review_id": "errBJoMAlM", "reviewer": "Reviewer_PtZE", "summary": "", "questions": "1. How did you determine the training/dev/test split as 76%, 12%, and 12%?\n2. Have you run experiments under different random seeds to observe the performance variance?\n3. Is it possible to release your proposed classifier and constructed dataset?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. How did you determine the training/dev/test split as 76%, 12%, and 12%?\n- A common approach to training/dev/test split is 80%, 10% and 10%. However, since we’re using the smallest version of T5 model (i.e. about 60M parameters), we reduce the training ratio by a little bit.\n---\n\n2. Have you run experiments under different random seeds to observe the performance variance?\n- In the case of T5-Sentinel, because we’re using the pre-trained model as a direct initialization value, different random seeds should not influence the weight initialization in the training process. So we did not try different random seeds on this one.\n- For T5-Hidden, an additional classifier head in the form of an MLP (Multi Layer Perceptron) is present. This MLP undergoes initialization through the Kaiming method (Kaiming et al., 2015), where randomness is involved. Our assessment involved testing T5-Hidden’s performance across five distinct random seeds and subsequently computing the evaluation metrics detailed in Table 1.\n- In the table attached below, the value of standard deviation for evaluation metrics is presented in the parentheses.\n\n|                 | AUC           | Accuracy      | F1            | Recall        | Precision     |\n|-----------------|---------------|---------------|---------------|---------------|---------------|\n| OpenAI          | 0.795         | 0.434         | 0.415         | 0.985         | 0.263         |\n| ZeroGPT         | 0.533         | 0.336         | 0.134         | 0.839         | 0.148         |\n| T5-Hidden       | 0.924 (0.024) | 0.894 (0.010) | 0.766 (0.052) | 0.849 (0.061) | 0.698 (0.079) |\n| T5-Sentinel     | 0.965         | 0.956         | 0.886         | 0.832         | 0.946         |\n---\n\n3. Is it possible to release your proposed classifier and constructed dataset?\n- After a thorough examination of the legal agreements governing ChatGPT, PaLM, LLaMA, and GPT-2, we are confident that there are no legal impediments to the public release of the constructed dataset, as long as researchers don’t use the model’s output to develop a competing product with the existing chat bot. \n- This restriction however, doesn’t undermine the reproducibility of our work, as we’re using the dataset to conduct research on identifying LLM-generated text. The dataset is presently available for anonymous access via Zenodo: https://zenodo.org/record/8285326. \n- We’re also aware that third-party classifiers mentioned in our paper like the one developed by OpenAI has removed its pubclic access, so we’ve included the raw responses from each third-party classification API in the Zenodo link as well.\n- For complete transparency, we have made the codebase accessible in an anonymous repository, encompassing model definitions, training procedures, and evaluation code. You can find it via https://anonymous.4open.science/r/Token-Prediction-As-Implicit-Classification-For-LLM/README.md. \n- After the anonymous review period, we plan to release the model weights of T5-Sentinel and T5-Hidden.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed T5-Sentinel shows superior performance in distinguishing human-generated and machine-generated text compared with other baseline methods.\n\n2. A dataset with 340k text samples is constructed, which may benefit future research in this field.", "reasons_to_reject": "1. The statistical significance is not reported in Table 1. For example, T5-Hidden contains an additional classifier head, and how different random initialization (over different random seeds) of the additional classifier would affect the performance is unclear.\n\n2. As one contribution, the release of the dataset is subject to the data and other LLM models' licenses. The failure of data release may diminish the value of this work.", "questions_to_authors": "1. How did you determine the training/dev/test split as 76%, 12%, and 12%?\n2. Have you run experiments under different random seeds to observe the performance variance?\n3. Is it possible to release your proposed classifier and constructed dataset?", "paper_topic_and_main_contributions": "The paper focuses on classifying machine-generated text. To this end, the authors proposed T5-Sentinel together with T5-Hidden detectors to distinguish human-generated text and several other LLM-generated texts. The experimental result shows that the proposed T5-Sentinel classifier outperforms T5-Hidden and other baselines. Furthermore, the dataset OpenLLMText containing 340k text samples was collected for the experimental purpose. ", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Please consider moving some tables and figures from Appendix back to the main body of the paper because some figures, like Figure 9, are necessary to get a basic understanding of the content, and readers have to go back and forth between the main content and the Appendix.  "}}, {"review_id": "lM5ylFhPMA", "reviewer": "Reviewer_K9Y3", "summary": "", "questions": "A. Will the constructed dataset and predictions by models tested in the paper become publicly available?\n\nB. Which size of T5 is used in terms of the number of parameters? Do the different sizes of T5 affect the tendency that the proposed method outperforms other baselines?\n\nC. Similarly, how is the proposed method compared to the previous work, such as Bakhtin et al. (2019) and Solaiman et al. (2019)?\n\nD. How can the comparison method, T5-Hidden, be visualized by the t-SNE plot compared to the proposed method? Does the comparison method also indicate the ability to distinguish textual content from different sources by the plot?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "A. Will the constructed dataset and predictions by models tested in the paper become publicly available?\n- After a thorough examination of the legal agreements governing ChatGPT, PaLM, LLaMA, and GPT-2, we are confident that there are no legal impediments to the public release of the constructed dataset, as long as researchers don’t use the model’s output to develop a competing product with the existing chat bot.\n- This restriction however, doesn’t undermine the reproducibility of our work, as we’re using the dataset to conduct research on identifying LLM-generated text. The dataset is presently available for anonymous access via Zenodo: https://zenodo.org/record/8285326.\n- We’re also aware that third-party classifiers mentioned in our paper like the one developed by OpenAI has removed its pubclic access, so we’ve included the raw responses from each third-party classification API in the Zenodo link as well.\n- For complete transparency, we have made the codebase accessible in an anonymous repository, encompassing model definitions, training procedures, and evaluation code. You can find it via **https://anonymous.4open.science/r/Token-Prediction-As-Implicit-Classification-For-LLM/README.md**.\n- After the anonymous review period, we plan to release the model weights of T5-Sentinel and T5-Hidden.\n---\nB. Which size of T5 is used in terms of the number of parameters? Do the different sizes of T5 affect the tendency that the proposed method outperforms other baselines?\n- We used the smallest version of T5 model, T5-small as the backbone of our experiment, which about contains 60M parameters.\n- We did not try other versions of the T5 model. Our rationale stems from the conviction that employing the most compact T5 version suffices to showcase the intuitive and yet efficacious nature of token prediction in discerning LLM-generated text. Furthermore, the training dataset we’ve collected and the computing resources available to us may not fully support fine-tuning on larger models. Even the second smallest version of T5 model contains about 220M parameters, which is almost 4 times larger.\n---\nC. Similarly, how is the proposed method compared to the previous work, such as Bakhtin et al. (2019) and Solaiman et al. (2019)?\n- We managed to reproduce the work of Solaiman et al. (2019) and ran evaluation on the OpenLLMText dataset. Results show that the detector proposed by Solaiman et al. performs well only in detecting the content generated by GPT2. Yet, the model we proposed, T5-Sentinel, still outperforms the detector by Solaiman et al. by a large margin on this task.\n- We suspect the performance degradation in other tasks are due to the difference in how text is generated. In task Human v. GPT3.5, Human v. PaLM and Human v. LLaMA, the generated text are rephrased / continued from provided human written sample, while in GPT2 output dataset, the samples are randomly generated from random hidden state vector.\n\n| Task            | Human v. GPT3.5 |      |      | Human v. PaLM |      |      | Human v. LLaMA |      |      | Human v. GPT2 |      |      |\n|-----------------|-----------------|------|------|---------------|------|------|----------------|------|------|---------------|------|------|\n| Metric          | AUC             | Acc  | F1   | AUC           | Acc  | F1   | AUC            | Acc  | F1   | AUC           | Acc  | F1   |\n| OpenAI          | .761            | .569 | .694 | .829          | .659 | .743 | .676           | .573 | .709 | .901          | .768 | .809 |\n| ZeroGPT         | .576            | .493 | .555 | .735          | .662 | .649 | .367           | .375 | .519 | .435          | .382 | .504 |\n| Solaiman et al. | .501            | .499 | .005 | .508          | .501 | .013 | .524           | .533 | .027 | .870          | .748 | .666 |\n| T5-Hidden       | .971            | .922 | .916 | .964          | .914 | .908 | .806           | .746 | .779 | .965          | .910 | .903 |\n| T5-Sentinel     | .970            | .914 | .906 | .962          | .906 | .898 | .964           | .903 | .901 | .965          | .912 | .904 |\n---\nD. How can the comparison method, T5-Hidden, be visualized by the t-SNE plot compared to the proposed method? Does the comparison method also indicate the ability to distinguish textual content from different sources by the plot?\n- We have plotted the t-SNE for T5-Hidden (with perplexity=100, same as the t-SNE plot for T5-Sentinel in the paper). Results show that T5-Hidden can clearly classify the text from Human, PaLM, GPT3.5 and GPT2. However, the hidden state vector of LLaMA from T5-Hidden is not mapped to split up with other classes correctly. This plot aligns well with the result reported in Table 5, appendix B.\n- We’ll also add this plot (https://markdown-img-1304853431.file.myqcloud.com/Screenshot%202023-08-29%20at%2019.04.25.png) to the appendix in the camera-ready version of our paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed method uses a next-token prediction from T5, which can leverage the prior knowledge obtained during pre-training.\n2. The paper compares the proposed method with not only the standard classifier-based method of T5 but also third-party methods.", "reasons_to_reject": "1. The paper, especially the result section, is not self-contained; multiple main results that verify the main claim of the paper, such as Figures 10-12 and Table 6, are placed in Appendix.\n2. It is unclear whether the constructed dataset and predictions by models tested in the paper will become publicly available.\n    - Given that [OpenAI’s text classification became unavailable as of July 20, 2023](https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text), it is difficult for other researchers to reproduce the experiments. Availability of these data is encouraged and valuable for making the experiment reproducible and advancing the research on generated text identification.", "questions_to_authors": "A. Will the constructed dataset and predictions by models tested in the paper become publicly available?\n\nB. Which size of T5 is used in terms of the number of parameters? Do the different sizes of T5 affect the tendency that the proposed method outperforms other baselines?\n\nC. Similarly, how is the proposed method compared to the previous work, such as Bakhtin et al. (2019) and Solaiman et al. (2019)?\n\nD. How can the comparison method, T5-Hidden, be visualized by the t-SNE plot compared to the proposed method? Does the comparison method also indicate the ability to distinguish textual content from different sources by the plot?", "paper_topic_and_main_contributions": "This paper addresses a text identification task to identify texts generated by large language models (LLMs). The paper proposes a T5-based method that uses a next-token prediction head to solve the task. To evaluate the proposed method, the paper also constructs a dataset called OpenLLMText that consists of texts created by humans and 4 LLMs (GPT3.5, PaLM, LLaMA, and GPT2), totaling ~340K text samples. Experimental results demonstrate the effectiveness of the proposed method over the standard classifier-based method as well as two third-party APIs (OpenAI and ZeroGPT) and provide interpretability analyses of the proposed method.\n\nUpdated: I have read the author rebuttal and keep the scores unchanged. While it is very useful to make the datasets, code, and models available, the effectiveness of the proposed method (T5-Sentinel) is limited since there is no statistical difference between T5-Sentinel and its strong baseline (T5-Hidden) on three out of four binary classification tasks (see [the discussion](https://openreview.net/forum?id=B3SjWgXHzM&noteId=qTlHpAtqwr) with the table along with standard deviations).", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. Section 5.1 may be put in Appendix because three out of four figures and tables in the section are placed in Appendix and the paper hardly discusses the results in detail."}, "extra_text": {}}, {"review_id": "r64u4dHH4v", "reviewer": "Reviewer_GnKq", "summary": "", "questions": "1. Do you have any plan to publish the OpenLLMText dataset?\n\n2. You concluded that T5-Sentinel outperforms T5-Hidden according to the result shown in Table 1. However, T5-Hidden is better in recall. I found that you used the same threshold of 0.5 for probability (mentioned in section 5.1) for both T5-Sentinel and T5-Hidden. Maybe you should try different values of the threshold for each of T5-Sentinel and T5-Hidden to identify the best threshold for each, which would give a more precise comparison result because the optimal value of the threshold may differ between these two. Could you share any information on this if you have?\n\n3. Many people will be interested in the t-SNE plot for T5-\"Hidden\" to understand the difference between T5-Sentinel and it. Could you show any result?\n\n4. In \"3.1 Data Collection,\" the text samples of GPT2-1B may be different in some way from the other four sets of samples (Human, GPT3.5, PaLM, and LLaMA-7B) because the other four datasets are based on the same Human text samples. Could you let me know if you have any insights on this?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. Do you have any plan to publish the OpenLLMText dataset?\n- After a thorough examination of the legal agreements governing ChatGPT, PaLM, LLaMA, and GPT-2, we are confident that there are no legal impediments to the public release of the constructed dataset, as long as researchers don’t use the model’s output to develop a competing product with the existing chat bot.\n- This restriction however, doesn’t undermine the reproducibility of our work, as we’re using the dataset to conduct research on identifying LLM-generated text. The dataset is presently available for anonymous access via Zenodo: https://zenodo.org/record/8285326.\n- We’re also aware that third-party classifiers mentioned in our paper like the one developed by OpenAI has removed its pubclic access, so we’ve included the raw responses from each third-party classification API in the Zenodo link as well.\n- For complete transparency, we have made the codebase accessible in an anonymous repository, encompassing model definitions, training procedures, and evaluation code. You can find it via **https://anonymous.4open.science/r/Token-Prediction-As-Implicit-Classification-For-LLM/README.md**.\n- After the anonymous review period, we plan to release the model weights of T5-Sentinel and T5-Hidden.\n---\n2. You concluded that T5-Sentinel outperforms T5-Hidden according to the result shown in Table 1. However, T5-Hidden is better in recall. I found that you used the same threshold of 0.5 for probability (mentioned in section 5.1) for both T5-Sentinel and T5-Hidden. Maybe you should try different values of the threshold for each of T5-Sentinel and T5-Hidden to identify the best threshold for each, which would give a more precise comparison result because the optimal value of the threshold may differ between these two. Could you share any information on this if you have?\n- We’ve already tried out different threshold ranging from 0 to 1 with step size of 0.1 on the prediction of both T5-Hidden and T5-Sentinel. Please refer to Figure 10 in Appendix B to see the resulting true positive rate and false positive rates under each threshold value are plotted as the ROC curve. And it is clear that T5-Sentinel out-performs T5-Hidden under all threshold value in terms of ROC curve.\n---\n3. Many people will be interested in the t-SNE plot for T5-\"Hidden\" to understand the difference between T5-Sentinel and it. Could you show any result?\n- We have plotted the t-SNE for T5-Hidden (with perplexity=100, same as the t-SNE plot for T5-Sentinel in the paper). Results show that T5-Hidden can clearly classify the text from Human, PaLM, GPT3.5 and GPT2. However, the hidden state vector of LLaMA from T5-Hidden is not mapped to split up with other classes correctly. This plot aligns well with the result reported in Table 5, appendix B.\n- We’ll also add this plot (https://markdown-img-1304853431.file.myqcloud.com/Screenshot%202023-08-29%20at%2019.04.25.png) to the appendix in the camera-ready version of our paper.\n---\n4. In \"3.1 Data Collection,\" the text samples of GPT2-1B may be different in some way from the other four sets of samples (Human, GPT3.5, PaLM, and LLaMA-7B) because the other four datasets are based on the same Human text samples. Could you let me know if you have any insights on this?\n- Yes, this is a potential problem in the data collection process. While using rephrasing / continuation prompt to construct the dataset for generated content can make our detector focus more on the syntactic and wording difference between different sources instead of the semantic difference between samples, we found that it is hard to instruct the GPT2-XL to follow the prompt and provide high-quality generation result based on human sample provided. Therefore, we used the open-source dataset released by OpenAI where content are generated from randomly initialized hidden state vector instead.\n- Since the task we are studying is to detect if a provided sample is generated or not and the source of sample, we believe the GPT2-output dataset still fits the definition of task here.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The research topic is hot recently and would attract many audiences.\nThe dataset the authors created would potentially accelerate research and development in this field.", "reasons_to_reject": "There are a very important but questionable point for which the authors do not have a plausible explanation. (Please see my 2nd question in Questions For The Authors)\nSince the authors do not state whether or not they will make the dataset publicly available, no replication study is possible though the experimental setting is clear.", "questions_to_authors": "1. Do you have any plan to publish the OpenLLMText dataset?\n\n2. You concluded that T5-Sentinel outperforms T5-Hidden according to the result shown in Table 1. However, T5-Hidden is better in recall. I found that you used the same threshold of 0.5 for probability (mentioned in section 5.1) for both T5-Sentinel and T5-Hidden. Maybe you should try different values of the threshold for each of T5-Sentinel and T5-Hidden to identify the best threshold for each, which would give a more precise comparison result because the optimal value of the threshold may differ between these two. Could you share any information on this if you have?\n\n3. Many people will be interested in the t-SNE plot for T5-\"Hidden\" to understand the difference between T5-Sentinel and it. Could you show any result?\n\n4. In \"3.1 Data Collection,\" the text samples of GPT2-1B may be different in some way from the other four sets of samples (Human, GPT3.5, PaLM, and LLaMA-7B) because the other four datasets are based on the same Human text samples. Could you let me know if you have any insights on this?", "paper_topic_and_main_contributions": "This paper discusses a recent hot topic of identifying whether or not a generative language model was used to generate a given text.\nThe proposed method reduces the problem into one of predicting the next token given a target text based on another generative model (The authors use T5 for this), which enables not only binary classification but also prediction of which of the candidate models was used.\nExperimental results show that the proposed method significantly outperforms two baseline classifiers (OpenAI and ZeroGPT) and a straightforward approach of adding a classification layer on top of T5 model, in terms of F1, Accuracy, and AUC. ", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Throughout the paper, many contents in Appendix are mentioned as if they were in the main body of the paper. A few examples: \"As illustrated with Figure 9\" in line 171 (Page 3), \"As presented in Table 4\" in line 187 (Page 3). I would say that this is not in a polite manner, and I would recommend writing like \"... Please refer to Figure 9 in Appendix for details,\" etc."}}]

Author rebuttal:

Meta-review: The paper presents the T5-Sentinel method for distinguishing human-generated and machine-generated text and introduces a dataset for this purpose. Reviewers agree that T5-Sentinel demonstrates superior performance in distinguishing human-generated and machine-generated text compared to baseline methods. Additionally, the construction of a dataset with 340k text samples is seen as a valuable contribution to the field. This dataset is expected to benefit future research in the area, making it another positive aspect of the paper. The relevance of the research topic to current trends is acknowledged, and it is noted that the paper is likely to attract a wide audience.

Reviewers raise concerns about the lack of reporting statistical significance in Table 1 but the authors addressed this concern during the rebuttal period. They note that certain aspects of the method, such as T5-Hidden containing an additional classifier head, could benefit from a discussion of how different random initializations might affect performance.

The potential issue of data and model licenses affecting the release of the dataset is raised as the main concern that especially impacts reproducibility. To my understanding,  there are no legal drawbacks to the public release of the constructed dataset and the model weights and the authors mention they will release them after the anonymity period. Another concern is about the paper's self-containment, particularly in the results section. Reviewers note that important results verifying the main claim of the paper are placed in the Appendix, making it less accessible to readers. The paper should ensure that key results are presented in a more prominent manner.

In summary, the paper has several positive aspects, including superior performance and dataset contribution. However, it needs to address concerns related to statistical significance, self-containment, and replication guarantees. Ensuring the clarity of data release plans and addressing these concerns could strengthen the paper's overall contribution and impact.
Decision: Accept-Main