Token Prediction as Implicit Classification to Identify LLM -Generated Text (2023)

Abstract: This paper introduces a novel approach for identifying the possible large language models (LLMs) involved in text generation. Instead of adding an additional classification layer to a base LM, we reframe the classification task as a next-token prediction task and directly fine-tune the base LM to perform it. We utilize the Text-to-Text Transfer Transformer (T5) model as the backbone for our experiments. We compared our approach to the more direct approach of utilizing hidden states for classification. Evaluation shows the exceptional performance of our method in the text classification task, highlighting its simplicity and efficiency. Furthermore, interpretability studies on the features extracted by our model reveal its ability to differentiate distinctive writing styles among various LLMs even in the absence of an explicit classifier. We also collected a dataset named OpenLLMText, containing approximately 340k text samples from human and LLMs, including GPT3.5, PaLM, LLaMA, and GPT2.

## introduction in recent years, generative llms have gained recognition for their impressive ability to produce coherent language across different domains. consequently, detecting machine-generated text has become increasingly vital, especially when ensuring the authenticity of information is critical, such as legal proceedings. traditionally, techniques like logistic regression and support vector machines (svm) have been used for detection tasks, as explained by @xcite . the analysis of textual features like perplexity is proven to be effective as well @xcite . recent advancements have introduced the use of language model itself to detect generated text, such as the ai text classifier released by @xcite and @xcite . however, the exponential growth in the number of parameters from hundreds of millions to † two authors contribute equally to this work. ⋆ the implementation of the classification model, training process, and dataset collection is publicly available on https: //github.com/markchenyutian/t5-sentinel-public hundreds of billions has significantly improved the text generation quality, presenting an unprecedented challenge to the detection task. to overcome this challenge, we propose using the inherent next-token prediction capability of the base lm for detection task, aiming not just to determine whether or not the text is generated but also to identify its source. ## method our approach can be formulated as follows. let σ represent the set of all tokens. the base lm can be interpreted as a function lm : σ * × σ → r. given a string @xmath0, lm(s, σ) estimates the probability of the next token being σ. let y denote the set of labels in openllmtext, which contains "human", "gpt-3.5", etc. we establish a bijection f : @xmath1, where y ⊂ σ acts as a proxy for the labels. by doing so, we reformulate the multi-class classification task σ * → y into a next-token prediction task σ * → y. hence, the multi-class classification task can be solved directly using lm: adamw optimizer @xcite with a mini-batch size of 128. the learning rate is 1 × 10 -4 with weight decay of 5 × 10 -5 , and we train for 15 epochs. ## evaluation t5-sentinel and t5-hidden are evaluated on the test subset of openllmtext dataset with receiver operating characteristic (roc) curve, area under roc curve (auc) and f1 score. ## interpretability study our interpretability studies, including a dataset ablation study and integrated gradient analysis, show that t5-sentinel does not rely on unexpected shortcuts. additionally, we employ t-distributed stochastic neighbor embedding (t-sne) projection (van der maaten and hinton, 2008) on the hidden states of the last decoder block of t5-sentinel. the resulted t-sne plot, shown in figure 7 , demonstrates the model's ability to distinguish textual contents from different sources, corroborating the evaluation results discussed earlier. for comparison, we also plotted the t-sne plot of t5-hidden on the test subset of openllmtext in figure 8 , results show that the t5-sentinel cannot distinguish llama with other source of sample correctly. this aligns with the evaluation reported in ## conclusion in conclusion, this paper demonstrates the effectiveness of involving next-token prediction in identifying possible llms that generate the text. we make contributions by collecting and releasing the openllmtext dataset, transferring the classification task into the next-token prediction task, conducting experiments with t5 model to create the t5-sentinel, and providing insight on the differences of writing styles between llms through interpretability studies. in addition, we provide compelling evidence that our approach surpasses t5-hidden and other existing detectors. as it eliminates the requirement for an explicit classifier, our approach stands out for its efficiency, simplicity, and practicality.