Detecting Machine-Generated Long-Form Content with Latent-Space Variables (2024)

Abstract: The increasing capability of large language models (LLMs) to generate fluent long-form texts is presenting new challenges in distinguishing these outputs from those of humans. Existing zero-shot detectors that primarily focus on token-level distributions are vulnerable to real-world domain shift including different decoding strategies, variations in prompts, and attacks. We propose a more robust method that incorporates abstract elements—such as topic or event transitions—as key deciding factors, by training a latent-space model on sequences of events or topics derived from human-written texts. On three different domains, machine generations which are originally inseparable from humans’ on the token level can be better distinguished with our latent-space model, leading to a 31% improvement over strong baselines such as DetectGPT. Our analysis further reveals that unlike humans, modern LLMs such as GPT-4 selecting event triggers and transitions differently, and inherent disparity regardless of the generation configurations adopted in real-time.

## introduction in today's digital world, large language models (llms) such as gpt-4 have transformed various daily tasks with their human-like text generation capability, such as drafting emails and essays. however, their potential misuse poses substantial risks including impersonation, misinformation, and academic dishonesty @xcite . this highlights the need for effective detection mechanisms. existing ai content detectors can be categorized into 1) a priori methods such as watermarking @xcite , 2) parameterized methods such as fine-tuned classifiers @xcite , and 3) zero-shot methods that rely on certain statistical differences @xcite @xcite . this paper focuses on the last due to its general pertinence in practice: end users may still choose non-watermarked llms outside the distribution of the fine-tuned classifiers @xcite . existing zero-shot methods to distinguish machine-generated texts (mgts) from humanwritten texts (hwts) typically assume an unchanging relationship between machine and human outputs, ignoring potential distribution shifts resulted from changes in generation setups @xcite or adversarial attacks @xcite . as is shown in figure 1 (a), prior zeroshot detection methods usually assume that mgts consistently exhibit higher log-likelihood (or conditional curvature) than hwts. however, certain changes-including increased decoding temperature, paraphrasing, and word substitution-can alter the distribution of mgts @xcite , making mgts and hwts inseparable by loglikelihood after domain shift. in this work, we further identify that complex prompts (e.g., ask to prepare an outline before generation) can alter the typical relationship between mgts and hwts. consequently, when considering mgts from diverse sources in real world, existing detectors become less effective. towards more robust detection, we aim to answer the research question: is there a shared feature among all mgts, regardless of the generation configurations and adversarial attacks? @xcite ; @xcite ; @xcite argue that machine-generated long texts are fluent at word level, but lack discourse coherence in terms of topic or event transitions. we hence hypothesize that mgts and hwts are more separable in such latent space illustrated in figure 1(b) , because those underlying features can not be easily captured by next token probabilities (i.e., optimization in observation foot_0 space). in addition, paraphrase or edit attacks change original texts with semantically similar yet lexically different alternatives. while they effectively alter token-level distributions, the high-level, hidden representations remain similar. we test our hypothesis on three writing tasks: creative scripts, news, and academic essays. we explored latent variables like part-of-speech tags, topics, verbs, and event sequences. specifically, we first train a lightweight model (62m transformer, half the size of gpt-2-small) on the latent variables inferred from hwts, and then compare the latent distributions between hwts and mgts at test time. we find detectors in observation and latent space exhibit complementary strengths ( § 3.2) in differentiating mgts from various configurations. integrating the criteria from both types yields the best performance. we explore five features to represent the underlying structure, and find that event trigger derived from information extraction models @xcite is the most effective in separating hwts from mgts, outperforming strong token-space detector @xcite by 31% in auroc. our analysis in § 5 further reveals that llms such as gpt-4 exhibit a different preference from human in choosing event triggers (for creative writing) and event transitions (for news and science), and such a disparity cannot be bridged through explicit planning of these latent structures. to sum up, we identify key factors that deceive existing detectors in real-world scenarios. we then demonstrate a significant discrepancy in hidden structures between current llms and humans, especially the selection and transitions of event triggers. building on these insights, we propose a novel detection framework that employs latent variables to robustly differentiate between human-and machine-generated texts. ## preliminary: fragility of existing zero-shot detectors in this section, we introduce two popular lines of zero-shot detection methods (logit-based and perturbation-based), and then illustrate how they are fragile to manipulations in decoding, variations in prompting style, and adversarial attacks. ## machine-content detection with latent variables in § 3.1, we formulate the next-token-prediction process with latent variables and introduce a neural model to learn the distributions of these variable. next, in § 3.2, we propose a simple but effective method to combine the benefits of existing samplespace detectors with our latent model. ## conclusion and discussion we propose a novel zero-shot detection framework that employs latent features such as sequences of events. our method leverages the limitations of current llms in replicating authentic human-like discourse, despite their ability to generate locally convincing language. experimental results demonstrate that our detector is highly robust across various real-world generation settings and attacks. similar to existing works, our detection methods also rely on certain assumptions about how machine-generated content differs from humancreated content. we specifically assumes that text generators struggle to replicate human-like highlevel structures. however, we believe that it will take longer for llms to catch-up with humans on high-level structures than for them to mimic token-level distributions. therefore, our method is likely to remain relevant for a longer period than the token-level approaches. furthermore, we hope the latent-space statistics can serve as a valuable indicator to guide current llms in improving highlevel content generation.