Reverse Engineering Configurations of Neural Text Generation Models (2020)

Abstract: Recent advances in neural text generation modeling have resulted in a number of societal concerns related to how such approaches might be used in malicious ways. It is therefore desirable to develop a deeper understanding of the fundamental properties of such models. The study of artifacts that emerge in machine generated text as a result of modeling choices is a nascent research area. To this end, the extent and degree to which these artifacts surface in generated text is still unclear. In the spirit of better understanding generative text models and their artifacts, we propose the new task of distinguishing which of several variants of a given model generated some piece of text. Specifically, we conduct an extensive suite of diagnostic tests to observe whether modeling choices (e.g., sampling methods, top-k probabilities, model architectures, etc.) leave detectable artifacts in the text they generate. Our key finding, which is backed by a rigorous set of experiments, is that such artifacts are present and that different modeling choices can be inferred by looking at generated text alone. This suggests that neural text generators may actually be more sensitive to various modeling choices than previously thought.

## introduction the task of generating plausible sounding text from large generative neural networks has garnered significant attention recently @xcite @xcite . the study of these models has been a keen area of interest for many, resulting in research pertaining to the behavior of generation methods @xcite @xcite as well as modeling techniques @xcite @xcite . this paper presents a focused empirical study of text generation artifacts, i.e., detectable 'signatures' that originate from certain modeling or decoding choices. there is a growing body of research that has focused on discriminating between human and machine generated texts @xcite @xcite . there is also extensive past research on authorship attribution @xcite @xcite , for which it was always assumed that the authors were humans. this work takes a much more fine-grained approach by learning to distinguish between text generated by different machine variants. do certain modeling choices leave more artifacts than others? in short, given a piece of generated text, can we determine the model configuration that generated this text? the utility of our study manifests in multiple ways. first, the unraveling of artifacts in generated text enables better understanding of neural text generators, revealing potential fundamental weaknesses in modeling or generation schemes. our study provides relative comparisons of the extent to which artifacts emerge from different modeling choices. second, this research advances tracking the provenance and origination of machine generated texts, which has a range of useful applications pertaining to online trust and safety, thereby helping to mitigate the overall risk of these models in the wild. to the best of our knowledge, this is the first systematic and fine-grained study of detectable artifacts present in neural generated text. ## methodology in this section, we introduce our experimental settings and setup. ## insights and findings this section presents the insights and findings uncovered by our experiments. a secondary finding is that discriminating p or k values that are close together is a significantly more challenging task than those that are far apart (i.e., task p1 vs p2). this empirically shows that generated text moves along some form of ordering and magnitude, i.e., s(a, b) ≤ s(b, c) if a -@xmath0, b, @xmath1, y) is the accuracy score obtained by classifying between configurations x, y. ## discussion this section discusses the implications of our results and findings. ## conclusion we studied machine generated text and found that modeling choices leave artifacts, i.e., it is possible to predict modeling choices such as parameterization/sampling choices by looking at generated text alone. we proposed the novel task of machine configuration detection (mcd) which aided in the discovery of these artifacts. we believe our work paves the way for better understanding of neural text generation models and understanding that modeling choices reveals the model configurations is a first crucial step.