An Application of Pseudo-log-likelihoods to Natural Language Scoring

Language models built using semi-supervised machine learning on large corpora of natural language have very quickly enveloped the fields of natural language generation and understanding . In this paper we apply a zero-shot approach independently developed by a number of researchers now gaining recognition as a significant alternative to fine-tuning for evaluation on common sense tasks . A language model with relatively few parameters and training steps ( albert-xxlarge-v2 ) compared to a more recent language model ( T5 ) can outperform it on a recent large data set ( TimeDial ) , while displaying robustness in its performance across a similar class of language tasks . Surprisingly , this result is achieved by using a hyperparameter-free zero-shot method with the smaller model , compared to finetuning to the larger model . We argue that robustness of the smaller model ought to be understood in terms of compositionality , in a sense that we draw from recent literature on a class of similar models . We identify a practical cost for our method and model : high GPU-time for natural language evaluation . The zero-shot measurement technique that produces remarkable stability , both for ALBERT and other BERT variants , is an application of pseudo-log-likelihoods to masked language models for the relative measurement of probability for substitution alternatives in forced choice language tasks such as the Winograd Schema Challenge , Winogrande , CommonsenseQA , and others . One contribution of this paper is to bring together a number of similar , but independent strands of research . We produce some absolute state-of-the-art ( SOTA ) results for common sense reasoning in binary choice tasks , performing better than any published result in the literature , including fine-tuned efforts . In others our results are SOTA relative to published methods similar to our own – in some cases by wide margins , but below SOTA absolute for fine-tuned alternatives . In addition we show a remarkable consistency of the model ’ s performance under adversarial settings , which we argue is best explained by the model ’ s compositionality of representations . 1 INTRODUCTION . Computational linguistics has made major strides in the adoption of machine learning techniques applied to unstructured corpora consisting of human generated natural language text . For example , some methods take advantage of the frequencies of words in natural human text to productive ends ( Collobert et al. , 2011 ; Mikolov et al. , 2013a ; b ; Peters et al. , 2018 ) . N-gram models providing frequencies of pairs , triples , etc . of words in natural text provided further gains on related tasks . However , a very influential paper in 2018 , signalling a major shift in the application of machine learning to natural text , advocated for an architecture that has “ a more structured memory for handling long-term dependencies in text , compared to alternatives like recurrent networks , resulting in robust transfer performance across diverse tasks. ” ( Radford et al. , 2018 ) This culminated in the application of the Transformer ( Vaswani et al. , 2017 ) to the creation of representations through language prediction tasks , motivated by the importance of long-term dependencies in natural language text for not only the choice of model , but also training data ; as Radford et al . note , “ Crucially , [ BooksCorpus ] , a common corpus for a multitude of emerging transformer models , contains long stretches of contiguous text , which allows the generative model to learn to condition on long-range information. ” Why might ‘ long stretches of contiguous text ’ , via learning conditioned on that text , lead to success at diverse tasks like natural language inference , question answering , sentence similarity , and classification ( Radford et al. , 2018 , Table 1 ) ? After all , these tasks typically involve very short , independent sections of text . Solving the Winograd Schema Challenge ( WSC ) Levesque et al . ( 2012 ) seems to require vast amounts of common sense knowledge , and the job of learning long-term dependencies was supposed to help replacing actual knowledge of the world with the proxy knowledge that humangenerated text provides . Although language models do well at common sense benchmarks through fine-tuning , when we evaluate them using standard fine-tuning methods with a small , admittedly unreliable ‘ quick-probe ’ , they do not generalize well to new samples that we offer . On the other hand , a recent zero-shot technique using an idiosyncratic model with several unique architectural and training features shows remarkable consistency and absolute performance on our unreliable quick probe , but also on a family of challenging common sense problems . 1.1 SUMMARY OF CONTRIBUTIONS : . In this paper we investigate the properties of a language model with parameter sharing : albert-xxlarge-v2 , small in both parameter count and pre-training corpus relative to the field of language models generally . We find that pseudo-log-likelihoods ( PLL ) and token-normalied PLLs ( NormPLL ) methods for scoring natural language with this model performs at a mixture of outright state-of-the-art ( SOTA ) performance and robust , but SOTA performance just for zero-shot methods at a series of recent binary common sense language tasks . The combination of model and method is remarkable consistent , scoring around 75-80 % under conditions both designed to be adversarial against language models . The approach is also robust against accidental processes that reduce zero-shot performance in language models generally , such as semantically and syntactically noisy data . To our knowledge , our results are SOTA for any approach to the TimeDial ( Qin et al. , 2021 ) dataset ; SOTA for any zero-shot approach to solving the train-xl split of Winogrande ( Sakaguchi et al. , 2020 ) ; SOTA for an average score on the perturbed Winograd set ( Abdou et al. , 2020 ) ; and , SOTA for any zero-shot approach to WSC , with the exception of a reported result in which training and testing sets were mixed . In other cases , our approach is SOTA for zero-shot and competitive with fine-tuned approaches . We provide an explanation for the results and their significance . 2 RELATED WORK . 2.1 BIDIRECTIONAL VS. UNIDIRECTIONAL MODELS . The two most recent GPT papers , ‘ Language Models are Unsupervised Multitask Learners ’ ( Radford et al. , 2019 ) and ‘ Language models are few-shot learners ’ ( Brown et al. , 2020 ) identify in their titles the nature or purpose of machine learning models for language with the purposes they put their GPT variants to in their paper . Emphatic titles aside , the most influential fine-tuning papers also advocate for few- and zero-shot results . A more important differentiator between GPT style and NormPLL-suitable models is the significant benefit of a bidirectional masked objective for success with PLL scoring methods over single-directional masked objectives , as Salazar et al . ( 2020 ) , Zhou et al . ( 2020 ) , and Ma et al . ( 2021 ) show . 2.2 THE ‘ QUICK-PROBE ASSUMPTION ’ . In his discussion of Winograd schemas , Dennett defines what he calls the ‘ quick-probe assumption ’ : success on a few Winograd schemas in a Turing test-style evaluation ought to indicate generalizability of a computer ’ s ability to make common sense judgements , not merely success at the few examples like it , or examples like it in some superficial way only ( Dennett , 1984 ) . One of us , skeptical of fine-tuning for success at tasks like the Winograd Schema Challenge and similar problems , hand-made a set of 20 sentence pairs1 prior to collaboration on the present paper . 1https : //anonymous.4open.science/r/NotSoFineTuning-4620/ winogradversarial/examples.json We have reproduced the dataset in its entirety in Appendix A.2 . The purpose of this set of Winograd-style pairs is to test whether fine-tuning can be attacked directly , as follows . Suppose a training set contains multiple complete pairs , such that reference is shifted every time a sentence has a twin that is different only in some modifier or short phrase . Then perhaps a pair in which reference isn ’ t shifted will be scored poorly , if the model is spuriously using the modifier trick . This can be an exploited trick ( at least in principle ) if , for example , one member of a Winograd schema pair is in the train set , and the other is in the test set 2 . Here is an example from this small , hand-made data set : 1 . This is why people are supposed to take salt tablets when < mask > sweat a lot . Answers : people , salt tablets 2 . This is why people are supposed to take salt tablets when < mask > sweat a little . Answers : people , salt tablets By substituting the answers in for the mask above we get two pairs of sentences for a model to score , or assess the relative likelihood of , resulting in two questions of the suitcase/trophy example above . The correct answer above for both examples is ‘ people ’ , since salt tablets don ’ t sweat . In Table 1 we compare the performance of a variety of models that have been fine-tuned on the Winogrande , a scaled WSC-variant debiased against RoBERTa ( Sakaguchi et al. , 2020 ) . We find that the BERT family of language models generally does poorly on this data set when evaluating its fine-tuned discriminator on the data set . On the other hand , using a method of scoring sentences using language models in a manner which is free of hyperparameters , we also score the models – in the second column , there is no training beyond the objective functions of the models during semi-supervised pre-training . Notice that a single model outperforms the others : albert-large . The albert-xxlarge-v2 variant scores an impressive 80 % on the Winogradversarial dataset we present . It is a well-defined question to ask whether this high value for that last variant is a statistical fluke , or evidence of a robust ability to score binary common sense sentence pairs at a rate of around 80 % . An anonymous reviewer points out that , on 20 coin flips , there is a greater than 25 % chance of achieving more than 12 heads , or 60 % accuracy . Therefore the results of Table 1 are not particularly meaningful . We agree : these results are not particularly meaningful , by themselves . This paper argues on the basis of new results on very large binary choice and similar data sets that the 80 % score achieved by the albert-xxlarge-v2 is due to its compositionality of representation and corresponding systematicity of behaviour . We also cite independent research that supports our interpretation of our results . 2.3 HYPERPARAMETERS AND ZERO-SHOT . A broad survey of machine learning research concludes that demonstrating an ability to innovate in model construction dominates work done in data set collection and cleaning ( Sambasivan et al. , 2021 ) . Resource constrained researchers are able to use platforms like huggingface to leverage pretraining with novel models contributed by other researchers . PLLs applied to language models for common sense tasks present both opportunities and challenges distinct from this standard approach 2This is in fact turns out to be the case in the WNLI dataset which is part of the general natural understaning benchmark of SuperGLUE Wang et al . ( 2019 ) to distributed work in NLP . Predictive language model scoring using a pre-trained deep learning model has been used since at least ( Linzen et al. , 2016 ) , although as we discuss below PLLs seem to display unique benefits for architectures with idiosyncratic features such as parameter sharing and bidrectionality . Despite its nascent application , scholarly literature has already recognized availability of GPU time for researchers as a limiting factor in applying NormPLLs ( our term ) for large language models . Laban et al . ( 2021 ) explicitly limit their investigation to the smallest , ‘ base ’ models of BERT and RoBERTa in their published results . As we demonstrate below , ALBERT requires an order of magnitude more GPU time for NormPLL scoring than BERT , but we nevertheless provide results for a number of important data sets using the ‘ xxlarge ’ variant of ALBERT . In Appendix C we compare the approach we share here to a wide range of common sense tasks , including COPA ( Roemmele et al. , 2011 ) . The website associated with the COPA dataset contains an ethics injunction for its users with specific imperatives : researchers should not peek at the data set before they evaluating their model on it ; and , researchers should only evaluate their method on COPA once.3 Our zero-shot method scores an impressive 80 % on COPA ; as we argue below , sensitivity of the method to even extra spaces around punctuation marks necessitates a certain amount of familiarity with the data . We see critical views of fine-tuning in industry . A recent white paper eschews fine-tuning and even few-shot evaluation for assessing the representational quality of a natural language model because of their potential for spurious results.4 Research parallelism can produce spurious results simply as a consequence of the number of hypotheses tested.5 It is beyond the scope of this paper , but there are number of methods , such as Bonferroni correction , that can be used in settings of multiple hypothesis testing . Regardless of one ’ s priors for the compliance of fine-tuning of language models by the research community with statistical best practices , one may find zero-shot measurements of language models more reliable simply because of the fewer points of possible p-hacking , intentional or otherwise .

Summary: The paper makes multiple independent contributions, including:  1. The addition of a new adversarial common-sense reasoning dataset dubbed “Winogradversarial” 2. The explicit gathering together of multiple research threads all exploring the use of pretrained language models for zero-shot prediction on language-understanding tasks 3. The singling-out of the albert-xxlarge-v2 architecture as performing especially well on zero-shot common-sense reasoning tasks, achieving SOTA on the TimeDial dataset  The overall message of the paper is that the existing framework of fine-tuning models for common-sense reasoning tasks is flawed and prone to overfitting, and that more robust results can be achieved using pretrained models in a zero-shot manner. In particular, doing so with the albert-xxlarge-v2 model achieves state-of-the-art performance on some benchmarks.