Are Some Words Worth More than Others? (2020)

Abstract: Current evaluation metrics for language modeling and generation rely heavily on the accuracy of predicted (or generated) words as compared to a reference ground truth. While important, token-level accuracy only captures one aspect of a language model’s behavior, and ignores linguistic properties of words that may allow some mis-predicted tokens to be useful in practice. Furthermore, statistics directly tied to prediction accuracy (including perplexity) may be confounded by the Zipfian nature of written language, as the majority of the prediction attempts will occur with frequently-occurring types. A model’s performance may vary greatly between high- and low-frequency words, which in practice could lead to failure modes such as repetitive and dull generated text being produced by a downstream consumer of a language model. To address this, we propose two new intrinsic evaluation measures within the framework of a simple word prediction task that are designed to give a more holistic picture of a language model’s performance. We evaluate several commonly-used large English language models using our proposed metrics, and demonstrate that our approach reveals functional differences in performance between the models that are obscured by more traditional metrics.

## introduction language models are foundational components in many nlp systems, and as such it is crucial to be able to empirically evaluate their behavior. traditionally, language models are evaluated using performance metrics that relate to the model's ability to accurately predict words given some context (e.g., perplexity). following the paradigm described by @xcite , this can be thought of as an intrinsic evaluation criterion (and perplexity an intrinsic metric), as it relates to the objective of the language model itself. in recent years, it has become common to also evaluate language models extrinsically, in terms of the model's function. this is done by measuring a model's performance when used as a component in a downstream task. 1 for example, @xcite evaluated bert by using it as the language model component in benchmark tasks such as question answering and "commonsense inference." foot_1 this shift towards extrinsic and task-oriented evaluation is welcome, and has the potential to make language model evaluation more ecologically valid. 3 as useful as task-oriented evaluation metrics are, however, we believe that this approach brings with it certain practical limitations, and that there remains a strong need for robust and meaningful intrinsic evaluation metrics that can be used to characterize and compare the performance of language models. in this work, we outline and propose a variation on the standard next-word-prediction language modeling task that is designed for use in evaluating and comparing language models and is robust to implementation differences (tokenization method, etc.) that complicate the comparison of modern models in terms of token-level predictions. our proposed metrics are richer and more meaningful measures than traditional intrinsic metrics such as perplexity, which is insensitive to which tokens are matched, and as such may be confounded by distributional properties of their evaluation corpora. our approach accounts not only for the accuracy of a model's word predictions, but also the diversity of types that it predicts, across different lexical frequency bins. we further propose a formulation for the next-word-prediction task that explicitly allows for language-and task-level details to be captured in the resulting metrics, thereby blurring the line between intrinsic and extrinsic language model evaluation. our methods provide greater ecological validity than traditional intrinsic evaluation methods, while still remaining simple to interpret and easy to calculate. ## methods in this section we describe a series of experiments in which we use our proposed evaluation metrics to explore the behavior of several widely-used and large-scale language models (obtained using the huggingface @xcite transformers library). specifically, we examine gpt-2 (alec et al., 2019) (gpt-2), gpt @xcite (openai-gpt), roberta @xcite (roberta-base), and bert @xcite (bert-base-uncased). ## a case study of paraphrasing in this section we will look at the impact of model inference performance on the particular downstream task of paraphrasing. to this end we employed a sota algorithm, bertscore @xcite , to compute similarity scores of sentence pairs in part by comparing embeddings derived from a language model. under bertscore, higher similarity scores indicate greater semantic similarity of a pair of sentences, such that one is a closer paraphrase of the other. we would like to stress that the critique that may be risen at the end of this section is not about bertscore tool as such, but are rather about a certain type of pattern that the models that are employed by this tool may have insufficiently learned. why paraphrasing? we choose the downstream task of paraphrasing to measure semantic similarity of sentence pairs as it can be easily manipulated to consider a single word modification. consider the following example sentence involving the word triple: (poodle, dog, cat) 14 (a) which dog has longer hair ? (b) which cat has longer hair ? (c) which poodle has longer hair ? the pair (a, b) ought to score lower (i.e., be considered by bertscore to be more dissimilar) than the pair (a,c), as a is a valid paraphrase of c while b is not. this, of course, assumes that the language model being used as the underlying source of embeddings for the bertscore algorithm has accurately captured the semantic meaning of the three words under consideration. if not, we may see an inversion of results such that (a,b) appears (incorrectly) more similar than (a,c), suggesting that the model in question should perhaps not be used for paraphrase-related tasks. to explore the impact of word frequency on model representations with our fine-tuned models, we have generated 50 rare, and 50 common triples elicited from wiki-103 trainset. each of the triples contains a rare/common word, its hypernym, and a sibling hypernym extracted from wordnet @xcite (using nltk @xcite ) (x r ,x h ,x a ), (x c ,x h ,x a ) respectively. for each word in a triple, we identified a sentence in which the rare word naturally occurs, and generated probe sentences in which we replace the rare word with x h and x a foot_13 . we then used bertscore to compare our sentences in terms of their similarity. in principle, we expect that bertscore(s(x h ),s(x r )) > bertscore(s(x h ),s(x a )). in other words, a similarity score for the pair made of a sentence with dog and the sentence with poodle is expected to be higher than than the pair made of a sentence with dog and a sentence with cat, as dog and a poodle are closer semantically, than a dog and a cat, and therefore would be a closer paraphrase of each other. alternatively, if the model's word representations are being confounded by lexical frequency, we may instead observe the opposite pattern (i.e., the sentences with the more common words mistakenly appearing to be more similar to one another, despite their semantic difference). we consider cases in which the model correctly identifies the paraphrase (e.g., if bertscore(s(dog), s(poodle)) > bertscore(s(dog), s(cat))) as hits, and misidentifications as misses. our "null hypothesis" is that there should not be any difference in hit rate between high-and low-frequency words (i.e., word frequency should not affect the model's ability to identify paraphrases). furthermore, we compare the performance of two fine-tuned models trained on wiki-103, bert and roberta, and (given the results of our earlier experiments) we hypothesize that if there is a difference in hit rate, roberta will prove more robust to the rare words condition, given its superior performance at predicting (and thus representing) rare words. we note that this is something of a "toy" experiment, given its small size, which limits the conclusions that we can draw. however, in we found that roberta and bert did not differ greatly in their performance, suggesting that with this method the strong effect of word frequency outweighed the between-model difference observed in our earlier experiments. this null result could easily be an artifact of our very small sample size of 100 probe sentences, though, and we also did notice a substantial number of misses with the set of common words. overall, despite this being based only on a small sample, it does seem that the lower performance of both models on the rare words is unlikely to be a coincidence. we hope to be able to experiment with a greater sample size to begin learning more about the degree to which rare-word inferences are reliable to produce outcomes aligned with human semantics on various downstream tasks. ## future work the paraphrasing task should be conducted at a larger scale. furthermore, we hope to continue evaluating language models' prediction diversity and its effects on additional downstream tasks (for example, tasks where human speech is anticipated), since prediction diversity evaluation may vary between one task to another. the unit of evaluation can go beyond words, and may be defined at various textual granularities, such as phrases, for instance, depending on prediction diversity desired. we also leave for future work questions of to what degree different tokenization approaches, or model size effect prediction diversity. in our experiments, we did observe differences in performance between models with different tokenization strategies (e.g. gpt-2 and roberta as compared with their architectural counterparts); however, these models also varied substantially from one another in other respects (size, etc.) and as such it is difficult to attribute this performance gap to tokenization alone. it may also be the case that the bigger the model (in terms of number of parameters), the more diverse it is likely to be; under this hypothesis, we would expect today's ever-larger models (e.g. gpt-3) to outperform their predecesors in terms of diversity. however, we do not believe that it is sustainable @xcite to rely on increasing model complexity as an approach to addressing the frequency-related challenges that we observed in our experiments, and believe that fundamentally different approaches to language model training are needed. ## conclusion we presented two types of evaluation techniques to learn about the performance of the model across its input distribution, revealing the easier and the challenging areas to learn. through this analysis we showed that current models are susceptible to frequency bias during training, and as a result under-performing when less frequent examples are encountered at test time, hurting the overall performance. in addition, we proposed a way to learn about the degree to which a model's prediction is semantically close to a target in cases where an exact match was not predicted, which may more accurately reflect a model's usefulness. thirdly, we showed how a downstream task of paraphrasing may be rendered less reliable, as the models employed struggle to produce semantically-useful representations when rare words are involved. we believe that language models should reflect the trained distribution more optimally than what we observed in our evaluation, and we should recognize their bias to frequency -making them unfair towards some words, and potentially harmful for our downstream tasks. we also believe it is important to take part in setting benchmarks for models' diversity. finally, distributional representation goes beyond words, and we hope to address more complicated representational tasks as well.