Do ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour (2022)

Abstract: Language models (LMs) trained on raw texts have no direct access to the physical world. Gordon and Van Durme (2013) point out that LMs can thus suffer from reporting bias: texts rarely report on common facts, instead focusing on the unusual aspects of a situation. If LMs are only trained on text corpora and naively memorise local co-occurrence statistics, they thus naturally would learn a biased view of the physical world. While prior studies have repeatedly verified that LMs of smaller scales (e.g., RoBERTa, GPT-2) amplify reporting bias, it remains unknown whether such trends continue when models are scaled up. We investigate reporting bias from the perspective of colour in larger language models (LLMs) such as PaLM and GPT-3. Specifically, we query LLMs for the typical colour of objects, which is one simple type of perceptually grounded physical common sense. Surprisingly, we find that LLMs significantly outperform smaller LMs in determining an object’s typical colour and more closely track human judgments, instead of overfitting to surface patterns stored in texts. This suggests that very large models of language alone are able to overcome certain types of reporting bias that are characterized by local co-occurrences.

## introduction large language models (llms) have been compared to hypothetical giant octopi living underwater that are exposed to a lot of language data @xcite . such octopi would struggle to understand what actually happens on land as they lack the physical perceptual experience of living there. as such, they may overfit to text-only corpora and thus amplify reporting bias @xcite rather than faithfully reflecting the physical world. figure 1 : on typical colour judgments, large language models (llms) greatly outperform small lms which previously were found to be no better than corpus statistics (google ngram). see in textual corpora, humans do not tend to mention what is commonly known, instead using language to express new information, which is likely less common. for example, when describing the colour of a banana: "green banana" has much higher frequency than "yellow banana" in the google books corpus. 2 it is natural to expect lms would overfit to such reporting bias since they are trained to memorise such co-occurrence statistics. to observe this, we can query widely used pretrained models, such as roberta large @xcite with our previous example. given the prompt "it is commonly known that most bananas have the @xmath0, roberta ranks "green" the highest. 3 this agrees with corpus statistics derived from raw text corpus such as the google ngram @xcite mentioned above. 4 @xcite test pretrained lms' perception of colours and confirm that they perform no better than naive co-occurrence statistics extracted from the corpus. in fact, naively using corpus statistics achieves around 40% accuracy on their proposed colour probing benchmark coda while the best lm performs similarly. @xcite extend the evaluation to a broader range of visual properties, confirming that reporting bias can negatively influence model performance and increasing model size does not help. @xcite repeat the reporting bias experiments of @xcite on pretrained lms and find that lms overestimate rare events and actions, also amplifying reporting bias. however, the lms tested by @xcite ; @xcite ; @xcite , i.e., gpt-2 @xcite , bert @xcite , roberta, and albert @xcite , usually have only several hundred million parameters and are of much smaller sizes than llms available now. in this work we probe t5 @xcite , gpt-3 @xcite , and palm @xcite of various sizes, with parameter counts ranging from 770m to 540b. surprisingly, we find that llms almost double the performance of small language models (slms) on the typical colour task (figure 1 ). 5 paik et al. (2021) point out that slms achieve poor performance on objects that typically only have one colour (such as bananas), possibly due to their true colour being an aspect of common sense and thus not frequently mentioned in the training corpus. we find in contrast that llms achieve surprisingly good performance in this category, @xmath1. after plotting accuracy against model size, we observe that scaling up is universally helpful for improving llms' performance on the colour probing benchmark (coda; @xcite . moreover, as llms are scaled their correlation to corpus ngram statistics plateau, suggesting that memorising (local) co-occurrence statistics cannot explain their success. 6 our study presents controlled analyses on the colour prediction task as a case study to show that scaling up llms could overcome surface-level pat-tern memorisation (i.e., text reporting bias in our case) and learn physical world common sense at least to some extent. this is an important and surprising finding as it provides a key evidence to counterargue the previous consensus that despite achieving better performance for a range of nlp tasks, larger lms are more prone to overfitting to corpus statistics and therefore amplifying the reporting bias. our study points out that this criticism on model scale is misleading as it is not based on the complete picture, and when the model capacity is increased to a significantly large scale such as palm-540b and gpt-3 davinci , they start to overcome reporting bias and are able to abstract physical common sense from text. ## method to test whether llms replicate corpus biases rather than human judgment in the typical colour task, we compare the models' output distributions with the corpora's distribution and the distribution of human judgments. visual perception provides an ideal testbed as corpus statistics can vary from physical facts; obvious facts are left unspoken. in this case, we focus on the typical colour task, largely following the setup by @xcite . given a query asking the colour of an object, the model must output a distribution over eleven possible colours. we then compare the output distribution to both corpus statistics and average human judgement to examine their respective correlations. in the following, we explain how we query the llms and use their predictions. we test llms in three setups: zero-shot, one-shot, and five-shot. zero-shot. we use the following prompt across all models: it is known that most {objectq} have the @xmath2. 7 after inputting the prompt, we compute next-token-prediction likelihood for all 11 colours in the coda label space and record the log-likelihood scores for all answers as the output distribution of the query: compared llms and their sizes. openai does not disclose the exact size of their text models ada, babbage, curie and davinci. according to blog.eleuther.ai/gpt3-model-sizes, they roughly correspond to 350m, 1.3b, 6.7b, and 175b, which we use as the models' parameter counts. for other models (i.e., t5 and palm), their number of parameters are made clear in the original papers. we list all compared models' sizes in the second column of ## results main results ( results on the "single" colour split ( the "single" split deserves extra attention as it has the highest human consensus and is also considered to be common sense knowledge, implying it is rarely stated in the corpus @xcite . while none of the slm baselines outperform the ngram baseline on acc @1 , the largest palm and gpt-3 surpass the ngram baseline by nearly 40%. furthermore, the llms' predictions correlate significantly more to human judgments. we also present an error analysis on the "single" split in appx. §b.5. out of the ten errors made by palm-540b, only one is a clear mistake where the model classifies picnic baskets as red. for other nine errors, the error seems to be associated with the ambiguous nature of the questions or the dataset construction process. correlation metrics (figure 2 ). initially increases but then plateaus and even decreases (on 5-shot: 42.2 → 51.9 → 57.1 → 55.9). on palm, ρ ngram decreases from the start as model size grows (on 5-shot: 62.0 → 61.0 → 60.1). on both models, ρ ngram initially is larger than ρ human . however, for model sizes above 10 11 parameters, both models' predictions have ρ @xmath3. this suggests that when lms are small, they can underfit corpus ngrams. when lms start to be scaled up, they increasingly fit the corpus. however, after a certain model size, additional scale does not lead to more overfitting to corpus statistics. on the contrary, as llms' predictions correlate more with human judgment, they also start to decorrelate with corpus statistics. comparing within model class for better control of confounders. though llms today are almost all transformer-based models with similar autoregressive pretraining objectives, we note that there are caveats preventing us from having a perfect control over design choices on pretraining corpora and specific architectures. in terms of pretraining data, within-family models of different sizes generally use the same training data (gpt-3models are however less transparent in this regard). however, it is unclear what differences there are across model families. in terms of model architectures, t5 is an encoder-decoder model while gpt-3 and palm are decoder-only models. palm has further modifications on top of the original transformer architecture such as using swiglue activation (shazeer, 2020) instead of the standard relu; using rope embeddings @xcite instead of the original relative position embeddings. as a result, more conclusive findings should be drawn within model classes, e.g. comparing palm-540b with its two smaller versions instead of gpt-3 models. colours live on a spectrum. the evidence we obtain does not reflect whether llms have a finegrained and holistic understanding of the nature of colour. that is, colours live on a continuous spectrum. llms could have solved coda by identifying the mappings between objects and colours but not colour's relative positions on the spectrum. one way to probe this is to examine if llms can resolve colour synonyms (e.g., do llms know that "scarlet" occupies a subspan of the colour red?). however, a rigorous and systematic study of this problem is beyond the scope of this study. ## conclusion and future work in this work, we examine llms ability to make typical colour judgments, a simple property of visual common sense. contradicting @xcite ; @xcite , we find that typical colour judgments do not follow an inverse scaling law, and scale is indeed quite critical for high accuracy on the task. while generalising from this task to visual reasoning as a whole is premature, we provide some evidence that larger models of language alone are able to overcome a basic type of reporting bias. future work will look at a wider range of physical properties @xcite and more carefully control for the data and model size. we also hope our work opens an avenue for empirically verifying on what level meaning acquisition is possible from a cognitive linguistic perspective (piantasodi and hill, 2022). a a more comprehensive table (