T he W orld of an O ctopus: H ow R eporting B ias I nfluences a L anguage M odel’s P erception of C olor (2021)

Abstract: Recent work has raised concerns about the inherent limitations of text-only pretraining. In this paper, we first demonstrate that reporting bias, the tendency of people to not state the obvious, is one of the causes of this limitation, and then investigate to what extent multimodal training can mitigate this issue. To accomplish this, we 1) generate the Color Dataset (CoDa), a dataset of human-perceived color distributions for 521 common objects; 2) use CoDa to analyze and compare the color distribution found in text, the distribution captured by language models, and a human’s perception of color; and 3) investigate the performance differences between text-only and multimodal models on CoDa. Our results show that the distribution of colors that a language model recovers correlates more strongly with the inaccurate distribution found in text than with the ground-truth, supporting the claim that reporting bias negatively impacts and inherently limits text-only training. We then demonstrate that multimodal models can leverage their visual training to mitigate these effects, providing a promising avenue for future research.

## introduction given sufficient scale, language models (lms) foot_0 are able to function as knowledge bases, yielding factoids and relational knowledge across a wide range of topics @xcite . however, subsequent work @xcite @xcite has raised concerns about the inherent limitations of text-only pretraining. motivated by these concerns and limitations, we identify and investigate how reporting bias, a concrete and measurable signal, correlates with these limitations and how multimodal training can mitigate these issues. grice's conversational maxim of quantity @xcite asserts that utterances only contain the required amount of information. this leads to explicit reporting of self-evident knowledge being rare, while less common facts, properties, or events are being reported at disproportionately high frequencies. for example, while most people agree that bananas are typically yellow, the bi-gram "green banana" is 332% more frequent in the google books ngram corpus @xcite than "yellow banana". 2 this reporting bias inevitably propagates from corpora to the models trained on them @xcite and affects a variety of concepts. one such concept that we would expect to be harmful in downstream applications, is easy to measure, and is solvable via visual input is color. for these reasons, we investigate the relationship between reporting bias and modern lms' perception of color. people's understanding of color is primarily derived from their experience in the world. every time we interact with an object, we update our understanding of the possible colors that object can take on. further, we can often apply meaning to the differences: a green banana is unripe, a yellow banana is ideal, and a brown banana may be past its prime. text-only lms do not share this embodied experience. similar to an octopus foot_3 they cannot see colors, and need to rely solely on the inaccurate reporting of colors in text. thus, we expect the colors lms associate with objects to differ drastically from a human's perception. to test this hypothesis, we construct the color dataset (coda) -a ground-truth dataset of color distributions for 521 well-known objects via crowdsourcing. we use this dataset to compare the color distributions found in text and those predicted from lms, finding that a lm's shortcomings in recovering color distributions correlates with the reporting bias for those objects. next, we hypothesize that models having access to multiple modalities, specifically vision and text, may be able to partially overcome these shortcoming by grounding the language to their limited visual experiences @xcite . to this end, we develop a unified framework for evaluating the color perception of text-only and multimodal architectures. our results support the hypothesis that multimodal training can mitigate the effect of reporting bias. contributions we make three contributions: 1) we introduce a dataset with human color distributions for 521 well-known objects. 2) we conduct an extensive analysis to identify how reporting bias affects lms' perception of color. 3) we demonstrate that multimodal training mitigates, but not eliminates, the impact of reporting bias. ## limitations while our work identifies issues with text-only training and motivates the use of multimodal signals during pretraining, in this section we outline some limitations of our approach. first, a number of recent papers have highlighted potential limitations of probing lms in certain ways @xcite . while we acknowledge that probing does not provide a full picture of the capabilities of lms, our hypothesis was supported by a range of different results from different approaches. in future work, we hope to leverage research @xcite ) that demonstrates effective methods for automatically producing templates optimized for specific models. in the current state, we cannot and do not state exactly what lms do and do not capture, rather we use our results to uphold and strengthen our original hypothesis that reporting bias hinders performance and that multimodal signals can help mitigate this problem. second, the bi-gram/tri-gram approach we use to quantify reporting bias only approximates the full set of object-color instances. to be more exact, a dependency parser would have to be run on every dataset. finally, although our results motivate the use of multimodal signals during pretraining, there are still challenges to overcome. as discussed by @xcite , the performance of multimodal models on classic nlp tasks often does not reflect the inherent advantages of these architectures, and many multimodal dataset are even more prone to reporting bias in text @xcite @xcite . further, while a visual signal is able to better impart a sense of color, it is not enough to endow models with the meaning behind those colors. humans easily learn that a green banana is not yet ripe, and that a brown banana is past its prime. for models to obtain this level of knowledge and reasoning they will likely require training signals from more modalities, and potentially fully embodied experiences. ## related work color-object relationships preexisting word association datasets often include object-color relationships as either having multiple equally likely pairings @xcite , or as probabilistic cue-target pairs @xcite . others such as @xcite take a norm completion approach, wherein participants are tasked with generating attributes given some concept. one can then extract the objectcolor relationships by counting the number of participants who reported a given color. however, the resulting "distribution" is an aggregate count over individuals, and does not necessarily reflect the distribution from the eyes of a single observer. thus, previous research into lms as knowledge bases has not been able to fully explore the extent to which they know color @xcite . previous work has shown the importance of color in visual perception and object recognition @xcite . more recently @xcite use time resolved neural imaging data to demonstrate how the typicality of object-color relationships influences object representations in visual processing. probing lms a wide range of papers have probed lms in a zero-shot fashion by looking at how they fill in a [mask] token in handcrafted @xcite @xcite @xcite or automatically generated @xcite template sentences. others, such as @xcite compare perplexities between minimal pairs of sentences. a different approach is to analyze the representation quality of lms for linguistic tasks by training a simple mlp on pretrained model representations @xcite . however, zhang and bowman (2018) demonstrate that the procedure of training an additional classifier may distort the results. an alternative approach introduced by @xcite is information-theoretic probing with mdl. this method builds on standard probing classifiers by not only measuring the final performance, but additionally measuring the amount of effort required to achieve that performance. probing multimodal lms often multimodal lms are used in the domain of visual question answering, where, given an image, the model is asked a question about concepts in the image @xcite . while it is often possible to simply use the text-only portion of these models for other tasks, this often leads to poor performance on solely language-based tasks @xcite . ## conclusion in this paper we investigate how reporting bias negatively effects a lm's perception of color. we do so by first creating coda, a dataset of 521 humanperceived color distributions for common objects. we then utilize this dataset to demonstrate that text-only models are inherently limited because of reporting bias. subsequently, we show that multimodal training mitigates these issues. overall, our results support the claims in bender and koller (2020) and @xcite ) that text-only train-ing is insufficient for language understanding and motivate further research on how to best employ multimodal training signals. kelly zhang and samuel bowman. 2018. language modeling teaches you more than translation does: lessons learned through auxiliary syntactic task analysis. in proceedings of the 2018 emnlp workshop blackboxnlp: analyzing and interpreting neural networks for nlp, pages 359-361, brussels, belgium. association for computational linguistics.