What do Toothbrushes do in the Kitchen? How Transformers Think our World is Structured (2022)

Abstract: Transformer-based models are now predominant in NLP.They outperform approaches based on static models in many respects. This success has in turn prompted research that reveals a number of biases in the language models generated by transformers. In this paper we utilize this research on biases to investigate to what extent transformer-based language models allow for extracting knowledge about object relations (X occurs in Y; X consists of Z; action A involves using X).To this end, we compare contextualized models with their static counterparts. We make this comparison dependent on the application of a number of similarity measures and classifiers. Our results are threefold:Firstly, we show that the models combined with the different similarity measures differ greatly in terms of the amount of knowledge they allow for extracting. Secondly, our results suggest that similarity measures perform much worse than classifier-based approaches. Thirdly, we show that, surprisingly, static models perform almost as well as contextualized models – in some cases even better.

## introduction few models have recently influenced nlp as much as transformers @xcite . hardly any new nlp system today is introduced without a transformer-based model such as bert @xcite or gpt @xcite . as a result, static models such as word2vec @xcite are increasingly being substituted. nevertheless, transformers are still far from being fully understood. thus, research studies are being conducted to find out how they work and what properties the language models they generate have. during training, transformers seem to capture both syntactic and semantic features @xcite . for example, dependency trees can be reconstructed from trained attention heads @xcite , syntactic trees can be reconstructed from word encodings @xcite , and these encodings can be clustered into representations of word senses @xcite . bert also seems to encode information about entity types and semantic roles @xcite . for an overview of this research see @xcite . since bert and other transformers are trained on various data crawled from the internet, they are sensitive to biases @xcite @xcite . in practice, instead of reproducing negative biases, they are expected to allow for the derivation of statements, such as that toothbrushes are spatially associated with bathrooms rather than living rooms. in this line of thinking, approaches such as the popularization of knowledge graphs can be located @xcite @xcite . our paper is situated in this context. more specifically, we examine the extent to which knowledge about spatial objects and their relations is implicitly encoded in these models. since the underlying texts are rather implicit regarding such information, it can be assumed that the object relations derivable from transformers are weakly encoded (cf. @xcite . reading, for example, the sentence: "after getting up, i ate an apple" one may assume that the narrator got up from his bed in the bedroom, went to the kitchen, took an apple, washed it in the sink, and finally ate it. the apple is also likely to have been peeled and cut. equally, however, nothing is said in the sentence about a bedroom or a kitchen. nevertheless, it is a well known approach to explore the usage regularities of words, currently most efficiently represented by neural networks, as a source for knowledge extraction (see, e.g. @xcite @xcite . in this work, we use a number of methods to identify biases in contextualized models and ask to what extent they can be used to extract object-based knowledge from these models. to this end, we consider three relations: it is assumed that (static or contextualized) models implicitly represent such relations, so that it is possible to identify probable targets starting from certain sources. that is, for a word like fridge (source), we expect it to be semantically more strongly associated with kitchen (target) than with words naming other rooms, since fridges are more likely to be found in kitchens than in other rooms, and that certain word representation models reflect this association. we also assume that this association is asymmetric and exists to a lesser extent from target to source (cf. @xcite ). the paper is organized as follows: related work is reported in sec. 2. the datasets we use are represented in sec. 3 and our method in sec. 4. our experiments are presented in sec. 5 and discussed in sec. 6. sec. 7 provides a conclusion. all used data, scripts and results are open source on github foot_0 . ## related work biases in nlp models are not a new problem that appeared with bert, but affect almost all models trained on language datasets @xcite . as such, there are methods for measuring social biases in static models such as word2vec @xcite . one of the best known approaches is weat @xcite . here, two groups of concepts are compared with two groups of attributes based on the difference between the sums of their cosine similarities (see section 4). this approach already points to a methodological premise that also guides our work: relations of entities are tentatively determined by similarity analyses of vectorial word representations. however, a direct comparison of word vectors is not possible with contextualized methods such as bert, where the vector representation of a word varies with the context of its occurrence (cf. @xcite . efforts to transfer the cosine-based approach from static to contextualized models have not been able to recreate similar performances @xcite . therefore, new approaches have been developed based on the specifics of contextualized models. for example, bert is trained using masked language modeling, where the model estimates the probability of masked words in sentences @xcite . the probability distribution for a masked word in a given context can then be used as information to characterize candidate words @xcite . sec. 4.3 describes this approach in more detail. an alternative approach is to examine the interpretability of models @xcite @xcite @xcite , which goes beyond the scope of this paper. in any event, both approaches share the same basic ideas, e.g., the probability prediction of mask tokens (cf. @xcite . work has also been done on how bert represents information about spatial objects. for example, bert has problems with certain object properties (e.g. cheap or cute) or implicit visual properties that are rarely expressed @xcite . problems are also encountered with extracting numerical commonsense knowledge, such as the typical number of tires on a car or the feet on a bird @xcite . more than that, the models seem to allow for extracting some object knowledge, but not with respect to properties based on their affordance (e.g. objects through which one can see are transparent @xcite ). even though these results seem to question the use of bert and its competitors for knowledge extraction, these models still perform better in downstream tasks than their static competitors @xcite @xcite . @xcite compared these models using different datasets and lexical relations. these include relations similar to those examined here (e.g. a pot is usually found in a kitchen), but beyond the level of detail achieved in our study. what will become increasingly important is the so-called grounding of language models @xcite : here, the models are trained not only on increasingly large text data, but also, for example, on images thus enabling better "understanding" of spatial relations @xcite . in this paper, we focus on models without grounding. ## approach we now present the static and contextualized models used in our study. ## experiments using the apparatus of section 4, we now evaluate the classes of word representation models (static, mlms and clms) in conjunction with the similarity measures and classifiers. the results for the static models are shown in ## discussion as good as the results obtained using classifiers are, they must be viewed with caution. one can attribute their success to the fine-tuning of numerous parameters (and ultimately to overfitting); however, one can also attribute this success to nonlinear structuring of the information encoded in language models. in other words, these models appear to encode object knowledge, but require a sophisticated apparatus to retrieve it. thus, they should not be considered as an alternative to unsupervised approaches. another issue is that our experiments do not yet allow for a comparison of model architectures, as the models studied differ significantly in terms of the size of their parameter spaces and training data. our experiments do suggest that certain smaller models come close to or even outperform the results of larger models. however, a comparison of model architectures would require controlling for these parameters. nevertheless, the results we have obtained are, in part, promising enough to encourage such research. finally, our experiments show that static models can perform better than contextualized models to some extent. this finding is conditioned by our experiments and their context of application. these observations that older models perform better on certain tasks are consistent with other work (e.g. lstms on small datasets for intent classification (ezen-can, 2020) or definiteness prediction @xcite . at this point, a much broader analysis is needed (considering more areas and object relations), which also exploits contextual knowledge represented in contextualized models more than has been done here and in related work. nevertheless, it is generally difficult to obtain data for such a broader analysis, and our experiments are already broader in scope and consider finer relationships than similar approaches. ## conclusion we evaluated static and contextualized models as potential resources for object-related knowledge extraction. to this end, we examined three datasets (to identify typical artifacts in rooms, objects of actions, or parts of objects). we also experimented with different similarity measures and classifiers to extract the information contained in the language models. in doing so, we have shown that the models in combination with the measures differ greatly in terms of the amount of knowledge they allow for extracting. there is a weak trend that bert-base is the best performer among contextualized models, and glove and fasttext among static models. secondly, our results suggest that approaches based on classifiers perform significantly better than similarity measures. thirdly, we have shown that static models perform almost as well as contextualized models -in some cases even better. this result shows that research on these models needs to be advanced. in future work we will also investigate how grounded language models perform on such datasets. however, as noted above, this requires a significant expansion of bias research, such as that conducted here to enable knowledge extraction.