Do Language Models Know the Way to R ome? (2021)

Abstract: The global geometry of language models is important for a range of applications, but language model probes tend to evaluate rather local relations, for which ground truths are easily obtained. In this paper we exploit the fact that in geography, ground truths are available beyond local relations. In a series of experiments, we evaluate the extent to which language model representations of city and country names are isomorphic to real-world geography, e.g., if you tell a language model where Paris and Berlin are, does it know the way to Rome? We find that language models generally encode limited geographic information, but with larger models performing the best, suggesting that geographic knowledge can be induced from higher-order co-occurrence statistics.

## introduction language models (lms) are fundamental building blocks in state-of-the-art natural language processing. these models are trained to predict words in context, and as a side product, they learn to compress higher-order co-occurrence statistics to represent the distributional properties of words and phrases. it is well-documented that the representations of modern language models encode some syntactic @xcite and semantic knowledge ( @xcite ), as well as some real-world knowledge @xcite @xcite , i.e., some knowledge base relations can be extracted directly from language models. much of this information was available in older language models such as word2vec, too @xcite . the probes that have been designed for the above studies, however, only probe for one-dimensional information (e.g., part of speech or semantic class) or very local relations (e.g., is daniel kehlmann a writer?). it is an open question whether higherorder co-occurrence statistics found in textual corpora can induce representations show isomorphism to human mental representations of entities and relations. to test this we operationalize a dataset of geographic information. geographic knowledge is a domain in which we can go beyond local relations, since we can access global representations of geographic knowledge (to the extent it is isomorphic to the physical world). cross-lingual language model alignment provides additional motivation for probing for geographic knowledge: we know that language models of different languages are sometimes nearisomorphic @xcite , and it is tempting to think that this is because language models reflect real-world structures, and that the (occasional) near-isomorphism of language models is simply the result of the fact that we use language(s) to talk about the same world. is the isomorphism of language models a result of them being in part isomorphic to the physical world? why would language models potentially encode physical geometry? higher-order co-occurrence statistics can be surprisingly informative. copenhagen and malmö are in different countries, but connected by a bridge, often referred to as the bridge between copenhagen and malmö. other cities belong to the same municipalities and their names therefore co-occur with the name of the municipality. other cities were, for instance, potentially impacted by the same natural disaster or part of the same development project. by distilling thousands, or maybe hundreds of thousands, of such co-occurrences, we conjecture that language models might be able to induce somewhat fine-grained maps of physical geometry. in this work, we study the extent to which geographical information is encoded by language models. based on geographic information from multiple sources, we train probing models on language model representations and evaluate their ability to predict the locations of cities and countries, as well as their ability to predict (one-dimensional) population numbers and (local) neighbor relations, relative to control probe performance levels. contributions we collect geographic information from multiple sources and design experimental protocols to probe language models for such information. we evaluate two bert models @xcite of different sizes, two roberta models @xcite of different sizes, as well as gpt-2 @xcite and word2vec @xcite across three different tasks and various protocols. our results show that modern language models do not encode much more geographic information than word2vec, but that larger (non-autoregressive) language models encode more information than their smaller counterparts, suggesting that such information is nevertheless available through higher-order co-occurrence statistics. ## related work word representations based on distributional statistics have been theorised to capture a wide range of information @xcite . to evaluate this, a considerable body of literature has made use of semantic similarity, relatedness, and analogy datasets @xcite @xcite @xcite @xcite . asking a broader question, @xcite investigated the types of semantic information which are encoded by different classes of word embedding models, finding that taxonomic properties (such as animacy) are well-modelled. in a similar direction, collell talleda and moens (2016) and @xcite draw on semantic norm datasets to test how well these models can encode a range of perceptual and conceptual features. in the context of the neural language models, several recent works such as @xcite and @xcite have attempted to extract factual and commonsense knowledge from them by posing knowledge base triplets as close statements which are used to query the models. most related to this work, @xcite investigate whether lms can learn physical commonsense through language. they find that lm representations do indeed encode information regarding object properties (e.g., bananas are yellow), and affordances (e.g. bananas can be eaten) but they do not to capture the more subtle interplay between the two. studies investigating the geometry of word representations have focused on intrinsic dimensionality and subspaces @xcite , embedding concentration in narrow cones and anisotropy @xcite , and comparisons to the geometry of cognitive measurements or perceptual spaces @xcite . in this work we investigate the degree of isomorphism between language model representations of geographic location names and their real-world counterparts. see vulić et al. (2020 ) for a general discussion of isomorphism across language models, what explains this, and what it depends on. ## methodology we probe language model representations of city and country names. below, we propose three classification/regression probing tasks, as an well as an analysis based on (relational) similarity. ## experimental settings and data language models we compare three different state-of-the-art contextualized lms, namely bert @xcite , roberta @xcite and gpt-2 @xcite . a major difference between gpt-2 and the two other models is that it is trained to be unidirectional (i.e. using next-token language modelling), whereas bert and roberta are trained to be bidirectionnal (i.e. using masked language modelling). for bert and roberta, we used both base (@xmath0, so we can study the influence of the model's size. we probe widely-used pre-trained versions of these models, as they have been trained in their respective original papers. both versions of bert and roberta have been pre-trained on the bookcorpus (800m words) and english wikipedia (2.5b words). gpt-2 is also trained on the book corpus. ## results in when predicting a location's gps coordinates, all models have an error reduction score significantly larger than 0, across both families of regressors and across cities and countries. the word2vec model especially has often the lowest error and a high per, showing no real weakness compared to contextualised models. gpt-2 has the lowest per and the worst error value, across almost all tasks and probes. the two regressors globally lead to the same error values, even though mlp allows a slightly bigger error reduction than lasso. we can also note that, for both bert and roberta, increasing model size (moving from base models to large), one increases the gap between the performance on the task and on the random control, reducing overall error. finally, we can also see that the models are generally not very accurate for predicting gps coordinates, leading to an average error of 2'500-5'000 kilometers. it is relatively accurate continent-wise, but not precise. in sum, we can conclude that it is non-trivial to learn linear maps from language models to physical world geo-coordinates, presumably because representations are non-isomorphic. on the other hand, the improvements from larger models seem to suggest that geographical knowledge is available from higher-order co-occurrence statistics. static model also provide a strong baseline compared to contextualised models. in task 2, the gap between the two regressors is more noticeable compared to task 1: mlp exhibits much higher error reductions between the original task and the random control, and generally allow for lower error numbers. we remark that our lasso model is particularly ineffective for country population prediction, leading only to low per and high errors. on this task and dataset, gpt-2 random control error is high, leading to an outlying per for this lm, even if its probing-task error is still the biggest. the scheme observed in task 1 between base and large version of bert and roberta seems to hold, with only an exception for countries with mlp. this overall indicates that a one-dimensional attribute is easier to learn than a two-dimensional space, giving more expressive models an upper hand. the results of the similarity analysis on city names are displayed in ## discussion tasks 1 to 3 suggest that the amount of geographic knowledge learned by the language models is limited. on the one hand, trained predictors for both gps coordinates and population are of poor quality, making average errors of thousands of kilometers or hundreds of millions of people. on the other hand, the probes are doing significantly better on these tasks than during random control, and information about country borders have been shown by results of task 3 to be embedded in pre-trained language models' representations. this suggests that approximate geographic positions and their neighborhood are learned during training by these models. similarity analysis also indicates that pretrained models have learned to associate representations of cities in the same country. in the end, language models, whether they are static or contextualized, seem to be able to extract limited geographical knowledge from higher-order co-occurrence statistics. the size of the language models, as well as the amount of data they were trained on, seem to be important for how well they encode this information. increasing the model's size leads to best performances and error reduction scores, indicating a better learning of geographic knowledge. overall, context-sensitive language models do not show significantly better performances than word2vec. finally, our experiments demonstrate the importance of probing for global, multi-dimensional relations that require near-isomorphism and are not easily saturated. in the future, it would be of interest to test models finetuned on corpora which contain a wealth of geographic information (e.g. a corpus of atlases) in order to evaluate a) whether representational alignment to real-world geography can be affected by seeing factual expressions such as "italy, france, and switzerland share borders." and b) how exactly representations of entities (e.g. france) alter when the model sees factual statements as the one above.