S ocio P robe: What, When, and Where Language Models Learn about Sociodemographics (2022)

Abstract: Pre-trained language models (PLMs) have outperformed other NLP models on a wide range of tasks. Opting for a more thorough understanding of their capabilities and inner workings, researchers have established the extend to which they capture lower-level knowledge like grammaticality, and mid-level semantic knowledge like factual understanding. However, there is still little understanding of their knowledge of higher-level aspects of language. In particular, despite the importance of sociodemographic aspects in shaping our language, the questions of whether, where, and how PLMs encode these aspects, e.g., gender or age, is still unexplored. We address this research gap by probing the sociodemographic knowledge of different single-GPU PLMs on multiple English data sets via traditional classifier probing and information-theoretic minimum description length probing. Our results show that PLMs do encode these sociodemographics, and that this knowledge is sometimes spread across the layers of some of the tested PLMs. We further conduct a multilingual analysis and investigate the effect of supplementary training to further explore to what extent, where, and with what amount of pre-training data the knowledge is encoded. Our overall results indicate that sociodemographic knowledge is still a major challenge for NLP. PLMs require large amounts of pre-training data to acquire the knowledge and models that excel in general language understanding do not seem to own more knowledge about these aspects.

## introduction when talking to somebody, we consciously choose how to represent ourselves, and we have a mental model of who our conversational partner is. at the same time, our language is littered with subconscious clues about our sociodemographic background that we cannot control (e.g., our age, education, regional origin, social class, etc). people use this information as an integral part of language, to better reach their audience, and to understand what they are saying (e.g., @xcite . in other words, we use sociodemographic knowledge to decide what to say (are we talking to a child or an adult, do i want to sound smart or relatable?) but do pre-trained language models (plms) have knowledge about sociodemographics? over the last years, plms like bert @xcite and roberta @xcite have achieved superior performance on a wide range of downstream tasks (e.g., @xcite . accordingly, they have become the de facto standard for most nlp tasks. consequently, many researchers have tried to shed light on plms' inner workings (cf. "bertology"; @xcite . they have systematically probed the models' capabilities to unveil which language aspects their internal representations capture. in particular, researchers have probed lower-level structural knowledge (e.g., @xcite @xcite , as well as mid-level knowledge, e.g., lexico-semantic knowledge (e.g., @xcite , and plms' factual understanding (e.g., @xcite . while these aspects are relatively well explored, we still know little about higher-level knowledge of plms: only a few works have attempted to quantify common sense knowledge in the models @xcite . probing of other higher-level aspects still remains underexploredhindering targeted progress in advancing humanlike natural language understanding. as recently pointed out by @xcite , sociodemographic aspects play a central role in language. however, they remain underexplored in nlp, despite promising initial findings (e.g., @xcite @xcite . importantly, we are not aware of any research assessing sociodemographic knowledge in plms. this lack is extremely surprising given the availability of resources, and the importance of these factors in truly understanding language. contributions. acknowledging the importance of sociodemographic factors in language, we address a research gap by proposing socioprobe, a novel perspective of probing plms for sociodemographic aspects. we demonstrate our approach along two dimensions, (binary) gender and age, using two established data sets, and with different widely-used easily-downloadable plms that can be run on a single gpu. to ensure validity of our findings, we combine "traditional" classifier probing @xcite and information-theoretic minimum distance length (mdl) probing @xcite . our experiments allow us to answer a series of research questions. we find that plms do represent sociodemographic knowledge, but that it is acquired in the later stages. this knowledge is also decoupled from overall performance: some models that excel in general language understanding do still not have more knowledge about sociodemographics encoded. we hope that this work inspires more research on the social aspects of nlp. our research code is publicly available at https://github.com/ milanlproc/socio-probe . ## research questions we pose five research questions (rqs): ## socioprobe we describe socioprobe, which we employ to explore the sociodemographic knowledge plms contain. guided by the availability of data sets, we focus on the dimensions of gender and age. note, however, that our overall methodology can be easily extended to other sociodemographic aspects. ## experiments we describe our experiments. ## conclusion sociodemographic aspects shape our language and are thus important factors to model in language technology. however, despite a plethora of works probing plms for various types of knowledge, we know little about these higher-level aspects of language. we present socioprobe to understand whether, when, and where plms encode sociodemographic knowledge in their representations. we find that sociodemograophic knowledge is located in plms, but much more diffuse than lower-level aspects. in the future, we will extend our analysis to languages other than english. we hope that our findings will fuel further research towards humanlike language understanding.