On learning and representing social meaning in NLP : a sociolinguistic perspective (2021)

Abstract: The field of NLP has made substantial progress in building meaning representations. However, an important aspect of linguistic meaning, social meaning, has been largely overlooked. We introduce the concept of social meaning to NLP and discuss how insights from sociolinguistics can inform work on representation learning in NLP. We also identify key challenges for this new line of research.

## introduction variation is inherent to language. any variety of language provides its users with a multitude of linguistic forms-e.g., speech sounds, words, grammatical constructions-to express the same referential meaning. consider, for example, the many ways of pronouncing a given word or the variety of words that can refer to a given concept. linguistic variation is the primary object of inquiry of sociolinguistics, which has a long history of describing and explaining variation in linguistic form across society and across levels of linguistic analysis @xcite . perhaps the most basic finding of the field is that linguistic variation allows for the expression of social meaning, information about the social background and identity of the language user. such sociolinguistic variation adds an additional layer of meaning onto the basic referential meaning communicated by any utterance or text. understanding the expression of social meaning based on linguistic variation is a crucial part of the linguistic knowledge of any language user, drawn upon continuously during both the production and processing of natural language. the relationship between variation and social meaning, however, has only begun to be explored computationally (e.g., @xcite ). studies have shown, for example, that words, capitalisation, or the language variety used can index political identity @xcite @xcite . despite general acceptance of the link between linguistic variation and social meaning in linguistics, nlp has largely ignored this relationship. nevertheless, the importance of linguistic variation more generally is increasingly being acknowledged in nlp @xcite . nlp tools are usually developed for standard varieties of language, and therefore tend to under-perform on texts written in varieties that diverge from the 'standard', including language identification @xcite , dependency parsing @xcite , and pos tagging @xcite . one approach to overcoming the challenges posed by linguistic variation is text normalisation @xcite . normalisation transforms non-standard texts into a more standardised form, which can then be analysed more accurately using nlp models trained on standard language data. text normalisation, however, removes rich social signals encoded via sociolinguistic variation. other approaches have also been explored to improve the robustness of nlp models across society, such as adapting them based on demographic factors @xcite or social network information @xcite . linguistic variation, however, should not simply be seen as a problem to be overcome in nlp. although variation poses a challenge for robust nlp, it also offers us a link to the social meaning being conveyed by any text. to build nlp models that are capable of understanding and generating natural language in the real world, sociolinguistic variation and its role in creating social meaning must be incorporated into our models. for example, over the last few years, research in nlp has been marked by substantial advancements in the area of representation learning, but although @xcite and @xcite have recently argued that the social nature of language must be considered in representation learning, the concept of social meaning is still largely overlooked. in this paper, we therefore introduce the concept of social meaning to nlp from a sociolinguistic perspective (section 2). we reflect on work on representation learning in nlp and how social meaning could play a role there (section 3), and we present example applications (section 4). finally, we identify key challenges moving forward for this important and new line of research in nlp for the robust processing of meaning (section 5). ## social meaning people use language to communicate a message. the same message can be packaged in various linguistic forms. for example, someone might say 'i'm not coming, pal'. but they could also refer to their friend as 'mate', 'buddy', 'bruv', or 'bro' for instance. or they could say 'i am not coming' or 'i ain't comin' to express that they are not joining that friend. with each of these options, or variants, the language user communicates the same referential meaning, that is, they refer to the exact same entity, action or idea in the real or an imagined world. the only difference is the linguistic form used to encode that message. to put it simply: these are different ways of saying the same thing @xcite . although this variation in form does not change the referential meaning of the message, it is not meaningless in itself. variation in form can also carry information about the social identity of a language user @xcite , which sociolinguists call the social meaning of linguistic variation. for example, @xcite define social meaning as all social attributes associated with a language feature and its users. these social attributes can be highly diverse and relate to any aspect of identity a language user may want to express through their linguistic output. linguistic variation can express broad social attributes like national background or social class. saying 'i left the diapers in the trunk' rather than 'i left the nappies in the boot', may suggest that the speaker is american rather than british. but linguistic variation can also be far more fine-grained and can be called upon directly by language users to construct local identities. a famous example is @xcite 's groundbreaking study on martha's vineyard, a small island off the northeast coast of the us. labov found that within the small island community there were differences in the way people pronounced the diphthongs /ay/ (as in 'right') and /aw/ (as in 'house'). the study shows that a more centralised pronunciation of the diphthongs was used by local fishermen who opposed the rise in tourism from the mainland on the island. conversely, islanders who were more oriented towards mainland culture used a more standard american pronunciation for these diphthongs. the pronunciation of these sounds was thus used in this particular community to express the local social meaning of island identity. the social meaning of linguistic variation is not fixed. over time, a linguistic variant can develop new meanings and lose others, while new forms can also emerge. a single linguistic feature can also be associated with multiple social meanings. which of those meanings is activated in interaction depends on the specific context in which that interaction takes place. campbell-kibler's research on the social meaning of the pronunciation of -ing in the us (e.g., 'coming' vs. 'comin') shows, for instance, that the variation can be linked to both social and regional identity. for example, velar pronunciation 'coming' sounds urban, while alveolar pronunciation 'comin' is perceived as sounding southern @xcite @xcite . information about the speaker can also influence the social meaning attached to variation in -ing pronunciation. experiments show that when a speaker is presented as a professor, they sound knowledgeable when using the velar pronunciation, while if the same speaker is presented as an experienced professional, they are perceived as knowledgeable when using the alveolar variant @xcite . the collection of social meanings a linguistic feature could potentially evoke is referred to as the indexical field of that feature @xcite , for a theoretical discussion of indexicality, see @xcite ). as the above examples suggest, social meaning can be attached to various types of linguistic features. in the friend, nappy and boot examples, there is variation on the level of the lexicon, while the 'i ain't comin' example shows morphosyntactic variation ('ain't' vs. 'am not') and variation in pronunciation @xcite . a language or language variety as a whole can also carry social meaning. think of the choice to use a standard variety or a local dialect to signal one's regional background or the use of loans from foreign languages to come across as cosmopolitan or fashionable @xcite . it is also important to acknowledge that there are other types of linguistic variation-and other traditions that analyse variation within linguisticsincluding variation across communicative contexts, as is commonly analysed in corpus linguistics (i.e. register variation) @xcite . for example, research on register variation has shown that texts that are intended to concisely convey detailed information, like academic writing, tend to have very complex noun phrases, as opposed to more interactive forms of communication, like face-to-face conversations, which tend to rely more on pronouns. crucially, register variation depends on the communicative goals, affordances, and constraints associated with the context in which language is used, as opposed to the social background or identity of the individual language users, although the relationship between social and situational variation is also complicated and not fully understood @xcite . linguistic variation and the expression of social meaning is thus a highly complex phenomenon, and one that sociolinguists are only beginning to fully grasp despite decades of research. nevertheless, we argue that language variation and social meaning must be considered when building nlp models: not simply to create more robust tools, but to better process the rich meanings of texts in general. moreover, we believe that methods from nlp could contribute substantially to our understanding of sociolinguistic variation. ## representing social meaning distributed representations map a word (or some other linguistic form) to a k-dimensional vector, also called an embedding. sometimes these representations are independent of linguistic context @xcite , but increasingly they are contextualised @xcite . these representations are shown to capture a range of linguistic phenomena @xcite @xcite . a key question in the development of representations is what aspects of meaning these representations should capture. indeed, recent reflections have drawn attention to challenges such as polysemy and hyponymy @xcite and construed meaning @xcite . however, even though @xcite ) note that '[l]inguistic meaning includes so-figure 1 : with all three utterances, the author asks how someone is doing, but the spelling variants carry different social meanings. for example, how should a spelling variant like doin be represented? providing it the same representation as doing would result in a loss of social meaning associated with g-dropping. ## applications because the expression of social meaning is a fundamental part of language use, it should be taken into consideration throughout model development, but it is especially relevant for computational sociolinguistics @xcite and computational social science @xcite . examples where social meaning is especially important are: ## conclusion despite the large body of work on meaning representations in nlp, social meaning has been overlooked in the development of representations. fully learning and representing the rich social meanings of linguistic variation will likely not be realised for years to come. yet even small steps in this direction will already benefit a wide array of nlp applications and support new directions in social science research. with this paper, we hope to encourage researchers to work on this challenging but important aspect of linguistic meaning.