SlowGuess/NAACL_2021_OCR huggingface dataset:
ing messages are usually full of positive words and phrases and rarely contain expressions carrying strong negative connotations. Product representatives are expected to communicate with customers in a professional manner rather than teasing or using slang and informal words. The scope of norms also include social commonsense about what is expected and "normal" in a given situation (Sap et al., 2019b), similar to scripts in Schank and Abelson (1975).
Social norms are related to the Gricean maxims of manner and quality: in some situations, it is very much expected to say too much and make unsupported claims, for example, when giving a laudatory speech or a eulogy; "Good evening. Martin didn't stand out while he was alive. Now he is dead. Thank you." is not much of a speech.
Applications Social norms are subtle constructs that are not easy to define, so we still do not have many computational techniques to reliably quantify them, let alone assessing whether certain model behaviors should be rewarded or sanctioned (Anastassacos et al., 2020). Consequently, most NLP models still fail to recognize social norms (for an exception, see Forbes et al. (2020)).
Failing to measure social norms, and to detect the alignment between expected or unexpected behaviors and models' actual behaviors, can introduce severe damage and negatively impact society, especially as more conversational agents or chatbots have been developed and deployed for real-world applications, such as customer services, travel or flight reservation, or therapy. In 2016, Microsoft released its now infamous chatbot on Twitter: Tay<sup>3</sup>. Microsoft initially expected Tay's language patterns to resemble a 19-year old American girl, but the chatbot quickly transformed into a fountain of racist, sexist, and abusive slurs, by interacting with people espousing these views. A similar issue played out recently with a Korean chatbot.<sup>4</sup>
Sap et al. (2019a) showed that lack of awareness of social norms around taboo words led to annotation bias being integrated into the models. However, norms are subject to change, as Danescu-Niculescu-Mizil et al. (2013b) have shown, and
<sup>3</sup>https://www.theguardian.com/world/2016/mar/29/microsoft-tay-tweets-antisemitic-racism
4https://www.theguardian.com/world/20 21/jan/14/time-to-properly-socialise-hate e-speech-ai-chatbot-pulled-from-facebook
2Note that the latter two show that human speakers depend on context as well, though.
can affect standing and integration of members.
2.6 Culture and Ideology
Language and culture are intertwined. Language reflects the society, ideology, cultural identity, and customs of communicators, as well as their values. It is therefore intertwined with social norms (Section 2.5). For example, in Japanese (Gao, 2005), the expression of hierarchy necessitates more fine-grained politeness and formality levels than in Western cultures. The terms of address also vary in terms of social and age differences, i.e., inferior members address superior ones with a relationship term instead of using personal names (see also Section 2.3). In many Asian cultures, family terms like "uncle" or "big sister" are used as honorifics. While it is common amongst native speakers of North American English to use "please" in requests even to close friends, such an act would be considered awkward, if not rude, in Arabic-speaking cultures (Kádár and Mills, 2011; Madaan et al., 2020).
Cultural norms can impose a hierarchy on Gricean maxims. For example, whether it is better to give made-up directions (which violates the maxim of relevance) instead of not saying anything (adhering to the maxim of quality) if you do not know the right answer.
Context and social and cultural norms can combine in unexpected ways, such as in the case of Korean Airline co-pilots not correcting pilot mistakes (a social and cultural taboo in ordinary contexts), which resulted in a series of accidents. Differing perceptions of the context, respect for seniority and age, and a hierarchical communication style can lead to one-way communication, in these cases resulting in the deaths of hundreds. The solution here was to change the context by making the working language English, which in turn removed associated social and cultural norms around hierarchical communication (Gladwell, 2008).
Applications Culture and ideology are probably the most complicated language constructs. Despite their substantial influence on communication interpretation and language understanding, most NLP models, like text generation or translation, have not included politeness or other similar subtle cultural signatures. A growing body of research has paid attention to the biases and cul
tural stereotypes encoded and amplified by current NLP models, e.g., inappropriate occupation predictions by large pretrained language models like "the black woman who worked as a babysitter" (Sheng et al., 2019). These findings call for work to look at the ideology, beliefs, and culture behind language content to mitigate biases and social stereotypes beyond data-level manifestations. The fact that embeddings reflect these stereotypes, cultural beliefs, and ideologies make them also an ideal diagnostic tool for social science scholars (Garg et al., 2018; Kozlowski et al., 2018). However, it also creates fundamental biases that cannot easily be mitigated (Gonen and Goldberg, 2019), which poses severe problems for their use in predictive models. Adding cultural awareness can also help counteract the overexposure (Hovy and Spruit, 2016) to the English language (Joshi et al., 2020)<sup>6</sup> and Anglo-Western culture.
2.7 Communicative Goal
Finally, communicative goals cover what people want to achieve with their language use, e.g., information, decision making, social chitchat, negotiation, etc. SFL represents this factor as multiple metafunctions of language. Two metafunctions are of particular relevance here: the interpersonal metafunction, whereby language enables us to enact social relationships, to cooperate, form bonds, negotiate, ask for things, and instruct; and the ideational metafunction, whereby language enables us to talk about inner and outer experiences, people and things, or circumstances in which events occur. Goals introduce an essential layer on top of content, and a good understanding of them can reveal the intent and implication behind the text structure. All of the Gricean maxims are used (or deliberately flaunted) in the service of achieving these goals. For example, when trying to convince someone to join us in a project, we might adhere to the maxims of relevance and concisely lay out the reasons we need them to join. However, to make it more likely that they agree, we might choose to exaggerate the expected payoff and to leave out some of the difficulties involved, which violates the maxims of quality and quantity, respectively.
Applications Communicative goals shape how speakers arrange their words and styles. For in
$^{6}$ https://thegradient.pub/the-benderrule-on-naming-the-languages-we-study-and-why-it-matters/
<sup>5</sup>https://www.cnbc.com/id/100869966
stance, text that aims to convince others often uses various persuasion strategies (Yang et al., 2019a; Chen and Yang, 2021), argumentation techniques (Stab and Gurevych, 2014), rhetorical structures (Rapp, 2011), and the exchange of social support (Wang and Jurgens, 2018; Yang et al., 2019b). Messages trying to entertain audiences need to be structured in ways that can trigger humor (Yang et al., 2015). People might use informal language or text with a high level of intimacy to indicate close relations (Pei and Jurgens, 2020) or reduce social distance between speakers and receivers (Bernstein, 1960; Keshavarz, 2001).
Therefore, it is essential for NLP systems like text generation models to be aware of communicative goals in order to arrange word choice, and styles to form a grammatically responsible and coherent text. Ongoing research has shown that style can be controlled independently of content(Prabhumoye et al., 2018; John et al., 2019). Some of the early work on NLP (Hovy, 1987) explicitly considered communicative goals in sentence generation, albeit modeled explicitly. More recently, Sap et al. (2020) modeled speaker intent to resolve conversational implicature.
3 Outlook and Challenges
Social Factors in Different NLP Tasks When and how, though, should we consider these various social factors for an NLP application? NLP practitioners should feel free to use our social factor taxonomy as a guide to examine what social factors should be used, and whether integrating each confers additional benefits (e.g., better design, performance, user experience, or cultural fit) for their use cases. Different NLP tasks will likely benefit differently from our social factor taxonomy.
There is some evidence that the earlier factors (such as speaker and receiver characteristics) can be applied to most tasks, as they are fundamental aspects of language. Social relations and context are likely to apply more to dialogue and text generation tasks than to, say, sentiment analysis. Lastly, "high-level" factors such as social norms and culture and ideology likely require more research to inform individual applications, but are likely to shape our community approaches. We would be well-advised to incorporate the findings of fields that have studied these issues for longer, such as philosophy, sociology, or sociolinguistics. As NLP tasks and algorithms are being now applied to different aspects of everyday interaction
and around the world, how we will equip NLP models with a grounding in social factors becomes extremely important, especially these two dimensions. Detailed modeling of these social factors is essential if NLP systems are to have any impact. It can also help avoid hegemonic approaches from assuming all conversations follow Western norms, culture, and ideology.
Real-world interaction involves more than the exchange of information or decision making via language; it involves a wide range of aspects related to social factors and interpersonal relations, reflected in rich modalities such as voice or facial expression. Though this work's focus is on the language side, we argue that the introduced taxonomy can be beneficial in broader scenarios for next-level multi-modal models.
Data, Ethics, and Privacy Our work here is related to some of the recent work on bias in NLP (Hovy and Spruit, 2016; Shah et al., 2020). On the one hand, the cooperative principle can be seen as a possible positive bias: a pre-existing expectation of how we interact, the violation of which signals an alternative approach. So far, models do not integrate this positive bias. On the other hand, work on speaker and receiver characteristics is affected by the models' predictive biases: exaggerating or overestimating one particular group's attributes can skew the results, for example, in the case of machine-translated texts sounding older and more male (Hovy et al., 2020). Recently, Blodgett et al. (2020) have discussed the role of "bias" conceptions, which serves as a meta-discussion of the conceptualization of social norms.
Integrating social factors into NLP poses a double challenge: on the one hand, it requires additional data to model those social factors. We need representative annotation samples for, e.g., the demographics and network information of speaker, receiver, and social relations, which requires us to collect and document our annotations (Bender and Friedman, 2018). Social media already contains some information from personal or socially grounded conversations, but other domains might suffer from data sparsity for these factors, and require advances in unsupervised learning or few-shot learning techniques.
On the other hand, collecting all this information raises questions about privacy, data protection, and ethics. Some data we need to collect to work with social factors might be personal or
protected data, which comes with risks for de-annotation and privacy leaks. Collecting sensitive data (i.e., membership in a protected category) requires the participants' approval and rigorous procedures to ensure that this information cannot be connected to them individually. These considerations also pose a challenge to data sharing; even if properly anonymized, data can contain clues as to participants' identity (Eckert and Dewes, 2017). We need to strengthen ethical considerations for this emerging direction to guide practice in the field and ensure our models are used in beneficial ways.
Evaluation and Metrics A central question in these efforts is How do we evaluate whether NLP models have learned the social factors of language, beyond performance improvements? Current models optimize performance metrics, but these metrics might fail to capture the nuances of NLP systems' understanding when considering social content. Thus, better metrics are needed to measure and visualize such additional benefits introduced by modeling language's social factors. These metrics will become essential to diagnose failure. Failed or improper incorporation of social factors could lead to awkward social consequences. E.g., a system misjudging its social relation to the speaker and being a bit too "chummy", or a conversational agent disrespecting social norms of turn taking and formality. To some extent, such problems might be unavoidable: interacting through language is always a trial-and-error process, even for humans. However, such "errors" become extremely important in high-stakes scenarios, such as inappropriate responses from conversational agents in mental health counseling applications. We need metrics to capture this failure and mechanisms to explain the decision-making process behind socially aware NLP models.
Multi-modal Social Interaction Real-world interaction involves more than the exchange of information or decision making via language; it involves a wide range of aspects related to social factors and interpersonal relations, reflected in rich modalities (Simmons et al., 2011) such as images, voice or facial expression. Though this work's focus is on the language side, we argue that the introduced taxonomy can be beneficial in broader scenarios for the next level multi-modal models.
In this work, we have argued that there are seven social factors of language that impact NLP applications: speaker, receiver characteristics, social relations, context, social norms, culture and ideology, and communicative goals. At present, NLP models often ignore these factors. We have shown that this ignorance limits the kinds of applications we can tackle. It can also can introduce mistakes, ranging from the hilarious to the severe. However, several extant approaches incorporate these social factors, all of them showing substantial improvements in a wide range of applications. By systematically addressing the social aspects of language as a field, we will improve the performances of existing NLP systems, open up new applications, and increase fairness and usability for all users.
This project has partially received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation program (No. 949944, INTEGRATOR). DY is supported in part by grants from Google and Salesforce. DH is the scientific director of the Data and Marketing Insights Unit at the Bocconi Institute for Data Science and Analysis. We would like to thank Maxwell Forbes, Christoph Purschke, and Maarten Sap for comments on the drafts, as well as the anonymous reviewers who suggested valuable additions.
Gavin Abercrombie and Dirk Hovy. 2016. Putting sarcasm detection into context: The effects of class imbalance and manual labelling on supervised machine classification of Twitter conversations. In Proceedings of the ACL 2016 Student Research Workshop, pages 107-113, Berlin, Germany. Association for Computational Linguistics.
Nicolas Anastassacos, Stephen Hailes, and Mirco Musolesi. 2020. Partner selection for the emergence of cooperation in multi-agent systems using reinforcement learning. In AAAI, pages 7047-7054.
Rafael E Banchs and Haizhou Li. 2012. Iris: a chat-oriented dialogue system based on the vector space model. In Proceedings of the ACL 2012 System Demonstrations, pages 37-42. Association for Computational Linguistics.
Justin Basilico and Thomas Hofmann. 2004. Unifying collaborative and content-based filtering. In Proceedings of the twenty-first international conference on Machine learning, page 9.
Emily M Bender and Batya Friedman. 2018. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science. Transactions of the Association for Computational Linguistics, 6:587-604.
Emily M. Bender and Alexander Koller. 2020. Climbing towards NLU: On meaning, form, and understanding in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185-5198, Online. Association for Computational Linguistics.
Adrian Benton, Margaret Mitchell, and Dirk Hovy. 2017. Multitask learning for mental health conditions with limited social media data. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 152-162, Valencia, Spain. Association for Computational Linguistics.
Basil Bernstein. 1960. Language and social class. The British journal of sociology, 11(3):271-276.
Yonatan Bisk, Ari Holtzman, Jesse Thomason, Jacob Andreas, Yoshua Bengio, Joyce Chai, Mirella Lapata, Angeliki Lazaridou, Jonathan May, Aleksandr Nisnevich, Nicolas Pinto, and Joseph Turian. 2020. Experience grounds language. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8718-8735, Online. Association for Computational Linguistics.
Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. Language (technology) is power: A critical survey of "bias" in NLP. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5454-5476, Online. Association for Computational Linguistics.
Su Lin Blodgett, Lisa Green, and Brendan O'Connor. 2016. Demographic dialectal variation in social media: A case study of African-American English. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119-1130, Austin, Texas. Association for Computational Linguistics.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165.
Ozan Caglayan, Pranava Swaroop Madhyastha, Lucia Specia, and Loic Barrault. 2019. Probing the need for visual context in multimodal machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4159-4170.
Amanda Cercas Curry and Verena Rieser. 2018. #MeToo Alexa: How conversational systems respond to sexual harassment. In Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing, pages 7-14, New Orleans, Louisiana, USA. Association for Computational Linguistics.
Amanda Cercas Curry, Judy Robertson, and Verena Rieser. 2020. Conversational assistants and gender stereotypes: Public perceptions and desiderata for voice personas. In Proceedings of the Second Workshop on Gender Bias in Natural Language Processing, pages 72-78, Barcelona, Spain (Online). Association for Computational Linguistics.
Eshwar Chandrasekharan, Mattia Samory, Shagun Jhaver, Hunter Charvat, Amy Bruckman, Cliff Lampe, Jacob Eisenstein, and Eric Gilbert. 2018. The internet's hidden rules: An empirical study of reddit norm violations at micro, meso, and macro scales. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW):1-25.
Jiaao Chen and Diyi Yang. 2021. Weakly-supervised hierarchical models for predicting persuasive strategies in good-faith textual requests. AAAI.
Herbert H. Clark and Michael F. Schober. 1992. Asking questions and influencing answers. Questions about Questions: Inquiries into the Cognitive Bases of Surveys, pages 15-48.
Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of machine learning research, 12(ArtICLE):2493-2537.
Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013a. A computational approach to politeness with application to social factors. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 250-259, Sofia, Bulgaria. Association for Computational Linguistics.
Cristian Danescu-Niculescu-Mizil, Robert West, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013b. No country for old members: User lifecycle and linguistic change in online communities. In Proceedings of the 22nd international conference on World Wide Web, pages 307-318.
Marco Del Tredici, Diego Marcheggiani, Sabine Schulte im Walde, and Raquel Fernandez. 2019. You shall know a user by the company it keeps: Dynamic representations for social media users in nlp. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4701-4711.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Emily Dinan, Angela Fan, Ledell Wu, Jason Weston, Douwe Kiela, and Adina Williams. 2020. Multidimensional gender bias classification. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 314-331, Online. Association for Computational Linguistics.
Rachel Dorn. 2019. Dialect-specific models for automatic speech recognition of African American Vernacular English. In Proceedings of the Student Research Workshop Associated with RANLP 2019, pages 16-20, Varna, Bulgaria. INCOMA Ltd.
Penelope Eckert. 2012. Three waves of variation study: The emergence of meaning in the study of sociolinguistic variation. Annual review of Anthropology, 41:87-100.
Svea Eckert and Andreas Dewes. 2017. Dark data. Presentation at DEFCON, 25.
Marcello Federico. 1996. Bayesian estimation methods for n-gram language model adaptation. In Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP'96, volume 1, pages 240-243. IEEE.
Ernst Fehr and Urs Fischbacher. 2004. Social norms and human cooperation. Trends in cognitive sciences, 8(4):185-190.
Charles J Fillmore. 1982. Frame semantics. In Linguistics in the morning calm, pages 111-137.
Lucie Flek. 2020. Returning the N to NLP: Towards contextually personalized classification models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7828-7838, Online. Association for Computational Linguistics.
Maxwell Forbes, Jena D. Hwang, Vered Shwartz, Maarten Sap, and Yejin Choi. 2020. Social chemistry 101: Learning to reason about social and moral norms. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 653-670, Online. Association for Computational Linguistics.
Tommaso Fornaciari and Dirk Hovy. 2019. Dense node representation for geolocation. In Proceedings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019), pages 224-230, Hong Kong, China. Association for Computational Linguistics.