NORMSAGE : Multi-Lingual Multi-Cultural Norm Discovery from Conversations On-the-Fly (2023)

Abstract: Knowledge of norms is needed to understand and reason about acceptable behavior in human communication and interactions across sociocultural scenarios. Most computational research on norms has focused on a single culture, and manually built datasets, from non-conversational settings. We address these limitations by proposing a new framework, NormSage, to automatically extract culture-specific norms from multi-lingual conversations. NormSage uses GPT-3 prompting to 1) extract candidate norms directly from conversations and 2) provide explainable self-verification to ensure correctness and relevance. Comprehensive empirical results show the promise of our approach to extract high-quality culture-aware norms from multi-lingual conversations (English and Chinese), across several quality metrics. Further, our relevance verification can be extended to assess the adherence and violation of any norm with respect to a conversation on-the-fly, along with textual explanation. NormSage achieves an AUC of 94.6% in this grounding setup, with generated explanations matching human-written quality.

## introduction norms are rules that embody the shared standards of behaviors amongst cultural groups and societies @xcite . these include social conventions (e.g., it's good to shake hand with your opponent even if you lost); behavior guidances (e.g., it's wrong to hurt a pet); and general concepts (e.g., it's nice to be smart) @xcite . knowledge of norms in general, and sociocultural norms in particular, is essential if we are to equip ai systems with capability to understand and reason about acceptable behavior in communication and interaction across cultures. yet, research on computational modeling of social norms is still in its early stages. social-chem-101 @xcite and the moral integrity corpus @xcite introduce large crowdsourced datasets of social norms and moral judgements conceptualized as free-text rules-of-thumb (rot) grounded in given situations. while this has led to significant advances in computational modeling of social norms, there are two important limitations. first, while models trained on these grounded rots can adapt to new situations, they often struggle to uncover new norms and to model richer context. second, the norms covered in these studies are primarily redditbased and us-centric @xcite . in this paper, we propose to address current shortcomings with the following contributions: norms manifest themselves in day-to-day conversations, either by being implicitly adhered to or violated. for example, we observe in fig 1 input a conversation between a taiwanese-american parent and son about "not wasting food". we define a new task aimed at extracting norms from conversation context, beyond single statements and question-answer pairs from recollected summaries. to support this task, we also introduce a collection of conversations in english and chinese spanning multiple genres (e.g., tv shows, videos, text messages), topics, and cultures. the culture-specific norms are also promising in quality, with humans being able to identify the culture from these norms at 80% accuracy. ## evaluation and results we organize this section as follows. in sec 4.1, we detail experiments on norm discovery from conversations, including multi-lingual and cross-culture scenarios. then, in sec 4.2 and sec 4.3, we analyze the effectiveness of self-verification mechanisms for norm discovery correctness, and relevance grounding between the norm and conversation, respectively. finally, in sec 4.4, we discuss the cost and resource contributions. ## related work the domain of norms is closely related to behavioral psychology and moral judgement. early studies investigated the pragmatic cooperative principles @xcite , politeness implicatures @xcite , and relationship between norms and law @xcite governing human behavior. as judgements of behavior are communicated through linguistics, @xcite introduced a lexicon of evocative words based on moral foundation theory, which later attempts utilize for predicting the moral value from text messages @xcite . recent approaches explore modeling moral and ethical judgement of real-life anecdotes from reddit @xcite @xcite , with delphi @xcite unifying the moral judgement prediction on these related benchmarks. related is another line of work modeling legal judgement on judicial corpora @xcite . norm discovery is a unique, emerging task, which aims to catalogue the underlying principles behind behavioral judgements, and can be seen as similar to distilling reactions, explanations, and implications from situations @xcite @xcite . separate from explicit information extraction @xcite , such as the extraction of events/entities/relations which may tend to be overly granular and situation-specific, norm discovery involves inferring the underlying stand-alone abstracted social or cultural rules. towards this end, @xcite ; @xcite are the main existing norm discovery approaches. each presents a large-scale catalogue of manually curated rule-of-thumbs from reddit post headers, and trains a language model to generate rule-of-thumbs based on this data. in contrast, our work focuses on norm discovery from conversations on-the-fly and without needing manual curation. modeling the social and moral dynamics in human interaction and communication have diverse applications, such as the detection of cyberbullying @xcite and hate speech @xcite , detoxification @xcite , debiasing @xcite @xcite , model alignment @xcite , bipartisan news framing @xcite , social media understanding @xcite @xcite , and situational report generation @xcite , amongst other interdisciplinary human-centered nlp applications @xcite . in particular, discovering norms is essential for explicitly detecting norm adherence and violations instances (our work), as well as implicitly guiding dialogues @xcite . from a technical perspective, our norm discovery approach based on language model prompting and knowledge elicitation can be seen as a form of prompt engineering (le scao and rush, 2021), where we prefix a question with an elaborated scene, with the underlying core intuition based on leveraging the zero-shot question answering capability of language models @xcite . the norm grounding with explanation task is intuitively similar to the explainable natural language inference problem setting @xcite . our proposed framework, normsage, achieves norm discovery and grounding without intensive prompt-tuning @xcite or finetuning @xcite . ## conclusions we introduced a new nlp paradigm of guiding cross-culture communication with on-the-fly sociocultural-aware norm discovery, violation detection, and explanations. this conversationbased norm discovery and grounding problem goes beyond the us-centric data predominent in prior work. to address the new challenges, we present a framework called normsage that leverages knowledge elicitation from large language model (lm) prompting and incorporates novel selfverification mechanisms. our approach surpasses baselines by improving the discovery of dialogue norms across diverse social and cultural groups, including multi-lingual conversations and culturespecific norms, while providing natural language explanations for interpretable norm discovery and violation detection. our work has broad impacts as it empowers the discovery of norms which may differ from each other across cultures, thus enabling individuals to navigate communication barriers more effectively and find common grounds that avoid norm violations.