Multi-Cultural Norm Base: Frame-based Norm Discovery in Multi-Cultural Settings (2024)

Abstract: Sociocultural norms serve as guiding principles for personal conduct in social interactions within a particular society or culture. The study of norm discovery has seen significant development over the last few years, with various interesting approaches. However, it is difficult to adopt these approaches to discover norms in a new culture, as they rely either on human annotations or real-world dialogue contents. This paper presents a robust automatic norm discovery pipeline, which utilizes the cultural knowledge of GPT-3.5 Turbo (ChatGPT) along with several social factors. By using these social factors and ChatGPT, our pipeline avoids the use of human dialogues that tend to be limited to specific scenarios, as well as the use of human annotations that make it difficult and costly to enlarge the dataset. The resulting database - Multi-cultural Norm Base (MNB) - covers 6 distinct cultures, with over 150k sociocultural norm statements in total. A state-of-the-art Large Language Model (LLM), Llama 3, fine-tuned with our proposed dataset, shows remarkable results on various downstream tasks, outperforming models fine-tuned on other datasets significantly.

## introduction sociocultural norms are informal rules or guidelines that dictate acceptable behavior within a particular society or culture @xcite . these norms encompass a wide range of behaviors, including manners, customs, values, and traditions. they govern how individuals interact with one another and shape societal expectations regarding appropriate conduct in various contexts. with the rapid development of ai in the last decade, it is crucial to define effective methods for discovering and assessing the cultural knowledge of ai systems, especially the knowledge of sociocultural norms. the study of cultural norm discovery has witnessed significant development in recent years. social-chem-101 @xcite , one of the earliest corpora, introduces social norms represented in a rule of thumb (rot) format. norm-bank @xcite is another large-scale corpus of norms that contains situational norms within a multivalent sociocultural frame. while these datasets have high-quality samples and can be applied to many culture-related tasks, they are constructed by humans, which is very time-consuming and costly. in response to this problem, @xcite introduced normsage, a norm dataset constructed with a fully automated pipeline. norm statements in normsage are extracted by prompting large language models (llms) with dialoguebased contents. the norms are then fed to a selfverification process to ensure their quality. while normsage showcases a promising direction for automatic norm discovery, it is based on real dialogue data, which may not be available in different cultures and can be limited to specific domains. moreover, social norms, relevant to specific frames, should possess the flexibility to be applicable across diverse dialogues, instead of being bound to a single specific conversation. to address the above challenges, in this paper, we present an automated frame-based pipeline for norm dataset construction using chatgpt in a multi-cultural setting. socio-cultural norms are often strongly associated with several social factors @xcite , and we refer to the combination of social factors as situational frames. norms in the proposed dataset are generated by prompting chatgpt with situational frames as the context, instead of using real-world dialogue content like existing works. these frames consist of carefully chosen social factors (culture, social relation, power distance, and so on) which help to align the norm generation process. in this way, we will not have to collect dialogue data for specific cultures and can easily expand the dataset. once the norms are extracted, we evaluate them both intrinsically and extrinsically. for the former, we use human evaluation to assess the quality of the extracted norm statements. for the latter, we employ the constructed norm database in various downstream tasks to prove the adaptability as well as the performance of our proposed dataset. to summarise, our contributions are as follows: ## building multi-cultural norm base in this section, we describe our proposed automatic pipeline for collecting socio-cultural norms for various cultures. the following subsections will discuss the overall pipeline, as well as provide a detailed explanation for each step in the pipeline. for simplicity, the term socio-cultural norm will be referred to as norm or social norm for short. ## experiments to demonstrate the quality of our proposed method and dataset, we carry out experiments with our data and other related datasets. our experiments are divided into two types: intrinsic evaluation and extrinsic evaluation. for intrinsic evaluation, we examine the quality of the constructed norm knowledge base and the norm extraction method. in the case of extrinsic evaluation, we demonstrate the applicability of our proposed dataset across different downstream tasks and compare the performance with other datasets. ## conclusions in this paper, we propose an automatic norm discovery pipeline using chatgpt for the multi-cultural setting. the pipeline extracts norm statements upon situational frames filled with crucial social factors. as real dialogues are not always available and can be limited to some domains, we have showcased that it is possible to extract meaningful norm statements only from social factors. our derived norm database has shown its effectiveness in the experiments, achieving remarkable results on several downstream tasks and outperforming other norm datasets. in the future, we plan to expand the data with coverage to more cultures and implement large language models embedded with explicit cultural knowledge.