FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models
Year: 2023

Abstract:
It is common sense that one should prefer to eat a salad with a fork rather than with a chainsaw. However, for eating a bowl of rice, the choice between a fork and a pair of chopsticks is culturally relative. We introduce FORK, a small, manually-curated set of CommonsenseQA-style questions for probing cultural biases and assumptions present in commonsense reasoning systems, with a specific focus on food-related customs. We test several CommonsenseQA systems on FORK, and while we see high performance on questions about the US culture, the poor performance of these systems on questions about non-US cultures highlights systematic cultural biases aligned with US over non-US cultures.

heading: 1 Introduction
text: Effective communication in natural language requires a shared base of knowledge between interlocutors. While this shared knowledge between communicators may be specific to individuals in a shared situation (e.g., Dhruv and Mei know they are sitting at a table in a restaurant), or to individuals with specialized knowledge (they are discussing backpropagation), some types of knowledge are sufficiently generic to be shared by most people in the world (e.g., objects fall when they are dropped). This latter category of commonsense knowledge has for decades been a holy grail of research in artificial intelligence and natural language understanding (McCarthy, 1959). If machines are to understand (and produce) human language competently, they must at a bare minimum share this commonsense knowledge with humans.
A question elided by this notion of commonsense knowledge is who counts as “most people”? What may appear as universal “common sense” to AI researchers in one cultural context may in fact not be so universal. Early efforts to schematize commonsense knowledge as scripts, or stereotyped
sequences of events, provide a nice illustration of such unintended cultural biases: the famous “Restaurant script” (Schank and Abelson, 1975) prototypically includes a LEAVE TIP event, though tipping is not customary at restaurants in many countries outside the United States.
More recent AI research on commonsense knowledge acquisition has relied on crowd sourcing (Regneri et al., 2010; Sap et al., 2019), corpus statistics (Lin and Pantel, 2001; Van Durme and Schubert, 2008), and language modeling (Rudinger et al., 2015; Liu et al., 2022) in place of expertcrafted knowledge. However, each of these methods carries the potential to encode cultural bias into data and models for commonsense reasoning, whether through the implicit cultural perspectives of corpus texts, crowd source workers, or AI researchers themselves.
In this work, we seek to investigate cultural biases or assumptions present in commonsense reasoning systems. Culture, like commonsense knowledge, is vast. By one definition,1 culture “encompasses the social behaviour and norms found in human societies, as well as the knowledge, beliefs, arts, laws, customs, capabilities, and habits of the individuals in these groups.” From the social sciences, Kendall (2015) defines culture as encom-
1https://en.wikipedia.org/wiki/Culture
9952
passing both material as well as non-material aspects, such as beliefs and linguistic practices. To limit the scope of our investigation, however, we focus on a single topic common to all human cultures but widely varying across them: food.
We introduce FORK (Food ORiented cultural commonsense Knowledge), a manually-curated set of CommonsenseQA-style (Talmor et al., 2019) test questions for probing culinary cultural biases and assumptions present in commonsense reasoning systems. For the purpose of this work, we say that a commonsense question-answering system is culturally biased if (1) in response to questions with culturally-dependent answers, it exhibits systematic preference for answers consistent with one cultural setting over others; or (2) for questions with explicit cultural contexts, it exhibits systematically higher accuracy for some cultural contexts over others. Figure 1 contains an example of three interrelated test questions in FORK we use to detect cultural bias. For Q1, a model that prefers A1 to A2 exhibits cultural bias in favor of the United States (US) over China. While there exists no tidy mapping between human cultures and countries, in this work, we use countries as a coarse-grained proxy for culture (see: § 7).
FORK contains questions pertaining to the food and culinary cultures of the US, China, Japan, and India with questions spanning topics of restaurant tipping, eating utensils, and other culinary customs. We test multiple encoder-based CommonsenseQA models on FORK, demonstrating systematic cultural biases favoring the US over non-US countries.
To summarize, our contributions are: 1. FORK: a “bite-sized” manually curated test
set of 184 CommonsenseQA-style questions which can be used for probing culinary cultural biases and assumptions in commonsense reasoning systems. 2. A systematic evaluation of several encoder based models on FORK to demonstrate systematic cultural assumptions aligned with US over non-US cultures.

heading: 2 Dataset
text: Since FORK aims to test the culinary cultural specificity of commonsense reasoning models, we choose the format to be along the lines of Commonsense QA (Talmor et al., 2019). Each question in FORK has two options, only one of which is cor-
rect. One of the options pertains to the US culture, while the other to non-US. The questions are manually written by the first author of this paper. The source of content used to formulate the questions is information gathered from Google searches, blog posts, traveler guides, etc. Upon publication, we will release FORK publicly.
There are three types of questions in FORK: • Underspecified: The question asked is about
culinary customs and practices of no particular country or culture, and we hypothesize that English models will default to a US-centric interpretation in such a case. (See Fig. 1, Q1.) • Implicit: The question asked is about culinary customs and practices in context of a particular country but no country is mentioned explicitly in that question. Rather, the cultural setting is established implicitly with context cues. (See: Fig. 1, Q2.) • Explicit: The question asked is about culinary customs and practices in context of a particular country and that country (or well-known city therein) is explicitly mentioned in that question. (See: Fig. 1, Q3.)
We assign a theme to each question, and questions in FORK span over three distinct culinary themes: eating utensils, tipping and general custom/culture. We also tag each Underspecified question-answer pair, and each implicit and explicit question, with a corresponding country. We present a brief overview of the distribution in Table 1, and a full demographic distribution in Table 3.
It is important to note that these country labels should not be construed as exclusive of other cultures or countries that may share the relevant attribute. Cultural customs and countries have a many-to-many relation, and our labels are intended to highlight particular points of contrast between the US and other countries. What we measure as US-oriented cultural bias could also be construed as, e.g., Canada-oriented cultural bias only to the extent that US-labeled questions are also applicable to Canada.
The questions in FORK can either be a single
sentence or a two sentence question. The questions consisting of two sentences help to provide context in case of Implicit and Explicit questions. We also follow a template-based approach where a question about the same theme is asked multiple times, varying only by e.g. the name of a dish, city, country, etc.
In total, FORK consists of 184 manually curated questions, with 91.84% of questions pertaining to China, Japan, India and the US, and a small number of additional questions for other countries. Researching and writing questions was a slow manual process, so we chose to focus on producing more questions for fewer countries, to yield more robust results.

heading: 2.1 Validation Study
text: In order to ensure that the manually curated questions are valid for probing culinary cultural differences, we conduct a validation study with six annotators, two each from China, the USA and India. This pool of annotators comprised of five graduate students, and one professor. We ask the annotators to answer questions in FORK and present statistics in Table 2.
For US, both annotators disagreed on the same question, while for India, the difference was on questions pertaining on tipping. Feedback from annotators observed that the tipping culture varied across the country. For China, the human annotators noted that some practices were untrue for the regions they were from, but true for other regions in China.
Additionally, the differences in customs and practices within the same country reiterate our note above that cultural customs and countries have a many-to-many relation. We have used country as a proxy variable for culture because there are no clear distinct boundaries across cultures, and using this proxy boundary allows to probe differences at a US vs non-US level. This highlights a need for future work to investigate cultural differences within a country, based on regional or other demographic dimensions.

heading: 3 Experiments
text: We summarize our experimental set up, models, and the evaluation strategy used in this work.

heading: 3.1 Experimental Setup
text: In this work, we test seven encoder-based models on FORK and report their performance. We test two variants of BERT (Devlin et al., 2019): bert-base and bert-large, two variants of RoBERTa (Liu et al., 2019): roberta-base and roberta-large, DistilBERT (Sanh et al., 2019), and two variants of DeBERTaV3 (He et al., 2021): deberta-v3-base and deberta-v3-large.
All models are finetuned on the CommonsenseQA train fold for 3 epochs. We run a grid search for the hyper-parameters and report them in Appendix A.2.

heading: 3.2 Evaluation Strategy
text: We evaluate the culinary cultural contingency of the models tested as follows. For the questions tagged as Underspecified, we look at the number of times a "US" answer is chosen over a "non-US" answer. Here, "US" answer refers to an answer that would be appropriate or likely in the US context, and "non-US" answer refers to an answer that is more appropriate for a country on context outside the US. For Implicit and Explicit questions, we take a look at the responses for both US and non-US questions, and the percentage accuracy for US vs non-US answers.
Additionally, we also compare the number of times a non-US answer is chosen for Underspecified, non-US Implicit and non-US Explicit questions to better determine the bias between US and non-US cultures.

heading: 4 Results and Discussion
text: Figure 2a shows the percentage of time when a US answer is chosen over a non-US answer for Underspecified questions. We observe that roberta-large, and bert-large report the top two percentages of choosing a US answer over a non-US answer with values of 90.32% and 83.87% respectively. Fig 2a shows that all models, except for DistilBERT, preferred US answers over non-
US answers for a majority of Underspecified questions.
Figure 2b shows the percentage accuracy for US and non-US Implicit questions. We observe that roberta-large and bert-large report the top two accuracies of 78.57% and 71.42% respectively, when answering US Implicit Questions. In contrast, for non-US Implicit questions, only two models, DistilBert and bert-base cross the 50% accuracy mark, with bert-large having the lowest accuracy of 26.78%.
Figure 3a shows the percentage accuracy for US and non-US Explicit questions. Here, roberta-large, and bert-large report the top two accuracies of 100% and 92.30% respectively when answering US Explicit Questions. In contrast, for non-US Explicit questions, roberta-base reports the best accuracy at 64.28% while roberta-large performs the worst, achieving 22.85%.
Figure 3b shows the percentage times a nonUS answer is chosen for Underspecified, non-US Implicit and non-US Explicit questions. For Underspecified questions, only DistilBert crosses the 50% mark, with 51.62% accuracy. The performance for non-US Implicit and non-US Explicit questions has been discussed above. We report all the model accuracies on FORK in Tables 4 and 5 in Appendix A.2.
In addition to aggregating US versus non-US results, we break down accuracy results for China, India, and Japan for Implicit and Explicit questions in Table 6 in Appendix A.2.
We observe that (China, Explicit) and (China, Implicit) questions have the lowest average accuracy across models, at 36.57% and 41.80%, respectively. The best performance is reported for (USA, Explicit) at 74.72%.

heading: 4.1 Statistical Significance of Results
text: In order to make sure that our findings our statistically significant, despite the small number of questions in FORK, we conduct several statistical significance tests.
For Underspecified questions, we conduct the binomial test for all 7 model prediction results separately. Only roberta-base and DistilBert report a p-value greater than 0.05 in this setting.
For Implicit and Explicit questions, we conduct the chi-squared test on all 7 model prediction results separately to determine the statistical significance of our findings. For Implicit questions, only bert-base and deberta-v3-base report a p-value greater than 0.05 respectively. None of the models reported a p-value greater than 0.05 for Explicit questions.

heading: 5 Related Works
text: A growing body of work aims to detect social biases in NLP models with respect to demographic attributes like gender and race (Rudinger et al., 2018; Zhao et al., 2018; Nangia et al., 2020; Li et al., 2020; Sap et al., 2020). More recent is the growing attention towards cultural biases in NLP and AI technology at large. Hershcovich et al. (2022) propose a framework that allows one to understand the challenges of cultural diversity in NLP applications. Wikipedia has been shown to embed latent cultural biases (Callahan and Herring, 2011) and Tian et al. (2021) propose a methodology to develop culturally aware models for English, Chinese and Japanese using distributional perspectives on controversial topics from Wikipedia across these languages. Acharya et al. (2020) explore cultural biases, but along the rituals like birth, coming of age, marriage etc. in the US and in India. Chen and Henning (1985) investigate cultural bias in language proficiency tests and identify items of bias against non-native English speakers. To the best of our knowledge, this is the first work to analyze cultural bias in commonsense reasoning from the angle of culinary customs.

heading: 6 Conclusion
text: We have introduced FORK, a dataset to measure culinary cultural bias in commonsense models. Confirming our hypothesis, we find that models default to US cultural contexts in underspecified questions, and perform markedly better on implicit and explicit questions about US culture than nonUS. A likely source of bias is the English, USproduced texts which models are pretrained on. We believe the results support our hypothesis that English Language Models LMS trained on texts (many of which are produced for a US audience) would reflect US (or broadly Western) cultural assumptions. We hypothesize that the Underspecified setting had the lowest “accuracy” for non-US countries because the experimental design forced the model to choose between US and non-US interpretations of the same question. For Implicit and Explicit settings, we speculate that the non-US accuracy is generally higher for Explicit than Implicit because the former made it easier for models to determine the cultural setting.
Potential mitigation techniques to eliminate such biases may involve better curation of training data, training separate models for different cultural contexts, training models to better recognize cultural cues or ask for clarification in ambiguous settings, among many other possibilities. We believe this is an open research question, and we hope this paper will inspire future research to address it.
The topic of cultural bias is vast, and we choose a narrow scope to avoid biting off more than we can chew. Future work will explore strategies for cultural awareness of commonsense models, analysis of cultural assumptions in non-English models, and analysis of other aspects of culture beyond the culinary.

heading: 7 Limitations
text: The term culture has many meanings, and before attempting to incorporate commonsense with culture, one needs to establish a well defined definition and boundary along which test cases and examples would be constructed. By focusing exclusively on food and culinary customs, we have greatly restricted our domain of inquiry. However, culinary topics are universal, and span multiple domains of common sense reasoning (physical, interpersonal, societal). Nonetheless, we hope this work will inspire future work to investigate cultural bias along many axes beyond the culinary.
Incomplete representation of all cultures: There are limitations with using countries as a proxy for culture. As noted in § 2, mappings between cultures and countries are many-to-many, not one-to-one. The majority of questions in our test set FORK focus on culinary cultures and customs of only a few countries, and we do not expect the results to generalize to all the countries of the world. We choose to focus only on one topic and a small number of countries so that we may initiate research on this broad, challenging problem with a narrower, more well-defined task. We selected these cultures based on the cultural backgrounds of the authors and authors’ colleagues who were available to provide direct feedback on/validate the data. We hope this work paves the way for follow-up work investigating a broader set of cultures.
Small Annotator Pool: The validation study in § 2.1 is done on a small pool of annotators from a few countries represented in FORK. While the study gave useful feedback about the dataset and question quality, a larger and more diverse set of annotators would reflect a broader range of perspectives within each country, and further reduce the potential for biases or inaccuracies in our data.

heading: 8 Ethics Statement
text: Our paper has demonstrated systematic cultural biases in commonsense reasoning models’ understanding of culinary scenarios. While our end goal is to develop methods of evaluating cultural biases in models between the US and other countries, we acknowledge a number of risks involved in this endeavor. In particular, we note that any attempts to define, characterize, or delineate different cultures, particularly those to which the authors do not belong, creates a potential for oversimplifying the representations of those cultures and failing to represent minority populations therein. To mitigate this, we had a small number of annotators from the US, China, and India validate the questions, but these annotators do not represent the full diversity of each of these countries. We also caution that, while this dataset may be used to demonstrate the presence of cultural biases in commonsense reasoning models, it cannot be used to prove the absence thereof.

heading: 9 Acknowledgements
text: We would like to thank the anonymous reviewers for their valuable feedback on this paper. Addi-
tionally, we would like to thank Antoine Bosselut, Linda Zou, Peter Rankel, Abhilasha Sancheti, Haozhe An, Elijah Rippeth, Rupak Sarkar, Ishani Mondal, and Rebecca Knowles for their helpful comments and suggestions.

heading: A Appendix
text: A.1 Details about FORK We present the full details about our test bed FORK. FORK has a total of 184 questions with 169 (91.84%) questions talking about the culinary culture in the US, China, Japan and India. We present the full demographic distribution of questions in Table 3
A.2 Details about Model Parameters and Performance on FORK
We fine-tune each of the models mentioned earlier on the CommonsenseQA train set with a grid search for hyper-parameters [batch size = 16 (10 for the DeBERTaV3 models), learning rate = {{2, 3, 4, 5, 6, 7}e−4,−5,−6,−7}, epoch = {3, 5, 10}]. Training for 3 epochs gives us the best performance on the CommonsenseQA validation fold.
For the BERT(Devlin et al., 2019) models, we end up using a learning rate of 4e−5, and 6e−5 for bert-base and bert-large respectively. For RoBERTA models (Liu et al., 2019), we use 3e−5, and 7e−6 for roberta-base and roberta-large respectively. For DistilBERT (Sanh et al., 2019), and two variants of DeBERTaV3 (He et al., 2021): deberta-v3-base and deberta-v3-large a learning rate of 3e−5, 2e−5, and 7e−6 gives us the best performance respectively. We used the RTX A6000 GPU and finetuning the models took approximately 4 hours. Performance statistics of all the fine-tuned models on FORK are reported in tables 4, 5 and 6.
Table 4 shows the percentage times each model chooses a US answer and non-US answer when answering Underspecified questions from FORK. roberta-large chooses a US answer 90.32% of
the time while all other models, except DistilBERT, choose US answers more than 50% of the time. Overall, we can observe that the models tend to choose a US answer more as compared to a nonUS answer.
Table 5 shows the percentage accuracy of each model when answering Explicit and Implicit questions from FORK. For Explicit questions, we can see that one model achieves a 100% while answering questions about the US, while the highest accuracy for non-US questions is at 64.28%. Similarly for Implicit questions, the highest accuracy for US questions is 78.57% while it is at 53.57% for non US questions.
Table 6 shows statistics for a (Country, Type) pair that has at least 10 questions in FORK and the performance of the fine-tuned models. We observe that the average accuracy for (US, Explicit) is the highest followed by (US, Underspecified) at 74.72% and 70.04% respectively. In contrast, only (India, Explicit) gets an average accuracy higher than 50%.
All these observations clearly highlight the existence of systematic cultural assumptions aligned with US over non-US countries.
A.3 Licenses We have used BERT, RoBERTa, DistilBERT and DeBERTaV3 in this work. All these models use Apache License Version 2.0.2 The CommonsenseQA Dataset is under the MIT License.3 We are granted permission to use and modify these models for our experiments as per the terms of these licenses.
2https://www.apache.org/licenses/LICENSE-2.0 3https://opensource.org/licenses/MIT
ACL 2023 Responsible NLP Checklist

heading: A For every submission:

heading: 3 A1. Did you describe the limitations of your work?
text: Section 7: Limitations

heading: 3 A2. Did you discuss any potential risks of your work?
text: Section 8: Ethics Statement

heading: 3 A3. Do the abstract and introduction summarize the paper’s main claims?
text: Yes, Abstract and Section 1: Introduction
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts? Section 2: Dataset and Section 3: Experiments

heading: 3 B1. Did you cite the creators of artifacts you used?
text: Section 2: Dataset and Section 3: Experiments

heading: 3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts?
text: Appendix A.3

heading: 3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided
text: that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Appendix A.3. We haven’t mentioned license information for our artifact. We will do that when we release the full artifact.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? Section 4: Results and Discussion and Appendix A.1
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. Section 2: Dataset and Appendix A.1
C 3 Did you run computational experiments? Section 3: Experiments

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? Section 3: Experiments and Appendix A.2
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? Appendix A.2

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? Section 4: Results and Discussion and Appendix A.2
C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Not applicable. Left blank.
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? Section 2: Dataset
7 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No, not included in the paper. The task was to choose between two options for a question.
7 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No, not required for this study. Annotators were colleagues of the authors.
7 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Not Applicable. No PII data collected.
7 D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not required.

heading: 3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data?
text: Yes, Section 2 Dataset

heading: Future Work
text: Nonetheless, we hope this work will inspire future work to investigate cultural bias along many axes beyond the culinary. Incomplete representation of all cultures: There are limitations with using countries as a proxy for culture. As noted in § 2, mappings between cultures and countries are many-to-many, not one-to-one. The majority of questions in our test set FORK focus on culinary cultures and customs of only a few countries, and we do not expect the results to generalize to all the countries of the world. We choose to focus only on one topic and a small number of countries so that we may initiate research on this broad, challenging problem with a narrower, more well-defined task. We selected these cultures based on the cultural backgrounds of the authors and authors’ colleagues who were available to provide direct feedback on/validate the data. We hope this work paves the way for follow-up work investigating a broader set of cultures. Small Annotator Pool: The validation study in § 2. 1 is done on a small pool of annotators from a few countries represented in FORK. While the study gave useful feedback about the dataset and question quality, a larger and more diverse set of annotators would reflect a broader range of perspectives within each country, and further reduce the potential for biases or inaccuracies in our data