SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Religion: hindu, hindus, islam, muslim, muslims, christians, christian, jain, jains, buddhist, buddhists, sikh, sikhs
Region: Andamanese, Nicobarese, Andhrulu, Teluguvaaru, Arunachalis, Assamese, Biharis, Chandigarhis, Chhattisgarhis, Dadran, Nagar Havelian, Damanese, Diuese, Delhiites, Delhians, Delhites, Goans, Goenkars, Gujaratis, Haryanvis, Himachalis, Jammuite, Kashimiris, Jharkhandis, Karnatakaans, Canarese, Kannadiga, Kannadigas, Keralites, Malayalis, Mallus, Mallu, Ladakhi, Ladakhis, Laccadivians, Madhya Pradeshis, Maharashtraians, Marathi, Manipuris, Meiteis, Meghalayans, Mizos, Nagas,Nagalanders, Odias, Odias, Odishans, Orissans, Pondicherrians, Pondicherrians, Punjabis, Rajasthanans, Sikkimese, Tamils, Tamilians, Tamizhan, Telanganites, Teluguvaaru, Tripuris, Tripurans, Uttar Pradeshis, Uttarakhandis, West Bengalis, Bengalis, marwadi, marwadis, south indian, south indians, north indian, north indians, UP, U.P, North East Indian, West Indian, East Indian
B.2 Word Lists for Offensive Stereotype Analysis
Annotations for offensiveness of attributes in stereotypes collected from existing work [22] and their respective dataset (https://github.com/google-research-datasets/seegull, Creative Commons license, and intended usage of evaluation of data and models).
Word lists used for analysis of offensive stereotypes from SPICE are listed below. The identity words were specified by us to retrieve stereotypes for respective groups and communities. The attribute words are the ones that were part of the stereotypes retrieved as a result.
Identity words: Brahmin, Brahmins, Dalit, Dalits, SC, ST, Hindu, Hindus, Muslim, Muslims, Islam, Christian, Christians, Christianity, Sikh, Sikhs, Sikhism.
All stereotypes in SPICE with these identity words were considered for this analysis. The attributes in the stereotypes were used to compute offensive score [22].
C Data and Analysis
C.1 Gender Stereotypes in Data
Some examples of gender stereotypes as submitted by male and female identifying participants are listed in Table 9.
Table 9: Some gender based stereotypes volunteered by participants identifying as male or female.
Participant	Identity	Attributes
Female	Female	bad driver, housewife, nurturing, inferior
Male	dominant, don’t cry
Male	Female	nurse, housework
Male	strong, doctor, sports
C.2 Regional Stereotypes in Data
Table 10 illustrates stereotypes about other regions ("North India", "North East India", etc.) as volunteered by participants located in a given region.
Table 10: Some stereotypes captured in each region about people belonging to other regions of the country. Here, 'region' refers to the where a stereotype was collected in.
Region	Stereotype
North	North East Indian: Chinese, chinki, Nepali; South Indian: dark skinned, engineer
South	North Indian: athlete, brave
West	North Indian: migrant; North East Indian: Nepali, Chinese; South Indian: intelligent, Christian
East	North Indian: casteist; North East Indian: not Indian, Chinese; South Indian: dark, intelligent
D Code, External Data, and License
All experiments and analysis in this work is done on a Google Colab CPU. Since the datasets, and models used are relatively small, greater compute is not needed.
The code used in this work for demonstrating the utility of SPICE dataset in evaluating models is adapted from existing code base. We use the task of NLI stereotype evaluations through https://github.com/sunipa/On-Measuring-and-Mitigating-Biased-Inferences-of-WordEmbeddings (work cited as stated by authors). This work and the word lists provided by authors for this task are available by Creative Commons license.
This code is used with BART, RoBERTa, XLNET, and ELECTRA in the work. These models, already fine tuned on SNLI [10] and MultiNLI [43] datasets and available from HuggingFace. The attached code document (Spice code.pdf) details the exact models called from HuggingFace, their paths, and how these models were called. We will make this available publicly via a GitHub link on publication.
The datasets used for comparison of SPICE in Sections 5, and sec 6 are from papers [5] and [22], and are also available by Apache license 2.0. Additionally, we use their data cards to ensure adherence to intended usage of their data, which is evaluation of models.