SlowGuess/ACL_2020_OCR huggingface dataset:
Cecilia L. Ridgeway. 2011. Framed by Gender: How Gender Inequality Persists in the Modern World. Oxford University Press.
Cecilia L. Ridgeway and Lynn Smith-Lovin. 1999. The gender system and interaction. Annual review of sociology, 25(1):191-216.
Kimberly B. Rogers, Tobias Schröder, and Wolfgang Scholl. 2013. The Affective Structure of Stereotype Content Behavior and Emotion in Intergroup Context. Social Psychology Quarterly, 76(2):125-150.
L. Smith-Lovin and W. Douglas. 1992. An affect control analysis of two religious subcultures. Social perspectives on emotion, 1:217-47.
L. Smith-Lovin and Dawn T. Robinson. 2015. Interpreting and Responding to Events in Arabic Culture. Final Report to Office of Naval Research, Grant N00014-09-1-0556.
Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. ConceptNet 5.5: An open multilingual graph of general knowledge. pages 4444-4451.
Robyn Speer, Joshua Chin, Andrew Lin, Sara Jewett, and Lance Nathan. 2018. Luminosoin-sight/wordfreq: v2.2.
Arthur Spirling and P Rodriguez. 2019. Word embeddings: What works, what doesn't, and how to tell the difference for applied research. Technical report, Working paper.
Leon Sterling and Kuldar Taveter. 2009. The art of agent-oriented modeling. MIT press.
Chris Sweeney and Maryam Najafian. 2019. A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1662-1667, Florence, Italy. Association for Computational Linguistics.
Nathaniel Swinger, Maria De-Arteaga, Neil Thomas Heffernan IV, Mark DM Leiserson, and Adam Tauman Kalai. 2019. What Are the Biases in My Word Embedding? In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES '19, pages 305-311, New York, NY, USA. ACM.
Alexander Todorov, Christopher Y. Olivola, Ron Dotsch, and Peter Mende-Siedlecki. 2015. Social Attributions from Faces: Determinants, Consequences, Accuracy, and Functional Significance. Annual Review of Psychology, 66(1):519-545.
Nicholas Winter, Tyler Burleigh, Ryan Kennedy, and Scott Clifford. 2019. A Simplified Protocol to Screen Out VPS and International Respondents Using Qualtrics. SSRN Scholarly Paper ID 3327274, Social Science Research Network, Rochester, NY.
4402
A Embedding Models
We use twelve publicly available embedding models. We use all public GloVe (Pennington et al., 2014) models $^{7}$ , FastText (Mikolov et al., 2018) models $^{8}$ , the original Word2Vec (Mikolov et al., 2013) model $^{9}$ , and v19.08 of the NumberBatch (Speer et al., 2017) model. $^{10}$
B Word Position Measurement Models
Table 3 outlines the word position measurement models used in the present work. The table provides information on the authors of the measure, whether or not embeddings are normalized before analysis, how words are measured once a direction has been specified, how a direction is specified, and whether or not the method is "multi-class," described further below.
Notationally, we have tried to remain as close to the original works as possible. Therefore,  $w$  is the identity to be measured, and  $b$  is the vector indicating the direction along which it is to be measured. For Garg et al. (2018),  $b_{l}$  and  $b_{r}$  represent words in the left-hand dimension-inducing word set (e.g. "man" and "him" for gender) and  $b_{r}$  the right-hand of the dimension-inducing word sets (e.g. "woman" and "her" for gender). The variables  $p_{i,l}$  and  $p_{i,r}$  have similar meanings for Swinger et al. (2019) and Bolukbasi et al. (2016).
We use the approaches of Garg et al. (2018), Kozlowski et al. (2019), Ethayarajh et al. (2019), and Swinger et al. (2019) exactly as described in the original texts, except for one modification. In the case where a paired set of terms is required—all cases except Garg et al. (2018) and Swinger et al. (2019)—and we have a multi-class measurement (e.g. we measure four different dimensions of racialized beliefs), we first identify a default dimension and then compare all other dimensions to it. For race, we follow theory on perceptions of default race categories and assign White to be the default race (MacLin and Malpass, 2001), and the comparison point for White, following Kozlowski et al. (2019), to be Black. For the associative dimensions, we select family for the default, and
compare family to politics.
In addition, we consider the possibility that the computationally appealing approach from (Ethayarajh et al., 2019) may be improved by using a different direction specification approach. Therefore, we consider two additional word position measurement models, Ethayarajh et al. (2019) + Garg et al. (2018), and Ethayarajh et al. (2019) + Kozlowski et al. (2019), that are the same as the original model but using the direction specification method in these two papers instead of the method from Bolukbasi et al. (2016), as was done in the original paper.
C Further Details - Identity Selection for Our Survey Data
In selecting identities for each cluster, we also ensured that the words selected were a) expressed most frequently as identities, b) were in a standard set of lexical resources and thus common English terms and c) were used relatively frequently. To ensure the identities were used first and foremost as an identity (and not, e.g., as a verb or place name), we first used both the NLP python library spacy and Wordnet to identify any identities for which the dominant sense was a verb (e.g. suspect, accused) and removed these from consideration. To ensure that the identities were in common lexical databases, we removed words which were not in Wordnet as a noun or an adjective. Finally, to ensure that identity words were used frequently, we checked that they were used frequently in either a fairly informal medium, Twitter, or in a fairly formal medium, Wikipedia. To check the former case, we use the frequency counts of words from 56M tweets given by Owoputi et al. (2013) and retain only those identities used in more than 2500 tweets. To check Wikipedia, we first extract all 532,051 "clean" pages from a Wikipedia dump from December, 2015. A clean page is a page that is not labeled as a stub, that was still active one month after the dump was created, and that also has more than 50 views over 2 year span, where we pull one random hour for each day.
D Further Details - Belief Measurement Data
D.1 Measurement
The slider bar for the affective dimensions gives labels at different points, ranging from "Infinitely" to "Slightly" on both ends, with a "Neutral" option
$^{7}$ https://nlp.stanford.edu/projects/glove/ $^{8}$ https://fasttext.cc/docs/en/english-vectors.html $^{9}$ https://code.google.com/archive/p/word2vec/ $^{10}$ https://github.com/commonsense/conceptnet-numberbatch
Measure	Normalized?	Position Measure	Direction-Specification	Multiclass
Ethayarajh et al. (2019)	N	⟨w,b⟩/||b||	Same as Bolukbasi et al. (2016)	N
Kozlowski et al. (2019)	Y	⟨w,b⟩/||b||||w||	∑pi∈P pi,l−pi,r/||P||	N
Bolukbasi et al. (2016)	Y	⟨w,b⟩/||b||||w||	SVD (c (pi,j − μpij pi ∈ P))	N
Swinger et al. (2019)	Y	avgpi∈P ⟨w,pi,l⟩/||w||||pi,l|| - avgpi∈P ⟨w,pi,r⟩/||w||||pi,r||	N/A	Y
Garg et al. (2018)	Y	||w−br||−||w−bl||	bl := ∑pi∈pr pi/||P||	Y
Table 3: Details of the prior work on word position measurement models from which we draw. We use each model listed here, as well as using the approach of Ethayarajh et al. (2019) but using direction specification as described by Garg et al. (2018) and Kozlowski et al. (2019).
in the middle. The age slider had the following qualitative labels, spaced equally across the slider bar: "Baby, Child, Teenager, 20s, 30s, 40s, 50s, 60s, 70s, 80s, 90s,  $> = 100$ " . The gender slider had the following labels, spaced equally across the slider: "Always Male, Mostly Male, Equally Male or Female, Mostly Female, Always Female".
For the race/ethnicity beliefs, order of the sliders was randomized, and the starting value for each was set to  $20\%$ . With respect to discussions about the 2020 census, most importantly, demographers have pushed to include Hispanic or Latino as a racial category rather than to split it out into its own separate question.
For the associative belief question, presentation order was randomized. The form of the question is drawn from other studies seeking to elicit cognitive associations between a term and a set of other concepts, e.g. from Hill et al. (2015). The specific institutions were originally drawn from the clustering used to determine our identities. However, we added the education and religion institutions after determining they would be necessary for a more complete meaning space, as suggested by the institutional settings with which identities are commonly associated as discussed by MacKinnon and Heise (2010).
Finally, pilot testing suggested that respondents became confused when provided with certain identities that had meanings that were used to construct the question - for example, on the race question, respondents became confused when being asked "Of all white people, what percentage of them do you think are ... [White]?" We, therefore, removed
the gender question from the identities guy, boy, girl, lady, man, and woman, and removed the race question from the identities Asian, White person, Black person, Arab, and Hispanic.
D.2 Participant Sample
In total, 252 Mechanical Turk workers completed the survey. These had greater than a  $95\%$  completion rate and had completed over 1,000 HITs. These workers also were located within the United States. In order to ensure this was the case, we leveraged a tool provided by Winter et al. (2019) to ensure that participants' IP addresses were located within the United States and that they were not using a VPN. We further ensured competency and attention by including two attention check questions. Five respondents were rejected because they failed attention checks. Sample demographics were not collected for Task 1, because we make the deliberate assumption that measurements from surveys of any individuals within a national culture can serve as the foundation for a meaning space. However, as noted, we do ensure cultural expertise by ensuring participants are native English speakers located in the U.S.
D.3 Ethical Approval
This study was approved by the Institutional Review board of the University at Buffalo.
D.4 Summary Results
Figure 6 provides  $95\%$  bootstrapped confidence intervals for the Evaluation, Potency, and Activity dimensions for each identity in the survey. The
measurements in our survey are compared to results from Smith-Lovin and Robinson (2015). The vast majority of our estimates overlap closely with theirs, signifying the broad generality of the measurement tools used by ACT across individuals within a national culture. Where differences arise, we do not believe one dataset or the other appears to be universally more accurate, and further, given the number of comparisons (171, 3 per each of 57 identities), we should expect even by chance some larger differences in the measurements.
Figure 7 provides full results for associative meanings. The figure shows that in general, identities assumed to cluster within a particular institution were rated as having the highest associations with that institution. However, it is also clear that other identities were also strongly aligned with the various institutions in ways that did not follow exactly from the construction of our identity set. For example, the identity thug was included as a negative affect term, but was perceived to have a strong association to the judicial institution.
Figure 8 provides full results for the expected age and gender of each identity. Note that some identities with a denotative meaning aligned with these dimensions were included, because pilot tests did not suggest confusion for these identities. We therefore attempted to include as many identities as possible in the actual measurement.
Figure 9 provides full results for the race question we asked. All identities were assumed by participants to be more White than expected by chance. Recall that the racial question was not posed for the denotatively aligned racial identities we studied.
E Further Details - Identity Labeling Task
E.1 Participant Sample
We collected valid data from 402 Mechanical Turk workers were located in the United States, had greater than a  $95\%$  completion rate and had completed over 1,000 HITs. To assess accuracy for respondents, we randomly sampled 5 questions from each respondent and ensured that answers did not appear to be entered randomly. The sample's gender was  $53.7\%$  female, and  $45.6\%$  male ( $.7\%$  did not say). A total of  $89.4\%$  had at least some college or vocational training,  $53.8\%$  of the sample had completed at least a Bachelor's degree, and  $16.1\%$  had a post-graduate degree. Almost all  $(96.7\%)$  of the sample were born in and had lived
between  $75 - 100\%$  of their life in the United States. With respect to age,  $9.0\%$  of the sample was aged 18-24,  $17.2\%$  aged 25-29,  $33.3\%$  aged 30-39, and  $40.0\%$  aged 40 or older ( $.5\%$  did not say). Finally, the sample was largely white,  $83.3\%$  of participants were White or Caucasian.
E.2 Ethical Approval
The survey carried out was approved by the Institutional Review board of Carnegie Mellon University.
E.3 Additional Implementation Details
As noted in the text, and replicated here for clarity, from the 57 identities in Table 1, we create survey questions for the identity labeling task as follows: for a given identity, we generate 14 random sets of the 56 other identities; each set contains four identities. We then generate one IsA and one Seen-With question for each of these sets, where these four identities constitute the possible answers to the question, and the given identity is used in the question text. This process is then repeated ten times for each identity. This process generates exactly ten questions for each of the  $3,192^{11}$  identity pairs for each type of question.
The intention was, therefore, to have exactly ten questions for each identity pair for each question type where the first identity in the pair is shown in the question and the second identity in the pair is shown as a possible answer. In each case, by construction, the other possible answers were randomly selected. Unfortunately, our survey suffered from a bug with the Qualtrics software used, where the option to present questions an even number of times fails in unclear cases. Due to this error, some of our identity pairs were not seen exactly ten times. Specifically,  $3.4\%$  were asked less than 6 times,  $40\%$  were asked less than ten times, and  $40.4\%$  were asked more than ten times. While this does not affect our analyses, because they do not rely on any exact number of questions per identity pairing, it is important to note for purposes of any future work with the dataset.
Such issues aside, the process described generated 15,960 questions. These questions produced a total of 16,080 responses (a small number of questions—120, or  $0.7\%$ —were asked more than once in attempts to address the Qualtrics bug) that were split evenly, 40 questions per respondent.
157*56=3,192
4405
E.4 Results
Figure 10 presents full results for the regression models described in the text to identify the importance of each dimension for identity labeling.
F Additional Details - RQ2
Figures 11-14 present results from generalized additive models with the four dependent variables described in the main text on the rank-level outcome variable at the belief level. The models explain  $31.8\%$ ,  $34.2\%$ ,  $16.9\%$ , and  $21.6\%$  of the deviance for the data from this paper, Bolukbasi et al. (2016), Agarwal et al. (2019), and Smith-Lovin and Robinson (2015), respectively. Across the four datasets, the only consistent predictor is the distance of the survey-based belief measure from the median. Note, however, that in the data from Smith-Lovin and Robinson (2015), this pattern does not hold at the extremes. Further analyses suggest this is due to a small number of outliers on the extremes of the Evaluation dimension, and does not appear to reflect any interesting trend worth additional consideration. Additionally, we note that, the authors of Bolukbasi et al. (2016) could only provide us with a mean per-identity estimate, and thus no information on the variance of those estimates is used in our results.
Figure 6:  $95\%$  bootstrapped confidence intervals for estimates of the Evaluation, Potency, and Activity dimensions. Results from the current study are in red, results from a prior study from Smith-Lovin and Robinson (2015) are given for comparison in cyan.
Figure 7: Associations for each identity to each of the institutional settings we study. Results are given with  $95\%$  bootstrapped confidence intervals, and each institutional setting is a different subplot. Red estimates represent identities that were assumed to be clustered on the given institutional setting
4408
Figure 8: On the left,  $95\%$  confidence intervals for the expected age of each identity. On the right,  $95\%$  confidence intervals for the expected gender of each identity, assuming a continuous dimension for gender. Note the gender labels are slightly shifted inward so labels are readable, meaning some points extend beyond the label itself.
Figure 9:  $95\%$  confidence intervals for the perceived percentage of each identity that are also of a given race. The five racial/ethnic categories we consider are represented as separate subplots, the red line at  $20\%$  gives the expected value of all races were equally represented for the identity.
Figure 10: Coefficients for the regression models described in the main text for IsA and SeenWith questions for each dimension. Blue bars represent error bars for each dimension on IsA questions (horizontal) and SeenWith (vertical). Red dotted lines represent zero, any error bars that cross this line are not significant
Figure 11: Regression results for survey data from this paper on the belief-level measure
Figure 12: Regression results for survey data from Bolukbasi et al. (2016) on the belief-level measure
4413
Figure 13: Regression results for survey data from Agarwal et al. (2019) on the belief-level measure
4414
Figure 14: Regression results for survey data from Smith-Lovin and Robinson (2015) on the belief-level measure