Social-Group-Agnostic Bias Mitigation via the Stereotype Content Model
Year: 2023

Abstract:
Existing bias mitigation methods require socialgroup-specific word pairs (e.g., “man” – “woman”) for each social attribute (e.g., gender), restricting the bias mitigation to only one specified social attribute. Further, this constraint renders such methods impractical and costly for mitigating bias in understudied and/or unmarked social groups. We propose that the Stereotype Content Model (SCM) — a theoretical framework developed in social psychology for understanding the content of stereotyping — can help debiasing efforts to become social-group-agnostic by capturing the underlying connection between bias and stereotypes. SCM proposes that the content of stereotypes map to two psychological dimensions of warmth and competence. Using only pairs of terms for these two dimensions (e.g., warmth: “genuine” – “fake”; competence: “smart” – “stupid”), we perform debiasing with established methods on both pretrained word embeddings and large language models. We demonstrate that our social-groupagnostic, SCM-based debiasing technique performs comparably to group-specific debiasing on multiple bias benchmarks, but has theoretical and practical advantages over existing approaches.

heading: 1 Introduction
text: The societal impacts of Natural Language Processing (NLP) have stimulated research on measuring and mitigating the unintended social-group biases encoded in language models (Hovy and Spruit, 2016). However, the majority of this important line of work is atheoretical in nature and “fails to engage critically with what constitutes ‘bias’ in the first place” (Blodgett et al., 2020). The bias found in language models is rooted in human biases (Caliskan and Lewis, 2022); thus, to alleviate such biases, we should ground our debiasing approaches in social psychological theories of stereotyping. These theories can help us shed light on the
underlying structure of language-embedded biases rather than attending to ad hoc superficial patterns (Osborne et al., 2022).
Although there is a multitude of approaches to bias mitigation (Bolukbasi et al., 2016; Zhao et al., 2018; Dev and Phillips, 2019; Kaneko and Bollegala, 2021; Solaiman and Dennison, 2021), most of these approaches are group-specific. Such methods, which debias along subspaces defined by social groups or attributes (e.g., gender or race), are not only atheoretical but also unscalable. Resources developed for bias mitigation on one social group or attribute (e.g., gender) do not axiomatically translate easily into other groups or attributes (e.g., age).
4123
For example, previous works’ focus on gender bias has driven the development of resources that are only applicable to gender debiasing (e.g., equality word sets for gender), but biases associated with other social groups and/or attributes remain understudied. Beyond the challenge of creating such resources for a given attribute, to achieve an “unbiased” model with group-specific debiasing, one would have to iterate over all social groups. This approach is practically impossible and arguably would result in significant degrading in the expressiveness of the model. Furthermore, groupspecific debiasing is limited in terms of effectiveness: stereotypic relations in distributed representations are deep-rooted, and thus may not be easily removed using explicit sets of group-specific words (Agarwal et al., 2019; Gonen and Goldberg, 2019). In contrast, a social-group-agnostic approach would not have such restrictions.
The Stereotype Content Model (SCM; Fiske et al., 2002) is a theoretical framework developed in social psychology to understand the content and function of stereotypes in interpersonal and intergroup interactions. The SCM proposes that human stereotypes are captured by two primary dimensions of social perception: warmth (e.g., trustworthiness, friendliness) and competence (e.g., capability, assertiveness). From a socio-functional, pragmatic perspective, people’s perception of others’ intent (i.e., warmth) and capability to act upon their intentions (i.e., competence) affect their subsequent emotion and behavior (Cuddy et al., 2009). Warmth divides people (or groups of people under a social identity) into “friends” or “foes,” while competence contains perceptions of social groups’ status. Depending on historical processes, various social groups may be located in different stereotypic quadrants (high vs. low on warmth and competence) based on this two-dimensional model (Charlesworth et al., 2022). While there are alternative theories related to social evaluation and stereotyping (e.g., Abele and Wojciszke, 2007; Ellemers et al., 2013; Koch et al., 2016; Yzerbyt, 2018) Ellemers et al. (2020) demonstrate the possibility of establishing a theoretical alignment between them.
In this work, we propose SCM-based bias mitigation and demonstrate that by relying on a theoretical understanding of social stereotypes to define the bias subspace (rather than group-specific subspaces), bias in pre-trained word embeddings and large language models can be adequately mitigated
across multiple social attributes (Figure 1). We chose to focus on SCM not only because of the availability of validated linguistic resources (Nicolas et al., 2021), but also due to SCM’s prominence, parsimony, and robustness across languages and cultures (e.g., Bye and Herrebrøden, 2018; Grigoryev et al., 2019; Sharifian et al., 2022; Liang et al., 2022). Specifically, we confirm that by debiasing with respect to the subspace defined by warmth and competence, our SCM-based approach performs comparably with group-specific debiasing for a given group (e.g., SCM-based debiasing is comparable to race-debiasing on race, see §3.5). We also show that SCM-based debiasing simultaneously reduces bias for understudied attributes such as religion and age (§3.5 and §4.4). Finally, we show that for both word embeddings and large language models SCM-based debiasing retains original model’s expressiveness (§3.6 and §4.4). Overall, our results confirm the viability of a theory-based, social-group-agnostic approach to bias mitigation1.

heading: 2 Background

heading: 2.1 Bias Mitigation in Word Embedding
text: Part of our work builds on post hoc bias mitigation which aims to remove biases by modifying pre-trained word embeddings and language models. Most efforts we review are restricted to genderrelated debiasing (e.g., Bolukbasi et al., 2016; Zhao et al., 2018; Dev and Phillips, 2019; Ravfogel et al., 2020); importantly, we focus our work on other social categories as well, bringing attention to these understudied groups and attributes. Originally, Bolukbasi et al. (2016) proposed Hard Debiasing (HD) for gender bias. HD removes the gender component from inherently non-gendered words and enforces an equidistance property for inherently gendered word pairs (equality sets). Two followups to this work include: Manzini et al. (2019), which formulated a multiclass version of HD for attributes such as race; and Dev and Phillips (2019), which introduced Partial Projection (PP), a method that does not require equality sets and is more effective than HD in reducing bias. Extending these approaches to other social attributes is not trivial because a set of definitional word pairs has to be curated for each social group/attribute; this curation is a non-trivial task as the list of words required is
1https://github.com/Ali-Omrani/Social-Group-AgnosticBias-Mitigation
dynamic and context-dependent. Gonen and Goldberg (2019) demonstrated that gender bias in word embeddings is deeper than previously thought, and methods based on projecting words onto a “gender dimension” only hide bias superficially. They showed that after debiasing, most words maintain their relative position in the debiased subspace. Our work is motivated by this important criticism. Specifically, we argue that our theory-driven approach removes the actual psychological ‘bias’ subspace, rather than subspaces, often superficially, containing bias for specific groups.

heading: 2.2 Bias Mitigation in Language Models
text: Fine-tuning and prompt-tuning are two major paradigms for using pre-trained language models in NLP, and bias mitigation approaches have been proposed based on both. Kaneko and Bollegala (2021) introduced Debiasing Pre-trained Contextualized Embeddings (DPCE), a finetuning method for bias mitigation in language models with a loss term that aims to find a balance between bias mitigation and language modeling ability. Other fine-tuning methods include AutoDebias (Guo et al., 2022), which samples the language model to find examples for finetuning-based debiasing. Promp-tuning (Li and Liang, 2021; Liu et al., 2021; Lester et al., 2021) is mainly done through either discrete prompts, which consist of text (i.e., tokens), or continuous prompts, which consist of a continuous array of numbers prefixed to a language model and trained. It has been shown that bias in language models can be mitigated by providing descriptions of desired and undesired behavior as discrete prompts (Schütz et al., 2021; Askell et al., 2021; Solaiman and Dennison, 2021). More recently Yang et al. (2023) proposed “A DEbiasing PrompT” (ADEPT) that outperforms DPCE by using continuous prompt tuning to mitigate biases in a language model. Yu et al. (2023) proposed partitioned contrastive gradient unlearning (PCGU), a gray-box method for debiasing pretrained masked language models by optimizing only the weights that contribute most to a specific domain of bias.

heading: 2.3 The SCM and Language
text: SCM is a well-established theoretical framework of stereotyping, and has begun to be applied in NLP. Recently, Nicolas et al. (2021) developed dictionaries to measure warmth and competence in textual data. Each dictionary was initialized with a set of seed words from the literature which was further
expanded using WordNet (Miller, 1995) to increase the coverage of stereotypes collected from a sample of Americans. Cao et al. (2022) employ AgencyBelief-Communion (ABC; Koch et al., 2016), an alternative theory from social psychology for stereotype content, to discover stereotyping in language models. Fraser et al. (2021) demonstrated that, in word embeddings, SCM dictionaries capture the group stereotypes documented in social psychological research. Recently, Davani et al. (2023) applied SCM dictionaries to quantify social group stereotypes embedded in language, demonstrating that patterns of prediction biases can be explained using social groups’ warmth and competence embedded in language. Contemporary to our work, Ungless et al. (2022) explore the idea of using SCM for bias mitigation but fall short of evaluating their models on the necessary benchmarks.

heading: 3 SCM-Based Bias Mitigation for Static Word Embeddings
text: Before we discuss our proposed method, we briefly review the algorithms and benchmarks on bias mitigation in word embeddings. There are two components to each post hoc bias mitigation approach for static word embeddings: the Bias Subspace, which determines the subspace over which the algorithms operate, and the Algorithm, which is how the word embeddings are modified with respect to the bias subspace. In this section, we review the concept of bias subspaces, established algorithms for debiasing, and how bias is quantified in word embeddings. Finally, we introduce our socialgroup-agnostic framework; SCM-based debiasing.

heading: 3.1 Identifying a Bias Subspace
text: Post hoc word embedding debiasing algorithms operate over a subspace of bias in the embedding space. Given a set D = {(d+1 , d−1 ), ..., (d+n , d−n )} of word pairs that define the bias concept (e.g. “father”–“mother” for binary gender) the bias subspace vB is the first k principal components of matrix C, constructed from stacking the difference in embeddings of d+i and d − i .

heading: 3.2 Debiasing Algorithms
text: Method definitions below use the following notation: W denotes vocabulary, w and w’ denote the embedding of word w before and after debiasing.
Hard Debiasing (HD) An established approach for mitigating bias in word embeddings is Hard
Debiasing (HD; Bolukbasi et al., 2016). For gender, HD removes the gender subspace from words that are not inherently gendered by projecting them orthogonal to gender subspace. For word pairs that are inherently gendered, HD equalizes them, modifying the embeddings such that they are equidistant from the inherently non-gendered words.
Subtraction (Sub) Sub was introduced as a baseline by Dev and Phillips (2019) wherein the bias subpspace vB is subtracted from all word vectors. Formally, for all w ∈ W ,w′ := w − vB . Linear Projection (LP) To mitigate the bias with respect to bias dimension vB , Linear Projection (LP) projects every word w ∈ W to be orthogonal to vB . Formally, w′ := w−πB(w) where πB(w) = ⟨w, vB⟩vB is the projection of w onto vB . Partial Projection (PP) To improve on LP, Partial Projection (PP) was developed to allow the extent of projection to vary based on the component of the given word vector which is orthogonal to the bias subspace. Intuitively, only words with unintended bias (e.g., “nurse” or “doctor”), and not words which are definitional to the bias concept (e.g., “man” or “woman”) will have a large orthogonal component to the bias subspace vB . For all words w ∈ W ,
w′ = µ+ r(w) + β · f(∥r(w)∥) · vB β = ⟨w, vB⟩ − ⟨µ, vB⟩
where µ is the mean embedding of words used to define vB , r(w) = w − ⟨w, vB⟩vB is the biasorthogonal component, and f(.) is a smoothing function which helps to remove unintended bias and keep definitional bias. We use f(η) = 1
(η+1)2
(see Dev and Phillips, 2019).

heading: 3.3 Static Word Embedding Benchmarks
text: Embedding Coherence Test Given a set of tuples A = {(a11, a21, ...), (a12, a21, ...)} where aij denotes the jth word for ith subgroup of an attribute (e.g., {(“father”, “mother”), ...} for binary gender), and a set of professions P = {p1, ..., pm}, the Embedding Coherence Test (ECT; Dev and Phillips, 2019) is the Spearman rank correlation between the rank order of cosine similarities of professions with each subgroup’s average embedding. Bias is completely removed when subgroups have identical ordering of associations with professions (ECT = 1).
Embedding Quality Test Word analogies are one of the main methods for evaluating word embeddings (Mikolov et al., 2013). The EQT (Dev and Phillips, 2019) quantifies the improvement in unbiased analogy generation after debiasing. Similar to ECT, EQT requires a set of word pairs A and a set of professions P . For each word pair (a+i , a − i ) the analogy a+i : a − i :: pj is completed, if the answer is pj or plurals or synonyms of pj (via NLTK; Bird et al. 2009), it is counted as unbiased. EQT is the ratio of unbiased analogies to all analogies. An ideal unbiased model would achieve EQT= 1 while lower values indicate a more biased model.

heading: 3.4 Proposed Method
text: To identify a group-agnostic bias subspace, we use the warmth and competence dictionaries from (Nicolas et al., 2021). To construct the poles of the dimensions, “high” and “low” word pairs (e.g., “able”–“unable” for competence and “sociable”– “unsociable” for warmth) were selected by downsampling to 15 word pairs, per dimension. We use word pairs for each SCM dimension to identify an SCM subspace (see Section 3.1), and subsequently apply the methods from Sec. 3.2.
We test whether SCM-based debiasing can substitute group-specific debiasing simultaneously for gender, race, and age. This is broken down into two related research questions. First, does SCM-based debiasing remove a comparable amount of bias relative to group-specific debiasing? And second, does SCM-based debiasing have more or less of a negative effect on embedding utility (Bolukbasi et al., 2016)? We compare SCM-based debiasing to group-specific debiasing using previous debiasing methods, specifically HD, Sub, LP, and PP (Section 3.2), and evaluate bias as measured by ECT and EQT following Dev and Phillips (2019). In addition, we evaluate the performance of each set of debiased embeddings on established word embedding benchmarks (Jastrzebski et al., 2017).

heading: 3.5 Results - Bias Reduction
text: We investigate whether SCM-based debiasing can simultaneously debias word embeddings with respect to gender, race, and age. For a given bias dimension, we established baselines by applying HD, Sub, LP, and PP using the respective word pair list (e.g., for gender bias we used gender word pairs), denoted with the subscript “same.” To place an upper bound on removed bias, we perform PP using gender, race, and age word lists (PPG+R+A).
For race and age we used the lists from Caliskan et al. (2017), while gender lists were taken from Bolukbasi et al. (2016). All methods were repeatedly applied using 30 different word pair samples, and we report each measure’s average and compare values using 95% confidence intervals. Implementation details are provided in the Appendix.
Table 1 shows the results of our experiments. Overall, SCM-based debiasing performs comparably to social-group-specific debiasing across methods. Specifically for ECT, SCM-based debiasing was either better than, or not statistically different from, LPsame and Subsame, while SCM-based debiasing was only slightly out-performed by PPsame (0.01–0.03). In other words, these results demonstrate that warmth and competence dimensions can simultaneously capture gender, race, and age bias in word embeddings. For the EQT, results are somewhat similar to those of ECT; however, we caution against interpreting small differences in EQT due to its definition of biased analogies relying on NLTK to compile comprehensive sets of synonyms and plural forms of words (Dev and Phillips, 2019).

heading: 3.6 Results - Word Embedding Utility
text: Table 1 shows that PPG+R+A outperformed all other methods on bias evaluations. However, one tradeoff is the reduction in word embedding utility. Table 2 shows that PPSCM preserves more embedding utility than PPG+R+A, using established benchmarks
for analogy and similarity (Jastrzebski et al., 2017). Due to the information removed in the debiasing process, as the number of social attributes increases, the quality of embeddings for group-specific debiasing deteriorates; however, this is not the case for PPSCM showing that PPSCM preserves some of the definitional biases (e.g. gender bias of actor vs. actress). These results indicate that our proposed approach for SCM-based bias mitigation is a better solution especially when our goal is to remove social biases for as many groups as possible.

heading: 4 SCM-Based Bias Mitigation for Contextualize Language Models
text: Similar to the previous section, before discussing our proposed method, we briefly review bias mitigation efforts in language models.

heading: 4.1 Methods of post-hoc bias mitigation
text: Similar to bias mitigation in static word embeddings, mitigating bias in language models requires a definition of the bias subspace. The bias subspace is defined via a set of attribute word tuples A = {(a11, a21, ...), (a12, a21, ...)} where aij denotes the jth word for ith subgroup of an attribute (e.g., for religion { (“Muslim”-“Jewish”-“Chrisitan”), ...}). In addition, bias mitigation for language models requires a set of neutral target words T (e.g., occupations such as “doctor”, “nurse”, etc.) Given a language model Mθ the goal is to find Mθ′ such that the difference in association of each neutral target word with all subgroups is minimized.
Our goal is to provide a theory-driven framework for bias mitigation that generalizes to many social groups. Therefore, we chose to focus on DPCE and ADEPT, two top-performing post-hoc methods that do not require significant hand-designed resource development, as our baselines.
Problem Definition: Both DPCE and ADEPT view bias mitigation as a downstream task and use a loss L = Lbias+Lrepresentation that balances bias mitigation – via Lbias – and preserves a model’s representational power via Lrepresentation. The formal definitions of Lbias and Lrepresenation are provided when we discuss ADEPT and DPCE. In addition, both algorithms collect a set of sentences Sw for each word w to capture the contextual representation of w. We use Ei(w, s; θ) to show the embedding of a word w in sentence s in the ith layer of the model parameterized by θ. Then the layer-prototype of a word w in ith layer of a model, ei(w), is defined as the average of Ei(w, s; θ) for all s ∈ Sw. The prototype of a word is then defined as the average of all layer-prototypes. DPCE: proposed by Kaneko and Bollegala (2021) is a fine-tuning approach for mitigating biases in a language model. Lbias for DPCE is designed to minimize the inner product of attribute word layerprototypes with embeddings of target words across all model layers.
Lbias = ∑
t∈T
∑
s∈St
∑ a∈A (ei(a) TEi(t; s; θe)) 2
Lrepresentation is defined to minimize the L2 norm of embeddings before and after debiasing across all layers.
Lrepresentation =
∑
s∈Sa
∑
w∈s
N∑
i=1
||(Ei(w, s; θ)− Ei(w, s; θ′)||2
ADEPT: Yang et al. (2023) proposed “A Debiasing PrompT” framework (ADEPT). Let ptj |ai quantify how much of attribute ai’s (e.g., male gender’s) information can be recovered from neutral target word tj . Also let P a i = [pt1|ai , pt2|ai , ...]. ADEPT’s Lbias is designed to reduce the difference, measured by Jensen-Shannon divergence, between relative distances of different attributes to neutral target words. Formally,
Lbias = ∑
i,j∈{1,...,d},i<j {JS(P a(i)||P a(j))}
Lrepresentation for ADEPT is defined to minimize the KL divergence of embeddings before and after debiasing across all layers.
Lrepresentation = KL(Mθ(S)||Mθ′(S))

heading: 4.2 Bias Benchmarks

heading: 4.2.1 SEAT
text: The Sentence Encoder Association Test (SEAT; May et al., 2019) is the extended version of the Word Embedding Association Test (WEAT; Caliskan et al., 2017), which places the WEAT words to the pre-determined sentences and computes the effect size and p-value. Effect sizes closer to zero indicate lower magnitude of bias.

heading: 4.2.2 CrowS-Pairs
text: Crowdsourced Stereotype Pairs (CrowS-Pairs; Nangia et al., 2020) evaluates whether a model gives a higher probability to the stereotypical sentences over the anti-stereotypical sentences. CrowS-Pairs test set consists of pairs of sentences that target explicit expressions of stereotypes by changing the stereotyped word with an anti-stereotype word. The ideal model should achieve a score of 50.

heading: 4.2.3 StereoSet
text: StereoSet (Nadeem et al., 2021) provides three scores, Stereotype Score (SS), Language Modeling Score (LMS), and Idealized CAT (ICAT) Score. For each item in Stereoset, the model should choose between stereotypical, anti-stereotypical or unrelated variations. The SS is the percentage of sentences in which the model prefers stereotypical ones. The LMS is the percentage of sentences the model prefers, stereotypical or anti-stereotypical sentences, over unrelated. The ideal model should get a SS of 50% and LMS of 100%. The ICAT combines the SS and LMS, and the ideal model should get 100% on ICAT.

heading: 4.3 Proposed Method
text: Similar to the SCM-based bias mitigation for static word embeddings, we propose that SCM’s warmth and competence can be used to define a socialgroup-agnostic bias subspace in language models. We operationalize the warmth and competence dimensions by 16 pairs of “high” and “low” words for each dimension from Nicolas et al. (2021). We hypothesize that similar to static word embeddings, SCM-based debiasing in contextualized language models will: 1. reduce bias for multiple social groups comparable to group-specific bias mitigation and generalizes to understudied social groups
or attributes (§4.4), and 2. maintain the expressiveness of language models (§4.5). To test our hypotheses, we mitigate bias with DPCE and ADEPT relying on SCM and compare it with social-groupspecific debiasing on gender and religion on multiple bias benchmarks (§4.2). In addition to bias benchmarks, we compare the models on the GLUE benchmark (Wang et al., 2018) to evaluate whether the debiased models have the same expressiveness as the original model.
We debias BERTLARGE (Devlin et al., 2019) with DPCE and ADEPT algorithms. All DPCE experiments were conducted on a GeForce GTX 3090 Ti GPU with the same hyperparameters reported in Kaneko and Bollegala (2021). Our experiments show that the DPCE algorithm is sensitive to the number of sentences collected for each at-
2https://www.statmt.org/wmt20/ translation-task.html.
3https://huggingface.co/datasets/bookcorpus
tribute and neutral word sentences. For each setting (gender, religion, and SCM), we run DPCE with |S| = {100, 500, 1000, 5000} sentences and chose the model with the highest overall ICAT score (|S| = 100 for religion and |S| = 500 for SCM and gender, see §B.3 for results). All ADEPT experiments use the same hyperparameters proposed by Yang et al. (2023) and were run on a 16GB Tesla V100. The original ADEPT algorithm was proposed for debiasing in a single bias dimension at a time, which we follow for each of the genderand religion-debiased models. For the SCM-based model, we adopt a coordinate descent-based iteration approach to debias the warmth and competence dimensions together: at each epoch, we first debias the warmth dimension with respect to the neutral words and then debias the competence dimension with respect to the neutral words (see §B.4).

heading: 4.4 Results - Bias Reduction
text: We compare variations of each algorithm (DPCE or ADEPT) separately to disentangle the effect of SCM-based debiasing from the debiasing algorithm (i.e., fine-tuning or prompt-tuning). We evaluate biases of our models using three benchmarks (§4.2) spanning four social groups/attributes of race, gender, religion, and profession. DPCE + SCM: On Stereoset, our results show that SCM-based bias mitigation achieves a higher ICAT score compared to group-specific debiasing for all categories of gender, profession, race, religion, and overall (Table 4). Table 3 shows our result on SEAT also reflects the same pattern. With the exception of S6, in all cases, SCM-based DPCE results in smaller effect sizes (or is insignificant). Finally, as shown in Table 4, for CrowS-pairs, SCM-based bias mitigation with DPCE achieves a better score on two of the three categories. Altogether, these results demonstrate that for DPCE, SCM-based debiasing can mitigate social biases on multiple social groups/attributes on par with or better than group-specific debiasing even when evaluated on the explicitly targeted group. ADEPT + SCM: SCM-based bias mitigation with ADEPT performs better than mitigating biases on religion in almost all cases for Stereoset, except for SS on gender. When compared to mitigating gender bias using ADEPT, SCM achieves comparable LMS, SS, and ICAT scores overall. Surprisingly, we observe that SCM-based deibiasing, compared to other models, achieves better scores on race on
SEAT. However, it is the gender-debiased model that achieves the best results on race for stereoset. This could be in part caused by the fact that race items in stereoset contain multiple instances of gender related stereotypes (e.g. “woman often notice how <blank> the egyptian men was” with “sexiest”, “generous”, and “lobster”). On CrowS-Pairs, SCM achieves better or comparable scores on race and religion to the best models, but mitigating biases on gender seems to be more effective for gender. As for SEAT, on average ADEPT with SCM gets lower effect sizes for race and comparable effect sizes for religion, but ADEPT with gender results in lower effect sizes on gender.
Across all benchmarks, our SCM-based approach consistently achieves a better score compared to at least one of the group-specific models. This suggests that our SCM-base approach is a better solution for minimizing the potential harms of language models for all social groups/attributes, and their potential intersections.

heading: 4.5 Results - Natural Language Understanding
text: Following Kaneko and Bollegala (2021), we used five tasks from GLUE benchmark, Stanford Sentiment Treebank (SST-2; Socher et al., 2013),
Microsoft Research Paraphrase Corpus (MRPC; Dolan and Brockett, 2005), Semantic Textual Similarity Benchmark (STS-B; Cer et al., 2017), Recognising Textual Entailment (RTE; Dagan et al., 2006; Haim et al., 2006; Giampiccolo et al., 2007; Bentivogli et al., 2009), and Winograd Schema Challenge (WNLI; Levesque et al., 2012). Table 5 shows that debiased models preserve their expressiveness. All models achieve comparable or better scores than the original model on all five tasks. We speculate that this is due to the Lrepresentation loss component in both algorithms. This analysis suggest that for LMs SCM-based debiasing not only doesn’t deteriorate the models but also improves their performance on some benchmarks.

heading: 5 Conclusion
text: In this work, we demonstrated the viability of a theory-driven approach to debiasing pre-trained word embeddings and language models. By replacing the ad hoc, social-group-specific component of existing debiasing techniques with a general, theory-driven, social-group-agnostic counterpart, we have solved two concrete problems with prior debiasing work and opened the door for more research into theory-driven approaches. First, SCMbased debiasing was shown to sufficiently reduce bias across social attributes, without relying on any manually constructed group-specific resources. Second, it is evident from our results that SCMbased debiasing is scalable with respect to generic social attributes – and embeddings or LMs debiased with respect to SCM can be thought of as generally debiased. Importantly, SCM-based debiasing results in improvements on the quality of the respective word embeddings or language models.

heading: 6 Acknowledgement
text: We would like to thank our anonymous reviewers for their feedback. This research was supported by NSF CAREER BCS-1846531 and DARPA INCAS HR001121C0165. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of DARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for governmental purposes notwithstanding any copyright annotation therein.

heading: 7 Limitations
text: The word embeddings and language models used in this work are trained on contemporary English language, and our social contexts overly contain explicit stereotypes encoded in English. Stereotypes for a specific group can be quite different depending on the language and culture. Although out of the scope of the present work, cross-societal differences in human stereotyping have been shown to be explainable using the SCM framework (Cuddy et al., 2009). Thus, it is fair to posit that our SCMbased framework generalizes to social group biases beyond those in English. Future research is encouraged to replicate our study in non-English languages.
Furthermore, we would like to point out that there exists a catalogue of bias measurements for
word embeddings and language models in the field. However, the current catalogue is far from comprehensive in covering social groups even in the contemporary English/American context, with few resources for the intersectionality of groups and attributes (Subramanian et al., 2021; Dhamala et al., 2021). Additionally, some of these measures have been shown to fail robustness checks. Although our current work uses some of the most recently developed ECT and EQT, we believe that few, if any, of these measurements are completely sound nor complete. In our experiments for language models, we tried to measure bias for the same social group or attribute using multiple benchmarks but still found some substantial differences in results across benchmarks. Therefore, we caution against interpreting low bias measurements as evidence of complete bias removal. While developing a new bias measurement scale is not within the scope of this work, we are optimistic that the social psychological theory in which our approach is grounded provides the bedrock for the current evidence of SCM efficacy to hold on future benchmarks.
Unlike bias mitigation methods for static word embeddings, such as partial projection, the post hoc methods of debiasing for large language models can’t be trivially applied to mitigate biases for multiple social attributes simultaneously. For DPCE, the formulation allows for mitigating biases on multiple social attributes, but collecting enough sentences from each attribute that do not include any words from other attributes or neutral words (i.e. mutually exclusive sentences) was not possible with the corpora we experimented with. This problem is exacerbated as the number of social attributes grow due to the mutual exclusivity condition. For ADEPT on the other hand, the formulation did not trivially handle multiple dimensions. Hence, we employed a coordinate-descent modification in our experiments to apply ADEPT to SCM (more info in §B.4). We encourage future work to devise data-efficient methods that can mitigate biases on multiple dimensions at the same time.

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
6
7 A2. Did you discuss any potential risks of your work? Our work is on a theory driven, generalizable bias mitigation for language models and in fact addresses some of the potential risks of previous methods of debiasing such as ignoring understudied social groups.
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 3 Did you use or create scientific artifacts?

heading: 3,4, appendix

heading: 3 B1. Did you cite the creators of artifacts you used?
text: 3,4,appendix
7 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? We used open-sourced or publicly available corpora, pre-trained word embeddings, and language models
7 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? We only tested pre-trained word embeddings and language models on common NLP benchmarks
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.

heading: 3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and
text: linguistic phenomena, demographic groups represented, etc.? 3, 4, appendix
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
C 3 Did you run computational experiments? 3,4,appendix

heading: 3 C1. Did you report the number of parameters in the models used, the total computational budget
text: (e.g., GPU hours), and computing infrastructure used? 3,4,appendix

heading: 3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found
text: hyperparameter values? 3,4,appendix

heading: 3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary
text: statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 3,4,appendix

heading: 3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did
text: you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 3,4,appendix
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Future research is encouraged to replicate our study in non-English languages. Furthermore, we would like to point out that there exists a catalogue of bias measurements for
word embeddings and language models in the field. However, the current catalogue is far from comprehensive in covering social groups even in the contemporary English/American context, with few resources for the intersectionality of groups and attributes (Subramanian et al. , 2021; Dhamala et al. , 2021). Additionally, some of these measures have been shown to fail robustness checks. Although our current work uses some of the most recently developed ECT and EQT, we believe that few, if any, of these measurements are completely sound nor complete. In our experiments for language models, we tried to measure bias for the same social group or attribute using multiple benchmarks but still found some substantial differences in results across benchmarks. Therefore, we caution against interpreting low bias measurements as evidence of complete bias removal. While developing a new bias measurement scale is not within the scope of this work, we are optimistic that the social psychological theory in which our approach is grounded provides the bedrock for the current evidence of SCM efficacy to hold on future benchmarks. Unlike bias mitigation methods for static word embeddings, such as partial projection, the post hoc methods of debiasing for large language models can’t be trivially applied to mitigate biases for multiple social attributes simultaneously. For DPCE, the formulation allows for mitigating biases on multiple social attributes, but collecting enough sentences from each attribute that do not include any words from other attributes or neutral words (i. e. mutually exclusive sentences) was not possible with the corpora we experimented with. This problem is exacerbated as the number of social attributes grow due to the mutual exclusivity condition. For ADEPT on the other hand, the formulation did not trivially handle multiple dimensions. Hence, we employed a coordinate-descent modification in our experiments to apply ADEPT to SCM (more info in §B. 4). We encourage future work to devise data-efficient methods that can mitigate biases on multiple dimensions at the same time