Bi-directional Bias Attribution: Debiasing Large Language Models without Modifying Prompts
Authors: Yujie Lin
Kunquan Li
YiXuan Liao
Xiaoxin Chen
Jinsong Su
Venue: ICLR 2026 Poster (2026)
Keywords: Large language models; Algorithmic fairness; Social bias
Primary area: alignment, fairness, safety, privacy, and societal considerations
TL;DR: Mitigating social bias in large language models without modifying prompts or finetuning.

Abstract: Large language models (LLMs) have demonstrated impressive capabilities across a wide range of natural language processing tasks. However, their outputs often exhibit social biases, raising fairness concerns. Existing debiasing methods, such as fine-tuning on additional datasets or prompt engineering, face scalability issues or compromise user experience in multi-turn interactions. To address these challenges, we propose a framework for detecting stereotype-inducing words and attributing neuron-level bias in LLMs, without the need for fine-tuning or prompt modification. Our framework first identifies stereotype-inducing adjectives and nouns via comparative analysis across demographic groups. We then attribute biased behavior to specific neurons using two attribution strategies based on integrated gradients. Finally, we mitigate bias by directly intervening on their activations at the projection layer. Experiments on three widely used LLMs demonstrate that our method effectively reduces bias while preserving overall model performance.

Published as a conference paper at ICLR 2026
BI-DIRECTIONAL BIAS ATTRIBUTION: DEBIASING LARGE LANGUAGE MODELS WITHOUT MODIFYING PROMPTS
Yujie Lin' Kunquan Li Yixuan Liao"
Xiaoxin Chen²
Jinsong Su
1 School of Informatics, Xiamen University, China Vivo AI Lab, China 3 Keyy Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China [linyujie, Sete,iike likunqquan] @stu xmu edu. en, jssu@xmu. edu.
ABSTRACT
Large language models (LLMs) have demonstrated impressive capabilities across wide range of natural language processing tasks. However, their outputs often exhibit social biases, raising fairesss concerns. Existing debiasing methods, such as fine-tuning on additional datasets or prompt engineering, face scalability is- sues or compromise user experience in multi-turn interactions. To address these challenges, we propose framework for detecting stereotype-induccin words and attributing neuron-level bias in LLMs, without the need for fine-tuning prompt modification. Our framework first identifies stereotype stereotype-inducing inducing adjectives and nouns via comparative analysis across demographic groups. We then attribute biased behavior to specific neurons using two attribution strategies based on in- tegrated gradients. Finally, mitigate bias by directly intervening their acti- vations at the projection layer. Experiments on three widely used LLMSs demon- strate that our method effectively reduces bias while preserving overall model performance. Code is available at the github link: / com/ SMUDeeppi11/Bl-cdreett.ooo
INTRODUCTION
Large language models (LLMs) (Achiam al., 2023 Dubey et al. 2024 have achieved remark- able performance across a wide range of natural language processing tasks. However, mounting evidence shows that these models can perpetuate and even amplify societal biases, such as gender, racial, religious, and occupational stereotypes (Nadeem et al. 2020). Such biases become especially problematic when LLMs are utilized critical applications, including content generation, decision- support systems, and interactive dialogues (Liang et al.) 2021 Parrish al. 2021 Gallegos et al.) 2024a). As LLMS grow in scale and generalization capacity, understanding and mitigating their internal sources of biased behavior becomes increasingly critical.
During the period of masked language models (Devlin et al. 2019 Liu et al., 2019). some approaches attempted to mitigate bias by fine-tuning models using existing synthesized datasets (Liang et al.] 2020) Guo et al. 2022). However, with the advent of large language mod- els, such methods have become increasingly impractical due to their substantial demands on time and computational resources. To address these limitations, recent efforts primarily focus on prompt- based debiasing, such explicitly instructing the model avoid relying on certain biased attributes in its response (Furniturewala et al., 2024). or analyzing the initial output to identify bias patterns before prompting the model to answer again (Gallegos al.) 2024b] Li et al. 2024). Nonetheless, modifying user prompts may negatively impact user experience, especially in multi-turn interactions where repeated rewriting significantly increases context length and inference cost. These challenges motivate us to develop debiasing approach that requires neither model fine-tuning nor prompt modification.
In this paper, we propose framework for stereotype detection and bias attribution in LLMs, with the goal of identifying biased neurons and applying interventions an interpretable man-
& Corresponding author.

---

Published as a conference paper at ICLR 2026
ner. In this work, we define stereotype cues as adjectives nouns that obviously induce skewed predictions toward specifie demographic groups. For example, when no additional gender infor- mation is provided in the context, LLMs may tend to associate doctor with being male; in this case, "doctor" serves as a stereotype cue. Our framework is built on two key stages: (i) Stereo- type Cue Selection via Entropy Minimization. By constructing sentence templates and computing entropy over the model's predicted distribution across demographic groups, we identify the most bias-inducing cues in a model-specific and attribute-specifie way. (ii) Forward and Backward Bias Attribution via Integrated Gradients. To trace biased outputs back to specific neurons in the LLM, we design two attribution strategies. The Forward-IG strategy is to construct prompts in which the subject contains unknown demographic group, and let the LLM predict the demographic group. Forward-IG quantifies neuron-level bias contributions when the LLM predicts skewed demographic groups from stereotype-laden prompts. Conversely, the Backward-IG strategy is to construct a se- ries of sentence subsets, where each subset contains sentences whose subjects belong to different demographic groups, in order to examine the relationship between the model's outputs and demo- graphic information. Backward-IG identifies neurons that drive differences in generated outputs across demographic groups. Overall, these two attribution strategies provide parallel perspectives: Forward-IG captures neuron contributions when the model infers demographic groups from stereo- type cues, while Backward-IG highlights neurons responsible for group-dependerd disparities in generated text. Together, they enable a comprehensive identification of bias-related neurons that directly shape the model's outputs. After identifying biased neurons, we intervene by fixing their activation values at the projection layer. the final layer before token prediction. By combining attri- bution and intervention, our framework offers a comprehensive pipeline for debiasing large language models at the neuron level. This contributes to the broader goal of building more trustworthy LLMs.
To summarize, our main contributions are summarized as three-fold:
We introduce entropy-based method to identify stereotype cues that elicit biased model behavior, covering both adjective and noun forms.
We propose Forward-IG and Backward-IG, two gradient-based attribution strategies for identifying neurons responsible for biased generation, respectively. Then we present an effective intervention that directly modifies the projection layer activations. improving fair- ness with minimal degradation to model performance. Moreover, we theoretically establish the intrinsic connection between bias reduction and output variation.
We conduct extensive experiments across four demographic attributes using three widely- used LLMs, providing insights into internal bias mechanisms.
2 BACKGROUND
In this section, we first decompose debiasing LLMs into two distinct subproblems, and then provide a brief overview of the attribution method IG and its bias attribution variant IG².
2.1 PROBLEM DEFINITION
Definition 1 (Demographic-Innaatee Invariant Generation (DIG)). Let X be the prompt space, y the output space, D the demographic attribute space (e.g., gender, race), and 0 parameterizes a language model inducing Poly x) over Let g:D X be a prompt generator that injects demographic information d E 'D into prompts (e.g.. "Her mother was very... ... or "Ethiopian men are ..."). We say the model satisfies demographic-invarre invvrrian generation if:
Poly 9(d)) Poly 9(d')) Vd, e D.
(1)
That means the model's output distribution should remain approximately unchanged when only the demographic information in the prompt varies.
Definition 2 (Stereotype-Free Free Inference (SFD). Let x E X be a prompt containing stereotype cues (e.g., words like "doctor", "nurse", " "CEO") which may be related to demographic attributes. Given the demographic label space D, the model's conditional prediction of demographic identities from such prompts should not be biased. We say the model can address stereotype-free inference if:
Po(d] x) R Po(d') | 2x) Yd, d' E DD
(2)
where Po 2) is the probability of the model associating x with the demographic group d (e.g., by predicting "man" ""mmann"" " for "The doctor is likely a.").
2

---

Published as a conference paper at ICLR 2026
In other words, the model should not systematically favor one demographic another when interpreting stereotype- sereottpe--rone prone prompts.
2.2 INTEGRATED GRADIENT AND INTEGRATED GAP GRADIENT
This section details two feature attribution methods: Integrated Gradient and Integrated Gap Gra- dient, with the latter specifically designed for bias analysis in language models.
Integrated Gradients (IG) Sundararajan et al. (0177) attribute model predictions input features by integrating gradients along a straight path from a input baseline x' to the input æ. For a model F Rd R, the attribution score for the i-th feature is
aF('' + +a(x - x²)) da.
IG(z))
(x: of.
ēx;
<==
(Z+)) represents the contribution of the i-th input feature to the model's prediction F(z) relative to baseline z'. Here, the term x' captures the magnitude of change in the feature from the baseline, while the integral computes the average gradient of the model's output with respect to x; along the straight-line path between x' and x. This approach ensures that the attribution is sensitive to both the scale of the feature variation and the model's 's response to incremental changes in the input.
Integrated Gap Gradients (IG?) Liu ct al. (2024) extend the idea of IG to analyze the internal mechanisms responsible for biased behaviors in language models. While IG attributes the output of a model to its input features, IG² instead attributes the prediction gap between binary demographic
pairs (e.g., female vs. male) to internal neurons, enabling the identification of social bias neurons.
Formally, given a pair of demographics d1 and d2, and the j-th neuron ()) in the I-th FFN layer of
model and h (0's initial activation F.g IG? computes the attribution score
(1)) 1 P(d a P(dz aai = S
(0²(1,0)= 10
da,,
(4)
ahg
where P(d: (1)) denotes the model's prediction probability for demographic d; when neuron
h takes the value an; This formulation directly attributes the difference in model confidence between demographic groups to individual neuron activations, revealing their contribution to biased behavior. These identified neurons, termed social bias neurons, can then be suppressed to miti- gate bias without requiring model retraining. However, although IG² has demonstrated success on masked language models such as BERT (Devlin et al., 2019). applying this neuron-suppression based debiasing approach to modern large language models still faces several challenges. First, in the lower layers of deep language models, the contribution of individual neuron activations to the final output tends to be marginal, as their influence is increasingly transformed and potentially suppressed by the model's 's subsequent non-linear operations. This constraint undermines the effec- tiveness of interventions aimed at modifying the model's token generation probabilities. Second, as shown in Equation 4 IG² can only capture bias relationships between specific demographic pairs (e.g., predefined biased pairs such as "driver-doctor" or "waiter-lwyerr". However, cross-par ppis-pair bias relationships (e.g., between "driver" and "waiter")) are not considered, which may lead unreliable attribution of model bias. Additionally, key unresolved issue is how systematically identify input words that reliably trigger biased responses, as such triggers are crucial for enabling precise bias attribution and improving the efficacy of debiasing strategies. To handle these challenges, our goal is to design an effective solution that jointly tackles the DIG and SFI problems.
3 METHODOLOGY
In this section, we describe our debiasing method in detail. As shown in Figure I our method mainly consists of the following steps: stereotype cue selection and two attribution strategies.
3.1 STEREOTYPE CUE SELECTION
In this work, we define "stercotype cues" as adjectives or nouns that are likely to trigger biased model outputs. Unlike (Guo et al. 2022) where both demographic attributes and stereotype cue
3

---

Published as a conference paper at ICLR 2026
Stage 1: Stereotype Cue Detection
Predict the demographic group via LLMs
Candidate Biased Words
Fill in the templates
e.g. "aggressive", "ambiftous" : "anatyrical" "assertive", "aulloritative" , "bold", "boxsy". "brilfiom"" : "caring". , "charminng.
Calculate eutropy and select top-K stercotype caes
egg "gentle", "nurturing" "sweer" "xenvitii"" "soff"
e.g, the gender of this aggressive person is |Demographic Group!
Stage Bias Attribution and Modifying Biased Neurons
Fill in the templates with stereotype cues
Caculate Forward-IG LLM's outputs
Modify the activations of biased neurons
probubilitios denognahic groups
e.g. the gender of this gentle person is (Demographic Group)
Figure I: Overview of our method (illustrated with Forward-IG). We first identify the words that trigger biased behavior in the model, then use these words to elicit such behaviors. Based on this, we attribute the biased responses to the most influential neurons and subsequently modify their values. In the bottom-right figure, the gray neurons denote the bias-related neurons after modification. The bar chart presents the debiasing performance of Llama-3.1 on StereoSet. The x-axis corresponds to four types of bias, while the y-axis represents the SS score, where values closer to 50% indicate greater fairness. Our method (gray bars) achieves results demonstrates improved faimess. Table 1: Examples of templates for two types of stereotype cues.
Category
Template Examples
Adjective
The (Demographic Attrrbbute of this (Stereotype_ Adfjectv person is DDemographi__rroug The [Demographic Attribute] of this [Sterentype Noun] is [Demographic Group].
Noun
words are predefined before generating the connecting tokens, our work focuses automatically identifying the cues that most effectively elicit model biases. Different from (Liu et al., 2024) that defines set of adjective-based templates, to cover noun-based constructions as well. Table | provides examples of the templates, and the
modify some of these templates and extend them
complete list can be found in Appendix AA.8 We first utilize GPT-4 (Achiam al. (023) help us identify adjectives that are potentially associated with various stereotypes. These adjectives and nouns are then used to construct the candidate list of stereotype cues.
Entropy-Baset Bias Quantification. The core intuition is that a stereotype cue exhibits stronger bias induction it causes the model to generate highly skewed predictions in favor of specific demographic groups. We access this via Shannon entropy over the model's conditional probability distribution over demographic groups. Formally, given a candidate stereotype cue w (adjective or noun) and a set of demographic groups D = {di, d2....). we compute entropy H (Pagg) where denotes the average of pl d& Replace(t, (1, w)) computed over all templates. Here, p(d.) Replaccel w)) represents the model's predicted probability distribution of demographic group d, given prompts containing w and Reploce (t, w) denotes the sentence constructed by inserting the cue w into a predefined template t. Lower entropy values indicate more concentrated probability distributions, signaling stronger bias induction by the cue w.
Cue Selection. The stereotype cue selection process involves four key substages (Appendix A.5) as outlined below: (i) Candidate Pool Initialization. We first collect the candidate lists of adjectives and nouns, ensuring these words are commonly used expressions that are likely induce model bi- ases. These adjective and noun lists are denoted as Vadi (adjectives) and Vnown (nouns), respectively. (ii) Probability Collection. For each candidate cue &Vddi U Vnoon. we generate prompts using the templates specified in Table I (with [Stereotype Adjective] or [Stereotype Noun] placeholders replaced by w)). For each generated prompt, we query the language model to obtain predicted prob- abilities over the demographic groups. This is done by constraining generation to the set D and extracting softmax probabilities from the model's final layer. (iii) Aggregate Entropy Calculation. For each cue w, we compute the average probability distribution across all templates, and calculate the entropy of this aggregated distribution. This averaging mitigates complate-speciiii noise, ensur- ing robust bias assessment. (iv) Ranking and Selection. Cues sorted by their entropy values in the ascending order. We conduct stereotype cue selection on the four demographic attributes in the StereoSet dataset. Table 2 reports the top five cues with I. 1lama-3.1 (Dubey al. 2024).
4

---

Published as a conference paper at ICLR 2026
Table 2: Selected cue examples across four demographic attributes.
Demographies
Types of Cues
Top-5 Selected Cues
Adjective
gentle, nurturing, sensitive, soft, sweet cheerleader, nurturer, breaker-breaker, confidante, delegate warmongering, imperialist, tribal, underdeveloped, primitive commissar, chauvinist, shaman, propagandist, tribesperson calculating, sensitive, empathic, anxious, emotional robot, assistant, counselor, bureaucrat, sidekick extremist, ascetic, monastic, evangelical, self-denying meditator, pietist, exorcist, extremist, apostle
Gender
Noun
Llama-3.1
Adjective
Nationality
Noun
Adjective
Profession
Noun
Adjective
Religion
Noun
3.2 FORWARD BIAS ATTRIBUTION
This paper refers to the causal direction consistent with the SFI problem (from prompts to demo- graphics) as the forward direction, where the input prompt contains stereotype cues and the model asked to predict which specific demographic group the sample belongs to. The direction asso- ciated with the DIG problem is referred to as the backward direction. We first replace the corre- sponding slots in the templates with the demographic attribute terms and the selected stereotype cues in Section 3.1 generating sentences such as "The gender of this sensitive person is (Demo- graphic_Group) I" These sentences collectively form synthetic dataset DS For each sample in DS,, we construct a corresponding prompt (see Appendix A. and use it to prompt the model to predict the sample's demographic group. To effectively improve the fairness of the model's output probabilities, we attribute the model bias to the input neurons of the projection layer in the LLM (i.e., the layer that maps high-dimensional representations to logits). This allows us to identify bias-related neurons and intervene accordingly. Specifically, for the j-th input neuron hy of the pro- jection layer and its initial activation value hj, propose Forward-IG to quantify the variation in the outputs across all demographic groups for the neuron hy:
[[(p(d(ah)) da,
Forward-IG(h,) hh Jn-0
(5)
Ōhg
where H(-) denotes the entropy function, and a E [o, 1] a scaling variable that gradually changes the value of neuron hy from 0 its original activation hy. The smaller H Hpp(d ah₂)) is, the larger (p(d ah,))] becomes, indicating that the model is certain about which specific demo- graphic group the sample belongs to. Such strong certainty toward particular demographic group reflects the model's bias. By integrating the gradients, Forward-IG accumulates this certainty along the interpolation path, thereby quantifying the contribution of each neuron to biased predictions. Since the integral in Equation (3) cannot computed analytically, we follow the approach of IG and IG2 and approximate it using a Riemann sum:
Ratey [[H(p(d axh;)))
Forward-IG(h,)
73 stpp
where os Katep and Rstep is the number of approximation steps. Note that Forward-IG is com- puted only for the bias contribution of neurons with respect to a single sample in DSy. Therefore, we identify biased neurons based on the average Forward-IG scores across all samples in DS,. To access this, we first rank all neurons in the descending order according to their average Forward-IG values. Then, we seleet the top N 3M neurons, where M is the total number of neurons in the relevant layer of the model and / [o, 1] is a proportion parameter. Once the bias neurons are selected, proceed disrupt their activation values. In this work, we fix the values of these bias neurons constant C. Mathematically, for each neuron h2, its updated value hy is defined
hy C, if hy is the top-N neurons
(n otherwise.
This operation effectively breaks the contribution of these neurons to the model's 's biased behavior. This approach of disrupting bias neurons' activation values based on the Forward-IG value selection provides a practical way to address both DIG and SFI problems in large language models, which is further validated in the subsequent experimental section.
5

---

Published as a conference paper at ICLR 2026
Table 3: Examples of generated sentence sets for each demographic group.
Demographies
Demographic Group
Sentence Subset
The gender of this [Stereotype_Nove is male. The gender of this [Stereotype_Nore is female.
Gender
Male, Female
3.3 BACKWARD BIAS ATTRIBUTION
Backward bias attribution identifies biased neurons through induced differences in the model's generation outputs, which occur in response to prompts containing different demographic groups. Specifically, for given template and a fixed demographic attribute, we construct a subset of sen- Bermogrpphie.Grouplll tences as shown in Table where the placeholder token [Demographic. Group] in & is replaced with na demographic groups associated with that attribute. All sentence subsets constitute the dataset DSs where each subset contains Nd sentences. For each subset in DS,. we construct nd prompts to predict the stereotypical adjectives or nouns within the sentences. The candidate options the stereotype cues selected in Section 3.1 We aim to identify the biased neurons responsible for the model producing different probability distributions over stereotypical cues for different groups. For each sentence subset, we compute Backward-IG I-G in the following:
aJSD(pI (wllh,)) .... Pno (w) (u)))))) da,
(ack hg
(7)
a-=
where (.)(()) denotes the Jensen-Shannon Divergence and w denotes the stereotype cue. (.) quantifies the divergence among probability distributions over stereotype cues and is formulated in Appendix AA.4 The Backward-IG score quantifies how much neuron's 's activation contributes to disparities in model outputs across demographic groups, measured via JSD over the predicted distributions of stereotype cues. As in the forward case, we interpolate neuron activations from zero to their original values and accumulate the gradients of JSD along this path to estimate each neuron's 'ss contribution. After obtaining Backward-IG scores for all neurons, we calculate average scores over subsets in DS, and select the top N as biased neurons. Similar to forward attribution, these neurons are then intervened on by fixing their activation values to a constant C, effectively neutralizing their influence on group-dependent output variation. Backward-IG also uses Riemann sum approximation.
3.4 THE RELATIONSHIP BETWEEN BIAS VARIATION AND OUTPUT VARIATION: A THEORETICAL ANALYSIS
Theorem 1 (Bias Change under Autribution-Guided Modification). Let y denote the model output of the projection layer Proj(-) and B R* R be a differentiable bias function, such as the reciprocal of entropy or the Pensen-Shamme divergence introduced above. Suppose the hidden representation h is modified along the path:
M((1) = hhhhh, I [0,
(8)
with Ah defined by citributiio-guuidee projection on a subset of neurons S. Then the change in bias satisfies
|AB| < ())( + 62g)] All.. Ae [o, 1).
(9)
where Ay y1)=(1)- (0) and y(t Proj (7+ +aA) t [o, 1] lies along the modification path.
Equation 9 aligns with our intuition: the larger the Ay. the greater the upper bound of AB. In the extreme case where the model completely loses its modeling capability, it produces random outputs for inputs from any demographic group. thereby exhibiting minimal bias. Specifically, the bias change AB equals the directional projection of the output shift Ay onto the local bias gradient V (gy(0)) + 0Ay). A detailed proof of Theorem can be found in Appendix A.6
4 EXPERIMENTS
In this section, we conduct experiments two debiasing strategies. For brevity, refer to the strategies based on Forward-IG and Backward-IG as forward bias attribution (FBA) and backward bias attribution (BBA), respectively.

---

Published as a conference paper at ICLR 2026
Table 4: Similarity between selected cues or all candidates and gender-associated vocabularies.
Llama-3.1
Llama-3.2
Adjective
Simto (%)
Simn (%)
Difl (%)
Simm (%)
Sinr (%)
Diff (%)
Top-5 Selected Cues
8.92
4.60
4.32
6.04
7.21
6.44
All Candidates
6.37
4.68
2.79
7.76
11.07
6.11
Noun
Simm (%)
Sim; (%)
Diff (%)
Simm (%)
Sime (%)
Diff (%)
Top-5 Selected Cues
7.60
6.97
3,36
8.23
15.07
9.41
All Candidates
7.37
5,60
2.89
8.55
12.14
7.20
4.1
EXPERIMENTAL SETTINGS
We conduct experiments on three widely used large language models, Llama3. (Llama-3.1) 3.1),
Llama3.2-3B (Llama-3.2) (Dubey et al.) (024) and Mistral-7B-v0.3 (Mistral-v0.33 (Jiang et al.]
2023 Biased neurons are identified and perturbed using Forward-IG and Backward-IG, respec- tively. We evaluate the effectiveness of both attribution methods on the DIG and SFI tasks. Due to space constraints, all results Mistral-v0.3 are presented Appendix A.7
Baseline Methods. We categorize the basclines into three types:
training- based
method: Auto-Debias (Guo et al.] 2022); (ii) prompt engincering-
methods, including Prefix Prompting (Furniturewala et al. 2024), Self-Debiasing (Gallegos al. (024b), and DDP (Li
al. 2024); and (iii) a neuron attribution-based method: IG (Liu et al.
2024). Detailed descrip-
tions of all baseline methods are provided in Appendix A.3
4.2 EVALUATION OF STEREOTYPE CUE SELECTION
We design the stereotype- cue selection procedure by grounding in the target model's own em- bedding space rather than relying preconceived human assumptions about stereotypical lan- guage. Using gender bias as an illustrative case, we define the male-associated vocabulary as Wm, m {"male", "man"), and the female-associated vocabulary as W, ("female", "woman") For the top five terms identified by the stereotype-cue selection method, we compute their average cosine similarity with the male and female vocabularies, denoted by Simm and Sim,, respectively. We additionally calculate the average absolute difference between these two similarities,
DifT
cos(es, Wm. - cos(oee ei, Wt),
No
where e, denotes the embedding of the i-th selected term and N, debotes the number of candidate words. It can be observed in Table|4that the selected terms exhibit larger Diff, indicating that each term is more strongly associated with either male- or female-related vocabulary. This suggests that cue selection method effectively identifies terms that elicit more biased model behavior.
4.3 DIG TASK (STEREOSIT)
StereoSet (Nadeem et al. 2020) is a benchmark designed evaluate stereotypical bias in pre- trained language models. It covers four demographic domains: gender, nationality, profession, and religion. The dataset consists of original sentence and option triplets, each composed of (i) stereo- type option that reflects a common social bias, (ii) an anti-stereotype option that counters the bias, and (iii) an unrelated option that is semantically incoherent or irrelevant. The primary task is sess whether language model favors stereotypical content over unbiased or unrelated alternatives. thereby quantifying bias while controlling for contextual relevance. For example, tuple from the gender domain might include:
Original template: Girls tend to be more [BLANK] than boys. Stereotype: soft Anti-Stereotype: determined Unrelated: fish
Evaluation Metrics. We employ three complementary metrics: SS, LMS, and ICAT. SS measures the proportion of instances where the model prefers the stereotype option the anti-stereotype

---

Published as a conference paper at ICLR 2026
Table 5: Evaluation results across four demographic attributes on the StereoSet dataset.
Gender
Nationality
Profession
Religion
Llama-3.1
SS (%) - 50%
LMS (%) 1
ICAT (%) f
Base
7 100.0 / 45.31
68.43 1443/98.117 61.95
76.96 /97.53 / 44.94
56.94 / /9.4//9.14/// /
Auto-Debias
79.69 100.0 / 40.62
69.07 0 /98. 13 60.71
77.33 / /9.002 44,44
70.08 99,114 54.81
Prefix Prompting
81.89 99.22 35.94
65.67/97.55
66.94
73,42 /9.72/97.73// 51.85
54.79 / /92.41 83.54
Self-Debiasing
77.97 / 99.19 40.63
65.24 /9. 64.03
69.17 /9..00/22//// 56.79
63.38 89.87 65.82
DDP
77.17 99.22 45,31
65.45/ / 96.88 66.94
73.91 / 96.54/ 50.37
64.18 84.81 60.76
IG
77.34 / 100.0 / 45.31
69.64 /97.92 59.46
76.52 / 97.78 45.93
61.64 / 92.41 / 70.89
FBA
68.75 100.0/ 62.50
64.88 / 99.09 68. 19
67.87 / 96.05 / 61.73
51.35 / 93.67 / 991.14
BBA
69.84 /98.44/ 59.38
63.18/ 95.43 / 70.27
71.58 95.56 / 54.32
49.31 92.41 99.114
Gender
Nationality
Profession
Religion
Llama-3.2
SS (%) - 50%
LMS (%) 1
ICAT (%)
Base
82.40 / 19..66 34.38
66.31 96.88 65.28
70.89 99.53 56.79
57.89 / /9. 81.01
Auto-Debias
80.00 97.66 39.06
65.59 96.67 / 66.53
70.38 99.53 57.78
59.21 / 96. 78.48
Prefix Prompting
79.20 97.66 40.62
65.08 / 95.84 66.94
67.69 / /2.22 62.22
56.58 / 96.20 83.54
Self-Dehiasing
73.11 /92.97 50.00
72.17 / 18.15//1 49.06
63.94 / 63.21
64.06 / 81.01 / 58.23
DDP
77.17 /9.221 45.31
70.57 /9..51 57.38
71.13 /97.53 56.30
63.51 / 93.67 68.35
IG
44.71 / 66.41 / 59.38
64.29 / 96.05 / 68.61
68.45 / 97.04 61.23
55,26 / /96220 86.08
FBA 67.46 98.44 64.06
69.60 / 97. 97.66 / 59.38
58.52 / 95.22 79.00
61.88 / 94.57 72.
51.95 / 97, 46 93.67
BBA
62.93 96.47 / 71.52
62.21 / 96.05 72.59
53.33 / 94.94 88.61
one. A higher SS indicates a stronger tendency to favor stereotypical associations. Ideally, a fair model should have SS close to 50%, suggesting systematic preference for either stereotypes or anti-stereotypes. LMS evaluates the model's ability to prefer meaningful content over incoherent or irrelevant options. A higher LMS reflects better language modeling capability. A desirable model should have an LMS close to 100%, indicating that it consistently favors contextually relevant com- pletions over unrelated ones. ICAT integrates both fairness and fluency by rewarding models that maintain low stereotype bias while preserving high linguistic coherence. The optimal ICAT score is 100%, which would indicate perfect fairness and fluency.
Overall Performance. From Table 5 we observe that on Llama-3.1, baseline methods are largely ineffective in mitigating model bias and even exacerbate it in certain domains. On Llama-3.2, some prompt modification approaches begin to show partial effectiveness, but only Prefix Prompting con- sistently reduces bias across all domains, and the reduction is marginal. Notably, the IG method is effective only on Llama-3.2, where it substantially lowers LMS in the gender domain to bring SS closer to 50%. However, such behavior is unacceptable in practical applications. In contrast, FBA and BBA achieve effective bias reduction while incurring little to no loss modeling capability, ultimately yielding the best overall performance as reflected by the highest ICAT scores.
4.4
DIG TASK (BBQ)
BBQ (Parrish et al. 2021) is a large-scale benchmarking resource designed to evaluate social bias and robust reasoning in question-ansee answering (QA) systems. Developed probe how QA models handle sensitive demographic attributes, BBQ focuses on whether models rely on stereotypical assumptions or demonstrate contextually grounded reasoning.
Evaluation Metries. BBQ includes two types of questions: those posed under ambiguous context and those under disambiguated context. In the ambiguous setting. models are expected to select the "unknown" option rather than rely on demographic stereotypes. In the disambiguated setting, models should choose the correct answer based on the explicit contextual evidence. Evaluation conducted using accuracy on the ambiguous questions (Accamb) and on the disambiguated questions (Accas). Since BBQ does not include a dedicated profession domain, we conduct evaluations on the gender, nationality, and religion domains. The results for each domain are provided in the appendix, and here we report the averaged performance across these three domains. BBQ includes two types of questions: those posed under ambiguous context and those under disambiguated context. In the ambiguous setting, models are expected to select the "unknown" option rather than rely on demo- graphic stereotypes. In the disambiguated setting, models should choose the correct answer based
8

---

Published as a conference paper at ICLR 2026
Table 6: Evaluation results on the BBQ dataset.
Llama-3.1
Llama-3.2
AcCamb
Accais
Average
AcCamb 1
Accais
Average
Base
54,50
91.18
72.84
58.95
84.61
71.78
Auto-Debias
54.61
91.06
72.84
58.53
84,62
71.58
Prefix Prompting
75.13
86.08
80.61
82.81
69.03
75.92
Saf-Debiasing
87.73
73.02
80.38
66.06
74.17
70.11
DDP
56.87
89.93
73.40
60.38
83.40
71,89
IG²
59.00
89,46
74.23
52.61
75.84
64.22
FBA
66.45
88.87
77.66
68.78
81.59
75.19
74.46
88,95
81.70
68.08
79.95
74.02
Table 7: Evaluation results the WinoBias dataset.
Llama-3,1
Llama-3.2
P stereo
Panti
Pother
Cap
P Sterren
Panti
Pother
Gap |
Base
62.63
37.37
0.00
25.26
95.71
4.29
0.00
91.42
Auto-Debias
61.49
38.51
0.00
22.98
96.46
3.54
0.00
92.92
Prefix Prompting
58.08
41.92
0.00
16.16
96.21
3.79
0.00
92.42
Self-Debiasing
85.61
14.39
0.00
71.22
88.64
11.11
0.25
77.53
IG
57.32
42.68
0.00
14.64
43.18
43.43
13.39
0.25
FBA
52.53
47.47
0.00
5,06
59.34
40.66
0.00
18.68
BBA
50.51
49.49
0.00
1.02
51.52
48.48
0.00
3.04
on the explicit contextual evidence. Evaluation is conducted using accuracy the ambiguous ques- tions (A (Accamh) and on the disambiguated questions (Accaи). Since BBQ does not include a dedicated profession domain, conduct evaluations the gender, nationality, and religion domains. The results for each domain are provided in Appendix A.7 and here we report the averaged performance across these three domains.
Overall Performance. As shown in Table 6 prompt-modification approaches (e.g., Prefix Prompt- ing, Self-Debiasing) boost Accamb but significantly undermine model accuracy when the context unambiguous, resulting in sharp drop in Accais In comparison, FBA and BBA reduce bias effec- tively in ambiguous scenarios with only minimal impact on Accais- This indicates that our method is more moderate: when clear contextual guidance is available, still tends to produce the correct answer rather than forcibly selecting the debiased "unknown" option.
4.5 SFI TASK (WINOBIAS)
WinoBias (Zhao et al., 2018) is dataset designed measure gender bias. We adopt cloze- style version of WinoBia to evaluate the SFI task) Specifically, WinoBias requires the model to predict the demographic subject (e.g., he she) or modifier in sentences like "The developer argued with the designer because [MASK] did not like the design". Specially, since DDP's prompt construction is not applicable this dataset, it is not adopted in the SFI task.
Evaluation Metrics. Similar to StereoSet, WinoBias also provides stereotype option and an anti-stereotype option. We denote the probabilities of the model selecting these options as Pstereo and Panti. respectively, while Pather = Pstereo Panti represents the probability of selecting any other token. lower Pother indicates better language modeling capability, and smaller gap between Pstereo and Panti reflects greater fairnsss in the model's behavior.
Overall Performance. Table 7 shows that BBA, while keeping Pater 0 (i.e., without sacrificing language modeling capability), achieves the best and second-best gap values on the two models, re- spectively. We find that, compared with the DIG task, IG demonstrates promising effectiveness on the SFI task. However, a limitation that, on Llama-3.2 2, although IG² reduces gap nearly results in substantially higher Pother value, indicating severe degradation in the model's language

---

Published as a conference paper at ICLR 2026
Gender
Nationality
Profession
Religion
100
100
100
90
80
Score
70
80
Score
Score
60
70
70
60
60
30
(a) Results when neurons are randomly selected and modified according to the FBA modification ratio. The dashed line denotes the FBA's results.
100
100
FBA
BBA
selection
w/o selection
Score
LMS Score
70
60
40
Gender
Nationality
Profession
Religion
Gender
Nationality
Profession
Religion
(b) FBA results without stercotype cue detection. The dashed line in the left panel indicates the idcal value.
Figure 2: Ablation results of Llama-3.1 on StereoSet: (a) w/o attribution, and (b) w/o selection.
modeling capability. We further find that BBA appears more suitable than FBA for addressing the SFI problem, and does not exhibit clear advantage in the DIG task.
4.6 ABLATION STUDY
We conducted two types of ablation studies for FBA and BBA: (i) w/o attribution. Removing the attribution strategy, where neurons the projection layer are randomly selected for intervention under the same hyperparameter settings our method; and (ii) w/o selection. Removing the stereo- type cue selection algorithm, where candidate words are grouped and the first word of each group chosen as stereotype cues. the main text, we only report the results of the FBA method on Llama-3.1, The results for the other two models as well as for the BBA method are provided in Appendix A.7
W/o attribution. To mitigate the unreliability caused by randomly selecting neurons, we conducted 50 tests for the samples in each domain, as shown in Figure 2a The violin plots illustrate the results of random neuron selection, while the dashed line on each violin corresponds to the FBA results. Except for the profession domain, FBA achieves a win-win outcome compared random selection, namely SS values closer to 50% and higher LMS. In the profession domain, FBA exhibits only marginal decline in LMS, while still obtaining substantial debiasing gains.
W/o selection. As shown in Figure 2b without selecting the stereotype cues. SS values overall de- viate further from 50%, indicating that our selection algorithm successfully identifies words that are more likely trigger biased model outputs. Surprisingly, in the absence of the selection algorithm, LMS also shows a slight decrease. This demonstrates that the stereotype cue selection algorithm does negatively impact the model's language modeling capability.
CONCLUSION
We presented neuron-level debiasing framework for large language models that integrates stereo- type cue detection, gradient-based bias attribution, and targeted projection-layer intervention. Our approach mitigates demographic bias without requiring fine-tuning or any modification to user prompts, while preserving core language modeling capabilities. By showing that biased behav- iors can be localized to specific, identifiable subsets of neurons, our work offers a practical and interpretable pathway toward building fairer and more trustworthy L.Mss.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
The project was supported by Natural Science Foundation Fujian Province of China (No. (No.2024101100) and the Public Technology Service Platform Project of Xiamen (No.3502220231044)) We also thank the reviewers for their insightful comments.
ETHICS STATEMENT
This work examines social bias in large language models. The analysis may involve examples containing stereotypes sensitive content, which are used solely for research purposes. Our aim is to understand and mitigate bias, not reinforce it.
REPRODUCIBILITY STATEMENT
We provide anonymous link to code in the abstract, and the proof of Theorem 1 is included in Appendix A.6
REFERENCES
Alok Abhishek, Lisa Erickson, and Tushar Bandopadhyay. Beats: Bias evaluation and assessment test suite for large language models. arXiv preprint arXiv:2503.2433 2025.
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303 230708347... 2023.
Solon Barocas, Moritz Hardt, and Arvind Narayanan. Fairness and machine learning: Limitations and opportunities. MIT press, 2023.
Emily M Bender, Timnit Gebru, Angelina McMillan- Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference fairness, accountability, and transparency, pp. 610-623, 2021.
Su Lin Blodgett, Solon Barocas, Hal Daumé lii, and Hanna Wallach. Language (technology) is power: A critical survey of bias" in nlp. arXiv preprint arXiv:2005.1 14050, 2020.
Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam Kalai. Man to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016.
Maria De-Arteaga, Alexey Romanov, Hanna Wallach, Jennifer Chayes, Christian Borgs, Alexan- dra Chouldechova, Sahin Geyik, Krishnaram Kenthapadi, and Adam Tauman Kalai. Bias in bios: A case study of semantic representation bias high-stakes setting. proceedings of the Conference on Fairness, Accountability, and Transparency, pp. 120-128, 2019.
Sunipa Dev, Tao Li, Jeff M Phillips, and Vivek Srikumar. On measuring and mitigating biased inferences of word embeddings. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 7659-7666, 2020.
Jacob Devlin, Ming- Wei Chang. Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pp. 4171 4186, 2019.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings the 3rd innovations in theoretical computer science conference, pp. 214-226, 2012.

---

Published as a conference paper at ICLR 2026
Shaz Furniturewala, Surgan Jandial, Abhinav Java, Pragyan Banerjee, Simra Shahid, Sumit Bhatia, and Kokil Jaidka. Thinking fair and slow: On the efficacy of structured prompts for debiasing language models. arXiv preprint arXiv:2405. 1004144400.1031,,, 2024.
Isabel O Gallegos. Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim. Sungchul Kim, Franck Demnn-- court, Tong Yu, Ruiyi Zhang, and Nesreen K Ahmed. Bias and faimess in large language models: A survey. Computational Linguisties, 30(3):1097-11 179, 2024a.
Isabel O Gallegos. Ryan A Rossi, Joe Barrow, Md Mehrab Tanjim, Tong Yu, Hanich Deilamsalchy, Ruiyi Zhang. Sungchul Kim, and Franck Demoncoutt. Self-debiasing large language models: Zero-shot recognition and reduction of stereotypes. arXiv preprint a arXiv:2402.0198. 2024b.
Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H Chi, and Alex Beutel. Counterfac- tual fairness in text classification through robustness. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp. 219-226, 2019.
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Real- toxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.1 1462, 2020.
Yue Guo, Yi Yang, and Ahmed Abbasi. Auto-debias: Debiasing masked language models with automated biased prompts. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1012-1023, 2022.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
Jacqueline He, Mengzhou Xia, Christiane Fellbaum, and Dangi Chen. Mabel: Attenuating gender bias using textual entailment data, arXiv preprint 2xXv:22210.14977 2022.
Tiancheng Hu, Yara Kyrychenko, Steve Rathje, Nigel Collier, Sander van der Linden, and Jon Roozenbeek. Generative language models exhibit social identity biases. Nature Computational Science, 5(1):65-75, 2025.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL https: Slarxiv.orgaae 22100 06825
Matthew Joseph, Michael Kearns, Jamie H Morgenstem, and Aaron Roth. Fairness in learning: Classic and contextual bandits. Advances in neural information processing systems, 29, 2016.
Falaah Arif Khan, Nivedha Sivakumar, Yinong Oliver Wang, Katherine Metcalf, Cezanne Camacho, Barry-John Theobald, Luca Zappella, and Nicholas Apostoloff. Investigating intersectional bias in large language models using confidence disparities in coreference resolution. arXiv preprint arXiv:2508.07111 2025.
Jon Kleinberg. Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determi- nation of risk scores. arXiv preprint arXiv: (609.668077 2016.
Jingling Li, Zeyu Tang, Xiaoyu Liu, Peter Spirtes, Kun Zhang, Liu Leqi, and Yang Liu. Prompting fairness: Integrating causality to debias large language models. arXiv preprint arXiv:2403.08743, 2024.
Paul Pu Liang, Irene Mengze Li. Emily Zheng, Yao Chong Lim, Ruslan Salakhutdinov, and Louis- Philippe Morency. Towards debiasing sentence representations, arXiv preprint arXiv:2007.08 100, 2020.
Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. Towards under- standing and mitigating social biases in language models. International conference on machine learning, pp. 6565-6576 PMLR, 2021.
12

---

Published as a conference paper at ICLR 2026
Yujie Lin, Chen Zhao, Minglai Shao, Baoluo Meng. Xujiang Zhao, and Haifeng Chen. Towards counterfactual fairness-waare domain generalization in changing environments. arXiv preprint D2Xiv:2399133055, 2023.
Yujie Lin, Dong Li, Minglai Shao, Guihong Wan, and Chen Zhao. Fade: Towards fairness-aware generation for domain generalization via classifier-guided score-based diffusion models. arXiv preprint arXiv:2406.094495 2024.
Yujie Lin, Jiayao Ma, Qingguo Hu, Derek Wong, and Jinsong Su. Biopro: On difference- gender fairness for vision-language models. arXiv preprint arXiv:2512.00807, 2025.
Yan Liu, Yu Liu, Xiaokang Chen, Pin- Yu Chen, Daoguang Zan, Min-Yen Kan, and Tsung-Yi Yi Ho. The devil is in the neurons: Interpreting and mitigating social biases in language models. In The Twelfth International Conference Learning Representations, 2024.
Yinhan Liu, Myle Out, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 19w.111071.11919 2019.
Moin Nadeem, Anna Bethke, and Siva Reddy. Stereoset: Measuring stereotypical bias in pretrained language models. arXiv preprint arXiv:2004.09456 2020.
Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R Bowman. Crows-pairs: A challenge dataset for measuring social biases masked language models. arXiv preprint 2000000000003 2020.
Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thomp- son, Phu Mon Htut, and Samuel R Bowman. Bbq: A hand-built bias benchmark for question answering. arXiv preprint arXiv:21 2021.
Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q Weinberger. On fairness and calibration. Advances in neural information processing systems, 30, 2017.
Chahat Raj. Anjishnu Mukherjee, Aylin Caliskan, Antonios Anastasopoulos, and Ziwei Zhu. Break- ing bias, building bridges: Evaluation and mitigation of social biases in Ilms via contact hypoth- esis. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, volume 7, pp. 1180-1189, 2024.
Danielle Saunders and Bill Byrne. Reducing gender bias in neural machine translation as a domain adaptation problem. arXiv preprint arXiv:2004.04490 04498, 2020.
Andrew D Selbst, Danah Boyd, Sorelle Friedler, Suresh Venkatasubrammanian, and Janet Vertesi. Fairness and abstraction in sociotechnical systems. In Proceedings of the conference on fairness, accountability, and transparency, pp. 59-68, 2019.
Minglai Shao, Dong Li, Chen Zhao, Xintao Wu, Yujie Lin, and Qin Tian. Supervised algorithmic 2024. 2024.
fairness in distribution shifts: A survey. arXiv preprint arXiv:2402.01327,
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. The woman worked babysitter: On biases in language generation. arXiv preprint arXiv: 1909.01326, 2019.
Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. Societal biases in language generation: Progress and challenges. arXiv preprint arXiv:2105.0403 04400444. 2021.
Irene Solaiman and Christy Dennison. Process for adapting language models to society (palms) with values-targeted datasets. Advances in Neural Information Processing Systems, 34:5861- 5873, 2021.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. International conference machine learning, 2017.
Xiaoyue Wang. Xin Liu, Lijie Wang, Yaoxiang Wang, Jinsong Su, and Hua Wu. Ibadr: an iterative bias-aware dataset refinement framework for debiasing nlu models. arXiv preprint arXiv:2311.000 0,0092,, 2023.
13

---

Published as a conference paper at ICLR 2026
Xiaoyue Wang, Xin Liu, Lijie Wang. Suhang Wu, Jinsong Su, and Hua Wu. A simple yet effective self-debiasing framework for transformer models. Artificial Intelligence, 339:104258,, 2025.
Yongkai Wu, Lu Zhang, Xintao Wu, and Hanghang Tong, Pe-fairness: A unified framework for mea- suring causality-based fairness. Advances in neural information processing systems, 32, 2019.
Zhenjie Xu, Wenqing Chen, Tang, Xuanying Li, Cheng Hu, Zhixuan Chu, Kui Ren, Zibin Zheng, and Zhichao Lu. Mitigating social bias in large language models: A multi-objective approach within a multi-agent framework. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 25579-25587, 2025.
Chen Zhao, Feng Chen, and Bhavani Thuraisingham. Fairness-aware online meta-learning. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, pp. 2294 2304, 2021.
Chen Zhao, Feng Mi, Xintao Wu, Kai Jiang, Latifur Khan, and Feng Chen. Adaptive fairness-awan online meta-learning for changing environments. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, pp. 2565-2575, 2022.
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai- Chang. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:
Human Language Technologies, Volume 2 (Short Papers), pp. 15-20, 2018
14

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
2
Background
Problem Definition
2.2
Integrated Gradient and Integrated Gap Gradient
3
Methodology
Stereotype Cue Selection
3.2
Forward Bias Attribution
Backward Bias Attribution
3.4
The Relationship Between Bias Variation and Output Variation: A Theoretical Anal-
ysis
Experiments
4.1
Experimental Settings
Evaluation of Stereotype Cue Selection
4.3
DIG Task (StereoSet)
4.4
DIG Task (B3Q)
4.5
SFI Task (WinoBias)
4.6
Ablation Study
10
Conclusion
10
Appendix
16
A.
Use of LLMs.
16
Related Works
16
A.3
Simple Introduction for Our Baselines
17
Jensen-Shannon Divergence
17
A.5
Stereotype Cue Selection Algorithm
17
Derivation Theorem
17
A.6.I
Definitions and Preliminaries
17
A.6.2
Step I: Exact Expression of Bias Reduction
18
A.6.3
Step 2:
Exact Expression of Output Distribution Variation
19
A.6.4
Step 3: Relationship Between Bias Reduction and Output Variation
19
A.6.5
Step 4: Final Relationship
20
A.7
Complete Experiments
20
A.7.1
Complete Results across Domains the BBQ Dataset
20
A.7.2
Results on the Bias- Dataset
20
A.7.3
Results the Bias-NLT
NL. Dataset
21
15

---

Published as a conference paper at ICLR 2026
A.7.4
Analysis of Lower-Layer Neuron Contributions
22
A.7.5
Experimental Results in Mistral-v0.3
22
A.7.6
Extra Ablation Results on StereoSet
24
A.7.7
Ablation Results on WinoBias
26
A.7.8
Hyperparameter Settings and Sensitivity Analysis
27
................................
A.8
Complete Templates for Two Types of Stereotype Cues
A.9
Demographie Groups for All Demographic Attributes
30
A.10 Prompts for Constructing Questions
A
APPENDIX
A.1
USE OF LLMS.
For each demographic attribute, we employed GPT- (Achiam al.) 2023) to assist in generating potentially biased words. In addition, we used LLMs to check the manuscript for typographical errors.
A.2 RELATED WORKS
Social Bias LLMs. Unlike the bias between the training and test distributions (Wang al. 2023 2025) this paper primarily investigates social bias (Zhao et al. [2221] 2022 Lin et al. [223]] Shao ct al. 2024: Lin et al.) 2025 2024). Early studies revealed biased associations in distributional representations (c.g., gendered analogies in word embeddings), and proposed geometric debiasing methods to reduce such effects (Bolukbasi et al.) 2016). Subsequent task-specific benchmarks ex- posed bias in core NLP systems. For example, gendered errors in coreference resolution and occu- pation classification highlighted how downstream models can reproduce and even magnify societal imbalances (Zhao open-ended models produce differing "regard" and disparate toxicity across demographic groups, and large-scale prompt-based evaluations revealed neural models' propensity for toxic degeneration under realistic prompts (Sheng et al.) 2019, Gehman et al. 2020). Too quantify stereotyping more broadly, community benchmarks such as CrowS-Pait and StereoSet were introduced; evaluations on these datasets demonstrate that both masked and autoregressive LMs often prefer stereotyped continuations (Nangia et al. 2020 Nadeem et al. 2020).
2018 De-Arteaga al. 2019). Work focused on generation showed that
Beyond empirical measurement, critical analyses highlight that the scale, opacity, and data prac- tices of modern LLMs generate significant socio-technical risks. These range from environmental and labor concerns to the reproduction of harmful narratives, thereby motivating calls for greater transparency, staged release strategies, and more comprehensive evaluation protocols (Bender et al.) 2021). More recent work has expanded the scope of bias evaluations: large-scale audits show that generative models systematically encode social identity biases across dozens of systems (Hu et al., 2025), and new resources such Winoldentity allow for fine-grained assessment of intersectional stereotypes across multiple demographic attributes (Khan et al. 2025). Complementary test suites (e.g., BEATS) propose unified frameworks assess bias and fairness conjunction with factuality and safety, reflecting the need for multidimensional evaluations of LLM behavior (Abhishek et al.) 2025]. At the same time, mitigation research has moved beyond prompt editing and fine-tuning, For instance, socially grounded approaches inspired by the contact hypothesis simulate intergroup ex- posure to reduce biased outputs (Raj et al. 20244) while multi-agent causal intervention frameworks seek to minimize stercotyping without degrading task performance (Xu et al.) 2025). Together, these lines of inquiry highlight both the persistence social bias in modern LLMs and the growing sophistication of evaluation and mitigation strategies.
Fairness-aware Learning, Parallel to work documenting bias in LLMs, the broader machine learn- ing community has developed rich body of research fairness-awan learning. Early studies formalized group faimess criteria such as demographic parity, equalized odds, and calibration, and explored algorithmic strategies balance predictive performance with faimess constraints (Dwork)
16

---

Published as a conference paper at ICLR 2026
et al. 2012: Hardt et al. 2016 Pleiss et al. 2017). Subsequent research introduced individual fair- ness notions grounded in similarity metrics, emphasizing that similar individuals should receive similar outcomes (Dwork al., 2012 Joseph al. 2016). Beyond static definitions, scholars high- lighted tensions among fairness criteria, impossibility results, and trade-offs with accuracy, mo- tivating the development of context-sensitive approaches (Kleinberg al. 2016; |arrcaas et al.) 2023). To address distributional challenges, fairness-aware methods increasingly account for do- main shifts and long-tail groups. particular, techniques such as reweighting, adversarial learning, and causal inference have been proposed to achieve robust fairness under covariate shift and label imbalance (Saunders & Byme, 2020 Wu et al.) 2019) Garg al. 2019). More recent directions extend fairness considerations to large-scale generative systems: approaches include counterfactual data augmentation, representation regularization, and fairress-constraineed decoding strategies tai- lored for pre-trained LMs (Saunders & Byrne 2020, [Sheng et al. [021] (Solaiman & Dennison, 2021I. At the same time, interdisciplinary critiques stress that fairness cannot be reduced to quanti- lative metrics alone: fairness-awan learning must also grapple with the structural and sociocultural dimensions of algorithmic maciiionnmmking (Selbst et al. 2019
Blodgett et al.) 2020).
A.3 SIMPLE INTRODUCTION FOR OUR BASELINES
Auto-Debias (Guo al.) 2022) is two-stage fine-tuning method for masked language models (MLMs). Without external corpora, the approach uses beam search to automatically discover prompts that maximally expose gender racial bias in cloze-style completions.
Prefix Prompting (Furniturewala et al. 2024) uses simple instructions or role-play prefixes that ask the model to be fair.
Self-Debiasing (Gallegos et al. 2024b) ask the model to explain which answer choices rely on invalid assumptions before answering
DDP (Li et al. 2024) develops a causality- prompting framework. A causal graph models how selection mechanisms in training data create spurious dependencies between social category and model decisions.
A.4
DIVERGENCE
For a set probability distributions (P1.P2....,,2 over the same probability space, Jensen- Shannon Divergence (JSD) is defined as
1 n
JSD(P..... Pn)
Σ KL
P1.P2..... Pn
(10)
n
where KLD(.)) denotes the Kullback-Leibler divergence.
A.5
STEREOTYPE CUE SELECTION ALGORITHM
Cue Selection Pipeline. The process consists four steps:
Candidate Initialization: Collect adjective and noun sets Vadj Vnoun-
Probability Collection: Generate prompts with templates and obtain p(d:l.) from the model.
Entropy Calculation: Aggregate probabilities across templates and compute entropy.
Ranking and Selection: Rank cues by entropy (ascending) and select those with strongest
bias induction.
A.6 DERIVATION OF THEOREM 1
A.6.1 DEFINITIONS AND PRELIMINARIES
Assumptions. We assume that Proj Rd - R and B gg* - R are continuously differentiable (C') on open sets containing the paths
(Te 1Ah / [0, I]}, (v(1) Proj(h + (AM) Le [0. 1]]-
17

---

Published as a conference paper at ICLR 2026
Algorithm Stereotype Cue Selection
Require: Pre-trained language model M. Templates Tadi, Tooun Candidate cues Vauii and Vnouns Demographic groups D, Entropy function (-))
Ensure: Top k biased adjectives and nouns
function COMPUTEENTROPIESS:T))
2: E < 0
3: for w
4:
Pagg +- 0
5:
for t E T do
6:
prompt Replace(tt, w)
7:
p - (prompt)
8:
Pagg Pagg + P/[T]
9: end for
E E H (Pags)
11: end for
return E
13: end function
14: Eadi -COMPUTEENTROPIES! Vadi Tad))
15: Emoun < COMPUTEENTROPIES V noun: Toum
16: Select top cues from Eadj and Enou based on lowest entropy
Hence B is continuous gradient field, and the line integral of V is path-independent.
1. Function Definitions:
Model output function: y = (h)j(h), where e Id denotes the input to the final hidden layer.
Bias function: B : R + R, which takes the output distribution input.
Composite function: F(h) = B (Proj(h)), mapping to the bias value.
2. Modification Procedure:
The modification set S (l....,,d d} (with cardinality |S] = (dd) contains neurons with highest positive attribution values.
Modified input:
C
jes S j$
h
(1)
h
Modification vector: Ah mad
Tí with
CC
hig
j 5
Ah,
j $ 5
A.6.2 STEP 1: EXACT EXPRESSION OF BIAS REDUCTION
By definition of the composite function the bias reduction is given by:
AB = modm ((h)
3
Using the integral form of the Mean Value Theorem along the path T J mod To Ah, we have:
AB
VF + tAh) Ah dt
(4)
J0
Expanding the dot product:
d
éF
AB
Σ
Ahj
(Th + tah) dt
(5)
oh,
18

---

Published as a conference paper at ICLR 2026
Since Ah 0 for j $ S, this reduces to:
| OF Thg
AB
(h 12h) dt
JES
A.6.3 STEP 2: EXACT EXPRESSION OF OUTPUT DISTRIBUTION VARIATION
For each component = Proj. (h), the change is:
Ayj Proji(h mod) Proji(A) (
Applying the integral form of the Mean Value Theorem:
Ayi Proji (1+ (AN) Ah dt
(8)
Expanding:
PProji (T tAh) dt
Ay: = Σ
Ahj
(9)
0
==1
Again, due sparsity of Ah:
PProj (h +t+) dt
Ay, Σ RT))
(10)
Th
.0
A.6.4 STEP 3: RELATIONSHIP BETWEEN BIAS REDUCTION AND OUTPUT VARIATION
Using the chain rule the composite function F = B Proj, we denote the components of the projection function by ym Projm Then
aF
²B 6ym
aB @Proj..
Σ
Σ
(11)
Thy
pym Ōhg
pym
Ōhg
m=1
Substituting (II) into (6):
['
k
AB Jym
PProjmn
AB EC h2)
Σ
(y(t))
(h + tah) dt
(12)
Ōh,
JES
with v(() Proj(h (Ah).
Interchanging the order of summation and integration:
AB 6ym
PProjm
AB
Σ
(v(t))
[c-- Fuy)
(T + tat)) dt
(13)
Th,
m-1
jES
Define the output-space velocity vector
d Croj(h+tAi)) = [C-
iProjm (Tr + tan) Th,
G(() = dt
(14)
JES
Then (13) becomes
v(1) V B dy.
AB B dt
(15)
JO
Jy(()
Since V is conservative vector field, the line integral depends only on the endpoints y(0), y(1)
19

---

Published as a conference paper at ICLR 2026
A.6.5 STEP 4: FINAL RELATIONSHIP
We can therefore replace the original curved path y(t) with the straight-line path in output space:
v(0) sAy, [0, 1],
(16)
where Ay =1)) (0).. Thus,
AB VB(u(s)) T As
(17)
Define
((s) = VB(u(s)) Ay,
which is continuous on [0, 1]. By the Mean Value Theorem for integrals, there exists some 0 e [0. 1]
such that
AB x(0) BBy((0) + ADy) Ay.
(18)
Final Result.
ABB BB(((0) 0Ay) Ayl < [V 0Ay)] || e [o, 1].
(19)
This completes the proof.
A.7 COMPLETE EXPERIMENTS
A.7.1
COMPLETE RESULTS ACROSS DOMAINS ON THE BEQ DATASET
Table 8: Evaluation result across three demographic attributes on the BBQ dataset.
Gender
Nationality
Religion
Llama-3.1
Accand (%) / Accau (%) Average (%)
Base
69.07 / 84.64 76.86
33.77 99.22 65.00
60.67 0667/92..67 76.67
Auto-Debias
68.86 84.29/ 76.58
33.63 96.23 64.93
61.33/ 61.33/92.6 77.00
Prefix Prompting
87.86 79.43 83.65
59.87 144.81/77.33
77.67 84.00/ 80.84
Self- Seff-Debiasing
86.50 58.64 72.57
0000/86.5//7...
88.00 73.67 80.84
DDP
81.50 / 83.71 82.61
35.45 0045/94.40 /9.///442/// 64.94
53.67 /9 72.67
IG
75.29 81.07 78.18
33.38 95.97 64.68
68.33 91.33 79.83 72.33 89.00/ 80.67
FBA
73.64 83.07 78.36
53.38 94.54 73.96
BBA
70.71 / 84.86 77.79
74.67 92.98 83.83
78.00 / 89.00/ / 83.50
Gender
Nationality
Religion
Llama-3.2
Accamb (%) / Accais (%) 1 Average (%) 1
Base
68.21 / 76.00/ 72.11 72.11
51.30/ 94.16 72.73
57.33 / 83.67 70.50
Auto-Debias
76.64 72.32
50.26 93.90 72.08
57.33
83.33 70.33
Prefix Prompting
86.71 62.57 74.64
78.05 85.19 81.62
83.67 59.33 71.50
Self-Debiasing
65.21 / 69.36 67.29
64.29 / 84.81 74.55
68.67 / 68.33 68.50
DDP
78.35 73.07 75..71
52.47 / 99.1//9.12/// 72.80
50.33 / 84,00 67.17
IG²
25.50 58.71 /22.11
67.66 92.47 80.07
64.67 76.33 70.50
FBA
72.00 73.57 72.79
67.01 99.21 / 79.61
67.33 / 79.00 73.17
BBA
72.43 /71.14//11777
59.48 93.38 / 76.43
72.33 / 75.33 73.83
A.7.2 RESULTS ON THE BIAS -N-BIOSS DATASET
Bias-in- Mlas-in-Bios (De-Artaaga et al. 2019) is a thirdperson biography dataset annotated by occupa- tion and gender. We use LLMs predict individual's 's profession given their biography.
Metric. For the Bias-in-Bios dataset, we adopt the five evaluation metrics from (He et al. 2022
including: (1) Accall overall accuracy; (2) Accms accuracy male-labeled instances; (3) Accy,
accuracy on female-labeled instances; (4) Gap-TPE the difference in true positive rate (TPR) between male- and female-labeled instances; (5) RMS-TPR, the noot-mean-square of the TPR gap
20

---

Published as a conference paper at ICLR 2026
Table Evaluation results on the Bias- in-Bio dataset.
Llama-3.1
Accall
Accm
Accy
Gap-TPR
RMS-TPR
Base
72.44
69.16
76.39
6.05
28.50
Auto-Debias
72.31
68.94
76.39
6.14
28.43
Prefix Prompting
73.72
69.83
78,41
7.30
28.85
Self-Debiasing
76.46
73.93
79.47
5.54
25.87
DDP
77.11
74.30
80.47
5.72
30.41
IG
71.44
68.60
74.88
4.42
27.83
FBA
73.35
69.93
77.51
5.86
26.43
BBA
71.21
71.60
70.59
1.39
6.53
Llama-3.2
Accall
Accm
Accy
Gap-TPR
RMS-TPP TPR
Base
71.47
68.68
74.84
4.50
25.61
Auto-Debias
71.78
69.20
74.87
4.29
27.29
Prefix Prompting
70.47
67.43
74.12
5.38
30.88
Self-Debiasing
70.70
67.93
74.03
5.44
30.23
DDP
54.92
53.92
56.14
1.20
29.08
IG
72.04
69.87
74.62
3.79
25.13
FBA
77.32
76.19
78.54
2.38
14.90
BBA
75.00
70.01
79.41
1.02
10.48
across all occupation classes, We selected lightweight version of the Bias-in -issinniosss datase for testing. The experimental results on Bias-in-Bios are shown in Table
We observe that BBA achieves significantly stronger debiasing effect Bias- -n-BBis com- pared to all other methods (including FBA). Mcanwhile, on Llama-3.2, both FBA and BBA improve all accuracy metrics.
A.7.3 RESULTS ON THE BIAS -NLI DATASET
Bias-NLI Dev et al. 2020) is an NLJ dataset consisting of neutral sentence pairs. It is systemat- ically constructed by populating sentence templates with gendered word and occupation word with strong gender connotation (e.g., The woman ate bagel; The nurse ate bagel).
Metrics. For the -NI dataset, we evaluate large language models directly using prompt rather than performing classification with a fine-tuned BERT model as in (He et al. 2022). We compute the probabilities of the three labels (entailment, neutral, contradiction), denoted P. Pn,
and Pe. A higher value of Pn indicates that the model is more fair.
Because the Bias dataset is exceptionally large, we selected the first 1,000 samples for ing. We find that 1lama-332 is almost unable to perform correct linguistic reasoning on this dataset, as shown in the following Table 10
Table 10: Evaluation results on the Bias as-NLI dataset (Llama-3.2).
Llama-3.2
Pe
Pa
Pe
base
22.7
0.3
77.0
The results on Llama-3.2 show abnormally low P., primarily focus on the results obtained with 1 (Table
Both FBA and BBA substantially increase P., and the accuracy of FBA is nearly 100%. This demonstrates that our method enables the model to correctly rule out stereotype-driven reasoning errors when inferring the relationship between sentences.
21

---

Published as a conference paper at ICLR 2026
Table 11: Evaluation results on the Bias -LII dataset (Llama-3.1)
Llama-3.1
Pa
Pa
Pa
base
17.4
81.8
0.8
Auto-Debias
15.3
84.0
0.7
Prefix Prompting
15.1
84.3
0.6
Self-Debiasing
0.6
75.9
23.5
DDP
38.4
61.1
0.5
IG2
12.5
86.9
0.6
FBA
0.9
99.1
0.0
BBA
0.4
97.5
2.1
Table 12: Forward-IG and Backward-IG magnitude across layers for Llama-3. I and Llama-3..
Llama-3.1
Gender
Nationality
Profession
Religion
Forward-IG (last layer)
1.72 x 10 3
4.49 x 10-4
1.14 x 10-2
3.81 x 10 3
Forward-IG (first layer)
9.94 x 10-8
8.14 x 10
7.31 x 10
1.71 x 10
Backward-IG (last layer)
6.89 10-5
3.57 10-6
2.09 10-5
4.25 x 10
Backward-IG (first layer)
1.71 x 10-7
2.85 x 10-*
2.93 x 10
1.82 x 10
Llama-3,2
Gender
Nationality
Profession
Religion
Forward-IG (last layer)
9.03 x 10
7.68 x 10
3.10 x 10-3
1.31 x 10-3
Forward-IG (first layer)
9.97 10
8.83 10-7
6.09 10-6
2.53 10-5 5
Backward-IG (last layer)
7.61 x 10-6
2.62 x 10-6
1.10 x 10-5
6.67 x 10-5
Backward-IG (first layer)
5.82 10-
2.85 x 10- 2.47 x 10
6.12 x 10
Table 13: Evaluation results (StereoSet) for FBA and BBA across layers on Llama-3. and
Llama-3.2.
Llama-3.1
Gender
Nationality
Profession
Religion
FBA (last layer)
68.75/100/6..50
64.88/97. 10166.99988999
11.88/97.99/8.11
67.87 67.87/96.05/61.77
51.35/93.67/91.14
67/91.14
FBA (first layer)
75.59/99.22/45
64.96/97.30/68.19
72.41/97.5355383
52.78/91. 14//66...//6008
BBA (last layer)
69.84/98.44 44/59.38
69.84/98.44/59.36
13.18/95.43/70.22
5..88995.66/5.32
99.31/92.41/91.1
BBA (first layer)
13.02/98.44/53.13
66.67/96..67/44455
75.38/97.28/47..0
58.11/93.67778.4
Llama-3.2
Gender
Nationality
Profession
Religion
FBA (last layer)
69.60/97.66/59..
58.52/95.22/99.00
61 81.88944.7772.100
$1..5/97.46693..0
FBA (first layer)
70.97/96.88/56.
2.69//5.84/71.22
60.68/94.81/74.55
19.74/97.47/78..4 47/78.448
BBA (last layer)
67.46/98.44464.00
62.93/96.47771.52
62.21/96.00572..99
55.33/94 94/88.61
BBA (first layer)
71.20/97.66/56.25
57.36/96.0//81.91
64.87/96.30/67.6
56.00/94.93/83.55
A.7.4 ANALYSIS LOWER-LAYER NEURON CONTRIBUTIONS
We compute the mean Forward-IG and Backward-IG values for the neurons the first-layer hidden state and compare them with those from the final layer used in experiments, shown in Table 12 Most lower-layer neurons exhibit gradient signals that decay by more than two orders of magnitude, with the most severe diminishing by up signals prevent the model from achieving optimal debiasing performance. We verify this by applying neuron-level modifications to lower layers on the StereoSet benchmark (Table [3).
orders of magnitude. Such weakened gradient
A.7.5 EXPERIMENTAL RESULTS IN MISTRAL-VO.3
Tables 14 and 15 present the performance of Mistral-vOR datasets, respectively. On StereoSet, both FBA and BBA demonstrate certain effectiveness, par- ticularly in the domains of profession and religion, where FBA achieves notably strong performance. On winn non as, although we did not achieve the optimal Gap, our approach still significantly outper- forms Self- Debi-Debis,,ng which causes sharp increase in P Pother- Overall, BBA attains the second-best performance.
the StereoSet and WinoBiar
22

---

Published as a conference paper at ICLR 2026
Table 14: Evaluation results across four demographic domains on the StereoSet dataset.
Gender
Nationality
Profession
Religion
Mistral-v0.3
SS (%)
50%
LMS (%)
ICAT (%)
Base
72.36 (9.6//96.09)) 53.12
60.00 74.01
69.39 / 93.58 57.28
60.81 99.67 73.42
Auto-Debias
77.12/92.1942..11
61.50/91.27 70.27
70.08 /91.60 54.81
60.00 /94994/ 75.95
Prefix Prompting
72.27 /9..97 51.56
55.10/91. /9.68 82.33
64.27 99.59 66.17
52.86 / 88.61 /83.54
Self- Self-Debiasing
42.70 69.53 59.37
48.09 / 70.89 68.19
56.51 / 72.10 62.72
55.93 / 74.68 / 65.82
DDP
66.39 / 99.331 64.06
58.65 / / 76.50
66.93 / 94.07 62.22
63.24 / 86.08 / 63.29
IG
76.03 94.53 45.31
45.96 66.94 61.54
51.81 68. 65.68
51.79 70.89 68.35
FBA
68.33 93.75 59.38
55.11 / 89.40 80.25
65.80 94.57 64.69
55.40 93.67 83.54
BBA
68.33 93.75 59.38
54.19 84.41 77.34
59.08 / 85.68 70.12
54.93 89.87 81.01
Table 15: Evaluation results on the WinoBias dataset.
Mistral-v0.3
P Patereo
Ponti
Pother
Gap |
Base
61.11
38.64
0.25
22.47
Auto-Debias
56,69
41.16
2.15
15.53
Prefix Prompting
71.21
28.79
0.00
42.42
Self-Debiasing
49.49
43.43
7.08
6.06
IG2
50.76
48.48
0.76
2.28
FBA
58.59
40.66
0.75
17.93
BBA
55.81
43.43
0.76
12.38
23

---

Published as a conference paper at ICLR 2026
A.7.6 EXTRA ABLATION RESULTS ON STEREOSET
Some ablation results have already been presented in the main Figures 6344567 report the ablation results of all models on StereoSet. In most cases, our two attribution methods achieve simultaneous improvement in both SS and LMS. certain cases, when LMS values are comparable, our methods yield SS scores closer to 50%.
Gender
Nationality
Profession
Religion
100
90
80
80
Score
Score
Score
70
70
50
40
50
55
LMS
ICAT
55
LMS
ICAT
.5
LMS
ICAT
55
LMS
ICAT
Figure 3: Ablation results of Llama-3.1 on StereoSet: BBA w/o attribution.
Gender
Nationality
Profession
Religion
100
90
90
Score
70
Score
80
Score
80
70
70
40
50
55
LMS
ICAT
55
LMS
ICAT
LMS
ICAT
55
LMS
ICAT
Figure 4: Ablation results of Llama-3.2 on StereoSet: FBA w/o attribution.
Gender
Nationality
Profession
Religion
90
90
90
80
Score
Score
80
Score
700
60
60
30
50
Figure 5: Ablation results of Llama-3.2 on StereoSet: BBA w/o attribution.
24

---

Published as a conference paper at ICLR 2026
Gender
Nationality
Profession
Religion
100
100
90
90
90
80
80
Score
Score
Score
70
60
60
50
60
50
SS
LMS
ICAT
SS
LMS
ICAT
$S
LMS
ICAT
S$
LMS
ICAT
Figure 6: Ablation results of Mistral-v0.3 on StereoSet: FBA w/o attribution.
Gender
Nationality
Profession
Religion
100
100
90
90
80
80
Score
70
Score
Scora
70
70
60
60
50
55
LMS
ICAT
55
LMS
ICAT
55
LMS
ICAT
LMS
ICAT
Figure 7: Ablation results of Mistral-v0.3 StereoSet: BBA w/o attribution.
100
100
FBA
BBA
w/o
Score
LMS Score
70
60
40
Gender
Nationality
Profession
Religion
Gender
Nationality
Profession
Religion
Figure Ablation results of Llama-3.2 StereoSet: w/o selection.
100
100
selection
BBA w/o selection
Score
LMS Score
70
40
Gender
Nationality
Profession
Religion
Gender
Nationality
Profession
Religion
Figure 9: Ablation results of Mistral-v0.3 on StereoSet: w/o selection.
25

---

Published as a conference paper at ICLR 2026
A.7.7 ABLATION RESULTS ON WINCBIAS
Tables [16 17 report the ablation results of all models WinoBias. When our attribution strategy is replaced with randomly selected neurons, the Gap value rises sharply, indicating that the model bias is not alleviated all. Moreover, when stereotype cue selection is removed, decline in debiasing performance is observed across all settings except for the FBA method on Llama-3.2.
Table 16: Ablation results of Llama-3.1 on Wi WinoBias.
Llama-3.1
Psterea
Panti
Pother
Gap |
FBA
52.53
47.47
0.00
5.06
FBA w/o attribution
62.4
36.75
0.85
25.65
FBA w/o seletion
57.58
42.42
0.00
15.16
BBA
50.51
49.49
0.00
1.02
BBA w/o attribution
59.85
40.15
0.00
19.70
BBA w/o seletion
49.49
50.51
0.00
1.02
Table 17: Ablation results of Llama-3.2 on WinoBias.
Llama-3.2
Pstereo
Panti
Pother
Gap |
FBA
59.34
40.66
0.00
18.68
FBA w/o attribution
74.75
21.16
4.09
53.59
FBA w/o seletion
57.83
42.17
0.00
15.66
BBA
51.52
48.48
0.00
3.04
BBA w/o attribution
75.72
19.15
5.13
56.57
BBA w/o seletion
55.30
44.70
0.00
10.60
Table 18: Ablation results of Mistral-vO.3 on WinoBias.
Mistral-v0.3
Patereo
Panti
Pother
Gap |
FBA
58.59
40.66
0.75
17.93
FBA w/o attribution
64.48
31.20
4.32
33.28
FBA w/o seletion
57.32
39.14
3.54
18.18
BBA
55.81
43.43
0.76
12.38
BBA w/o attribution
61.50
37.66
0.84
23.84
BBA w/o seletion
62.88
35.10
2.02
27.78
26

---

Published as a conference paper at ICLR 2026
A.7.8 HYPERPARAMETER SETTINGS AND SENSITIVITY ANALYSIS
Since our method does rely on training set, split StereoSet and WinoBias into val- idation and test sets at 1:1 ratio. However, due to the limited number of samples in the religion domain of Stereošet, this domain could be partitioned. For the approximate computation of Forward-IG and Backward-IG, we set the number of approximation steps to Nstep = 20.
In the process of modifying neuron activations, two parameters are involved: the modification ra- tio 3 and the constant value C to which the activations are set. Arbitrarily setting the constant C' may exacerbate bias, and therefore we perform grid search over the parameters. Specifically, set the search range of 3 to [0.1, 0.2, 0.3, 0.4], and the range of C' to 2-2, -1, -,,0, 1, 2J.. Tables 19 20 22 23 25 26) present the grid search results FBA and BBA on the Llama-3.1 model with StereoSet. The gray cells indicate the parameters ultimately adopted. We observe that larger absolute values of 3 and tend to cause a more severe degradation of the model's ''s language modeling ability (i.e., LMS), while simultancously yielding stronger debiasing effects. This phe- nomenon is consistent with our previously established Theorem [
Table 19: Hyperparameter search of the FBA method on the gender domain (SS, LMS).
-2
-1
0
2
8
0.1
(68.00, 97.66)
(67.46, 98.44)
(73.44, 100.0)
(73.23, 99.22)
(68.75, 100.0)
0.2
(59.32, 92.19)
(68.80, 97.66)
(72.44, 99.22)
(74.19, 96.88)
(71.43, 92.97)
0.3
(58.18, 85.94)
(66.40, 97.66)
(78.05, 96.09)
(78.57, 98.44)
(60.91, 85.94)
0.4
(46.15, 71.09)
(67.74, 96.88)
(71.43, 98.44)
(68.85, 95.31)
(47.78, 70.31)
Table 20: Hyperparameter search of the FBA method the nationality domain (SS, LMS).
C
B
-1
2
0.1
(69.25, 96.67)
(68.51, 97.71)
(68.86, 98.13)
(69.02, 97.30)
(65.17, 97.30)
0.2
(68.20, 94.80)
(69.72, 97.51)
(69.43, 97.92)
(64.88, 97.09)
(58.76, 93.76)
0.3
(60.43, 86.69)
(68.13, 94.59)
(67.95, 97.30)
(64.78, 95.63)
(59.06, 88.36)
0.4
(52.52, 70.06)
(68.58, 93.97)
(68.71, 98.34)
(62.23, 95.22)
(50.43, 72.56)
Table 21: Hyperparameter search of the FBA method on the profession domain (SS, LMS).
-2
-1
2
B
0.1
(70.36, 95.80)
(75.96, 96.54)
(76.20, 97.53)
(73.05, 98.02)
(73.55, 98 98.02)
0.2
(68.60, 93.58)
(73.08, 96.30)
(75.76, 97.78)
(74.11, 97.28)
(69.41, 92,84)
0.3
(60.34, 87.16)
(67.87, 96.05)
(72.73, 97.78)
(71.25, 97.04)
(66.48, 88.40)
0.4
(55.31, 76.79)
(64.17, 92.35)
(72.91, 97.53)
(68.24 24, 94.07)
(60.47, 74.32)
27

---

Published as a conference paper at ICLR 2026
Table 22: Hyperparameter search the FBA method on the religion domain (SS, LMS).
C
-2
-1
2
B
0.1
(58.90, 92.41)
(61.64, 92.41)
(63.01, 92.41)
(59.15, 89.87)
(59.72, 91.14)
0.2
(51.35, 93.67)
(54.93, 89.87)
(65.28, 91.14)
(63.01, 92.41)
(63.38, 89.87)
0.3
(59.42, 87.34)
(58. 11, 93.67)
(55.41, 93.67)
(57.75, 89.87)
(58.46, 82.28)
0.4
(56.90, 73.42)
(62.50, 81.01)
(62.50, 91.14)
(61.11 91.14)
(56.67, 75.95)
Table 23: Hyperparameter search of the BBA method on the gender domain (SS, LMS).
-2
2
B
0.1
(73.02, 98.44)
(75.59, 99.22)
(75.59, 99.22)
(76.19, 98.44)
(74.59, 95,31)
0.2
(63.56, 92.19)
(69.84, 98.44)
(73.23, 99.22)
(73.81, 98.44)
(74.36, 91,41)
0.3
(58.82, 79.69)
(62.50, 93.75)
(71.20, 97.66)
(76.61, 96.88)
(69.81, 82.81)
0.4
(51.16, 67.19)
(61.95, 88.28)
(70.16, 96.88)
(75.63, 92.97)
(52.75, 71.09)
Table 24: Hyperparameter search of the BBA method the nationality domain (SS, LMS).
C
-1
2
B
0.1
(65.43, 95.01)
(66.88, 97.30)
(66.45, 97.30)
(66.60, 97.71)
(64.96, 97.30)
0.2
(60.92, 90.44)
(63.18, 95.43)
(65.45, 96.88)
(65.58, 95.43)
(63.35, 91,89)
0.3
(58.42, 81.50)
(65.19, 93.76)
(65.13, 94.80)
(64.33, 92.10)
(62.47, 82.54)
0.4
(49.12, 70.69)
(60.68, 85.65)
(61.28, 91.27)
(62.53, 85.45)
(56.73, 72.56)
Table 25: Hyperparameter search the BBA method on the profession domain (SS, LMS).
-2
-1
2
B
(73.91, 96.54)
(77.22, 97.53)
(75.76, 97.78)
(74.18, 97.53)
(72.89, 96.54)
0.2
(69.57, 90.86)
(74.17, 96.54)
(75.26, 96.79)
(72.42, 95.80)
(66.94, 91.11)
0.3
(65.88, 83.21)
(71.54, 94.57)
(71.58, 95.56)
(73.28, 93.33)
(62.54, 81.73)
0.4
(56.69, 70.12)
(69.36, 88.64)
(71.17, 95.06)
(66.00, 86.42)
(54.49, 74.32)
Table 26: Hyperparameter search of the BBA method the religion domain (SS, LMS).
C
B
0.1
(62.50, 91.14)
(61.11, 91.14)
(60.81, 93.67)
(59.46, 93.67)
(56.94, 91.14)
0.2
(59.42, 87.34)
(62.16, 93.67)
(57.53, 92.41)
(58.90, 92.41)
(49.32, 92.41)
0.3
(64.52, 78.48)
(63.89, 91.14)
(58.90, 92.41)
(S4.79, 92.41)
(49.25, 84.81)
0.4
(49.12, 72.15)
(53.62, 87.34)
(56.16, 92.41)
(52.17. 87.34)
(49. 72.15)
28

---

Published as a conference paper at ICLR 2026
A.8 COMPLETE TEMPLATES FOR Two TYPES OF STEREOTYPE CUES
Category
Template Examples
Adjective
The [DemographiccAttribuue of this [Stercotype_Adjeccivee] person is [Demographic_Grog] This Serreotyee_Ajjcttine person belongs to the AeemographicAAtribuu of [Demo- This person who is AStieeetype_Aljecciveee] is of the [Demographic_Attien A of [Demo- graphic graphic_Groupl. Group].
This person who is CeereotypeAAjecttiie is of the [Demographic Group] [Demo- AtppiccAttrbbute], This AAdieetty person is in the [Demographic.Attribbe) of [Demo- graphic_Grroupl
AeamoggraphicAAtrbuuee The [Stereotype Adjectivel person's [Demographic Attribute] is identified as [Demo-
This [Stereotype_Afficcivv)] person is in the (Demographie_Growg
graphicGGroup.
This [Stereotype Adjective] person's [ttribba Attribute] is [Demographic Group]. The [Demographic Attribuue of this person who is Adijeciv is identified as [Demographic_ Group].
This |Stereotype_Adjcee graphiccAtributt).
person identifies as [Demographic.Grrove in terms of [Demo-
This person who is (Steeeitype_Adjectvvee identifies with the Deemographic.Grogg [Demo- graphic_Attribute.
In terms of (Demographic Attribute], this [Stereotype Adjective] person is identified Demographic Group].
The [Demographic Attribute] identification of this person who is [Stereotype Adjective] is [eemographic
These ASijecitypeAAjjectvvve]]] people associate themselves with the [Demograpphic.Group] Cemmographic_Attribuue
In terms of DDemographic as [Demographic Grrupp.
[ttriipaphittttbuttt Attribute), these [Stereotype Adjective] people identify themselves
These [Stereotype SStfeeitye_AAjjeciivee]] people identify themselves as [Demographic Group] in relation
to (Demographic.Attribnn These people who are Stereotype.Adjecttii identify their [Demographic as [Demographic Group].
Noun
The [Demographicc Attribute] of this [Stereotype Noun] is [Demographic D Group]. This [Stereotype Noun] belongs to the [Demographic Attribute] of [Demographic Group]. This person, who is [Stereotype Noun]. is of the [Demographic Attribute] of [Demo- graphic_Groupl.
This [Stereotype Noun] is of the [Demographie Group] This [Stereotype_Nomn is in the (Demographic..Atrrutee of [Demographic_Groone This [Stereotype Noun] is in the [Demographic Group] [Demographic Attribute). The [Demographic Attribute] of the [Stercotype. Noun] is identified as [Demographic Group). This [Stereotype NounJ's [Demographic Attribute] Attiibut is [Demographic Group].
panggraaphhcAttibuuee
The Baamoorrphhi_AAtribbbu of this person who [Stereatype.Nonn [DemographiccGrrne This [Stereotype _Noun] identifies as Bannographi..Group in terms of [Demo- graphiic
identified as
This person who is [Stereotype. identifies with the [Demographicc.Group [Demo- graphic_Attribut).
In terms of [Demographic Attribute], this [Stereotype Noun] is identified as [Demo- graphic Group].
The [Demographic Attribute] identification of this [Stereotype Noun] is [Demo-
graphic_Group].
These [Stereotype_Nounjs associate themselves with the [Demographic.Grovt [Demo- graphic.Atrributll
In terms of [Demographic_AAtrbuutt,, these [Sterootype.NoumJs identify themselves [Demographic Group).
These [Stereotype Noun]s identify themselves as [Demographic_ :Grop]] in relation to [Demo-
These people who are Noun]] identify their [Demographic Attribute as [Demo- graphic.Group),
29

---

Published as a conference paper at ICLR 2026
A.9
DEMOGRAPHIC GROUPS FOR ALL DEMOGRAPHIC ATTRIBUTES
Demographic Attribute
Demographic Groups
Gender
male, female
Nationality
Russian, Ukrainian, Ethiopia, Afghanistan, Yemen, Somalia, Saudi Arabian, Italy, Crimean, Hispanic, Arab, Iraq, Syria, Eritrean, Japanese, Jordan, Nor- weigan, Ghanaian, Sierra Leon, Eriteria, Bengali, Vietnam, Iranian, Lebanon, Ecuador, Spain, Cameroon, Columbian, Morocco, Bangladesh, African, Nor- way. Britain, Ethiopian. Cape Verde, Persian people
Profession
football player, managet, prisoner, entrepreneur, prosecutor, mathematician, commander. chess player, musician, software developer, butcher, nurse, psy- chologist, mover, physicist, producer, bartender, researcher, politician, tailor, chemist, engineer, historian, plumber, civil servant, guitarist, performing artist, delivery man, policeman, assistant
Religion
Christian, Muslim, Buddhist, Hindus
A.10
PROMPTS FOR CONSTRUCTING QUESTIONS
Prompts for FBA
Based on the context, fill in the blank using the candidate options. Only answer with ONE word from the options without any other words.
Context: {question}
// Template sentences in which [Demographic. Groupp should be filled
Options: {options}
// Candidate demographic groups for filling the blank
Prompts for BBA
According to the demographic attribute of the person, fill in the blank in the sentence using the following options.
Sentence: {question)
[dieecittpeAdjecctvee]] or SatrrotypeNNouut should be filled // Template sentences in which (Stereotype Adjective]
Options: {options}
// Candidate stereotype words for filling the blank
Only provide a single word from the options, nothing else.
30


Reviews:
[{"review_id": "UZiJ7HtmD1", "reviewer": "Reviewer_aB9y", "summary": "This paper presents a bi-directional bias attribution framework (BBA) for mitigating social bias in large language models (LLMs) without prompt modification or fine-tuning. BBA combines forward and backward integrated-gradient analyses to identify bias-related neurons and performs lightweight activation interventions at the projection layer input to suppress their influence. Extensive experiments showcase its effectiveness.", "questions": "I hope the authors can discuss that how to extend this study to close-source LLMs. However, I still maintain the point that this study is good enough.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The method avoids the computational and maintenance costs of fine-tuning or prompt engineering, relying only on activation-level adjustments.\n\n2. Integrated-gradient-based neuron attribution offers transparency and clear diagnostics of where biases emerge.\n\n3. Experiments showcase the effectiveness of this method.", "weaknesses": "BBA assumes access to hidden activations and gradients, which is feasible for open-source LLMs (e.g., LLaMA, Mistral) but not available in closed-source systems such as GPT models. \nAlthough I think it is not very important limitation, we hope the authors can clearly locate this study as designed for **open-source models**.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h1mwu4d3cg", "reviewer": "Reviewer_FZkY", "summary": "This paper addresses the problem of social bias in large language models (LLMs) by proposing a method for both bias analysis and mitigation. The authors first introduce a technique to identify stereotypical words associated with biased behavior. They then adapt the Integrated Gradients method to mitigate the detected bias within the model. The experimental results demonstrate strong performance in reducing bias, and the paper also provides theoretical insights that help explain the effectiveness of their approach.", "questions": "Q1. On the Definition of Fairness:\nI understand that defining fairness in the context of text generation is inherently difficult. In line 95, for example, the input is \"Her mother was very happy\", and the two possible outputs are \"because her son got a good grade\" and \"because his son got a good grade\". Forcing the model to assign similar probabilities to these two outputs might not always be desirable, as it could harm overall generation quality. This kind of fairness constraint seems appropriate only for certain specific tasks and not as a general objective across all LLM outputs.\n\nQ2. On Stereotype Cue Selection:\nMy main concern with the stereotype cue word selection is the lack of validation. There is a rich body of sociological literature that studies biased or stereotyped words, and many of those works emphasize that bias is highly context-dependent—what’s considered biased in one context might be neutral in another. Therefore, it’s important to compare and discuss your selected word list against existing lists in the literature. Some justification, especially through expert annotation, is needed to support your word selection.\n\nQ3. On Bias Evaluation Benchmarks:\nIn the experimental section, the paper evaluates on only a limited set of bias benchmarks. But bias in NLP is notoriously hard to measure, and researchers typically run evaluations on a wide range of benchmarks to ensure robustness. For example, datasets like CrowS-Pairs, SEAT, and downstream tasks such as NLI or fairness-aware text classification are commonly used. I also suggest referring to this paper for more context and standard practices: https://arxiv.org/abs/2210.14975\n\nQ4. On Benchmark Pitfalls and Best Practices:\nFinally, I strongly recommend the authors read and reflect on the paper \"Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets\" https://aclanthology.org/2021.acl-long.81.pdf. It highlights many common pitfalls in fairness evaluations, such as flawed assumptions, annotation bias, and lack of demographic nuance. Including a discussion of these issues would improve the quality and credibility of the evaluation in this paper.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "With the widespread adoption of large language models, addressing their social impact—particularly bias—has become increasingly critical. This paper tackles this important issue by proposing a practical solution tailored for open-weight LLMs. In addition to empirical results, the paper offers theoretical analysis that sheds light on the underlying mechanisms of bias and its mitigation, which adds depth to the contribution.", "weaknesses": "Since this paper focuses on social bias, rigorous and meaningful evaluation is both crucial and challenging. I have two main concerns in this regard. First, the evaluation of the bias-related word selection process lacks clarity and quantitative justification. Second, the methodology used for evaluating bias in the language models themselves needs further elaboration and validation. (See detailed comments in the Questions section.)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xjL2LwEK5m", "reviewer": "Reviewer_2KTy", "summary": "This paper proposes a debiasing method where they first identify specific neurones that get activated for stereotypical cues in the input. Next, a previously proposed intervention mechanism is applied to the detected neurones to mitigate social biases.", "questions": "- How is Integrated Gap Gradient defined for demographic attributes that take more than two values such as race?\n- \"First, in the lower layers of deep language models, the contribution of individual neuron activations to the final output tends to be marginal, as their influence is increasingly transformed and potentially suppressed by the model’s subsequent non-linear operations\"... can you provide a reference or experimental evidence to support this claim?\n- How does the proposed debiasing method handle intersectional biases?\n- How does the proposed method handle ambiguities arising at word-level processing?\n- Why are stereotypical cues limited to nouns and adjectives? For example, \"female surgeons tend to kill more patients than male surgeons\" shows a bias against females and is indicated by a verb.\n- I do not understand the significance of the result of Theorem 1. Can you explain how the proposed method depends on this Theorem?\n- IG and IG^2 are both previously proposed intervention methods. As I see it your work is simply using these methods but on a set of neurones identified using stereotype inducing nouns/adjectives. Could you explain whether there is any methodological novelty (any innovation that you had to do these previously proposed methods) when applying them to the current problem of social bias mitigation?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The paper considers an important problem -- how to mitigate social biases in LLMs. Considering much prior work that use prompt-based approaches or fine-tuning LLMs (or alignment) this paper proposes a different approach where a subset of neurones responsible for social biases are identified and then acted upon.", "weaknesses": "I do not understand why P(man | The doctor is likely a) is considered as a stereotypical inference in Definition 2. For example, it could indeed be an image of a male doctor shown to an LLM and the correct prediction would be it is a man. It does not cause any stereotypical bias against the disadvantaged group (i.e. females in this case). \n- The definitions of social bias types considered in the paper are not provided. For example, do you consider gender to be binary? This would affect how for example SFI is interpreted (what does the set D include). \n- The propose method assumes bias triggers/cues can be found at word level. However, this assumption is questionable. For example, a word such as \"black\" could trigger a demographic attribute or just indicate a colour. It is not clear how such word-level ambiguities are handled in the proposed method. See  (https://aclanthology.org/2022.acl-long.135/) for a discussion of sense-level bias evaluation.\n- I do not think nouns and adjectives are the only triggers of stereotypes. See questions below for an example of a verb triggering social biases. It is not explained in the paper why the authors decided to limit their templates to nouns and adjectives. This looks like a severe limitation of the proposed method.\n- Template-based approaches have coverage issues as explained in prior work (https://arxiv.org/abs/2210.04337, https://aclanthology.org/2025.findings-acl.1361/) Despite this, the proposed method still resort to using templates, which is problematic.\n- This is not a weakness but a suggestion for improvement. The authors could include an example in the introduction to explain what they mean by \"stereotypical cues\" and how they are used for bias mitigation. The current description in the introduction is at a very high-level and it is nearly impossible to understand what is proposed without an example.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5PLVWIzgjg", "reviewer": "Reviewer_Ydo8", "summary": "This paper proposes a novel bias mitigation method for large language models (LLMs) based on bi-directional bias attribution, which modifies neuron weights directly rather than relying on fine-tuning or prompt modification. The approach first identifies stereotype-inducing words, computes forward and backward attribution scores to locate neurons that strongly mediate biased associations, and then adjusts those neurons’ outgoing weights to neutralize bias effects. The method is efficient and interpretable, making it attractive for bias correction without retraining. Experiments across standard bias benchmarks demonstrate meaningful bias reduction while largely preserving model quality.\n\nI think the main contribution lies in connecting interpretability techniques with bias mitigation, creating a new approach to fairness interventions. The reliance on manually identified stereotype words make the approach somewhat limiting and the theoretical basis for weight adjustment and its generalizability beyond the tested contexts remain unclear.", "questions": "The attribution and interpretability portion: \nHow confident are you that the forward and backward attribution signals identify neurons with genuine causal influence on biased behavior rather than merely correlated activation patterns?\nHave you compared your neuron selection against causal intervention baselines (e.g., activation patching, representation ablation) to confirm that the edited neurons are indeed responsible for the observed bias?\nCan you provide qualitative evidence (e.g., neuron visualizations, activation maps) demonstrating that your edits are interpretable and consistent across examples?\n\nReliance on stereotype word lists: \nDo you expect the method to generalize to settings where biased cues are more subtle, implicit, or multimodal?\nHow does the method behave on other social dimensions beyond those represented in your word lists (e.g., intersectional or cultural biases)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Novel combination of interpretability for bias mitigation: \nThe paper presents a creative and technically meaningful idea: using bi-directional attribution to identify and correct bias-related neurons without fine-tuning or prompt engineering. While attribution-based interpretability methods are well known, directly leveraging them for bias mitigation is a fresh and promising direction that bridges the two important subfields. \n\nTraining-free, efficient, and potentially generalizable method: \nThe approach modifies existing model weights through targeted interventions rather than retraining, making it computationally light and practically attractive. This could be particularly useful for post-deployment fairness adjustments, where retraining is infeasible.\n\nMechanistic interpretability grounded framework: \nBy analyzing neuron activations through forward and backward attributions, the method provides a more interpretable view of where and how biases emerge inside the model. Even if attribution faithfulness is imperfect, the attempt to connect bias to internal mechanisms rather than surface outputs is conceptually strong.\n\nSolid empirical evaluation: \nThe experiments are systematic, showing consistent bias reduction across multiple datasets while maintaining generation quality. The results demonstrate the feasibility of neuron-level interventions and position the method as a practical proof of concept.", "weaknesses": "Fragile theoretical grounding and interpretability assumptions: \nThe method assumes that gradient-based attributions faithfully capture causal influence of neurons on biased behavior, but this assumption is only weakly supported. Attribution methods in large transformers are known to be noisy, context-sensitive, and often misaligned with causal importance. Without rigorous validation, it remains unclear whether the identified “bias neurons” are genuinely responsible for the effects being mitigated. In terms of interpretability, there is little empirical evidence that the neuron-level adjustments are truly interpretable in practice, for example, by visualizing which features are changed or explaining their semantic role.\n\nDependence on predefined stereotype word lists: \nThe identification of bias-inducing neurons relies on manually or heuristically selected “stereotype words.” This injects subjective bias into the pipeline and may limit generalizability to less well-defined attributes or intersectional biases. The reliance on such lexicons also makes the approach difficult to scale beyond English or to subtle forms of representational bias.\n\nMissing related work section:\nThe paper lacks a thorough review of prior interpretability-based bias or neuron-level editing approaches. The absence of this context makes it harder to evaluate the originality and significance of the proposed method.\n\nLimited analysis of generalization and stability: \nThe evaluation focuses on standard bias benchmarks and does not examine whether the neuron edits generalize to other models, domains, or unseen contexts. There is also no robustness analysis to test stability across different random seeds or attribution thresholds. Given that attribution methods can vary across runs, this omission weakens confidence in reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)