FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA
Authors: Songqi Zhou
Zeyuan Liu
Benben Jiang
Venue: NeurIPS 2025 poster ()
Keywords: Fair machine learning
Fairness
Bias mitigation
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: FairNet introduces a dynamic, instance-level fairness correction method for machine learning models.

Abstract: Ensuring fairness in machine learning models is a critical challenge. Existing debiasing methods often compromise performance, rely on static correction strategies, and struggle with data sparsity, particularly within minority groups. Furthermore, their utilization of sensitive attributes is often suboptimal, either depending excessively on complete attribute labeling or disregarding these attributes entirely. To overcome these limitations, we propose FairNet, a novel framework for dynamic, instance-level fairness correction. FairNet integrates a bias detector with conditional low-rank adaptation (LoRA), which enables selective activation of the fairness correction mechanism exclusively for instances identified as biased, and thereby preserve performance on unbiased instances. A key contribution is a new contrastive loss function for training the LoRA module, specifically designed to minimize intra-class representation disparities across different sensitive groups and effectively address underfitting in minority groups. The FairNet framework can flexibly handle scenarios with complete, partial, or entirely absent sensitive attribute labels. Theoretical analysis confirms that, under moderate TPR/FPR for the bias detector, FairNet can enhance the performance of the worst group without diminishing overall model performance, and potentially yield slight performance improvements. Comprehensive empirical evaluations across diverse vision and language benchmarks validate the effectiveness of FairNet. Code is available at \url{https://github.com/SongqiZhou/FairNet}.

FairNet: Dynamic Fairness Correction without
Performance Loss via Contrastive Conditional LoRA
Songqi Zhou
Zeyuan Liu
Benben Jiang"
Department of Automation
Tsinghua University
Beijing, China
zhousonggi@tsiinguua edu. cn, 1iuzeyuan23tmail: tsinghua. edu. CI, bbj bbjiangOtsinghu. edu.
Abstract
Ensuring fairness in machine learning models is critical challenge. Existing debi- asing methods often compromise performance, rely static correction strategies, and struggle with data sparsity, particularly within minority groups. Furthermore, their utilization of sensitive attributes often suboptimal, either depending ex- cessively on complete attribute labeling or disregarding these attributes entirely. To overcome these limitations, we propose FairNet, a novel framework for dy- namic, instance-level fairness correction. FairNet integrates a bias detector with conditional low-rank adaptation (LoRA), which enables selective activation of the fairness correction mechanism exclusively for instances identified as biased, and thereby preserve performance on unbiased instances. key contribution a new contrastive loss function for training the LoRA module, specifically designed to minimize intra-class representation disparities across different sensitive groups and effectively address underfitting in minority groups. The FairNet framework can flexibly handle scenarios with complete, partial, or entirely absent sensitive attribute labels. Theoretical analysis confirms that, under moderate TPR/FRR for the bias detector, FairNet can enhance the performance of the worst group without diminishing overall model performance, and potentially yield slight performance improvements. Comprehensive empirical evaluations across diverse vision and language benchmarks validate the efffectiveness of FairNet. Code is available https:// /ttp:////tttbtt.ge Settubb.co//Sngggi/oo//aaaaaaaa
Introduction
The widespread implementation of machine learning (ML) in high-stakes domains such finance, hiring, criminal justice, and healthcare [23] (9) while promisingly increasing efficiency, has sparked significant concerns regarding fairness. primary issue the tendency of ML models inherit and potentially exacerbate latent social biases within training data 2 thus leading to discriminatory outcomes against protected groups (e.g., based on race or gender). Evidence of this includes automated financial systems that disproportionately deny creditworthy minority applicants [8), hiring tools that perpetuate gender stratification 4 and diagnostic systems that produce lower accuracy for underrepresented patient populations [28] These systematic biases erode trust in AI systems and pose considerable ethical and legal challenges 66 29). Consequently, confronting and mitigating algorithmic bias is no longer solely technical pursuit, but an urgent societal imperative.
"Corresonniigg author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

However, devising effective debiasing techniques remains challenge due to several persistent obsta- cles that limit their welldworld efficacy. A major impediment the well-documented "performance- where interventions aimed at fairness frequently diminish overall model performance (e.g. accuracy), thus limiting deployment. Additionally, current paradigms for bias mitigation, including pre-processing (3, in-processing [17] [3], and post-processing [16 35), predominantly rely on static global adjustments. Such Cone-size-fits-alll interventions disregard the instance-level nuances of bias, risking suboptimal outcomes like over-correction for some samples and under-correction for others. Compounding this, models trained via standard Empirical Risk Minimization (ERM) often struggle with data sparsity in minority subgroups, leading to underfitting and disproportionately high error rates for these populations [T], 21 38). Finally, existing methods lack flexibility in leveraging sensitive attributes: many demand complete annotations, which are often impractical or costly to obtain, while others entirely discard potentially valuable partial label information, thus limiting the potential for more targeted and effective bias mitigation 12 22
fairness trade-off' [32
14 30),
To address these limitations, we propose FairNet, novel framework for dynamic, instance-level fairness correction operating internally within the model. Unlike conventional static approaches, FairNet employs an internal, lightweight bias detector that analyzes instance representations during inference. Based this detection, parameter-efficient Low-Rank Adaptation (LoRA) modules [13 are conditionally activated to apply targeted, corrective adjustments directly at the representation level, enabling nuanced bias mitigation. These adjustments are applied selectively to instance representations identified as potentially biased. preserving performance on others and thus addressing the carooommmnnc--fiimeseeeee trade-off. We train the corrective LoRA module using a novel contrastive loss specifically designed to minimize intra-class representation gaps between sensitive attribute groups, thereby tackling minority subgroup underfitting. Furthermore, FairNet is architected for flexibility, effectively leveraging full, partial, or even absent sensitive attribute labels.
Complementing the framework's 's design, our theoretical analysis provides formal performance guarantees. Specifically, we establish that under moderate and practically achievable conditions on the bias detector's 's true positive rate (TPR) and false positive rate (FPR), FairNet can provably enhance performance on the perst-performing subgroup without sacrificing overall performance, and may even yield marginal gains. Our main contributions are:
1. A novel framework, FairNet, enabling dynamic, instance- faimess correction via conditional LoRA, offering new paradigm for selective bias mitigation operating directly within model representations.
2. A targeted contrastive loss function specifically designed to minimize intra-class repre- sentation discrepancies across sensitive groups, effectively mitigating the common issue of minority subgroup underfitting at the representation level.
3. An adaptable framework design demonstrating effectiveness and flexibility across varying levels of sensitive attribute label availability, including scenarios with full, partial, or entirely absent labels, enhancing practical applicability.
4. Theoretical guarantees coupled with comprehensive empirical validation across diverse vision and language benchmarks, demonstrating FairNet's ability to enhance worst-group accuracy without sacrificing overall performance.
2 Related Work
2.1 Diversity and Challenges of Fairness Definitions and Metrics
Fairness in machine learning is predominantly investigated within two paradigms: group fairness and individual fairness. Group fairness aims to achieve statistical parity across protected groups defined by sensitive attributes S. targeting conditions such Demographic Parity (DP) M. Equal Opportunity (EOP) [10], or Equalized Odds (EOD), which enforces equality of both true positive and false positive rates [10.. Evaluation metrics typically include disparities in these statistical rates across groups and measures like Worst- Worst-Group Accuracy (WGA) [1]. In contrast, individual faimess requires that the model generates similar outputs for individuals who are similar with respect to a task-relevant distance metric d(z,, x [5] Assessing individual fairness often involves imposing Lipschitz constraints or testing consistency in sampled pairs, but defining proper similarity metric remains a key challenge.

---

Given the inherent trade-off's between different faimess definitions [20] and the lack of a universally optimal notion. FairNet avoids directly optimizing for any single faimess criterion. Instead, identifies pefformance-vulleraae applies a Contrastive Conditional LoRA correction module to refine their internal representations and predictions. The objective of FairNet is to enhance performance for these disadvantaged subgroups. thereby indirectly improving metrics such as WGA and reducing EOD gaps, without enforcing a potentially abstract fairesss constraint that may not align with the specific application context.
instances (often associated with minority groups) and dynamically
2.2 Fairness Intervention Strategies and Performance-Fairee Trade-off'
Bias mitigation techniques are commonly categorized by their intervention stage within the machine learning pipeline: pre-processing (e.g., data rebalancing) (3], in-processing (e.g., fairness-aware regularization or adversarial training) [36], and post-processing (e.g., output adjustment) [10] 22].. Despite these methods targeting different stages of the ML pipeline, pervasive challenge is the well-documented performance-fatin accompanied by decline in predictive performance. This trade-off is especially pronounced for static, global interventions that apply uniform corrections, failing to capture the instance-level heterogeneity of bias and risking both over- and under-correction.
trade-off [7 where improvements in fairness are often
FairNet introduces a novel dynamic, instance-level intervention strategy. As an in-processing method, it employs lightweight bias detector that assesses instance representations during inference, selectively activating parameter-efficient Low-Rank Adaptation (LoRA) modules only for instances identified as potentially biased. This targeted correction the representation level minimizes disruption to unbiased instances. By focusing corrections where they are most needed, FairNet enhances fairness particullryy for underperforning subgroups (e.g., improving WGA) while largely preserving overall model performance, thus effectively mitigating the conventional trade-off.
2.3 Fairness under Varying Sensitive Attribute Constraints
Fairness algorithms vary in their reliance on sensitive attribute labels (S, e.g., race, gender). Many pre- and in in-processing methods, such as GroupDRO [31] require complete S labels to compute group statistics or implement group-aware aware regularization. However, full access to S is often impractical due to privacy concerns, cost, data scarcity. Other approaches attempt to function without any labels [11], relying on uncertainty or proxy attributes, though this can limit correction precision. third line of work explores the use of partial S label information |2 : for instance, JTT [22] leverages a small labeled validation set to guide the upweighting of challenging samples. Despite these advances, existing method provides unified solution that seamlessly accommodates all sensitive attribute scenarios-fully labeled, partially labeled, and unlabeled.
FairNet is designed with inherent flexibility bridge this gap. Its internal bias detector can be trained under full S supervision, adapt to partially labeled data by utilizing the available subset, or switch to unsupervised strategies (c.g.. representation- based outlier detection) when S is absent. This versatility allows FairNet to generalize beyond rigidly label-dependent methods, thereby enhancing its deployability in real-world scenarios.
3 Method
We introduce FairNet, framework designed to dynamically correct faimess biases at the instance level directly within a pre-trained model fo(-). FairNet aims to enhance fairness, particularly for minority subgroups, by selectively adjusting internal representations during inference, while minimizing the impact on overall task performance.
3.1 Framework Overview and Inference Process
FairNet enhances a base model fo by integrating two main components: Bias Detection modules (D))) positioned after intermediate layers I, and Conditional LoRA modules (L Joaa) associated
with layers j. The detectors (1) parameterized by é, monitor intermediate representations h,0) to identify samples that may belong minority sensitive group 1), and output risk score ps
3

---

The LoRA modules, parameterized by low-rank matrices A, E R and B, E R (with rank r (in k)). provide targeted adjustments AW; = B, to the model's weights W,- The core idea is that high-risk score detected (p.) > 7) triggers the activation of relevant LoRA modules during inference, applying corrective adjustments only when needed. The overall architecture depicted in Figure We summarize the notation in Supplementary Table
The inference process for given input sample proceeds as follows First, necessary intermediate representations (x)) are computed using the base model fa. Second, all bias detectors D evaluate
these representations to produce risk scores ps (z)) Third, based on these scores and a predefined threshold set of LoRA modules is identified for activation. Finally, forward pass is performed using the FairNet-enhanced model fFaieNet, where only the weights Wy corresponding to the activated
LoRA modules are adjusted (W, + I[p!5) (9) (x) > r) (B))))) to produce the final prediction Upred-
Minorty
Pretraned Weight
Check
Majority
Ja
?
Hactivated
Continue
Condition
No group .. Activtte
Backtrack
Training Stap1
Stap2
Step3
Step4
[Extrct Enbeddingg)
Majorty
Minority
Class
Class
Datector
Label guided
classification/ clusteriig
Figure 1: FairNet framework and conceptual training steps. (Top) The overall architecture featuring Bias Detectors and Contrastive Conditional LoRA (CL) modules. (Bottom) A four-step conceptual training process: (1) Base model preparation, (2) Bias Detector training, (3) Contrastive pair em- bedding preparation, and (4) Conditionally training LoRA modules with Contrastive Loss based on detector signals.
3.2 Bias Detection Module
The Bias Detector ( acts as internal monitor at layer l. It takes the intermediate representation
(x)) as input and outputs score pa (x) E [0, 1] indicating the likelihood of belonging to the minority group (s = 1). Detectors are designed as lightweight networks (e.g., MLPs possibly preceded by attention pooling [33 to minimize computational overhead.
Detectors are trained using a standard supervised loss, typically binary cross cross-entropy (BCE), on the subset of training data Praneled where sensitive attribute labels available:
I
dotector = (2.z)) lssa (D D (x)), s)]
This enables FairNet to operate under partial information settings. To counteract potential class imbalance within Prateted and ensure high sensitivity (TPR) minority group samples, techniques like weighted loss or worst-group loss optimization strategies [31] can be applied during detector training. For handling multiple sensitive attributes 8; (e.g., race 81, gender 82). separate detectors D are trained for each attribute using the corresponding available labels. Each detector (!)) independently assesses the risk related attribute 8;. Alternatively, single detector can be employed using multi-label prediction.

---

3.3 Contrastive Conditional LoRA Module
The Conditional LoRA module L cond Jora provides the mechanism for the correction of the targeted representation at the layer j. It uses the standard LoRA parameterizztion [13] but with two key distinctions: conditional activation (based on detector signals) and unique training objective.
The key innovation is that the LoRA parameters A,, B3 are trained exclusively via a novel contrastive loss, L (
comttasttvve This loss directly targets the fairness goal by aiming minimize intra-class, inter-group representation gaps. Specifically, it encourages the LdRA-adjusted representations (r) BaseOutput, (z) + [(trigger(() (B)-A)) (input,) to be similar for samples (Tiin Tmmj) that share the same true class label y but belong to different sensitive groups (s vs. 0). We employ triplet loss formulation:
L (5) contrastive (Fa: Tp. Fn) [D(2 (xa), 20) (xp))
D(2 (xa), (z.)) margin] +
(2)
Intra-class, Inter-group dist.
Anchor-Negative dist.
where is a minority anchor (label Va. 1), Tp is majority positive (label V/ = ya. 0), a negative sample, (): is distance metric (e.g., squared Euclidean distance), [++ max (0.-). and margin > 0 is hyperparameter. The selection of Zp (same class, different group) is crucial for achieving the targeted representation alignment. For multiple sensitive attributes 8;, separate
LoRA modules _ J (with parameters A,,i, By.;) are introduced. Each module I L Joraj is
trained using an attribute-specific contrastive loss _ contrastive_ which aims to align representations
across groups defined by attribute 8i, triggered by the corresponding detector D( p.
3.4 Model Training
The training process optimizes the base model parameters 0 (optional fine-tuning), detector parameters 6. and LoRA parameters Aj. By. This optimization proceeds through a multi-stage approach aimed minimizing a composite objective function:
Lioial = Lusk (frainea (x), y) + AD Σ detector + Ac
Σ [triigger (i))
(3)
contrastive
Here, Lusk (e.g., cross-entropy) ensures the model maintains task accuracy. L detector trains the bias detectors available labeled data Paaeled- The crucial term ( contrastive drives the fairness correction via representation alignment, and its contribution is gated by the indicator [(trigger())) which is 1 only if the contrastive loss of the j -th LoRA module was activated by downstream detector for the current batch anchors, and otherwise. Ap, Ac 2 0 are hyperparameters that balance these objectives.
The conceptual training encompasses four sequential stages as depicted Figure (Bottom): (1) Base Model Preparation, which involves initializing establishing the foundational model, potentially leveraging pre-trained parameters; (2) Bias Detector Training, entailing the development and training of specialized detector modules responsible for identifying inputs likely belonging to predefined (minority) subgroups based on intermediate representations; (3) Contrastive Pair Embedding Preparation, which consists of generating and processing embeddings from input samples to construct the necessary positive and negative pairs (or triplets) required for the contrastive learning objective; and (4) Conditional LoRA Module Training, which involves the fine-tuning of LoRA modules using a contrastive loss formulation, where the application of LoRA-basec adjustments is dynamically triggered by the signals originating from the trained bias detectors.
4 Theoretical Analysis
In this section, we provide a theoretical analysis FairNet. We first analyze how the proposed contrastive loss fosters representation faimesss. Then, we derive conditions under which FairNet can provably improve fairness- specificaily worst-group performance-
sacrificing, and potentially even enhancing, overall model accuracy. This analysis distinguishes our method from conventional approaches that are often subject to performance-fainnes trade-off. The detailed mathematical notations for all theorems provided in the Supplementary Material
5

---

4.1 Preliminaries
We consider data drawn from joint distribution P(X, ((X,Y.S) S),, where X X' are inputs, Y E y are task labels, and S E {0, 1] is binary sensitive attribute. We denote the majority group (S = 0) as G1 and the minority group (S 1) as G2, with P(S = 1) = p. Let M fo be the base pre-traines model and MrainNt be the model enhanced with FairNet's bias detectors
_ and conditional LoRA modules L L Jom (parameters A,. B,). Let 1 be the representation at layer in the base model, and 200 (z) be the representation at layer j potentially adjusted by L
We evaluate performance using group-conditional accuracy P(M,G) P( = YIG) and overall accuracy P(M) = (1 - P)M, G1) + pP(M, G2). where Y M (X). Fairnsss is assessed using metrics like WGA, WGA(M) = ( P(M, G), and conditional fairness metrics like
EOD. The bias detector D is characterized by its True Positive Rate (TPR, = P(D (1("))= >
T|S 1)) and False Positive Rate (FPRd P(D (h()) r|S = 0)), where is the activation threshold. The detailed derivations are provided in the Supplementary Material
4.2 Contrastive Loss and Representation Fairness
The core mechanism for improving fairness on FairNet is the custom contrastive loss I, T used to train the conditional LoRA modules. This loss targets intra-class, inter-group representation alignment. By minimizing the distance ((2(a) (xa), (x:p) between samples za (minority, label y) and Tp (majority, label y). the training process encourages the adjusted representations 200 to become invariant to the sensitive attribute s, conditional on the true class label y.
comttustive (Eq.
Successful minimization of (5)
contrastive implies that for a given class y. the distribution of repre- sentations P()())y y. S = 0) approaches P(z())|
S = 1). Theoretical results in fair representation learning suggest that, firstly, by enabling minority group representations to learn from those of the (often better-performinn cally enhanced (which in turn can improve WGA), and secondly, if representations are conditionally independent of the sensitive attribute given the true label, then any classifier relying solely these representations will satisfy conditional fairness criteria like EOp and EOD. Specifically, reducing the discrepancy between P = y, S 0) and P(=0)|y = y, = 1) expected bound the downstream EOD. Thus, FairNet's contrastive loss directly targets the representational disparities that underpin conditional fairness violations.
majority group, the performance of the minority group is typi-
4.3 Performance Preservation Analysis
A critical aspect of Fair is its ability to enhance fairness without degrading overall performance P(M). We analyze the change in overall performance AP = P P(M)) Let P (M,,RA G) denote the hypothetical accuracy if the contrastively trained LoRA modules were unconditionally applied to all samples from the group G. The actual performance of FairNet on each group depends on the detector's 's rates (TPRp, F PRR):
P(MPainNce- G1) = (1 FPRD) P(M, + FPRD F(M, P(MLORA, G1)
(4)
P(MPaiNer, G2) TPRn P(MLORA, G2) +2)+(() (1-TPR)) G(M,G2)
(5)
These follow from partitioning each group based whether the detector triggers the LoRA correction (correctly for G2 with probability TPRp, incorrectly for G₁ with probability FPR)).
The overall performance change is the weighted average of group performance changes:
AP = (P))PPPPPMMaaae G1) - P(M, G,)] + P|P G2) - P(M, G2)]
(6)
= (1-p) (1-p))PRDp P(MLOA G1) P(M, G1) + PTPpp (MMLLA G2) - ((M,G2))
(7)
To ensure non-decreasing performance (AP > 0), assuming P(M -2)-P(M- G2) > 0 (LoRA
improves minority performance) and FPRD > O, rearranging Eq. 7 yields the condition:
TPRp 1 1 P(M,G1) P(MLOR G1)
FPRD Z P(MLORA. G2) - P(M,G)
(8)

---

This condition relates the detector's quality (ratio FPRp/FPRD) to the relative population sizes (1-p)/p) and the differential impact of the LoRA correction on the two groups.
Why FairNet Satisfies the Condition: FairNet's design, particularly its contrastive LoRA training objective, makes it highly likely to satisfy Condition g
1. Large Positive Denominator: LOMLLRA, G2) P(M, G2) is expected to be significantly
positive. The base model M often performs poorly in the minority group G2. The contrastive loss Leontrastive explicitly trains the LoRA modules to improve G2 's representations by aligning them with G1 's (presumably better) representations within each class. This targeted improvement should substantially increase POMORAA G2) over P(M, G2).
2. Small Negative Numerator:
P(M,G) - P(MI.RA, G1) expected be small or even negative. Since the contrastive alignment target is the majority group's representation space, applying the resulting LoRA correction (which only happens for G1 samples due to detector false positives) should ideally have minimal negative impact on G1's performance. The correction might slighdl perturb G 's representations, potentially causing small performance drop (MMM G1) < P(M, G. but this effect is anticipated to be much smaller than the gain for G2. is even possible that the alignment process slightly regularizes or improves the robustness of G₃ 's representations, leading to P(MOOAA G1) 2 P(M, G1). making the numerator non-positive.
3. Achievable Condition:
Consequently, the ratio P.M.G.)-PMMaa..C G.) P(M P(M.G2) is expected to be a small positive value, zero, or negative. As long as the bias detector has reasonable discriminative ability, the inequality in Condition is likely to hold.
Contrast with Other Architectures: Many existing bias mitigation strategies lack an explicit internal detection mechanism, rendering them incapable of exploiting the theoretical performance preservation condition (Condition 8). which fundamentally hinges on the detector's 's efficacy.
Conventional bias mitigatttn methods uniformly apply global static interventions and thus can- not escape the inherent centormmnne--airneeee trade-off. Pre-processing techniques rebalance data via resampling or sereigghinn-theeeee perturbing the original distribution and impairing gener- alization-while in-processing approaches embed fairness constraints into training, which often conflict with the primary optimization objective and hinder the learning of optimal decision bound- aries. Post- processing adjustments rectify outputs to meet faimess metries but leave biased internal representations intact and may degrade predictive accuracy or impair calibration.
FairNet's architecture- explicitly integrating an internal bias detector with a conditional correction module-is expressly designed to circumvent this limitation.
Experiments
We empirically assess FairNet by (i) benchmarking its accuracy-fairne trade-off against strong SOTA baselines and (ii) running targeted ablations that isolate the impact of its key components. Additional experiments and analyses appear in the Supplemental Material
5.1 Experimental Setup
We evaluate FairNet on three diverse datasets: CelebA 24). MultiNLI (34,, and HateXplain [26] which represent different modalities and bias types. For CelebA, we predict the "Male" attribute while accounting for "Blond Hair" as a sensitive attribute, revealing imbalances across male and female images with blond hair. In MultiNI.I we predict entailment relations with focus on negation as a sensitive attribute, uncovering linguistic biases. HateXplain helps assess overlapping biases related to gender and race, focusing on hate speech prediction. We fine-tune VIT for vision tasks and BERT for language tasks; full experimental details are provided in the Supplementary Material
We evaluate three versions of FainNet based on the availability of sensitive attribute labels.
FairNet-Full assumes complete sensitive attribute labels (S) are available for both training and validation; its bias detector (D directly uses these ground-truth sensitive labels.
7

---

FairNet-Parrie is designed for scenarios where the training set contains only partial sensitive attribute labels (c.g. k% of samples are labeled with S), and the validation has no S labels. In this setting, the bias detector is trained on the laheled subset of the training data, and the contrastive LoRA module utilizes these available labels.
addresses cases where no sensitive attribute labels are available in either training or validation. Here, the bias detector (D)) employs unsupervised methods (outlier detection on h (())) to generate pseudo-sensitive attribute labels (3). which are then used by the L C Lood_Jora to form corrective pairs.
5.2 Comparison with Existing Methods
We benchmark FairNet against a representative spectrum of bias-mitigation baselines that span the three canonical intervention stages pee-proreesseeeee rre-processing, in-processing. and post-processing. We further stratify methods based on the degree of sensitive-attribute availability into three scenarios: (1)
no access to sensitive labels during both training and testing; (2) access to sensitive labels during training but not at test time; and (3) access to sensitive labels at both training and testing. To assess performance and fairness, we primarily report overall task accuracy (ACC), WGA evaluate the performance the most disadvantaged group, and the EOD to measure inter-group disparities in true positive and false positive rates across distinct datasets, CelebA and MultiNLI.
Table 1: Performance comparison across different datasets and methods
Group Info
CelebA %
MultiNLI %
Method
Train Test
ACC1
WGA
EOD
ACC1
WGAt
EOD
ERM
X x
95.8
77.9
10.6
82.6
67.3
12.5
Lu et al. 25
X
95.4
81.4
8.3
82.0
72.8
8.5
D3M
X 95.2
82.0
8.1
81.0
72.8
8.3
FairNet-Unlabel
x x/x
95.8
82.3
7.3
82.5
73.1
8.1
GroupDRO
V1x
94.0
87.4
4.7
80.8
78.2
5.5
DFR 19
partial x
94.3
86.0
7.7
81.2
74.1
6.7
Sebra 18
partial
94.8
85.2
8.1
81.5
74.2
6.5
FairNet-Partial
partial x
95.9
86.5
5.6
82.6
76.5
6.2
Eq
VIV
95.0
83.2
7.2
81.3
75.3
6.3
GSTAR [6
V1V
94.2
85.4
6.6
80.8
76.6
6.2
Faillet-Full
V1V
95.9
88.2
3.8
82.6
78.5
4.7
Bold indicates the global best performance; Underlined indicates the best in each category.
No Access to Sensitive Attributes: In the absence of sensitive attributes, FairNet--nlatie demon- strates strong performance. On CelebA, achieves an ACC 95.8%, matching ERM, while attaining the highest WGA (82.3%) and the lowest EOD (7.3%). On MultiNLI PairNet-Unlaber again achieves competitive ACC (82.5%, matching ERM) and secures the best WGA (73.1%) and EOD (8.1%) among methods in this category. These results highlight FairNet's capacity to improve fairness in the absence of explicit group supervision, without compromising overall performance.
Partial Access to Sensitive Attributes: When sensitive attributes are partially available during training. FairNet-Partial yields the highest ACC on both CelebA (95.9%) and MultiNLI (82.6%). On CelebA, its WGA (86.5%) and EOD (5.6%) are competitive, ranking second to GroupDRO (WGA 87.4%, EOD 4.7%) but notably improving over DFR and Sebra. On MultiNLI, FairNet- Partial achieves WGA of 76.5% and an EOD of 6.2%. again demonstrating a strong balance of accuracy and fairness, outperforming DFR and Sebra in these aspects.
Full Access to Sensitive Attributes: Given full access to sensitive labels during both training and testing, FairNet-Full consistently outperforms all other methods across all metrics. On CelebA it achieves state-of-the--rt results with ACC 95.9%, WGA 88. 28,, and HOD 3.8%. Similarly, on MultiNL.I, FairNet-Ful leads with ACC 82.6%, WGA 78.5%, and EOD 4. 4.7%. These results highlight FairNet's s efficacy when complete group information is available.

---

Overall Comparison: Across all evaluated scenarios, FairNet variants demonstrate a compelling balance between overall accuracy and fairness metries. FairNet effectively adapts to varying levels of sensitive attribute availability, consistently improving WGA and reducing EOD relative to baselines, often while maintaining or exceeding their ACC. The performance on fairness metrics generally improves with increased access to sensitive information, showcasing the robustness and versatility of the FairNet framework.
5.3 Efficacy in Mitigating Intersectional Bias
Beyond single-axis biases, real-world applications demand fairness across multiple, often intersecting, sensitive attributes such race and gender. To evaluate FairNet's capabilities in such complex scenarios, we conducted experiments on the HateXplain dataset, which contains multifaceted biases. Using FairNet-Partien we applied a progressive debiasing strategy: first targeting racial bias (related to the "African American" demographic) and subsequently targeting gender bias (related to the "Female" demographic).
Table 2: Performance and fairness during progressive debiasing on the HateXplain dataset.
DistiBBERT-base
BERT-base
Metric
ERM
FairNet Afr.
FairNet Fe.
ERM
FairNet Afr:
FairNet Fe.
DP (R)
38.2 + 1.4
33.7 + 1.4
32.8 = 1.1
27.1 + 0.9
14.0 = 1.0
12.4 + 0.7
EOp (R)
14.9 + 1.1
14.2 + 1.0
13.1 =11=100
13.0
8.4
7.2 + 1.0 9.8 1 0.5
EOD (R)
26.5 1.51:00 0.7
0.6
23.0 0.6
2.4
11.2 0.6
DP (G)
7.4 1.3
7.6
2.2
1.5
8.5
7.6 + 1.0
EOp (G)
1..
0.5
2.0
1
16.7 0.4
8.8 1.4
EOD (G)
11.3
11.2
7.4 0.6
1219
12.6 0.9
8.2 + 0.4
ACC1
79.5 19.5 0.2
79.6 + 0.2
79.7 0.3
79.8 + 0.3
79.6 0.5
79.7 + 0.4
Bold values indicate the best performance each category. while underlined values represent the second-best results. "R" refers to Race (African American vs. Other), and "G" refers to Gender (Female vs. Male).
The results, presented in Table demonstrate FairNet's s capacity for effective, progressive debiasing. Applying the correction for the "African American" group (FairNet Afr.) successfully reduces racial bias across both models. For DistilBERT-base, the EOD (R) improves from 26.5% to 24.4%, while for BERT-base, the reduction is more pronounced, from 20.1% to 11.2%.
Crucially, the subsequent application of a fairness module for the "Female" group (FairNet Fe.) not only preserves enhances the initial gains in racial fairness but also delivers substantial improvements in gender fairness. This progressive approach further lowers EOD (R) to 23.0% for DistilBERT and 9.8% for BERT-base. Simultaneously, it dramatically reduces gender- based disparity, with EOD (G) dropping from 11.3% to 7.4% on DistilBERT, and from 12.9% to 8.2% on BERT-base.
This multi-facette bias mitigation is achieved without compromising task performance. The overall accuracy is consistently maintained or slightly improved across all interventions for both models. These findings validate the modularity and adaptability of the FairNet architecture, showcasing its ability to address complex, multi-attribute fairness challenges a targeted manner while upholding overall model performance. This is vital characteristic for real-world deployments where fairness considerations often span several demographic dimensions.
5.4
Ablation Studies
To quantify the contribution of FairNet's core components the bias detector and the contrastive loss-we conduct ablation experiments on the CelebA and MultiNLI datasets. We evaluate the impact of removing each of these components individually, as well as removing both simultaneously. All experiments are conducted under partial sensitive-attribur setting, in which both the attribttt label and aesesitive-attributee primary controlled ablations focus
scenarios are treated as special cases of partial observation. The

---

1. Detector Ablation:
We disable the bias detector D so that conditional LoRA modules remain active on every input. This variant measures how selective gating via the detector preserves accuracy on non-biased instances while still applying corrections where required.
2. Contrastive-Loss Ablation:
contrastive with binary cross entropy loss on flagged instances. This experiment evaluates the role of the contrastive ob- jective in closing intra-class representation gaps and boosting minority- group performance.
We replace our contrastive loss L (
Table 3: Ablation study results of FairNet on CelebA and MultiNI.I datasets.
CelebA
MultiNLI
Method
ACC (%)
WGA (%)
EOD (%)
ACC (%)
WGA (%)
EOD (%)
FairNet-Partian
95.9
86.5
5.6
82.6
76.5
6.2
w/o detector
94.1
86.7
5.3
81.0
77.2
5.7
w/o contrastive loss
95.8
81.2
8.5
82.5
70.1
9.2
w/o both
94.3
82.3
7.8
81.3
71.8
8.9
ERM
95.8
77.9
10.6
82.6
67.3
12.5
Bold indicates the best; Underlined indicates second-best.
The results of our ablation studies (Table 3) reveal the consistent contributions of FairNet's com- ponents across both datasets. Disabling the bias detector ('w/o detector") leads to notable ACC drop on both CelebA (from 95.9% to 94.1%) and MultiNLI (from 82.6% to 81 81.0%). even as fair- ness metrics like WGA and EOD marginally improve. This highlights the detector's key role in maintaining high overall accuracy by applying corrections selectively. Conversely, removing the contrastive loss ("w/o contrastive loss") substantially degrades fairness on both datasets with only minor change in ACC. On CelebA, WGA falls from 86.5% to 81.2%, and MultiNLI it drops
from 76.5% 70.1% 1%. This confirms the critical function of L [(i) representations. The "w/o both" variant, lacking both core components, shows fairness performance superior to the ERM baseline but significantly lower than the full FairNet model. These findings affirm the synergistic importance of the bias detector for preserving accuracy and the contrastive loss for enhancing fairness.
contrastive in improving minority group
Conclusion
We propose FairNet, dynamic, instance-conditioned framework that reconciles fairness and ac- curacy by pairing lightweight bias detectors with contrastively trained conditional LoRA adapters. Unlike global debiasing schemes that impose uniform changes every example, FairNet activates corrective updates only for samples predicted to be vulnerable, thereby preserving performance for the majority group while narrowing critical error gaps for protected subgroups.
Theoretical analysis shows that, whenever the detector achieves a sufficiently high TPR /FPR ratio, FainNet provably improves worst-group performance without diminishing-ann often slightly improv- ing- ing-overall accuracy, thus breaking the long-assumed performance-fainee trade-off. Extensive experiments vision and language benchmarks and datasets confirm these guarantees under three practical settings: fully labeled, partially labeled, and unlabeled sensitive attributes. In nearly all cases, FairNet delivers the highest WGA and the lowest EOD while matching surpassing strong ERM baselines in overall accuracy. Ablation studies further reveal that both selective activation and the contrastive objective are indispensable to these gains.
FairNet's modular, ssstancecoonitiioned design by successfully decoupling fairness interventions from overall performance impacts impacts-represeeeee represents a significant advancement. Backed by both theoretical guarantees and extensive empirical validation, offers a practical pathway towards deploying Al systems that are both high-performing and equitable, effectively moving beyond traditional compromises. Future work will address complex intersectional biases and integrate advanced unsupervised detection methods, further broadening FairNet's 's impact on trustworthy Al.
10

---

Acknowledgements
This work supported by the "singhuaTooyol Joint Research Fund, the National Natural Science Foundation of China (62273197), and the Beijing Natural Science Foundation (L233027).
References
[1] Solon Barocas and Andrew D Selbst. Big data's disparate impact. Calif. L. Rev. 104:671, 2016.
[2] Tolga Bolukbasi, Kai- Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. Man computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29, 2016.
[3] Junyi Chai and Xiaoqian Wang- Fairnsss with adaptive weights. In International conference machine learning, pages 2853-2866. PMLR, 2022.
[4] Jeffrey Dastin. Amazon scraps secret ai recruiting tool that showed bias against women. In Ethics of data and analytics, pages 296-299, Auerbach Publications, 2022.
[5] Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 216-226, 2012.
[6] Luciano Floridi and Josh Cowls. A unified framework of five principles for ai in society. Machine learning and the city: Applications in architecture and urban design, pages 535-545, 2022.
[7] Sorelle A Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan Hamilton, and Derek Roth. A comparative study of fairness-enhancing interventions in machine learning. In Proceedings of the conference on fairness, accountability, and transparency, pages 229-338, 2019.
[8] Andreas Fuster, Paul unequal? the effects of machine learning on credit markets. The Journal of Finance, 77(1):5-47, 2022.
Tarun Ramadorai, and Ansgar Walther. Predictably
[9] Arjun Ghosh, Ankit Saini, and Himanshu Barad. Artificial intelligence in governance: recent trends, risks, challenges, innovative frameworks and future directions. Al & SOCIETY, pages 1-23, 2025.
[10] Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
[11) Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographies in repeated loss minimization. In International Conference on Machine Learning, pages 1929-1938. PMLR, 2018.
[12] Max Hort, Zhenpeng Chen, Jie M Zhang, Mark Harman, and Federica Sarro. Bias mitigation for machine learning classifiers: A comprehensive survey. ACM Journal on Responsible Computing, 1(2):1-52, 2024.
[13] Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference Learning Representations, 2022.
[14] Bar Iluz, Yanai Elazar, Asaf Ychudai, and Gabriel Stanovsky. Applying intrinsic debiasing on downstream tasks: Challenges and considerations for machine translation. In Proceedings of the 2024 Conference on Empirical Methods Natural Language Processing, pages 14914-14921. 2024.
[15] Saachi Jain, Kimia Hamidieh, Kristian Georgiev, Andrew Ilyas, Marzyeh Ghassemi, and Aleksander Madry. Improving subgroup robustness via data selection. Advances in Neural Information Processing Systems, 37:94490-94511, 2024.

---

[16] Taeuk Jang, Pengyi Shi, and Xiaoqian Wang. Group aware threshold adaptation for fair classification. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, 2022.
pages
[17] Zhimeng Jiang, Xiaotian Han, Chao Fan, Fan Yang, Ali Mostafavi. and Xia Hu. Generalized demographic parity for group fairesss. In International Conference on Learning Representations, 2022.
[18] Adarsh Kappiyath, Abhra Chaudhuri, AJAY KUMAR JAISWAL, Ziquan Liu, Yunpeng Li, Xiatian Zhu, and Lu Yin. Sebra: Debiasing through self-guided bias ranking, The Thirteenth International Conference on Learning Representations, 2025.
[19] Polina Kirichenko, Pavel Izmailov, and Andrew Gordon Wilson. Last layer re-training is sufficient for robustness to spurious correlations. In The Eleventh International Conference on Learning Representations.
[20] Jon Kleinberg, Sendhil Mullainathan, and Manish Raghavan. Inherent trade-offs in the fair determination of risk scores. In 8th Innovations in Theoretical Computer Science Conference für Informatik, 2017.
(ITCS 2017), pages 43-1. Schloss Dagstubl-Lcibniz-Zentee
[21] Hongkang Li, Shuai Zhang, Yihua Zhang. Meng Wang. Sijia Liu, and Pin-Ya Yu Chen. How does promoting the minority fraction affect generalization? a theoretical study of one-hidden-layer neural network on group imbalance. IEEE Journal of Selected Topics in Signal Processing, 2024.
[22] Evan Liu, Behzad Haghgoo, Annie Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Perey Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pages 6781-6792. PMLR, 2021.
[23] Zeyuan Liu, Xin Zhang, and Benben Jiang. Active learning with fairness-aware clustering for fair classification considering multiple sensitive attributes. Information Sciences, 647:119521, 2023.
[24] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of the IEEE International Conference Computer Vision, pages 3730-3738, 2015.
[25] Shenyu Lu, Yipei Wang, and Xiaoqian Wang. Debiasing attention mechanism in transformer without demographics. In The Twelfth International Conference on Learning Representations, 2023.
[26] Binny Mathew, Punyajoy Saha, Seid Muhie Yimam, Chris Biemann, Pawan Goyal, and Animesh Mukherjee. Hatexplain: A benchmark dataset for explainable hate speech detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 14867-14875, 2021.
[27] Ninarch Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey bias and faimess in machine learning. ACM computing surveys (CSUR), 54(6):1-35, 2021.
[28] Ziad Obermeyer, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. Dissecting racial bias in an algorithm used manage the health of populations. Science, 366(6464):447-453, 2019.
[29] Oluwatofunnni Oguntibeju. Mitigating artificial intelligence bias in financial systems: comparative analysis of debiasing techniques. Asian Journal of Research in Computer Science, 17112):165-178, 2024.
[30] Drago Plecko and Elias Bareinboim. Fairness-accuracy trade-offs: A causal perspective. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 26344-26353, 2025.
12

---

[31] Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang, Distributionally robust neural networks. In International Conference on Learning Representations, 2020.
[32] Hua Tang, Lu Cheng, Ninghao Liu, and Mengnan Du A theoretical approach to characterize the accuracy-fairness trade-off pareto frontier. CoRR, 2023.
[33] Mingyang Wan, Daochen Zha, Ninghao Liu, and Na Zou. In-processing modeling techniques for machine learning fairness: survey. ACM Transactions on Knowledge Discovery from Data, 17(3):127,, 2023.
[34] Adina Williams, Nikita Nangia, and Samuel R Bowman. broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long Papers), pages 1112-1122, 2018.
[35] Ruicheng Xian, Lang Yin, and Han Zhao. Fair and optimal classification via post-processing. In International conference on machine learning, pages 37977-38012. PMLR, 2023.
[36] Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez. and Krishna P Gummadi. Fairness beyond disparate treatment & disparate impact: Learning classification without dis- parate mistreatment. In Proceedings of the 26th International Conference on World Wide Web,
pages 1171-1180, 2017.
[37] Zhe Zhang, Shenhang Wang, and Gong Meng. A review on pre-processing methods for fairness in machine learning. In The International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery, pages 1185-1191. Springer, 2022.
[38] Xuan Zhao. Klaus Broelemann, Salvatore Ruggieri, and Gjergji Kasneci. Enhancing fairness through reweighting: A path to attain the sufficiency rule. In ECA/ 2024, pages 794-801. IOS Press, 2024.
13

---

Supplementary Material
A Notation
We summarize the notation used in the main paper in Supplementary Table
B Theorem Proofs and Derivations
This section provides detailed derivations and proofs for the theoretical claims made in Section 4 of the main paper.
B.1
Contrastive Loss and Representation Fairness (Main Paper Section 4.2)
The core idea behind FairNet's contrastive loss L (()
contrastivv (Equation in the main paper) is to align the representations of samples from the same class but different sensitive groups. The triplet loss formulation is:
I (i)
contrastive Ep, x.,) = (D(20) (z,))) - ((20)(x))) (x,,) + margin]+
where za is an anchor from the minority group (S 1) with label Y = y. Tp is positive sample from the majority group (S = 0) with the same label Y y, and zn is negative sample from any group with a different label Y * y. The LoRA-adjusted representation is 200
Minimizing this loss encourages (z2l)(zx), 20)) (zp)) to be small, ideally smaller than D(20) (xz), 2(),,) by at least the margin. This means that after the LoRA adjustment (which is primarily triggered for minority group samples or samples predicted as such by the detector), the representation (xa) for a minority sample becomes closer to the representation 200) (z,p) a majority sample of the same class. If this minimization is successful across many such triplcts, the distribution of adjusted representations for the minority group, P(z Y = y. S = 1), will be pushed towards the distribution of representations for the majority group. P(20) y. S = 0), for each class y.
Implications for Worst-Group Accuracy (WGA): Models often underperform for minority groups due data scarcity or skewed data distributions, leading to suboptimal representations for these groups. By aligning minority group representations (S = 1) with those of the majority group (S = 0) within the same class Y y. FairNet effectively allows the minority group to leverage the potentially better-leamed representational characteristies of the majority group. If the majority group's representations are more discriminative for the downstream task, this alignment can lead to improved classification accuracy for the minority group. Since WGA is defined as WGA min P(M, G). an improvement in the minority group's aceuracy (P(M, G2)) directly contributes to an increase in WGA, assuming the minority group is the worst-performing one.
Implications for Equalized Odds Difference (EOD): Equalized Odds requires that the prediction y is independent of the sensitive attribute S given the true label Y. This means P(Ý 1|Y y. = 0) P(ý 1|Y = y, 1) for y E (0, 1} (for binary classification and labels). the representations 2(6) upon which the final classifier operates are made conditionally independent of S given Y. i.e., P(- y, S 0) P(2 = y. 1), then any downstream classifier that only uses = will naturally tend to satisfy Equalized Odds. Achieving such representational fairness is a strong step towards satisfying fairness criteria like Equalized Odds. Reducing the discrepancy between P(= Y S = 0) and P(:6)|Y = S = 1) aims to reduce the statistical information about S in 200) that is not already captured by Y. This, in turn, is expected to reduce the AFOD, which measures the disparity in true positive rates and false positive rates between groups. While direct quantitative bound on EOD from the contrastive loss value is complex, the qualitative argument that by making the input representations to the final classification layers more similar across groups for a given true class, the classification outcomes will also become more similar, thus reducing BOD,
Performance Preservation Analysis (Main Paper Section 4.3)
We derive the equations presented for the performance preservation analysis.
14

---

Table 4: Summary of Notation
Symbol
Description
X,
Input space, input sample
Y
Task label space, a task label
8,8
Sensitive attribute space, a sensitive attribute (s 1 for minority, for majority)
fe(-)
Base pre-trainn pre-trained model with parameters 0
(x)
Intermediate representation of z at layer of the base model
D
Bias Detection module at layer with parameters
ps (x)
Risk from ( indicating likelihood of belonging to minority group
Activation threshold for LoRA modules based pa (x)
(1)
L (
Conditional LoRA module associated with layer j
-ond lora
W
Original weights of layer in the base model Low-rank matrices for LoRA module layer j (A, E IR B3 E
Aj, Bj
IRdx))
Rank LoRA matrices < min(d, k)) Weight adjustment from LoRA module, AW; BjA, FairNet-enhanced model Subset of training data with available sensitive attribute labels Standard supervised loss (e.g.. Binary Cross-Entropp i-th sensitive attribute (for multiple attributes)
AW;
FainNet
Piabeled
$i
Bias detector for sensitive attribute s;
(xx)
LoRA-adjusse representation at layer Anchor, positive, and negative samples for contrastive loss Distance metric for contrastive loss (e.g., squared Euclidean distance) Margin hyperparameter for triplet contrastive loss Task-specific loss (e.g., cross-entropy)
Ta: Zp. Zn
D(.-)
margin
Lunk
L detector
(I)
Loss for training bias detector ( (Eq.
L contrastive
(i)
L Contrastive loss for training LoRA module Loodd_orr (Eq. 2) Indicator function for activation of j-th LoRA module's 's contrastive loss during training
I (triggeer ())
Luoil
Total composite loss function for FairNet training (Eq. 3) Hyperparameters balancing terms in Looual
AD, Ag
P(X,Y,S)
Joint data distribution
X,
Input and task label spaces, respectively
Gr,G2
Majority group (S 0), Minority group =1
Prior probability of belonging to the minority group, P(S Base model fe FairNet model
MEaNNa
P(M.
Accuracy of model M on group G Overall accuracy of model M
P(M)
Y
Predicted label by model
WGA(M)
Worst-Group Accuracy of model M Equalized Odds
EOD
TPRD
True Positive Rate of the bias detector False Positive Rate of the bias detector Hypothetical accuracy if LoRA modules were unconditionally applied to group
FPRD
P(M G)
AP
Change in overall performance P(Mrainee) - P(M)
ACC
Overall task accuracy
Derivation of Equations 4 and 5: Performance of FairNet each group
15

---

The bias detector D produces binary signal indicating whether sample is suspected of being from disadvantaged group. We define TPRp P(D fires S 1) and FPRD = P(D fires S 0). The contrastive LoRA correction is conditionally activated only when the detector fires.
For group G1 (majority, =0):
With probability FPRp, the detector incorrectly fires, and LoRA is applied. The accuracy in this case is P(MLORA. G1).
With probability (1 FPRp), the detector correctly does fire, and the base model M is used. The accuracy is P(M,GG
So, the performance for group G1 (Equation 4) is:
P(MFaineer: G1) = FPRn P(MaRRA, Gi)+ + -FPRn) P(M,G) Gi)
For group G2 (minority, S=
With probability TPRp, the detector correctly fires, and LoRA is applied. The accuracy is P P(MLORA G2).
With probability (1 11-TPPR -PPP) the detector incorrectly does not fire, and the base model M is used. The accuracy is P(M,G2)
So, the performance for group G2 (Equation 5) is:
P(M G2) = TPRp P(MLORA, G2) - -PPD)) P(M,
Derivation of Equation 6 and 7: Overall performance change AP
The overall accuracy of model M' is given by:
P(M') = PMM G.) ( + pP(M', G2)
where p P(S= = 1). The change in overall performance is:
AP = P(M P(M)
Substituting the expressions for P ((MPaineer) and P(M):
AP (1 p)) P(Mratett G1) + PP(MaanNa G2]] - [[1 PM) PP((M pP(M, G2)l
= (1-p) (I-P)[P(Mraaeee G1) - P(M, G,)] + p|PP G2) - CPM, G2)l
Substitute Equations and into the terms above:
For the first term, representing the change in performance for group G1:
PMM G1) P(M,G) =PPRp P(MLORA, G1) (1 FPRD) P(M,G.) P(M,C.)
-FPR, P(ML G1) FPRn P(M,G.)
-FPR G1) P(M,Gi)
For the second term, representing the change in performance for group Gg: P(MraieNets G2) P(M,G2) = -PPRp P(MLORA, G2) + (1 - TPRp) P(M, P(M,G2)
=TPRp P(MLORA. G2) TPR, G2,.
TPRDIP((MLOR, G2) ((M,Gz)
Substituting these back into the expression for AP, get Equation 7:
AP = (1-p))PRp(P(MMon G:) P G1) PIPR P(M,
Derivation of Equation 8: Condition for AP 2 0
want to find the condition for non-decreasing performance, i.e., AP
(1 - PPPFFPRDDBB (P G1) - P(M, G1) PTPR G2) - P(M,
16

---

Assume PIMLORA. G2) G2M, > 0 (LoRA improves minority performance, which is the goal). Also assume FPRD > (the detector is not perfect and sometimes misfires on the majority group).
Rearranging the terms:
pTPRD (MMLLAA G2) ((M, 2 2 -11-))FPooP(MMaea G1) P(M, G1)]
PTPRD ITPRO(P(MLARE PMMMAAA G2) (2M, 2 (1 (1-p) ()) G1) P(MERAA; G:)]
Dividing by p. FPRD. [P(MOORA G2) - P(M,, Gz)] (this term is positive under our assumptions),
we obtain Equation 8:
TPRD (1 p) p) GI) - P G1)]
FPRD p [P(MORA, G2) - P(M, G2).
Justification for "Why FairNet Satisfies the Condition" (Elaboration)
The main paper (Section 4.3) provides three key arguments. We elaborate further:
1. Large Positive Denominator: P(MLORA. P(M, G2) is significantly positive. The base model M often shows disparate performance, with P(M, G2) being lower than G1). The contrastive LoRA modules are specifically trained to improve the rep- resentations of minority group samples (G2) by aligning them with the (presumably better) representations of majority group samples (G₂) within the same class. This targeted repre- sentational enhancement is designed to directly address the underfitting or misrepresentation of G2 samples. Therefore, P(MLARA. G2), the accuracy if LoRA were always applied to G2, is expected to be substantially higher than P.M, G2). The magnitude of this improvement, P(MLORA, G2) P(M, G2), forms the denominator and is expected to be a positive and non-trivial value.
2. Small Negative Numerator: P(M, G1) P(MLORA G1) is small or negative. The LoRA correction AW; is learned via the contrastive loss, which uses majority group (G1) representations as targets for alignment within each class. When this correction is (incorrectly, due to FPR)) applied to a G1 sample, it aims to shift its representation (()))) (z) to (z) in way that is consistent with the learned alignment. Ideally, if G1 representations are already optimal or near-optimal for the base model, applying a LoRA adjustment might slightly perturb them, potentially leading to a small drop in performance, making P(MERA, G1) < P(M, G1), and thus P(M, G.) P(M G1) a small positive value. However, it is also plausible that the contrastive learning process, by emphasizing robust intra-class similarities, could act as a form of regularization. The LoRA adjustment, even when applied to G1 samples, might push their representations towards more generalizable or robust manifold shared with G2 samples of the same class. This could potentially lead to no change or even slight improvement in G1 performance, i.c., P.MLORA. G1) > P(M, G1). In this scenario. P(M, G1) P(MLORA, G1) would be zero or negative. In either case, the impact on G2 expected be much less detrimental than the positive impact on G2, as the LoRA modules are not trained to specifically alter G1 representations away from their original effective state in a harmful way.
3. Achievable Condition:
Given point (denominator is significantly positive) and point 2 (numerator is small positive, zero, or negative), the fraction
P(M. G1) P(MLORA, G1)
P(MLOAA: G2) - G2)
will be small number (possibly close to zero or negative). The term
1-2 p
P
is the ratio of majority to minority group sizes, which is a constant for a given dataset.
If the numerator is negative or zero (i.e., LoRA does not hurt or helps Gi), then the right- hand side of Equation 8 is < O. Since TPRp/ /PRp//PRRp is always negateve, the condition holds trivially for any reasonable detector.
If the numerator is small and positive, the condition becomes
TPRp 2 small_positive value. FPRD
17

---

This is realistic condition to meet for bias detector that has even moderate discrim- inative power (i.e., T is reasonably high and F is reasonably low, making P FPRD significantly greater than 1). For instance, if the accuracy loss G1 due LoRA much smaller than the aceuracy gain on G2, then the ratio on the right side small.
Therefore, FairNet's design, which focuses improving G2 performance while minimally affecting G₁ (due to targeted training of LoRA and conditional application), makes Condition 8 practically achievable. This analysis supports the claim that FairNet improve fairness (specifically WGA by increasing P(M, G2)) without sacrificing, and potentially slightly enhancing, the overall performance of the model.
C Detailed Experimental Setup
This section provides comprehensive details of the experimental setup used to evaluate FairNet, covering datasets, base models, FairNet-speccfic configurations, training protocols, and evaluation metries.
C.1 Datasets
We evaluated FairNet on three publicly available datasets:
CelebA: A large-scale dataset of celebrity face attributes.
Task: Binary classification to predict the "Male" attribute.
Sensitive Attribute: "Blond Hair". The dataset exhibits imbalance where, for example, female images more likely to have blond hair than male images, creating spurious correlation that can lead to bias.
- -Splits: We used the standard training, validation, and test splits provided with the dataset.
Preprocessing: Standard image normalization (mean subtraction and division by standard deviation based on ImageNet statistics).
MultiNLI: A dataset for Natural Language Inference (NLI).
Task: Predict the relationship (entailment, contradiction, neutral) between premise and hypothesis.
Sensitive Attribute: Presence of negation cues (e.g., "not", "n", "never") in the hypothesis. Models can develop biases by associating negation with specific entailment labels (e.g., contradiction) without proper reasoning.
-Splits: We used the standard train,
and test-matched splits. Preprocessing: BERT tokenizer used convert text pairs into input IDs, attention masks, and token type IDs. Maximum sequence length was set according to typical BERT practices.
HateXplain: A dataset for hate speech detection with fine-grained annotations, including target communities.
Task: Multi-class classification to predict if post is hate speech, offensive, or normal. Sensitive Attributes: We focused on biases related to gender and race as indicated by the target communitics or demographics mentioned in the posts. This dataset allows for the study of overlapping biases.
-Splits: We used the standard train, validation-matcher and test-matched splits.
Preprocessing: Similar to MultiNLI, text was tokenized using BERT tokenizer.
C.2 Base Models
Vision Tasks (CelebA): We used Vision Transformer (ViT) with the following 8, num oum_attention. seatiiionheadsss heads 8, intermediate_size = 768, tmage_sizes 64, patch_size = 16). This model was trained from scratch (no pre-trained parameters) and then fine-tuned on the specific CclebA task.
configuration: ViTConfig (auu_hidden__aaeee
18

---

Language Tasks (MultiNLI, HateXplain): We used pre-trained models from the Hugging Face Transformers library, specifically DistilBERT- and HERT-Base-Uncaser These models, utilizing their pre-trained parameters, were then fine-tuned on the respective NIJ or hate speech detection task.
The base models were first fine-tuned on the target tasks using standard ERM to establish baseline performance before integrating FairNet components.
C.3 FairNet Implementation Details
C.3.1 Bias Detection Module (D()
Architecture: The bias detectors D operate on intermediate layer representations from
the base model, denoted as sequence of hidden states H (h1", .. ..... ( An
attention pooling mechanism is first applied to these representations to obtain fixed-size vector h (t) This pooled representation is computed as: "ooledd
Σ
og (
pooted
where the attention weights (; are typically derived from the hidden states using learnable scoring function, for example:
042 exp(s:)
with 8, v tanh Wanh b)
==1 exp(s;)
Here, W. and are learnable parameters of the attention mechanism. The resulting pooled representation h pooled is then fed into a lightweight Multi-Layer Perceptron (MLP). This MLP typically consisted of 1-2 hidden layers with ReLU activations and final sigmoid output layer for the bias risk score pa (z:). The hidden layer dimensions were kept small to ensure low overhead.
Placement (l): Detectors were typically placed at intermediate layers of the base model, as these layers often capture more abstract and potentially bias-encoding features. However, the specific layer(s) can be flexibly chosen. The paper's notation E, detcctrr implies that multiple detectors at different layers could be integrated; specific configurations would be subject to tuning.
Training:
FairNet-Full: When complete ground-truth sensitive attribute labels s are available. these are used directly as the basis for detection. The module D does not undergo a separate training phase; it acts as a conditional switch based on the known s. The bias risk score ps (æ:) can be considered itself (if s E {0, 1)) or indicator directly derived from s. Thus, the parameters & of the MLP (and attention mechanism applicable here) not trained in this setting group identity is deterministically known.
FairNet-Partial: Detectors were trained using Binary Cross-Entrops (BCE) loss only on the k% of training samples where was available. Weighted loss or focal loss was considered to handle imbalance in the labeled subset if the minority group was rare even within the labeled portion.
FairNet-Unlabeled. Unsupervised methods were used to generate pseudo-labels & for training the detector D Using methods like Local Outlier Factor (LOF) Isolation
Forest the intermediate representations (x:) (or the pooled ( to identify samples that representationally distant from the norm, under the hypothesis that minority group samples or those affected by bias might manifest as outliers. The
generated pseudo-labels & were then used to train the detector ( via BCE loss.
19

---

Threshold (T): The activation threshold T for triggering LoRA was selected based on validation set performance, aiming for good balance of detector TPRp and FPRp. Typical values ranged from 0.5 to 0.8. A grid search was often performed.
C.3.2
Contrastive Loss (T (T ccontrustive
Distance Metric (D(: In this work, Fuclidean distance was used as the distance metric.
Margin: The margin hyperparameter was typically set value between 0.1 and 1.0, tuned on a validation set.
Contrastive Elements and Representations: The contrastive loss aims to adjust the representation (()( of an anchor sample za from the minority group at layer j (where LoRA is applied). This is achieved by comparing (Ira) with pre-computed target representations for positive and negative examples, which are derived from the model trained in the first stage (the ERM baseline).
Anchor (za): An imput sample F. identified as belonging the minority group (either via ground-truth label s = 1 or predicted as such by the bias detector (()) ). Its representation (60 is the output of the j-th layer of the FairNet model being trained.
Positive Target Representation (Afagge)) For an anchor ta with task label Va. the positive target representation h is the pre-computed average embedding of samples from the majority group (s = 0) that share the same task label ya- This average is calculated using representations from the first-stage ERM model.
Negative Target Representation (hagge)):: For an anchor va with task label yo negative target representation hurget is the pre-computed average embedding of samples belonging to a different task label yn * Ya- This average is also calculated using representations from the first-stage ERM model.
Loss Calculation Strategy: For each anchor (z,). ). the contrastive loss is typically formulated as (xa)) h - (x,)) / - margin] During training, for given anchor, one more negative target representations (corresponding to different yn)) can be selected, for instance, by choosing the class Yn whose hureet is closest to (() (xa) (hard negative mining based on average embeddings) by randomly selecting from other classes
C.3.3 Multiple Sensitive Attributes
As described in Sections 3.2 and 3.3 of the main paper, for multiple attributes s;:
Separate detectors D d; were trained for each 8; using available labels for that attribute.
Separate LoRA modules [ Loraj (with parameters AJ.s, By,)) were trained.
Each (f)
Jerai was trained with an attribute-specific contrastive loss T, ( "cuntrastive triggered by its corresponding detector D The final weight adjustment would be a sum or a sequen-
tially applied set adjustments if multiple LoRAs are triggered for different attributes.
C.4 Evaluation Metrics
ACC: Standard classification accuracy on the test set.
WGA: The minimum accuracy observed across all defined sensitive groups. For binary sensitive attribute {0, 1}. WGA min(P(Ý Y = 0), P(Y Y = 1)). EOD: For binary classification and binary sensitive attribute, EOD is typically calculated as the average of the absolute difference in True Positive Rates (TPR) and False Positive Rates (FPR) between the groups:
ATPR = TPRs_0 TPRs=1
AFPR FPRs=0 FPRs=1
20

---

EOD = (ATPR AFPR)
2
Lower values indicate better fairness in terms of equalized odds. The paper reports EOD,, implying this or a similar definition where lower is better.
C.5 Baselines
The baselines mentioned in Table 2 of the main paper include:
ERM (Empirical Risk Minimization): Empirical Risk Minimization (standard model training without explicit fairness considerations).
This method focuses on mitigating bias in Transformer models by modifying components of the attention mechanism (queries, keys, and values). It aims to achieve fairness, for example, by normalizing attention weights or aligning value representations, without requiring access to sensitive demographic labels during deployment and potentially during training for the main debiasing logic.
Lu al.:
D3M: A data-centric debiasing approach that aims to improve subgroup robustness and reduce worst-group error. D3M identifies and removes small subset of training examples that are identified as disproportionately contributing to the model's s failure on minority sub- groups. This identification can be done using "datamodels" or influence- traccen techniques (like TRAK) to pinpoint detrimental samples, often without needing group labels for the selection process itself.
GroupDRO (Group Distributionally Robust Optimization): Optimizes for the worst- group loss, meaning it aims to maximize the performance of the group that the model performs worst This method explicitly requires group labels during training identify underperforming groups and upweight their loss.
DFR (Dehiasing via Feature Representation): A general category of methods that learn debiased feature representations. These techniques often involve an auxiliary component or loss function during training to remove information about sensitive attributes from the learned features, typically requiring partial or full access sensitive labels for training this debiasing component.
Sebra: An unsupervised debiasing technique that mitigates spurious correlations by au- tomatically ranking training data points based on their degree of "spuriosity" within their respective classes. Sebra leverages the observation that samples with stronger spurious cues are often learned more easily by standard ERM. This Gelf-Guided Bias Ranking" is then typically used within a contrastive learning framework to train more robust and fair model.
Eq.Odds: Refers fairness criterion where a predictor y satisfies Equalized Odds if Ý is independent of sensitive attribute A conditional on the true outcome Y. A common relaxation Equal Opportunity, which typically requires the True Positive Rate to be equal across different groups (i.c., P( = 1|A = 0, Y = 1) = P( = 1/A 1, = 1)). This can often be achieved through post-processing methods, such as applying different decision thresholds for different groups, which requires access to sensitive labels for the data on which adjustments are made (e.g., validation set test time).
GSTAR: A post-processing method aimed at mitigating discriminatory model behavior and satisfying multiple fairness constraints by learning adaptive classification thresholds for different demographic groups (i.e., groups defined by sensitive attributes). This method typically estimates confusion matrices based on the probability distributions of a classifi- cation model's 's output and uses this optimize thresholds for each group. This allows for improving fairness in a model-agnostic manner and can even be used to further improve the fairressssairness trade-off of existing fairness methods.
For methods not from our work, we either used publicly available implementations or re-implemented them following the original papers, tuning their hyperparameters on the respective validation sets.
21

---

C.6 Experimental Infrastructure
All experiments were conducted on a single NVIDIA 100 GPU with 80GB memory, accompanied by a 236-core CPU and 512GB of RAM. This computational setup ensures consistent and reproducible evaluation across all experiments.
D Further Empirical Investigations of FairNet
This section extends our primary empirical validation by presenting series of targeted experiments designed further probe the operational characteristics, robustness, and nuanced performance aspects of the FairNet framework. We assess the framework's ' performance across the spectrum of label availability
partial and fully unlabeled data to fully labeled setting. Furthermore, we conduct detailed analysis of the model's 's robustness to label noise and the critical role of the bias detector's activation threshold in calibrating the fairness-aceuracy trade-off. Finally, we evaluate the computational overhead to demonstrate its practicality.
D.1 Impact of Training Data Size for FairNet-Partinn
We conducted detailed investigation into the sensitivity of FairNet-Partial to the quantum of available sensitive attribute labels. This experiment, performed on the CelebA dataset, involved varying the percentage of training instances for which the Blond Hair sensitive attribute
accessible for training the bias detector (D and the contrastive conditional LoRA module (L Jocaa) The objective was to quantify the trade-offs between label availability, bias detector efficacy, and the resultant fairness and accuracy outcomes. The empirical results are presented in Table
Table 5: Impact of Different Training Data Sizes (Percentage Labeled Sensitive Attributes) on MhirNet-Partina 's Performance CelebA.
Size
Num
TPR (%)
FPR (%)
TPR/FPR
ACC (%)
WGA (%)
EOD (%)
ERM
95.7
77.7 1 0.8
10.2 = 0.7
0.1%
163
76.0 1.8
6.95 = 0.55
10.9
95.7
0.6
7.3 0.5
0.5%
813
86.7 1.2
7.73 = 0.48
11.2 0.9
9.2
0.5
7.0
1%
1,627
89.3
8.03 = 0.41
0.7
9.2
0.4
6.5
5%
8,134
94.0 0.5
8.11 0.35
11.6
9.1
8.3
6.6
10%
16,269
93.9 0.4
7.04 0.28
13.3 0.4
9.1
0.2
6.4
50%
81,344
95.0
1.55 =
21.0 0.3
9.1
0.2
6.3 0.2
100%
162,688
95.8 + 0.3
4.05
2.3
9.1
8.10
5.9
"TPR and FPR refer to the performance of the internal bias detector for the minority group. Num indicates the number of samples with sensitive attribute labels. Best values in each column are bolded.
The empirical evidence robustly demonstrates FairNet-Partial's capacity to yield substantial fairness enhancements even under conditions of severe label scarcity. With a mere 0.1% of training data endowed with sensitive attribute labels (equivalent to 163 instances), FairNet-Partia elevated the WGA to 82.1% 1% from the ERM baseline of 77.9%. Concurrently, the EOD was markedly reduced from 10.0 to 7.1. These improvements were achieved with negligible impact on ACC, which remained high 95.7% compared to ERM's 95.8%. This underscores the framework's 's practical utility in real-world scenarios where comprehensive annotation of sensitive attributes is often infeasible.
A monotonic improvement in the bias detector's efficacy is observed with an increasing fraction of labeled data. TPR for minority group detection progressively rises from 77.0% (at 0.1% labels) to a peak of 94.9% (at 50% labels). While the FPR does not show strictly monotonic decrease initially, it reaches its minimum of 3.45% when 100% of labels are available. Critically, the TPR/FPR ratio, key indicator of the detector's 's discriminative power and its reliability in triggering the conditional LoRA. exhibits substantial growth, particularly beyond the 10% label mark, culminating at an impressive 27.2 when all labels are utilized. This enhanced detector precision is theoretically linked to more effective and targeted application of fairness corrections.
Congruent with the improved detector performance, fairness metrics (WGA and EOD) exhibit consistent positive trend with increased label availability. WGA steadily climbs from 82.1% (0. % labels) to a maximum of 86.2% (100% labels). Similarly, EOD progressively decreases, indicating
22

---

enhanced fairness, from 7.1 (0.1% labels) to its lowest point of 5.8 (100% labels). This demonstrates that while SairNet-Partial is effective in low-label regimes, access more comprehensive sensitive attribute information allows the framework to further refine its internal models and achieve superior fairness outcomes.
Notably, the overall task accuracy remains remarkably stable and high across all levels of label availability, consistently hovering around 95.7%-95.889 and even marginally increasing to 95.9% when 50% or 100% of labels are present. This stability is a crucial finding, as it signifies that the dynamic, conditional application of LoRA modules, guided by the increasingly proficient bias detector, successfully mitigates bias without deleteriously affecting the model's s primary predictive capabilities. The peak fairness performance, characterized by the highest WGA (86.2%) and lowest EOD (5.8%), is achieved when the full set of sensitive attribute labels is available, coinciding with the highest ACC (95.9%) and the optimal TPR/PPR ratio for the detector.
In summary, these findings highlight FairNet-Partia robustness and data efficiency. The framework demonstrates strong capability enhance fairness even with minimal supervisory signals regarding sensitive attributes. As label availability increases, FairNet-Partial systematically leverages this additional information refine its bias detection and correction mechanisms, leading progressively better faimesss outcomes while consistently preserving, and in some instances marginally improving, overall model accuracy. This scalability and performance consistency across varying degrees of label availability underscores the practical applicability and methodological soundness of the FairNet approach.
D.2 Performance in Unsupervised Scenarios
To explore the viability of FairNet in settings where no sensitive attribute labels are available, we implemented FairNet-Unlabeled This variant replaces the supervised bias detector with an unsupervised one based on the Local Outlier Factor (LOF) algorithm, which identifies potential minority group instances from their representation embeddings. We evaluated this approach on the CelebA test set, with the performance detailed in Table B
Table B: Performance of FainNet-Unlabeled on the CelebA dataset.
Method
TPR (%)
FPR (%)
TPR/PPR
ACC (%)
WGA (%)
EOD (%)
FainNet-Unlabeled
74.2
7.71
9.62
95.8
81.9
7.5
ERM
95.8
77.9
10.6
"TPR and FPR refer to the unsupervised detector's performance on minority group.
The results show significant improvement in fairness over the ERM baseline, even without any sensitive labels for training. WGA increases from 77.9% to 81.9 81.9%, and EOD reduced from 10.6% to 7.5%, all while maintaining the same high level of accuracy (95.8%). The unsupervised detector achieves a respectable TPR/FPR ratio of 9.62, confirming its ability to effectively distinguish minority instances. This demonstrates the potential of FairNet to operate in challenging real-world scenarios where annotating sensitive attributes is impractical or impossible.
D.3 Ablation on Contrastive Loss in the Full-Label Setting
To isolate the contribution of the contrastive loss, we conducted a targeted ablation study in the full-label setting (FairNet-Full). In this scenario, the bias detector is unnecessary sensitive attributes are known for all instances. This allows for a direct and clean comparison between using contrastive loss and a standard binary cross-entropy (BCE) loss for the corrective LoRA module. The results are presented Table
The results confirm the critical role of our proposed loss function. While maintaining the same peak accuracy of 95.9%, introducing the contrastive loss boosts WGA substantially from 81.7% to 88.2% % and cuts EOD by more than half, from 8.2% to 3.8%. This significant gain in fairness, isolated from the influence of a detector, provides strong empirical evidence that the contrastive objective highly effective aligning intra-class representations and uplifting minority group performance.
23

---

Table C: Ablation study of the contrastive loss in the FairNet-Full setting on CelebA
Method
ACC (%)
WGA (%)
EOD (%)
FairNet-Full (with contrastive loss)
95,9
88.2
3.8
w/o contrastive loss (uses BCE) ERM
95.9
81.7
8.2
95.8
77.9
10.6
Bold indicates the best.
D.4 Robustness to Bias Detector Degradation via Label Noise
To rigorously test FairNet's robustness under more challenging conditions, conducted an additional sensitivity analysis. In this experiment, we intentionally degraded the bias detector's quality by injecting random noise into the sensitive attribute labels during its training phase. The noise level was varied from 0% (the original, clean data) to 100% (where every label is randomly flipped, rendering the labels useless). This procedure allows quality and observe how FairNet's performance responds to a progressively less reliable detector signal. The results, presented in Table demonstrate that FairNet's performance degrades gracefully. Critically, even in a worst-case scenario where the bias detector is rendered completely ineffective by noise, FairNet's performance does fall below the initial ERM baseline. This finding strongly confirms the inherent robustness and a key safety property of our method.
simulate scenarios of varying data
Table H: Impact of varying label noise FairNet's performance (CelebA)
Noise
TPR (%)
FPR (%)
TPR/FPR
ACC (%)
WGA (%)
EOD (%)
0%
94.1
3.45
27.2
95.9
86.2
5.8
20%
84.8
4.26
19.9
95.8
85.7
6,4
40%
78.5
4.83
16.3
95.8
85.0
6.9
60%
64.6
5.92
10.9
95.7
83.1
8.1
80%
51.2
7.17
7.14
95.7
81.2
9.2
100%
9.11
8.00
1.14
95.7
77.8
10.3
ERM
95.8
77.9
10.6
"TPR and FPR refer to the detector's performance the minority group.
D.5
Analysis of the Bias Detector Activation Threshold
The activation threshold, т. is critical hyperparameter within the FainNet framework, directly governing the trade-off between fairness enhancement and performance preservation. While its optimal value may exhibit dataset-dependency, we demonstrate that it functions as a simple and interpretable mechanism for practitioners calibrate model behavior.
To systematically investigate the influence of т, we conducted an ablation study on the CelebA dataset, with results presented in Table The experiment reveals clear and controllable relationship between the threshold value and the model's final performance characteristics.
The results in Table I out the Pareto frontier between accuracy and fairness. At one extreme, threshold of r = 0.0 equates to an unconditional application the LoRA correction, as both TPR and FPR 100%. This strategy yields the most substantial fairness improvements (highest WGA of 87.1% and lowest EOD 4.8%) ) but incurs a notable accuracy penalty, reducing ACC to 94. 94.1%. 1%. This outcome is expected, the corrective module is applied universally, including to instances that do not require it. At the opposite extreme, T = 1.0 deactivates the mechanism entirely, causing the model's 's performance revert to the ERM baseline.
Between these extremes, increasing makes the fairness intervention progressively more selective. This selectivity is reflected in the detector's metrics: as rises, both TPR and FPR decrease, while the TPR/FPR ratio-a key indicator of the detector's discriminative efficacy-initially increases, peaking around 0.8. A threshold of 7 0.5 demonstrates well-balanced configuration, achieving

---

Table I: Impact of the activation threshold on FairNet's performance (CelebA).
Threshold
TPR ( (%)
FPR (%)
TPR/FRR
ACC (%)
WGA (%)
EOD (%)
0.0
100.0
100.0
1.00
94.1
87.1
4.8
0.2
98.8
18.7
5.28
95.4
86.9
5.1
0,4
96.7
6.34
15.3
95.6
86.5
5,4
0.5
94.1
3,45
27.2
95.9
86.2
5.8
0.6
72.3
2.60
27.8
95.9
85.7
6.1
0.8
62.9
1.48
42.5
96.0
82.1
7.4
1.0
0.0
0.0
95.8
77.9
10.6
"TPR and FPR refer the detector's performance the minority group. Best values in each column are bolded.
significant fairness gains (WGA 86.2%, EOD 5.8%) while nearly matching the peak overall accuracy. Meanwhile, setting T = 0.8 prioritizes accuracy, achieving the highest ACC of 96.0% at the cost of some fairness gains.
Implications for Practical Deployment This inherent tunability is a significant practical advantage of FairNet. It provides straightforward lever for practitioners to adapt the model to specific deployment contexts and priorities. The process of selecting an appropriate threshold via grid search validation set is procedurally simple and computationally inexpensive, thus posing minimal barrier to real-worlh application. This allows for explicit calibration of the desired balance between maximizing overall performance and ensuring equitable outcomes for disadvantaged subgroups.
D.6 Impact of Threshold on Bias Detector Performance
analyzed the effect of varying activation threshold r on the bias detector's TPR and FPR, as well as the TPR/FPR ratio. This analysis was performed the HateXplain dataset for the "African American" and "Female" demographic groups The results are depicted in Figure
African American Group Analysis (Left Pair of Plots in Figure 2 The top-left plot illustrates the variation of TPR and FPR for the "African American" group as function of the threshold. As the threshold increases, both TPR and FPR decrease. The reduction in TPR suggests that higher threshold leads stricter classification, reducing the number of true positives. Meanwhile, the rapid decrease in FPR indicates fewer false positives.
The bottom-left plot shows the TPR/PPR ratio across different thresholds. This ratio peaks approximately threshold of 0.7-0.8, indicating an optimal balance between TPR and FPR. Beyond this peak, the ratio declines, suggesting diminishing benefits from further increasing the threshold due to disproportionate reduction in TPR compared to the decline in FPR. Therefore, this peak threshold be used to guide optimal threshold selection, ensuring fairness and maintaining model performance.
Female Group Analysis (Right Pair of Plots in Figure 2 TPR and FPR for the "Female" group, following a similar pattern to the "African American" " group. As the threshold increases, both TPR and FPR decrease, with higher thresholds making the model stricter, leading to a reduction in both true positives and false positives.
The top-right plot shows the changes in
The bottom-right plot depicts the TPR/FPR ratio, which also peaks around the 0.7-0.8 threshold range, indicating the threshold range that maximizes classification efficiency for the "Female" group. After this peak, the ratio starts to decline, suggesting that further increases in the threshold reduce classification effectiveness. Thus, selecting a threshold near this peak ensures optimal fairness while retaining classification accuracy.
Summary For both the "African American" and "Female" groups in the HateXplain dataset, the TPR/FPR ratio reaches its peak around a threshold of 0.7-0.8, indicating that this range provides an optimal balance between correctly identifying minority group instances (high TPR) and avoiding misclassification of majority group instances (low FPR), which is crucial for the conditional LoRA mechanism. For other datasets, a similar analysis can conducted determine the optimal threshold range that ensures FairNet (referred to as FairLoRA in the original figure caption's 's context) effectively
25

---

TPR and FPR vs Threshold (Arrican)
TPR and FPR vs Threshald (Female)
Tue Positive Rate
True Positive Rate
Positive
0.8
0.6
0.6
Rate
Rate
0.4
0.4
0.0
0.0
Threshold
Threshold
TPR / FPR (African)
TPR / FPR (Female)
Tru Positive Rate False Positive Rate
17.5
20
Rato
7.5
10
5.0
5
2.5
True Positive Rate / False Positive Rate
0.0
0.2
0.8
1.0
0.0
0.2
0.6
0.8
Threshold
Threshold
Figure 2: TPR and FPR Analysis with TPR/FPR Ratio for African American and Female Groups across Different Thresholds on the HateXplain dataset.
mitigates biases while maintaining overall model efficacy, aligning with the theoretical condition that links detector quality (TPRp/ performance preservation.
D.7 Computational Overhead
To evaluate the computational cost introduced by framework, conducted an efficiency analysis on both VIT and BERT models. The results are presented in Table
Table F: Computational overhead analysis of FairNet. Parameters are reported in millions (M), and FLOPs in giga-FLOPs (GFLOPs).
VIT
BERT
Metric
Base
FairNet
Base
FairNet
Parameters (M)
29.57
29.77
109.48
109.79
Minority Proportion
0.15
0.07
GFLOPs (Minority)
0.74
16.43
Majority Proportion
0.85
0.93
GFLOPs (Majority)
0.50
10.90
Total GFLOPs
0.49
0.53
10.88
11.29
The overhead is marginal. For large model like BERT-base, FairNet adds only 0.28% more parameters and increases the total GFLOPs by just 3.7%. The differentiated GFL.Pss for minority and majority groups stem from the conditional application of the LoRA module. This quantitative
26

---

analysis confirms that our framework is highly efficient and introduces minimal latency, making it practical solution for deployment in real-world applications.
27

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction claim the proposal of FairNet, novel framework for dynamic fairness correction using conditional LoRA and a new contrastive loss, its flexibilityy with attribute labels, theoretical guarantees for performance preservation, and empirical validation. These claims are substantiated the Method (Section 3), Theoretical Analysis (Section 4), and Experiments (Section 5) sections, which detail the framework, its theoretical underpinnings, and performance on various benchmarks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Justification: The paper discusses limitations primarily in the Conclusion (Section 6) by stating:
"Future work will address complex intersectional biases and integrate advanced unsupervised detection methods, further broadening FairNet's impact on trustwor- thy AI." This acknowledges current scope limitations regarding intersectional biases and the sophistication of unsupervised methods used.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best
28

---

judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
For the performance preservation analysis (Section 4.3), assumptions regarding the bias detector's TPR/FPR and the LoRA module's 's impact are stated. The detailed derivations for Equations 4-7 (labeled as Eq. 4, 5, 6. 8 in the paper, with Eq. being the main A A and Eq. 8 the condition) are provided in the Supplementary Material (Section B.2). The discussion on contrastive loss and representation faimess (Section 4.2) links the mechanism to established principles in fair representation learning, with further elaboration in Supplementary Material (Section B.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: The paper provides details datasets, base models, FairNet-specific configu- rations (bias detector architecture, contrastive loss details, handling of attribute scenarios), training protocols, and evaluation metries in Section 5.1 and extensively in the Supple- mentary Material (Section C). This includes data splits, some hyperparameter ranges and selection strategies (e.g., tuning on validation). The overall setup and component deserip- tions are detailed enough to guide reproduction efforts.
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
29

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
We use publicly-accessee datasets. We upload the codes and instructions to recover the results. Once the blind review period is finished, we'll open-source all codes, instructions, and model checkpoints.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to to reproduce the results. See the NeurIPS code and data submission guidelines (https
httpps.cc/pbblcc/uu.dddee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if' applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper specifies data splits (standard splits for public datasets), types of models, some hyperparameter ranges (e.g., for т, contrastive margin) and how they were chosen (tuned on validation sets, grid search). Details are provided in Section 5.1 and extensively in Supplementary Material Section C. The information provided generally sufficient to understand the experimental setup.
30

---

Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: The paper does report error bars suitably and correctly defined for the statistical significance of the experiments. For example, in Table 4, the performance and fairness metrics are reported with error margins. These error bars capture the variability due to factors such different train/test splits and model initializations. The paper also mentions that the results are averaged over multiple runs, and the error bars represent the standard deviation.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall
run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: We provide detailed information about the computational resources used for all experiments in Section C.6 Experimental Infrastructure. This includes the type of GPU (single NVIDIA A100 with 80GB memory), CPU (236 cores), and system memory (512GB RAM), ensuring that readers have sufficient information to reproduce our experimental setup.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
31

---

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttps://neuria
Answer: [Yes]
The research presented in this paper fully adheres to the NeurIPS Code of Ethics. We have carefully considered potential ethical implications, ensured transparency and reproducibility, and followed responsible practices in data usage, model development, and evaluation. Relevant ethical considerations, including fairness and societal impact, explicitly discussed in the main paper and supplementary material.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The paper presents FairNet as novel framework that can enhance fairness in machine learning models without compromising performance, which has positive societal implications for increasing trust in AI systems and addressing ethical and legal challenges.
Justification:
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there negative socictal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
32

---

The focus of the paper is on developing a faimess correction framework for machine learning models, and the content centered on improving fairesss and accuracy in model performance. The paper poses no such risks that would require safeguards against misuse.
Justification:
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: The paper mentions using publicly available datasets like CelebA, MultiNLI, and HateXplain, and all of them are properly cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, pappsssippeestteeeeeeeee thcode com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We will open source the code once the paper is accepted.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
33

---

14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
The paper does describe any experiments or research involving human subjects, thus there is no information regarding instructions, screenshots, or compensation provide.
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
It focuses on developing machine learning framework (FairNet) to address fairness in Al models, without conducting experiments that directly involve human participants.
Justification: The paper does not involve research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve the use of large language models (LLMs) as a core method. LLMs are used in the research methodology experiments described in the paper.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
34


Reviews:
[{"review_id": "HCv2U1wmlx", "reviewer": "Reviewer_niDK", "summary": "The paper introduces FairNet. FairNet is an in-processing method that attaches bias detection heads to intermediate representations and conditionally triggers low rank adaptation (LoRA) modules trained with a novel contrastive loss. The detector identifies potentially biased instances and only those examples receive a small weight update to remove bias, and the base model’s predictions for unbiased inputs remain untouched. Additionally, ablations confirm both the detector and the contrastive loss are essential.", "questions": "1. Is the detector threshold dataset dependent (Fig 2)? If yes, how could one tune it? and how difficult and sensitive is it? How this affect its application in real world applications?\n2. What is the FLOPs / latency overhead of detectors and LoRA? I believe it is affecting the test time, not only the train time.\n3. Have you tried two-attribute detectors (e.g., race and gender) and measured fairness gaps? Is it even possible?\n4. In FairNet unlabel, LOF is used for pseudo-labels. How stable is performance, especially for outlier samples?\n5. Does FairNet still preserve accuracy when test-set distribution shifts? Any comments on this?", "limitations": "Please see weaknesses and the questions.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. A single architecture handles fully labelled, partially labelled, and unlabeled settings.\n2. Experiments are done across vision and NLP, with baselines and ablations.\n3. A theoretical framework to generate a formal condition linking detector quality to non decreasing overall accuracy.\n4. Touching an important research question in fairness domain.\n\nWeaknesses:\n1. Guarantees are based on a moderate TPR/FPR for the bias detector; however, the paper gives no systematic error bars to confirm this in real world applications.\n2.  CelebA and other datasets are mid-scale; no large-scale dataset is tested.\n3.  No intersectional bias is discussed.\n4.  Adding multiple detectors may hurt the test time speed.\n5. Ablation study is done only for CelebA.", "was_revised": true, "final_justification": "I like the paper and I believe it has a simple yet effective idea. The authors addressed my concerns.", "rebuttal": "# Response to W1:\n\nThank you for this insightful comment. We agree that validating theoretical guarantees with systematic error bars is essential for connecting theory to practical performance.\n\nTo address this, we have updated Supplementary Table 5 to include standard deviations across five runs with different random seeds. The results confirm that key metrics—TPR, FPR, and the TPR/FPR ratio—remain stable within a narrow range, even under limited labeled data. This supports the practical reliability of the theoretical “moderate TPR/FPR” condition. （Due to space/word constraints, only a selection of results is presented here.）\n\n| **Size** | **Num** | **TPR (%)** | **FPR (%)** | **TPR/FPR** | **ACC (%)** | **WGA (%)** | **EOD (%)** |\n| -------: | ------: | ----------: | ----------: | ----------: | ----------: | ----------: | ----------: |\n|      ERM |       - |           - |           - |           - |  95.7 ± 0.2 |  77.7 ± 0.8 |  10.2 ± 0.7 |\n|     0.1% |     163 |  76.0 ± 1.8 | 6.95 ± 0.55 |  10.9 ± 1.2 |  95.7 ± 0.3 |  81.9 ± 0.6 |   7.3 ± 0.5 |\n\nAdditionally, we tested FairNet's robustness under label noise to simulate degraded detector performance. As shown below, FairNet maintains performance above the ERM baseline even when the detector fails entirely, confirming its robustness and safety guarantees.\n\n| **Noise** | **TPR (%)** | **FPR (%)** | **TPR/FPR** | **ACC (%)** | **WGA (%)** | **EOD (%)** |\n| --------: | ----------: | ----------: | ----------: | ----------: | ----------: | ----------: |\n|        0% |        94.1 |        3.45 |        27.2 |        95.9 |        86.2 |         5.8 |\n|       20% |        84.8 |        4.26 |        19.9 |        95.8 |        85.7 |         6.4 |\n|       40% |        78.5 |        4.83 |        16.3 |        95.8 |        85.0 |         6.9 |\n|       60% |        64.6 |        5.92 |        10.9 |        95.7 |        83.1 |         8.1 |\n|       80% |        51.2 |        7.17 |        7.14 |        95.7 |        81.2 |         9.2 |\n|      100% |        9.11 |        8.00 |        1.14 |        95.7 |        77.8 |        10.3 |\n|       ERM |           - |           - |           - |        95.8 |        77.9 |        10.6 |\n\n# Response to W2:\n\nThank you for the thoughtful comment. We acknowledge that our experiments use mid-scale datasets and appreciate the opportunity to clarify this choice.\n\nCelebA, MultiNLI, and HateXplain are widely-used benchmarks in fairness research, selected for their well-characterized biases. Their moderate size allowed us to perform controlled ablation studies (Table 2) and isolate the effects of individual components, which was essential for validating FairNet’s design.\n\nImportantly, FairNet is built for scalability. Both the bias detectors and LoRA modules are lightweight and parameter-efficient, ensuring low computational overhead. We agree that testing FairNet on large-scale datasets is a valuable direction for future work.\n\n# Response to W3:\n\nThank you for highlighting this important issue. While intersectional bias is noted as future work in our conclusion, we do include a foundational experiment addressing multiple sensitive attributes simultaneously.\n\nAs shown in Supplementary Section D.1 and Table 4 (p. 21), we evaluate FairNet on the HateXplain dataset considering both race (\"African American\") and gender (\"Female\"). FairNet reduced the EOD for race from 20.1 to 9.8 and for gender from 12.9 to 8.2, without harming overall accuracy. This demonstrates FairNet’s potential in handling multi-attribute fairness and lays groundwork for future exploration of intersectional bias.\n\n# Response to W4:\n\nThank you for raising this important concern about test-time efficiency.\n\nFairNet is designed to remain efficient, even when handling multiple biases. Users can either deploy separate lightweight detectors or use a single multi-label detector, with additional biases handled by adding LoRA modules—without a linear increase in computation.\n\nMoreover, the detectors operate on intermediate representations and consist of only an attention pooling layer and a small MLP, keeping their overhead minimal.\n\nIn practice, applying FairNet to BERT-base increases the total parameter count by only **0.28%** and test-time GFLOPs by just **3.7%**. These small increments ensure that test-time performance remains largely unaffected, even in multi-attribute settings.\n\n# Response to W5:\n\nThank you for the helpful suggestion. We initially focused the ablation study on CelebA due to its clearly defined attributes, which made it suitable for isolating the effects of each component.\n\nTo further demonstrate the generalizability of FairNet, we have now conducted a full ablation study on the **MultiNLI** dataset. As shown below, the results are consistent with those on CelebA, reinforcing the effectiveness and modularity of our framework.\n\n| **Method**           | **ACC (%)** | **WGA (%)** | **EOD (%)** |\n| -------------------- | ----------- | ----------- | ----------- |\n| FairNet-Partial      | **82.6**    | _76.5_      | _6.2_       |\n| w/o detector         | 81.0        | **77.2**    | **5.7**     |\n| w/o contrastive loss | _82.5_      | 70.1        | 9.2         |\n| w/o both             | 81.3        | 71.8        | 8.9         |\n| ERM                  | **82.6**    | 67.3        | 12.5        |\n\nThese findings confirm that the individual components of FairNet contribute meaningfully across different datasets.\n\n# Response to Q1:\n\nThank you for the insightful question. Yes, the detector threshold (τ) is dataset-dependent, but it is designed to be easily tunable.\n As shown below, varying τ controls the trade-off between fairness and accuracy: lower thresholds lead to more frequent bias correction (improving fairness), while higher thresholds prioritize accuracy.\n\nIn our experiments, we uniformly set **τ = 0.5** for all datasets to achieve a balanced trade-off, and found this setting to be robust in practice. The threshold can be tuned via grid search on a validation set with minimal overhead, allowing practitioners to align FairNet with different deployment goals.\n\n| **Threshold** | **TPR (%)** | **FPR (%)** | **TPR/FPR** | **ACC (%)** | **WGA (%)** | **EOD (%)** |\n| ------------: | ----------: | ----------: | ----------: | ----------: | ----------: | ----------: |\n|           0.0 |       100.0 |       100.0 |        1.00 |        94.1 |    **87.1** |     **4.8** |\n|           0.2 |        98.8 |        18.7 |        5.28 |        95.4 |        86.9 |         5.1 |\n|           0.5 |        94.1 |        3.45 |        27.2 |        95.9 |        86.2 |         5.8 |\n|           0.8 |        62.9 |        1.48 |        42.5 |    **96.0** |        82.1 |         7.4 |\n|           1.0 |         0.0 |         0.0 |           - |        95.8 |        77.9 |        10.6 |\n\n# Response to Q2:\n\nThank you for this excellent question. You are right—FairNet introduces some test-time overhead, and we have quantified this in detail.\n\n| **Model** | **Metric**     | **Base** | **+FairNet** |\n| --------- | -------------- | -------: | -----------: |\n| **ViT**   | Parameters (M) |    29.57 |        29.77 |\n|           | GFLOPs (Total) |     0.49 |         0.53 |\n| **BERT**  | Parameters (M) |   109.48 |       109.79 |\n|           | GFLOPs (Total) |    10.88 |        11.29 |\n\nFor **BERT-base**, FairNet adds only **0.28%** more parameters and increases test-time GFLOPs by just **3.7%**. ViT shows similarly small overhead. This confirms that FairNet remains highly efficient and suitable for real-world, latency-sensitive deployments.\n\n# Response to Q3:\n\nThank you for the thoughtful question. Yes, FairNet can handle multiple sensitive attributes, and we have implemented this capability in our experiments.\n\nOn the **HateXplain** dataset, we applied FairNet to address both **race** (\"African American\") and **gender** (\"Female\") biases simultaneously. As detailed in Supplementary Section D.1 and Table 4 (p. 21), FairNet reduced the EOD for race from **20.1** to **9.8** and for gender from **12.9** to **8.2**, with overall accuracy maintained (**79.8%** baseline vs. **79.7%** with FairNet).\n\nThese results demonstrate that FairNet is indeed capable of mitigating fairness gaps across multiple attributes at once.\n\n# Response to Q4:\n\nThank you for this important question. FairNet-Unlabeled relies on pseudo-labels from LOF, and we find its performance to be stable and effective in practice.\n\nOur results (Table 1, p. 8) show that on **CelebA**, FairNet-Unlabeled improves WGA from **77.9%** to **82.3%**, reduces EOD from **10.6%** to **7.3%**, while maintaining the same high accuracy (**95.8%**) as the ERM baseline—all without using any sensitive attribute labels.\n\nTo evaluate stability, we compared FairNet-Unlabeled to a supervised detector with noisy labels. Its performance is comparable to detectors trained with **60–80%** label noise. Moreover, even under **100% noise** (fully random pseudo-labels), FairNet does not fall below ERM baseline performance.\n\nThese findings confirm that FairNet-Unlabeled is not only effective, but also robust—even for outlier samples or noisy signals.\n\n# Response to Q5:\n\nThank you for this insightful question. While our study focuses on fairness under a fixed distribution, we agree that robustness to distribution shift is critical.\n\nAlthough we did not explicitly test FairNet under distribution shift, its design may help in such scenarios. In particular:\n\n- The **contrastive loss** promotes class-level representations that are invariant to sensitive attributes, reducing reliance on spurious correlations.\n- The **conditional LoRA module** enables flexible adjustments without overfitting to specific data patterns.\n\nThese properties could contribute to better generalization under distribution shift, though we have not yet validated this empirically. We appreciate the suggestion and consider this an important direction for future work."}, {"review_id": "5M3h7uaKCr", "reviewer": "Reviewer_hDZ7", "summary": "The paper proposes FairNet, an in-processing debiasing wrapper that inserts lightweight sensitive-attribute detectors and conditionally triggered LoRA adapters into a frozen backbone. A detector-gated contrastive loss pulls minority-group representations toward majority ones, and a simple bound shows worst-group accuracy can rise without harming overall accuracy provided the detector’s TPR/FPR exceeds a data-dependent threshold. Experiments on vision (CelebA) and language (MultiNLI, HateXplain) benchmarks deliver ≈3–4 pp gains in worst-group accuracy and equal-odds while matching ERM accuracy.", "questions": "1. For multiple sensitive attributes (e.g. intersecting race and gender), do you train separate detectors per attribute or one shared detector?\n2. For the unlabeled detector, which unsupervised method is used? How is the threshold chosen? And what are the resulting TPR/FPR on held out data?", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. Conditionally applying LoRA adapters is a conceptually simple idea that is easy to retrofit to existing networks.\n2. The performance preservation condition (eq. 8) makes it explicit when instance-level debiasing can avoid the usual accuracy-fairness trade-off.\n\nWeaknesses:\n1. Combining a learned detector with conditional parameter-efficient adapters is an incremental idea with limited novelty, similar to \"detect-then-correct\" idea that exist in selective prediction and test-time adaptation literature.", "was_revised": true, "final_justification": "I would like to thank the authors for their detailed response. They have adequately addressed my main concerns regarding novelty (W1) and I would like to maintain my score.", "rebuttal": "# Response to W1:\n\nThank you for this insightful comment, which gives us the opportunity to clarify FairNet's core contributions and distinguish it from related literature. While we agree that FairNet can be broadly categorized within the 'detect-then-correct' paradigm, its novelty lies in how it adapts this concept to the unique challenges of algorithmic fairness, its specific technical components, and its theoretical guarantees.\n\nWe would like to highlight three key areas of novelty that differentiate FairNet from prior work in selective prediction and test-time adaptation:\n\n1. **Novel Problem Formulation and Goal:**\n\n   Selective prediction aims to improve model reliability by abstaining from predictions on low-confidence samples. FairNet's goal is fundamentally different: it aims to always make a prediction while ensuring that the prediction is fair. Specifically, it works to improve the accuracy for the worst-performing subgroup (WGA) without degrading overall performance.\n\n   Test-time adaptation (TTA) primarily addresses domain shift between training and test sets. In contrast, FairNet is designed to correct biases that exist within the *training distribution* itself, targeting unfairness between subgroups rather than adapting to a new domain.\n\n2. **Unique Correction Mechanism and Training Objective:**\n\n   Unlike existing methods, FairNet's correction mechanism is a *conditional LoRA* module trained with a novel *contrastive loss function*. This loss is a core contribution and is specifically designed to minimize intra-class representation gaps between different sensitive groups. It directly targets the representational disparities that cause unfairness-a mechanism not present in standard TTA or selective prediction frameworks. Notably, the LoRA parameters are learned during the training phase, rather than updated on-the-fly at test time based on batch statistics as in many TTA methods.\n\n3. **Theoretical Guarantees for Fairness:**\n\n   FairNet is supported by a theoretical analysis that provides a formal guarantee for improving fairness without performance loss. We establish a clear condition (Equation 8) linking the bias detector's quality (TPR/FPR ratio) to a provable increase in worst-group performance. This formal underpinning for achieving fairness without the typical accuracy trade-off is a significant contribution that distinguishes FairNet from more heuristic 'detect-then-correct' approaches.\n\nIn summary, while the high-level concept has precedents, FairNet's novelty comes from its specific application to *instance-level fairness*, its unique contrastive training objective tailored for bias correction, and its formal theoretical guarantees. These contributions address a distinct and critical challenge in machine learning not covered by existing selective prediction or TTA literature.\n\n\n# Response to Q1:\n\nThank you for this excellent question. It allows us to describe our experimental setup and recommendations more clearly.\n\nWhile our framework supports both approaches, we used and would recommend a single, shared detector trained via multi-label learning to handle multiple sensitive attributes, primarily for its parameter efficiency. We encourage this method when multiple biases need to be addressed simultaneously.\n\nHowever, FairNet is flexible. For use cases that require more granular control, one can train separate detectors for each attribute. The advantage of this approach is that biases can be addressed progressively, ensuring that correcting for a new attribute does not negatively impact fairness guarantees already achieved for others. This method offers greater control at the cost of a modest increase in parameter overhead.\n\n\n# Response to Q2:\n\nThank you for these important questions regarding our unsupervised approach. We address each of them below.\n\n1. **Unsupervised Method Used**\n\n   For the FairNet-Unlabeled experiments, we utilized the *Local Outlier Factor (LOF)* method. As described in the Supplementary Material (Section C.3.1), LOF operates on the model's intermediate representations to identify samples that are outliers. This is based on the hypothesis that instances from underperforming minority groups often manifest as representational outliers in a standard ERM-trained model.\n\n2. **Threshold Selection**\n\n   The activation threshold ($\\tau$) is a key hyperparameter in our method, designed to balance the trade-off between accuracy and fairness. A lower threshold encourages more frequent bias correction, enhancing fairness, while a higher threshold tends to preserve overall accuracy.\n\n   In our experiments, we set $\\tau$ = 0.5 uniformly across all datasets to demonstrate the general applicability and robustness of our approach. Although $\\tau$ can be tuned on a validation set (e.g., via grid search) to better suit specific deployment objectives, we found the default value of 0.5 to yield a balanced performance in practice.\n\n   To further illustrate the impact of $\\tau$, we conducted an ablation study on the CelebA dataset (in a supervised setting for clearer interpretation). As shown in Table A, varying $\\tau$ results in different trade-offs between fairness and accuracy, highlighting its practical utility as a tunable parameter. \n\n   *Table A: Impact of activation threshold on FairNet’s performance (CelebA)* \n\n   | **Threshold** | **TPR (\\%)** | **FPR (\\%)** | **TPR/FPR** | **ACC (\\%)** | **WGA (\\%)** | **EOD (\\%)** |\n   | ------------- | ------------ | ------------ | ----------- | ------------ | ------------ | ------------ |\n   | 0.0           | 100.0        | 100.0        | 1.00        | 94.1         | **87.1**     | **4.8**      |\n   | 0.2           | 98.8         | 18.7         | 5.28        | 95.4         | 86.9         | 5.1          |\n   | 0.4           | 96.7         | 6.34         | 15.3        | 95.6         | 86.5         | 5.4          |\n   | 0.5           | 94.1         | 3.45         | 27.2        | 95.9         | 86.2         | 5.8          |\n   | 0.6           | 72.3         | 2.60         | 27.8        | 95.9         | 85.7         | 6.1          |\n   | 0.8           | 62.9         | 1.48         | 42.5        | **96.0**     | 82.1         | 7.4          |\n   | 1.0           | 0.0          | 0.0          | --          | 95.8         | 77.9         | 10.6         |\n\n\n   TPR and FPR refer to the detector’s performance on the minority group. Best values in each column are bolded.\n\n3. **Resulting TPR/FPR on Held-Out Data**\n\n   For the LOF-based unsupervised detector, we evaluated its performance on the held-out test set for CelebA. The resulting metrics are summarized in the following Table B.\n\n   *Table B: Performance of FairNet-Unlabeled (extension to Table 5)* \n\n   |                   | **TPR (%)** | **FPR (%)** | **TPR/FPR** | **ACC (%)** | **WGA (%)** | **EOD (%)** |\n   | ----------------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n   | FairNet-Unlabeled | 74.2        | 7.71        | 9.62        | 95.8        | 81.9        | 7.5         |\n   | ERM               | -           | -           | -           | 95.8        | 77.9        | 10.6        |\n\n\n   TPR and FPR refer to the detector’s performance on the minority group.\n\n   These results demonstrate that FairNet-Unlabeled can effectively improve fairness outcomes even without labeled sensitive attributes, highlighting the promise of unsupervised bias detection in real-world scenarios."}, {"review_id": "ImNjGCjepi", "reviewer": "Reviewer_ZBjU", "summary": "This paper proposes FairNet, a framework to dynamically correct fairness biases at the instance level.  At inference time,  it employs a lightweight bias detector that assesses the likelihood of the instance belonging to a “sensitive group”, and selectively activates Low-Rank Adaptation (LoRA) based correction for instances identified as potentially belonging to the “sensitive group”.\n\nAt training,  both the lightweight bias detector and the  LoRA based correction model are trained together with the base model.  The training can be done on data with or without the sensitive group labels. A contrastive loss is used for training the LoRA module.\n\nExperiments are run on vision and language benchmarks.", "questions": "With the additional bias detector and the LoRA based correction, computational cost will increase at inference time. Has any analysis been done on latency/computational cost?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is generally well written.  It tackles the important problem of fairness in machine learning models. It runs at instance level and dynamically corrects fairness biases when a “sensitive” attribute is detected.  In training, a contrastive loss is used to train the LoRA module for correction. It can use data with or without the sensitive group labels, and the original accuracy performance of the base model is preserved. A theoretical analysis is also provided.\n\nThe proposed approach seems to be solid and its effectiveness is demonstrated on vision and language datasets. \n\nOne concern of the approach is that, with the additional bias detector and the LoRA based correction, computational cost will increase at inference time. It would be desirable to provide some latency/computational cost analysis.", "was_revised": true, "final_justification": "This paper tackles the important problem of fairness in machine learning models. The proposed approach seems to be solid and its effectiveness is demonstrated on vision and language datasets. My questions are adequately addressed in the rebuttal.  Reviews from other reviewers are also generally positive. I would like to maintain my rating for acceptance.", "rebuttal": "# Response to W1:\n\nWe appreciate your observation, and you are absolutely right that our method introduces additional components during inference—specifically, a bias detector and a LoRA-based correction module—which lead to some increase in computational cost.\n\n*Table A: Computational overhead analysis of FairNet. Parameter counts are reported in millions (M), and floating-point operations (FLOPs) are in Giga-FLOPs (GFLOPs).*\n\n|                        | **ViT**      |               | **BERT**     |               |\n| ---------------------- | ------------ | ------------- | ------------ | ------------- |\n| **Metric**             | **Baseline** | **+ FairNet** | **Baseline** | **+ FairNet** |\n| Parameters (M)         | 29.57        | 29.77         | 109.48       | 109.79        |\n| Minority Routing Ratio | --           | 0.15          | --           | 0.07          |\n| GFLOPs (Minority Path) | --           | 0.74          | --           | 16.43         |\n| Majority Routing Ratio | --           | 0.85          | --           | 0.93          |\n| GFLOPs (Majority Path) | --           | 0.50          | --           | 10.90         |\n| **Total GFLOPs**       | **0.49**     | **0.53**      | **10.88**    | **11.29**     |\n\n\nTo address this concern, we performed a detailed analysis to quantify this overhead. As shown in the table A, the GFLOPs for minority and majority groups differ slightly because samples flagged as minority by the detector activate a conditional LoRA module, which adds a small computational cost due to the altered execution path.\n\nHowever, since the proportion of minority samples is typically low, the overall impact on inference efficiency remains minimal. For example, in the case of the BERT-base model, FairNet increases the parameter count by only 0.28%, and the total GFLOPs by just 3.7% (from 10.88 to 11.29 GFLOPs). A similar level of overhead is observed for ViT.\n\nThese results suggest that the computational overhead introduced by FairNet remains minimal, supporting its practicality for real-world deployment, including latency-sensitive applications.\n\n\n# Response to Q1\n\nThank you for raising this important concern about test-time efficiency when using multiple detectors. We agree that computational overhead is a critical consideration for real-world deployment, and we appreciate the opportunity to clarify how FairNet addresses this.\n\nFirst, the design of our bias detectors is explicitly optimized for efficiency. As noted in Section 3.2, each detector $D_{\\phi}^{(l)}$ is a lightweight module—typically a small MLP preceded by attention pooling on intermediate representations $h^{(l)}(x)$. This ensures that even with multiple detectors (e.g., for different sensitive attributes), the additional computational cost remains minimal. The attention pooling operation, which aggregates hidden states into a fixed-size vector, is designed to be computationally inexpensive, and the MLP layers use small hidden dimensions to avoid bottlenecks.\n\nSecond, the conditional activation of LoRA modules further mitigates efficiency concerns. Even with multiple detectors, LoRA adjustments are only applied when triggered by a high risk score, meaning that the majority of samples (i.e., unbiased instances) incur no additional LoRA computation beyond the base model. This selective correction ensures that the overall inference speed remains dominated by the base model’s runtime, with minimal overhead from detectors.\n\n**Planned Revision.** To clarify this aspect, we will include a new subsection in the supplementary material titled *\"Computational Complexity Analysis\"*. This section will provide quantitative results. As shown in the table A presented in 'Response to W1' (*Table A: Computational overhead analysis of FairNet*), FairNet adds only 0.67% and 0.28% to the total parameter count for ViT and BERT respectively. The total GFLOPs increase by just 8.2% (ViT) and 3.7% (BERT), confirming that FairNet maintains high efficiency and is well-suited for scalable deployment."}, {"review_id": "bDArYVAkSY", "reviewer": "Reviewer_33v7", "summary": "The authors propose a technique for adjusting classifier performance on disadvantaged and minority groups via low-rank adaptation. By training a discriminative model to predict when a data instance comes from a disadvantaged group, they can engage the LoRA augmentation on specifically those inputs and not degrade the performance of a fairness-agnostic model on members of the majority. They demonstrate this FairNet's performance on vision and text data, where they improve the model's general accuracy, its accuracy for the worst-performing group, and its equalized odds gap across groups.", "questions": "1. In your experiments, how many detectors are used? Based on your notation, each LoRA module is activated only if its corresponding Detector fires. So in Figure 1, should there be a detector at every blue layer? In your experiments, are the detectors generally consistent, with every layer making similar predictions about minority group membership? What would results look like if you ablated this choice of multiple detectors, and either aggregated the detectors' results to fire all LoRA modules or used a single detector?\n\n2. Line 337 \"We replace our contrastive loss with binary cross entropy loss\". Given that the binary cross $L_\\textnormal{task}$ already appears in the loss in Eq (3), would it be correct to rephrase this as \"we remove the contrastive loss from Eq. (3) and only train on $L_\\textnormal{task}$\", or is a loss being applied to each layer $(j)$? I would be interested to see what effect this ablation has in the fully-supervised CelebA setting. In that setting, if optimizing the contrastive loss leads to better minority accuracy (WGA) than directly optimizing the minority group's cross-entropy loss, there is an extremely strong argument for the claim that the minority group benefits from aligning with the majority group's representation. Even if the WGA is better for the $L_\\textnormal{task}$ optimizer in that setting, this ablation would help me understand why the contrastive loss is helpful in the detector setting, and the contrastive loss could be motivated instead by downstream fairness performance.", "limitations": "See Weakness 4.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The submission has strong experimental results, including strong supplementary results that cover intersections of sensitive attribute, effect of the activation threshold and more.\n\n2. The proposed technique is simple, lightweight, and adaptable. It is agnostic to model architecture, and can improve performance on specific groups without hurting performance on \n\nWeakness:\n1. There are moderate gaps between the motivation and application in some places. For example, I do not find the motivation provided in lines 84-90 regarding choice of fairness criteria persuasive. Other fairness criteria are described as \"potentially abstract [and] might not align with the specific application context.\" However, WGA and EOD are abstract and potentially-misaligned as well. The submission argues that it bypasses the fairness-accuracy tradeoff, but to me it seems that this is inherent to its choice of metrics: if a classifier is held constant for the majority group, improving its accuracy on the minority group (i.e. improving WGA) inherently improves the accuracy of the entire classifier. Thus I find this submission to be more focused on improving model performance on sub-populations than it is about fairness. In lines 222-227, authors argue that aligning representations has the effect of improving downstream fairness, but I do not see this in the experimental results. We do see EOD go down, but EOD trivially goes down when accuracy goes up. EOD is a fairness metric when it exposes asymmetry: e.g. group A is given a lot of false positive (bad outcomes) while group B is given more false negatives (good outcomes). By reporting an average EOD gap it's hard for the reader to tell if EOD shrinks because fairness is improving, or because accuracy is generally improving.\n\n2. Relatedly, I am unconvinced by section 4.3. When sensitive attributes are fully supervised, it seems trivial that holding a classifier constant for the majority group and training an augmented classifier for minority groups with worse performance would simultaneously improve WGA and total accuracy. From my reading, the inequality in Eq (8) boils down to: performance goes up for FairNet if the discriminator is accurate enough that performance gains on correctly-classified minority points outweigh the performance costs on majority points that are incorrectly-classified as members of the minority. \n\nI believe modest edits to the text and experiments can resolve these points. If the contrastive loss aligning the latent representations is having a desired effect on downstream fairness, then it should be demonstrated in the experiments and emphasized in the text over claims that WGA and accuracy can be improved in tandem, which seems evidently true. \n\n3. Two key components of the technique seem unexplained: (a) how contrastive pairs are chosen (e.g. lines 169-170 emphasizes that the selection is crucial, but I cannot find where the selection process is defined), and (b) how the FairNet-Unlabeled algorithm generates pseudo-sensitive attribute labels, beyond the high-level description given in lines 292-294.\n\n4. In the abstract, introduction, and conclusion, FairNet is often described as applying the LoRA augmentation to instances \"identified as (potentially) biased\" or \"predicted to be vulnerable\". I find this language misleading, as it implies that the model is identifying cases where unfair predictions are being made. Instead, the LoRA augmentation is applied to instances identified as being disadvantaged/members of a minority group. This is a large ethical limitation: in the fully-observed setting, this reduced to applying an augmented classifier to minority instances (i.e. violating group fairness) and in the partially-observed setting this reduces to predicting whether an instance is a member of the minority group to decide which classifier to apply, which comes with similar ethical concerns. I do not think these ethical concerns reduce this submission's contribution or render the technique inapplicable, but I think they should be made more clear.\n\nSmall notes:\n- The appendix notation table is wonderful; it would be helpful to reference it from the main text. In addition, some terminology is introduced in lines 200-205 but used much earlier, e.g. $x$ is used on line 138 without introduction.\n\n- The word \"jointly\" is confusing on line 176, when the various modules are being trained sequentially and not in a joint fashion. The use of snowflakes and fire in Fig 1 to demonstrate this is very nice, and would benefit from a single sentence in the figure caption defining what these two symbols mean.", "was_revised": true, "final_justification": "See rebuttal and response. Although the text would benefit from some revisions, I believe the approach and results are strong. The approach combines simple ideas into a lightweight module that can be broadly applied when ethical concerns are appropriately accounted for, as discussed below.", "rebuttal": "# Response to W1: \n\nThank you for your insightful comments regarding our framing of fairness and the fairness–accuracy tradeoff. \n\n**On Bypassing the Fairness–Accuracy Tradeoff:**   \n\nWe agree that improving WGA while holding majority group accuracy constant will mathematically lead to an improvement in overall accuracy. Our claim of bypassing the 'performance–fairness tradeoff' is made in the context of prior work, where fairness interventions often result in a net decrease in overall accuracy-typically because gains for minority groups are outweighed by significant performance degradation for majority groups. For example, in Table 1, GroupDRO achieves a high WGA on CelebA (87.4%) but sacrifices overall accuracy (94.0%), which is lower than the baseline ERM (95.8%). \n\nIn contrast, FairNet’s key contribution lies in its ability to substantially improve worst-group performance while preserving, and in some cases slightly enhancing the overall accuracy. This stands in contrast to the common pattern where fairness improvements come at the cost of top-line model performance. \n\n**On the EOD Metric and Representation Alignment:**   \n\nWe appreciate your observation that a reduction in average EOD may result from overall accuracy gains rather than targeted fairness improvements. To address this, we provide a more detailed view of group-wise error distributions in our multi-attribute debiasing experiments in the Supplementary (Table 4, page 21). We report disaggregated True Positive Rates (TPR) and False Positive Rates (FPR) across demographic groups. \n\nFor instance, in the case of DistilBERT-base, the baseline ERM model achieves a Female TPR of 67.5%. After applying FairNet targeted at the Female demographic (FairNet-Fe), the Female TPR increases substantially to 78.6%, while the Male TPR and FPRs for both groups remain largely unchanged. This indicates that FairNet is not merely improving overall accuracy; rather, it is directly correcting representational disparities by improving sensitivity for the disadvantaged group. Such targeted improvements are consistent with the fairness criterion of Equalized Odds, where the goal is to equalize TPR and FPR across groups. \n\nOur theoretical argument in lines 215-225 is that aligning latent representations provides a direct mechanism for achieving this form of conditional fairness. The experimental results discussed above offer concrete empirical support for that claim. \n\nThank you again for pointing out this ambiguity. We will revise the relevant sections in the manuscript to clarify our definitions.\n\n\n\n# Response to W2: \n\nThank you for your thoughtful analysis of our theoretical contribution in Section 4.3. Your interpretation is correct: Equation (8) formalizes an intuitive condition-overall performance will improve if the accuracy gains on correctly identified minority samples outweigh the performance degradation caused by misclassified majority samples. The main purpose of Equation (8) is to establish a condition under which fairness interventions do not degrade overall performance. Identifying such a no-degradation condition is a key conceptual contribution of our work. \n\nWe would like to clarify that our novelty lies not in the inequality itself, but in the architectural design that explicitly aims to satisfy this condition in practice. As discussed in lines 343-351, conventional debiasing methods are typically static and global. They lack an internal mechanism for selectively applying corrections. Thus, they cannot take advantage of the performance-preservation condition, which fundamentally depends on the selective power of the detector. \n\nThe contribution of Section 4.3 is to provide a formal guarantee that our proposed architecture, by decoupling bias detection from correction, can provably improve WGA without degrading overall accuracy. This theoretical result supports our broader claim that dynamic, instance-level approaches are more effective than static methods in balancing fairness and accuracy.\n\n\n\n# Response to W3: \n\nWe thank the reviewer for pointing out the lack of methodological clarity in our original manuscript. We will incorporate the following key details into the main text to improve transparency and reproducibility. \n\n**(a) Selection of sample pairs for contrastive learning:** As detailed in Section C.3.2 of the supplementary material (Lines 991–1021), we do not construct positive/negative pairs by selecting individual instances during training. Instead, we precompute target target representations using the baseline ERM model, and define contrastive targets as follows: \n\n- **Positive target:** For an anchor sample $x_a$ (from the minority group with label $y_a$), the positive target is defined as the average embedding of all majority group samples that share the same label $y_a$. \n- **Negative target:** Similarly, the negative target is computed as the average embedding of samples with labels $y_n \\ne y_a$.\n\nThe contrastive loss is then constructed to encourage the LoRA-adjusted representation of the anchor sample $x_a$ to be closer to the positive target and farther from the negative target. \n\n**(b) Pseudo-label generation in FairNet-Unlabeled:** In the unsupervised setting, we apply the Local Outlier Factor (LOF) algorithm on the intermediate representations $h^{(l)}(x)$ from the baseline model. Clustering is performed based on LOF scores, and the cluster with the smallest number of samples is treated as the minority class, while the rest are treated as majority class. These pseudo-labels are then used to train both the bias detector and the contrastive learning module with LoRA. This approach is based on the assumption that instances from underrepresented or minority groups tend to behave as outliers in the representation space. \n\n\n# Response to W4: \n\nThis is a very important and valid ethical concern. We thank the reviewer for encouraging us to use more precise language and to engage more directly with the ethical dimensions of our work. \n\nWe agree that the phrase 'identifying biased instances' may be misleading. A more accurate description, which we will adopt in the revised manuscript, is that the detector identifies instances predicted to belong to a pre-defined, performance-disadvantaged minority group. The 'bias' we aim to mitigate is a systemic property of the model’s performance on that group, rather than a property of any individual instance. The intervention is thus applied to instances from groups for which the model is known to have systematically lower accuracy. \n\nThis approach, which involves group-aware interventions, does raise important ethical questions around disparate treatment. However, it aligns with a well-established paradigm in the algorithmic fairness literature, shared by many in-processing (e.g., GroupDRO) and post-processing methods that leverage group information to promote equitable outcomes. The objective is to address observed disparities in performance metrics, which may necessitate treating different groups differently. \n\nWe believe this is a crucial discussion to include. Accordingly, we will add a dedicated paragraph to the limitations section or introduce a new ethics statement to explicitly address this issue.\n\n\n# Response to Q1: \n\nThis is an excellent question regarding the implementation details of our experimental setup. While our notation is general and permits a more complex configuration, in the experiments presented in this paper, we adopted a simplified yet effective design that balances performance and efficiency. \n\nSpecifically, we employ a single bias detector ($D_\\phi$), which operates on intermediate representations extracted from the encoder of the base model. The output of this single, strong detector is then used to conditionally activate all LoRA modules inserted at various intermediate layers. In other words, all LoRA modules are controlled by a unified prediction regarding minority group membership. This design avoids potential inconsistencies or conflicts that might arise from using multiple detectors across different layers, and it is highly effective in practice. \n\nWe appreciate your close reading. Our intent was merely to indicate the insertion point of the detector, not that a separate detector is placed at each layer. We will clarify this point in the revised manuscript to avoid ambiguity. \n\nFurthermore, we agree that ablation studies on the number and placement of detectors represent a compelling direction for future work. One could explore strategies such as majority voting across layer-specific detectors or introducing detectors tailored to individual layers. However, such designs would increase model complexity and inference cost, which pose significant challenges. We plan to explore these directions in future research.\n\n\n\n# Response to Q2: \n\nThank you for this thoughtful suggestion. \"we remove the contrastive loss from Eq.(3) and only train on $L_\\textnormal{task}$\" is indeed a more accurate description of our ablation setup. We will revise it in the manuscript for clarity. As you correctly inferred, the loss is not applied separately to each layer. \n\nYour suggestion to evaluate this ablation in the fully-supervised CelebA setting is particularly valuable. As shown in the table below, FairNet-Full allows us to isolate the effect of the contrastive loss more cleanly, without the confounding influence of the detector. Compared to using cross-entropy loss alone, the contrastive loss yields a notable improvement in WGA, supporting the hypothesis that alignment with the majority group's representation is beneficial. This result strengthens the empirical justification for our approach. \n\n| Method               | ACC (\\%) | WGA (\\%) | EOD (\\%) |\n| -------------------- | -------- | -------- | -------- |\n| FairNet-Full         | **95.9** | **88.2** | **3.8**  |\n| w/o contrastive loss | **95.9** | 81.7     | 8.2      |\n| ERM                  | 95.8     | 77.9     | 10.6     |"}, {"review_id": "eol7MMX78r", "reviewer": "Reviewer_2KFg", "summary": "The paper introduces FairNet, a dynamic, instance-level fairness correction framework that selectively applies representation-level adjustments using contrastive Conditional LoRA modules, guided by a lightweight bias detector. Unlike static global debiasing methods, FairNet activates corrections only for instances predicted to be biased, preserving overall performance. A novel contrastive loss aligns intra-class representations across sensitive groups to mitigate minority underfitting. FairNet supports full, partial, or no access to sensitive attribute labels. Theoretical analysis shows that, under reasonable bias detector performance, FairNet can improve worst-group accuracy without reducing overall accuracy. Experiments on vision and language datasets (CelebA, MultiNLI, HateXplain) validate its effectiveness.", "questions": "1.\tHow sensitive is FairNet to the accuracy of the bias detector? Have you evaluated how downstream performance is affected when the detector has suboptimal TPR/FPR, especially in the FairNet-Partial and FairNet-Unlabeled settings?\n2.\tHow do you choose the threshold for activating LoRA modules? Is it a fixed value, or tuned per dataset? How robust is the system to this choice?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1.\tI find the idea of selectively applying fairness correction via conditional LoRA to be a clear advancement over traditional static interventions. The framework addresses over-/under-correction by activating the correction mechanism only on potentially biased instances.\n2.\tThe design accommodates settings with full, partial, or no access to sensitive attributes, which enhances its applicability in real-world scenarios where such labels are often incomplete or missing.\n3.\tThe use of parameter-efficient LoRA modules and compact bias detectors allows FairNet to be integrated into existing models with relatively low computational overhead, making it more practical than full model retraining or architecture redesign.\n\n\n\nWeaknesses \n\n1.\tI am not convinced that we can improve the fairness while keeping the main performance. There is a theoretical analysis of the performance preservation analysis. However, the assumption is strong and makes the application domain more limited. Specifically, the theoretical guarantee relies heavily on the bias detector achieving a high TPR/FPR ratio and assumes that LoRA corrections will significantly help minority groups while having little or no negative effect on the majority. These conditions may not hold in realistic scenarios, particularly when biases are complex or intersectional.\n2.\tFairNet’s core mechanism depends on the accuracy of its internal bias detector. In low-label or unlabeled settings, this module may perform poorly, triggering LoRA on inappropriate samples or missing biased ones. I would like to see a deeper analysis of how sensitive the system is to detector quality, especially in the FairNet-Partial and FairNet-Unlabeled configurations.\n3.\tWhile the experiments show promising results on CelebA, MultiNLI, and HateXplain, I find the scope of evaluation too limited to support the strong claims made. All datasets focus on relatively simple binary or categorical sensitive attributes. There is no evaluation on more complex tasks, real-world large-scale datasets, or scenarios involving multiple/intersectional sensitive attributes. The paper would be significantly strengthened by a broader empirical validation, including robustness to detector failures, ablations on thresholding strategies, and analysis under real-world deployment constraints.\n4.\tThe paper does not adequately address the computational complexity introduced by the bias detectors and per-attribute LoRA modules. It remains unclear how well this approach scales in practice, especially when dealing with many sensitive attributes or high-dimensional inputs.", "was_revised": true, "final_justification": "We thank the authors for their extra experiments on this paper. It is much clear now. We suggest them to include these experiments in the next revision. We raised the score accordingly.", "rebuttal": "# Response to W1\n\nWe sincerely thank the reviewer for their insightful comments and valuable feedback.\n\nWe agree that the theoretical guarantee of performance preservation (Section 4.3) relies on strong assumptions—namely, that the bias detector achieves a high TPR/FPR ratio and the LoRA correction benefits minority groups with minimal adverse effects on the majority. FairNet is designed to satisfy these assumptions in practice.\n\n#### 1. Justifiability of Assumptions\n\nSection 4.3 (\"Why FairNet Satisfies the Condition\", p.6) outlines how these conditions are encouraged by our design:\n\n- **LoRA benefits minority groups.** The contrastive loss explicitly aligns minority and majority representations to enhance fairness.\n- **Limited impact on majority.** Soft-label guidance ensures only minor perturbations to majority samples, typically preserving or slightly improving their performance.\n\nFairNet is designed to satisfy theoretical conditions under realistic assumptions. Table B further shows that even with a suboptimal TPR/FPR ratio, FairNet performs no worse than the ERM baseline, demonstrating its robustness.\n\n#### 2. Intersectional Bias Evaluation\n\nWe further validated FairNet on the HateXplain dataset across race and gender attributes (Supplementary D.1, p.21). Results show consistent fairness gains (e.g., BERT-base EOD(Race): 20.1 → 9.8; EOD(Gender): 12.9 → 8.2) with stable accuracy, demonstrating robustness under intersectional settings.\n\n#### 3. Planned Revision\n\nWe will revise Section 4 to clarify that the theory assumes a binary group setting. While the assumptions may not always strictly hold, our experiments show that FairNet remains effective in more complex bias scenarios, and we plan to extend the theory accordingly in the future.\n\n# Response to W2:\n\nWe sincerely thank the reviewer for raising the important question concerning FairNet’s sensitivity to the quality of its internal bias detector. This is indeed a critical consideration.\n\nAs detailed in Section D.2 of the supplementary material (*\"Impact of Training Data Size for FairNet-Partial\"*, p. 22), we conducted a dedicated quantitative analysis. **Table 5** shows:\n\n- **Under minimal supervision (0.1% labeled):** The detector performs poorly (TPR/FPR = 10.9), yet FairNet-Partial achieves notable fairness gains: WGA improves from 77.9% to 82.1%, EOD drops from 10.0% to 7.1%, with accuracy maintained at 95.7%.\n- **With more labels:** Detector quality improves (TPR/FPR up to 27.2), and fairness metrics improve correspondingly—demonstrating a **smooth, monotonic scaling** with supervision quality.\n\n**1. FairNet-Unlabeled with weak detectors**  \nWe include an extension showing that even in fully unsupervised settings, FairNet is robust:\n\n*Table A: Performance of FairNet-Unlabeled (extension to Table 5)*\n\n|         | **TPR (%)** | **FPR (%)** | **TPR/FPR** | **ACC (%)** | **WGA (%)** | **EOD (%)** |\n| ------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n| Unlabel | 74.2        | 7.71        | 9.62        | 95.8        | 81.9        | 7.5         |\n| ERM     | -           | -           | -           | 95.8        | 77.9        | 10.6        |\n\n**2. Sensitivity to detector noise**  \nWe simulate degraded detector quality via label noise, as shown in Table B. Even at **100% noise** (TPR/FPR = 1.14), FairNet performs comparably to ERM:\n\n*Table B. Impact of varying label noise on FairNet’s performance*\n\n| **Noise** | **TPR (%)** | **FPR (%)** | **TPR/FPR** | **ACC (%)** | **WGA (%)** | **EOD (%)** |\n| --------- | ----------- | ----------- | ----------- | ----------- | ----------- | ----------- |\n| 0%        | 94.1        | 3.45        | 27.2        | 95.9        | 86.2        | 5.8         |\n| 20%       | 84.8        | 4.26        | 19.9        | 95.8        | 85.7        | 6.4         |\n| 40%       | 78.5        | 4.83        | 16.3        | 95.8        | 85.0        | 6.9         |\n| 60%       | 64.6        | 5.92        | 10.9        | 95.7        | 83.1        | 8.1         |\n| 80%       | 51.2        | 7.17        | 7.14        | 95.7        | 81.2        | 9.2         |\n| 100%      | 9.11        | 8.00        | 1.14        | 95.7        | 77.8        | 10.3        |\n| ERM       | -           | -           | -           | 95.8        | 77.9        | 10.6        |\n\nThis highlights a key **safety property**: even when the detector fails, FairNet’s performance does not degrade below the baseline.\n\n# Response to W3:\n\nWe sincerely thank the reviewer for highlighting the need for broader empirical evaluation. We agree that such validation is important and summarize below the extensions we have conducted to address these concerns:\n\n#### 1. Evaluation on Multiple and Intersectional Attributes  \n\nSection D.1 (p. 21) reports results on the HateXplain dataset across both race and gender attributes, showing that FairNet can reduce bias across multiple sensitive dimensions simultaneously. We will make this clearer in the revised version.\n\n#### 2. Robustness to Detector Failure  \n\n**Section D.2 (p. 22) and Table B** present a sensitivity analysis covering a wide range of detector qualities (TPR/FPR), including extreme cases where the detector is noisy or fails completely. Results show that FairNet degrades gracefully and never underperforms the ERM baseline.\n\n#### 3. Threshold Analysis  \n\n**Section D.3 (p. 23) and Table C** present an ablation study on the activation threshold $\\tau$. Lower thresholds trigger LoRA more frequently, enhancing fairness at a modest cost to accuracy; higher thresholds better preserve accuracy with reduced fairness improvement. We adopt a default setting of $\\tau = 0.5$, which offers a balanced trade-off. The method remains robust across a range of $\\tau$ values, allowing practitioners to adjust it to suit specific application needs.\n\n*Table C. Impact of activation threshold on FairNet’s performance*\n\n| Threshold | TPR (%) | FPR (%) | TPR/FPR | ACC (%)  | WGA (%)  | EOD (%) |\n| --------- | ------- | ------- | ------- | -------- | -------- | ------- |\n| 0.0       | 100.0   | 100.0   | 1.00    | 94.1     | **87.1** | **4.8** |\n| 0.2       | 98.8    | 18.7    | 5.28    | 95.4     | 86.9     | 5.1     |\n| 0.4       | 96.7    | 6.34    | 15.3    | 95.6     | 86.5     | 5.4     |\n| 0.5       | 94.1    | 3.45    | 27.2    | 95.9     | 86.2     | 5.8     |\n| 0.6       | 72.3    | 2.60    | 27.8    | 95.9     | 85.7     | 6.1     |\n| 0.8       | 62.9    | 1.48    | 42.5    | **96.0** | 82.1     | 7.4     |\n| 1.0       | 0.0     | 0.0     | —       | 95.8     | 77.9     | 10.6    |\n\n#### 4. Scalability and Deployment  \n\nFairNet introduces minimal overhead in terms of inference time and memory  (Table D). Its components are lightweight and designed to scale efficiently with additional attributes. We will clarify this in the revised paper to reinforce its practicality.\n\n# Response to W4:\n\nWe sincerely thank the reviewer for pointing out the lack of discussion on computational complexity and scalability.\n\nFairNet is designed to be lightweight and modular:\n\n#### Lightweight Components  \n\nThe bias detectors are implemented as small MLPs with negligible cost. LoRA adds fewer than 1% additional parameters, making it highly efficient.\n\n#### Linear Scalability  \n\nFairNet scales linearly with the number of sensitive attributes. Each attribute introduces one small detector and one LoRA module. These components are independent and can optionally be shared for multi-label scenarios. As a result, the total overhead remains modest, even with multiple attributes or high-dimensional inputs.\n\n#### Planned Revision  \n\nTo clarify this, we will include a new subsection in the supplementary material titled *\"Computational Complexity Analysis\"*, with the following quantitative results:\n\n*Table D. Computational overhead analysis of FairNet*  \n*Parameters are reported in millions (M), and FLOPs in giga-FLOPs (GFLOPs).*\n\n| **Model** | **Metric**     | **Base** | **+FairNet** |\n| --------- | -------------- | -------: | -----------: |\n| **ViT**   | Parameters (M) |    29.57 |        29.77 |\n|           | GFLOPs (Total) |     0.49 |         0.53 |\n| **BERT**  | Parameters (M) |   109.48 |       109.79 |\n|           | GFLOPs (Total) |    10.88 |        11.29 |\n\nFairNet increases parameter count by only 0.67% (ViT) and 0.28% (BERT), and GFLOPs by 8.2% and 3.7%, respectively—demonstrating that it remains efficient and scalable in practice.\n\n# Response to Q1:\n\nWe sincerely thank the reviewer for this thoughtful and important question. In brief, FairNet demonstrates strong robustness to the accuracy of the bias detector. Even under extreme scenarios—such as when the detector is weak (e.g., only 0.1% of the training data is labeled in a semi-supervised setting) or nearly fails completely (e.g., 80% label noise injection)—FairNet still achieves substantial fairness improvements without degrading performance below the baseline. This indicates a desirable *safety property* of the method.\n\nWe have provided detailed quantitative evidence in our response to *Weakness 2*, which includes results for both **FairNet-Partial** and **FairNet-Unlabeled**, as well as a comprehensive sensitivity analysis with noise-injected detectors (see Table B). We kindly refer the reviewer to that section for full details.\n\n# Response to Q2:\n\nWe sincerely thank the reviewer for this insightful question. Across all experiments, we use a fixed default activation threshold $\\tau = 0.5$ without dataset-specific tuning, which supports the generality of our method.\n\nThe system is also robust to the choice of this threshold. As shown in our response to *Weakness 3* (Point 3), the ablation study in Table C demonstrates that $\\tau$ provides a clear and predictable trade-off between task accuracy and fairness. Practitioners can flexibly adjust this value to align with application-specific goals. For a detailed discussion, we respectfully direct the reviewer to that section."}]

Author rebuttal:

Meta-review: This paper proposes FairNet, an in-processing, instance-level fairness method that inserts a lightweight bias detector into a frozen backbone and conditionally activates contrastively trained LoRA adapters only when needed. The authors theoretically show that under a TPR/FPR threshold on the detector, FairNet can improve WGA without decreasing overall accuracy. The authors then empirically show that FairNet variants improve WGA and reduce EOD on three datasets.

**Strengths**

The paper tackles a common problem in algorithmic fairness (improving worst-group performance without harming overall accuracy) using a simple modular mechanism. The framework’s ability to operate with full/partial/unlabeled sensitive attributes, plus low computational overhead, improves its practical applicability. Reviewers were generally happy with the experimental evaluations.

**Initial Weaknesses**

1. 2KFg and niDK raised concerns about the TPR/FPR assumption used in the theoretical results holding in practice.

2. 2KFg and niDK requested additional experimental evaluations, including larger-scale or intersectional analyses, sensitivity to $\tau$, computational overhead, and robustness to noisy/weak detectors.

3. 33v7 raised several issues with the clarity of the paper.

**Rebuttal Period**

The authors addressed the core concerns with substantive additions, including several new experimental results which address the primary concerns in W1 and W2, and various clarity changes/rephrasings which address W3. 

**Overall Evaluation**

All reviewers felt that their concerns have been addressed by the rebuttals, and all are leaning towards acceptance. As such, I recommend acceptance. The authors should be sure to integrate the new experimental results and promised clarity improvements to the camera ready.
Decision: Accept (poster)