Title: When majority rules, minority loses: bias amplification of gradient descent
Venue: NeurIPS 2025
Accepted: yes
Keywords: Bias

fairness training

gradient descent

unbalanced learning
Abstract: Despite growing empirical evidence of bias amplification in machine learning, its theoretical foundations remain poorly understood. We develop a formal framework for majority-minority learning tasks, showing how standard training can favor majority groups and produce stereotypical predictors that neglect minority-specific features. Assuming population and variance imbalance, our analysis reveals three key findings: (i) the close proximity between "full-data" and stereotypical predictors, (ii) the dominance of a region where training the entire model tends to merely learn the majority traits, and (iii) a lower bound on the additional training required. Our results are illustrated through experiments in deep learning for tabular and image classification tasks.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper develops a framework for majority-minority learning tasks, going beyond empirical evidence of bias amplification to provide theoretical foundations. It shows how standard training can favor majority groups and produce stereotypical predictors that neglect minority-specific features.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper conducts various experiments across image and tabular datasets to validate the effectiveness of the proposed method and introduces the \"fairness overcost ratio\" to quantify the additional computational cost required for fair training.\n- Based on both theory and experiments, the paper offers practical advice: extend training duration, use larger models, and carefully monitor minority-group performance to avoid stereotype gaps.\n\n**Weaknesses**\n\n- The paper lacks a discussion of the related works. Please add a section on related works in the main paper.\n- The theoretical part relies on relatively restrictive assumptions, such as in Theorem 1, which assumes, which assumes $\\\\tau < \\\\min\\\\left\\\\{ \\\\frac{c}{2}, \\\\frac{\\\\delta}{8}, \\\\frac{\\\\delta^2}{32M} \\\\right\\\\}$. Do the real-world models, like the models tested in the experiments, meet these assumptions?\n- The theoretical results are derived primarily for convex/quadratic losses, while the experiments are conducted using non-convex deep networks. Is there a theoretical gap between these settings?\n- The presentation could be improved.\n    - For example, Figure 1 could be better illustrated with a more detailed caption or elsewhere in the paper, better pointing out what Figure 1 wants to show.\n    - Section 2.4 (linear regression) appears abruptly after the general theoretical framework without a clear rationale. The authors don't justify why linear regression was chosen as the case study or explain what insights it provides beyond the general theory. This also creates poor logical flow: general theory → sudden linear case → back to general gradient dynamics.\n\nQUESTIONS:\n- In the experiments on the imbalanced CIFAR-10 dataset, the paper states that one class (A=0) is subsampled while others (A=1) remain unchanged. Which specific class is chosen? Does the choice of class influence the results?\n- Does Theorem 2 hold only under the linear regression model assumption? Why was linear regression selected as the case study?\n- Although the paper states that “assessing fairness on the test set would require extensive tuning and is left for future work,” and evaluates fairness only using training accuracy, are there any preliminary insights into the gap between training-set fairness and test-set fairness?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates theoretical causes of *bias amplification*, specifically in settings where class imbalance leads models to favor majority groups over minority ones. The authors formalize a majority–minority learning scenario in which the empirical loss decomposes into majority loss $L_1$ and minority loss $L_0$, with $L_0 \\ll L_1$. They prove a perturbation theorem: every critical point of the full loss $L = L_0 + L_1$ is close to a critical point of $L_1$ alone. These paired solutions are termed “representative” (trained on all data) and “stereotypical” (majority-only), and their distance, the “stereotype gap”, is bounded by a quantity that depends on the imbalance in population size and variance. This implies that most of parameter space is dominated by stereotypical solutions. The authors also derive a lower bound on the time required to escape these solutions during training. Experiments on tabular and vision tasks show that typical training procedures converge to stereotypical predictors, and that long training or dataset rebalancing is needed to recover features specific to the minority group.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n* The paper is generally well-written and clearly structured.\n* It tackles an important and under-explored theoretical question: why models trained on imbalanced data tend to amplify bias against minority groups.\n* The theoretical results are not purely abstract; they have practical implications, for instance, highlighting the potential risks of early stopping when validation metrics are dominated by majority-class performance.\n* The experimental evaluation spans a diverse range of settings, including multiple model architectures, various imbalance ratios, and both image and tabular datasets.\n\n**Weaknesses**\n\n* Although the paper claims to be among the few theoretical treatments of bias amplification (lines 28–30), it overlooks several relevant prior works that directly address this issue from a theoretical standpoint \\[1,2,3].\n* Key assumptions underlying the theoretical results (e.g., those in Theorem 1) are not discussed in terms of their realism or the conditions under which they are likely to hold in practice.\n* While the main theoretical insight is sound, the underlying intuition is fairly well-established in the field, so the paper may not significantly advance the state of the art.\n\nReferences:\n\n[1] Francazi, E., Baity-Jesi, M. and Lucchi, A., 2023, July. A theoretical analysis of the learning dynamics under class imbalance. In International Conference on Machine Learning (pp. 10285-10322). PMLR.\n\n[2] Subramonian, A., Bell, S.J., Sagun, L. and Dohmatob, E., 2025. An Effective Theory of Bias Amplification. In International Conference on Learning Representations.\n\n[3] Mannelli, S.S., Gerace, F., Rostamzadeh, N. and Saglietti, L., Bias-inducing geometries: exactly solvable data model with fairness implications. In ICML 2024 Workshop on Geometry-grounded Representation Learning and Generative Modeling.\n\nQUESTIONS:\n* Please expand the related work on the theory of bias amplification, including how this work connects to and extends prior theoretical and empirical studies.\n* Please include a brief description of the main assumptions (e.g., in Theorem 1) and discuss when they are expected to hold in practice.\n\nLIMITATIONS:\nThe authors adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

comments: SUMMARY:
This paper introduces a theoretical framework for analyzing majority–minority tasks. It shows the proximity between “full-data” and stereotypical predictors, analyzes the dominant region of the parameter space, and provides a lower bound on the additional budget required to “unlearn.” The authors also perform experiments to validate their theoretical results.

STRENGTHS AND WEAKNESSES:
**Strengths**

**Theoretical novelty**: The analysis of the distance between critical points in Theorem 1 is both novel and intuitive.

**Empirical validation and reproducibility**: The authors provide experiments on distances to critical points and include the linear regression case for clarity.

**Weaknesses**

**Unclear scope**: In the introduction, the authors situate this work in the bias literature, yet most experiments use an imbalanced-learning setup. It therefore remains unclear whether the results extend to learning from genuinely biased data.

**Restrictive binary setup**: The assumption of a binary majority/minority split is overly limiting, since real-world settings often involve intersectional minority groups.

**Missing test-set generalization**: It is surprising that test errors are not reported for the experiments presented in Table 1 and Figure 3.

QUESTIONS:
Although the theoretical framework appears model-agnostic, the paper discusses only deep-learning models—why exclude other model classes?

Question on Line 99: Why does ∇L₁ = 0 imply ∇L = 0 even when L₁ is much larger than L₀?

LIMITATIONS:
See weaknesses.
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates how gradient descent amplifies bias when training on imbalanced data with majority and minority groups. The authors develop a theoretical framework for majority-minority learning tasks where the total loss L = L1 + L0, with L0 (minority) being negligible compared to L1 (majority). They prove that critical points of the full loss L (representative predictors) are close to critical points of L1 (stereotypical predictors that ignore minority features), with distance bounded by a \"stereotype gap\" that depends on population and variance imbalance ratios. The analysis reveals that the parameter space is dominated by a \"majority-training zone\" where minimizing L essentially minimizes only L1. The authors derive lower bounds on the additional training time required to achieve fair predictions and validate their findings through experiments on CIFAR-10, EuroSAT, and Adult datasets.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:** \n- The paper is well written and easy to understand.\n- The concepts of \"stereotype gap\" and \"majority-training zone\" provide clear geometric intuition for why standard training amplifies bias in imbalanced settings.\n- The lower bounds on debiasing time offer concrete guidance on how much additional training is needed to achieve fairness.\n- The experiments are validated on different domains (vision and tabular datasets).\n\n**Weaknesses:**\n\n- Theorem Statement: For each pair of distinct elements θ,θ′∈crit L₁ (resp. crit L), we have ∥θ−θ′∥≥δ/32M. \nHowever the Proof Issue: The proof derives two different bounds for critical points of L1 and L:\n\n      - For crit L₁: ∥θ−θ′∥≥δ/2M\n\n      - For crit L: ∥θ−θ′∥≥δ/32M\nHowever, the theorem claims the same bound δ/(32M) for both. This contradicts the proof, which shows a stricter bound δ/(2M) for L1 and a looser bound δ/(32M) for L. This error affects the interpretation of critical point separation and needs correction.\n\n- While the paper identifies the problem and quantifies training time, it doesn't propose algorithmic solutions beyond \"train longer,\" missing opportunities for more efficient debiasing methods.\n\n- The figures lacks consistency and clarity. Several figures (e.g., Figure 1 and Figures 3–5) are not accompanied by sufficiently descriptive captions, and the figure references in the main text are very short.  This makes it difficult for the reader to connect theoretical claims with empirical evidence. For example, the figure showing the majority and minority training zones (Figure 1) is central to the paper’s geometric intuition, but its caption is concise and does not adequately explain the experimental setup or relevance\n\n- The paper lacks a dedicated related work section.\n\nQUESTIONS:\n1. The paper uses \"SGD with a constant learning rate... no weight decay or learning rate decay schedule was applied\" to match the theoretical setting. How would the results change with other optimization practices (Adam, learning rate schedules, weight decay, pretrained models)? Would the bias amplification be worse or better?\n\n2. Can the theoretical framework extend to settings with multiple minority groups?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-07"}