Title: On Optimal Steering to Achieve Exact Fairness
Venue: NeurIPS 2025
Accepted: yes
Keywords: steering

fairness

representation

bayes optimal classifiers

data bias

ideal distributions
Abstract: To fix the `bias in, bias out' problem in fair machine learning, it is important to steer feature distributions of data or internal representations of Large Language Models (LLMs) to \emph{ideal} ones that guarantee group-fair outcomes. Previous work on fair generative models and representation steering could greatly benefit from provable fairness guarantees on the model output. We define a distribution as \emph{ideal} if the minimizer of any cost-sensitive risk on it is guaranteed to have exact group-fair outcomes (e.g., demographic parity, equal opportunity)---in other words, it has no fairness-utility trade-off. We formulate an optimization program for optimal steering by finding the nearest \emph{ideal} distribution in KL-divergence, and provide efficient algorithms for it when the underlying distributions come from well-known parametric families (e.g., normal, log-normal).

Empirically, our optimal steering techniques on both synthetic and real-world datasets improve fairness without diminishing utility (and sometimes even improve utility). We demonstrate affine steering of LLM representations to reduce bias in multi-class classification, e.g., occupation prediction from a short biography in Bios dataset (De-Arteaga et al.). Furthermore, we steer internal representations of LLMs towards desired outputs so that it works equally well across different groups.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposed a preprocessing method to achieve fair classification by steering the feature distribution into an ideal distribution where all cost-sensitive Bayes optimal classifiers will ensure equal opportunity (EO) (also other metrics under a univariate Gaussian setting). The paper first defined the ideal distribution and characterized its property when the distribution is multivariate/univariate Gaussian. Then it formulated the feature steering as an optimization problem to minimize the KL divergence between the target distribution and the ideal distribution $\\tilde{D}$. Since the KL divergence constraint is convex and finding $\\tilde{D}$ can be tractable for certain distribution families, we can solve for $D$ in practice. Numerical experiments demonstrated the effectness of the method.\n\nSTRENGTHS AND WEAKNESSES:\n> Strengths\n\n- The problem formulation is clear and well-motivated.\n- The theoretical results are plausible.\n- The LLM experiments are interestiing.\n- The writing is clear.\n\n> Weaknesses\n\n- Most theoretical results of finding $\\tilde{D}$ in the main paper are based on Gaussian distribution. And the conditions are only sufficient for multivariate Gaussian. Also, the discussed classifiers are Bayes optimal ones. I understand it is somehow standard in theoretical papers, but not sure of the practical applicability. Is it possible to discuss more general distributions?\n\n- Theorem 4.1 and Prop 4.3 demonstrate how we can find $D$, but the discussions on accuracy guarantee are deferred to Appendix (prop 2.5). However, this trade-off can be very important while understanding whether the preprocessing methods should be used in practice. I suggest discussing this more in the main paper.\n\n- Similarly, in the experiment results of section 6, it would be better to show the accuracy comparison (though it is mentioned that accuracy of EF  Affirmative is at the same level of other methods).\n\n\n> Minor: I think the literature on long-term fairness can also be related to this work. For example, \"steering\" and long-term fairness in performative prediction happens naturally when agents respond to the decision-maker [1,2]. It would be interesting to consider whether your preprocessing method can also benefit long-term fairness.\n\n[1] Hardt M, Jagadeesan M, Mendler-Dünner C. Performative power[J]. Advances in Neural Information Processing Systems, 2022, 35: 22969-22981.\n\n[2] Jin, Kun, et al. \"Addressing Polarization and Unfairness in Performative Prediction.\" arXiv preprint arXiv:2406.16756 (2024).\n\nQUESTIONS:\nCould you provide the trade-off results on accuracy?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper first explores an ideal distribution under which group-aware classifiers can achieve absolute fairness metrics, along with the corresponding conditions required to reach such a distribution. The authors then propose a method that solves a KL divergence problem to approximate this ideal distribution. The results show that both the “Affirmative” approach and altering all subgroups lead to significant fairness improvements, while also enhancing classification performance. Based on this theory, the authors test LLMs on two tasks to validate their practical effectiveness in real-world scenarios.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n**1.**  The author finds the condition for ideal distribution.\n\n**2.** It is theoretically guaranteed to achieve EO fairness. \n\nWeakness: I have read this paper at least three times, below are the weaknesses I found. \n\n**1** Although this paper is theoretically based, its presentation lacks rigor. For example, in line 122, it introduces $h(\\cdot)$ without a definition. Line 148 should be argmin $L$, not the probability. Line 127, the second Probability expression, etc., makes it very hard to check the correctness. \n\n**2.** I also found this theory's condition is relatively strong, as the condition introduced in line 171, I don't believe real-world data can fit such a condition.\n\n**3.** It also assumes univariate Gaussion on condition $X$ in Proposition 3.3, Theroem 4.1. It is an oversimplification of the data distribution.\n\n**4.** The theoretical part is based on a Bayesian optimal classifier. I don’t quite understand the experiment, which involves an LLM-based representation steering task.\n\n**5** Did the author assume that Test and Training are IID?\n\nQUESTIONS:\n1. On the definition of the ideal distribution, I don't understand why DP can also be incorporated. As the optimal classifier and DP can be in conflict.\n\n2. How is it possible to make the prediction both fair and accurate? Does it mean the previous Pareto frontier works [1, 2] are not meaningful? \n\n\n[1] Kim, Joon Sik, Jiahao Chen, and Ameet Talwalkar. \"Fact: A diagnostic for group fairness trade-offs.\" International Conference on Machine Learning. PMLR, 2020.\n\n[2] Xian, Ruicheng, Lang Yin, and Han Zhao. \"Fair and optimal classification via post-processing.\" International conference on machine learning. PMLR, 2023.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work introduces a methodology to steer a data distribution towards an ideal one, formalised as a data distribution on which a Bayes-optimal classifier satisfies exact group fairness (e.g., no DP or EO difference). The framework optimises for the closest ideal distribution via minimal KL divergence, pushing the data (or latent representations) towards it. The paper provides the conditions under which a data distribution is ideal, especially for common parametric families (Gaussian, log-normal). The authors further provide efficient solutions for interventions: an “affirmative action” intervention, adjusting the under-privileged group only), and changing all the subgroups. \nResults on simulations show perfect fairness at low KL cost, and some real-world applications show bias reduction on multi-class classification by steering LLM representations, without sacrificing accuracy. \nThe main contributions are:\n- The definition and formalization of ideal distributions without a fairness-utility trade-off\n- The formulation of the problem of finding the nearest ideal distribution as a KL divergence minimization problem, with some efficient solutions through particular interventions\n- Empirical results showing bias reduction in synthetic and real-world cases, while preserving accuracy.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- Although the high-level idea of searching for an unbiased distribution has been explored theoretically before, with the existence of such distributions being proven, this work has a significant contribution by bridging a solid theoretical foundation on how to achieve such a distribution with provable fairness guarantees (even if constrained to some parametric families), and their practical application.\n- The paper has a strong theoretical foundation, from the formalisation of the ideal distribution (for any cost-sensitive Bayes-optimal classifier). Several theorems and propositions provide a framework to understand the necessary and sufficient conditions for the “ideal” property (in Gaussian conditions). The optimization formulation is also well defined. Although recognised as a non-convex problem, the authors provide solutions by restricting to some parametric families. The proposed interventions are also well supported, and it is shown that the fairness/accuracy bounds depend only on the KL divergence.\n- The paper demonstrates the framework’s potential in different and versatile scenarios, from synthetic data, real-world bias reduction (Bias-in-Bios dataset), and generative text-sentiment steering (for “joy”). The results are consistent and show the fairness improvements with negligible accuracy drops (or even improvements).\n\nWeaknesses\n- Not an actual weakness, but some limitations (expanded below) ar ethe parametric assumptions and the application to Bayes-optimal classifiers\n- The paper is dense in some parts, and there are some notation and clarity issues. A thorough revision is suggested to improve clarity and correctness at this level, but here are some catches:\n   - Typos: D’ vs D tilde, BR where BE should be (Fig 1), a “tildeh” for h tilde, etc.)\n   - Abstract: initial sentence is a strong statement since other fairness angles (other than group fairness) can also be applied; and the final sentences seem repetitive\n   - The optimization is for an ideal distribution, but some text parts give space to there being more of such distributions, although it is not clearly discussed.\n   - In Definition 2.1 regarding DP the positive rate is mentioned. Shouldn’t it be the positive rate difference?\n   - It’s not really clear in Section 6.2 how the simulation results are obtained exactly.\n- Some details are not clear on the optimization procedure in practice, and for the representation steering as well (e.g., is the downstream classifier retrained or is it inference-time like the post-training intervention?). Also, the alpha value influence (e.g., it is said that excessive nudging “distorts the steering vector” but the potential reason is not discussed).\n\nQUESTIONS:\n- This work is limited to common parametric families. What kind of applications still work under this assumption, and what others do not? A couple of examples would help. Although the checklist mentions that limitations are discussed in Section 7, this is not clearly stated (on the other hand, the future work is well discussed).\n- The authors mention the framework of ideal distributions can also be applied to audit deployed model for fairness, particularly when the evaluation data may itself be biased. Beyond the steering showed, how can this be achieved in practice? Use the KL gap as a proxy for what’s needed to debias the model? It's not trivial to understand from the text.\n- In the future work the authors mention normalising flows or mixture models as possibly integrated. Would this be a relaxation of the parametric condition (through intermediate transformations or mapping), or would this have more profound implications?\n- It’s not clear if the framework would generalise beyond one binary sensitive attribute.\n\nLIMITATIONS:\nThe authors could improve their limitations by explicating stating them as such: the parametric assumptions, for instance. The practical guidance is not discussed as well. For example, larger KL gaps should be harder to overcome, but what would be the practical consequences, limits, and constraints?", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes the concept of 'ideal distribution', defined as a distribution on which any trained model that minimizes loss will have exact fairness. The paper then provides theoretical analyses of tools to find the 'nearest' ideal distribution given a real-world distribution, as well as several practical approximations. Overall, the setup aims to show how a given input distribution of data can be 'steered' towards an ideal distribution such that the model trained on this new distribution is fair. Finally, it shows the benefits of their technique by steering LLM representations, achieving good fairness scores.\n\nSTRENGTHS AND WEAKNESSES:\nQuality: The paper appears technically sound to me. Most claims are well supported, and the methods used are appropriate.\n\nClarity: The paper is clearly written and easy to follow. My main concerns in clarity come from a lack of appropriate framing of the work in the context of existing fairness literature and a missing discussion of the same. Please see 'Questions' and 'Limitations' for more details.\n\nSignificance: I believe the results are impactful for the community, and I can imagine useful future work building on the results presented here. It takes a unique approach to fairness, and further exploration of the concept of 'ideal distributions' can be interesting for the field.\n\nOriginality: To my knowledge, the perspective of 'ideal distributions' and steering the underlying distribution to achieve fairness is novel. However, clearly there are connections to existing literature, something that I believe can benefit from better framing (see Questions and Limitations section for details).\n\nQUESTIONS:\nOne big 'discussion' missing in the paper is the gap between being able to steer underlying distributions and concerns of fairness in a real-world task. This includes both explicitly acknowledging that the distribution needs to change for training and inference, and discussing the impact of access to finite data instead of distributions (expanded further in the 'Limitations' section).\n\nCorrect me if I've misunderstood the paper, but given that the underlying distribution itself is steered, this technique is much closer to fair representation learning than any 'fairness pre-processing'. Consider the perspective of how a practitioner would take advantage of this technique. When using fair pre-processing techniques, like reweighing, massaging, sampling, etc., these techniques are only applied during training to learn a fair classifier, which can then be applied directly to the original distribution during inference. But similar to fair representation learning, the method proposed in this paper also changes the distribution itself, and its 'steering' is something that'll need to be applied not just to the training set but also to the input during inference.\n\nThis is a vital clarification, in my opinion. One, I suggest the authors improve the intutions provided in the introduction by not comparing their technique to pre-processing and instead of representation learning (or please correct me in the rebuttal if I'm wrong). And two, I suggest the authors provide a better discussion of how a practitioner can take advantage of this technique in classical settings (different from representation steering in LLMs). No additional experiments needed, but a discussion of how this technique fits within the broader landscape of fairness is important.\n\nLIMITATIONS:\nOne limitation of the work is the missing link between distribution-level analysis and real-world applications, where one has access to only a finite amount of data. \n\nAlthough misaligned distributions between the minority and the majority play an important role in the bias encoded in the ML model, this is not the only cause of bias. When there is a lack of enough data from the minority, even approximating the underlying distribution is error-prone, which can further exacerbate fairness concerns (and their variance).\n\nThe paper mostly focuses on distribution-level analysis, assuming access to the underlying distribution and directly steering this distribution. While they do provide empirical results with real-world datasets, a deeper analysis of this missing link and a better discussion of what's lost when you only have access to finite data would further strengthen the paper.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}