Title: On Group Sufficiency Under Label Bias
Venue: NeurIPS 2025
Accepted: yes
Keywords: algorithmic fairness

label noise

label bias

group sufficiency
Abstract: Real-world classification datasets often contain label bias, where observed labels differ systematically from the true labels at different rates for different demographic groups. Machine learning models trained on such datasets may then exhibit disparities in predictive performance across these groups. In this work, we characterize the problem of learning fair classification models with respect to the underlying ground truth labels when given only label biased data. We focus on the particular fairness definition of group sufficiency, i.e. equal calibration of risk scores across protected groups. We theoretically show that enforcing fairness with respect to label biased data necessarily results in group miscalibration with respect to the true labels. We then propose a regularizer which minimizes an upper bound on the sufficiency gap by penalizing a conditional mutual information term. Across experiments on eight tabular, image, and text datasets with both synthetic and real label noise, we find that our method reduces the sufficiency gap by up to 7.2% with no significant decrease in overall accuracy.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper studies the problem of mitigating bias in the presence of noisy labels. It specifically considers the fairness notion of **Sufficiency**, where the true outcomes are independent of the group conditioned on the prediction. First, the authors provide a theoretical analysis demonstrating that enforcing fairness with respect to noisy sensitive attributes necessarily results in a non-negative group sufficiency gap with respect to the true labels. They then propose a regularization based on mutual information that can upper bound the sufficiency gap, assuming access to noisy labels and a transition probability matrix $P(y_{\\text{noise}} \\mid y, \\text{g})$. The paper also presents an extensive experimental study on multiple datasets with both synthetic and real noisy data to evaluate their proposed regulation method.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**  \n1. The paper is well written, easy to follow, and presents a comprehensive literature review.  \n2. The concept of noisy labels is particularly interesting and effectively captures many practical challenges.  \n3. The authors contribute by focusing on the notion of *sufficiency*, which has received less attention compared to other fairness notions in the literature.  \n4. They provide a theoretical analysis that highlights the importance of their proposed regularization approach.  \n5. Extensive experiments are conducted to evaluate the performance of the method.\n\n**Weaknesses**  \n1. **Dependence on transition probabilities**: The method requires knowledge of the transition probabilities. While the authors acknowledge this limitation, and I understand that estimating such quantities may require additional information, there are existing methods that attempt to relax this assumption (e.g., reference [48]). The claim in line 126 that the method requires only noisy labels seems overstated, as the transition probabilities are also needed.\n\n2.  **Entropy term and conditioning**: My second concern is about the correctness of the proposed regularization method. I found it somewhat difficult to follow the derivation of the entropy term in Equation (4), particularly the use of \\( T_{g_i} \\) as the entropy is not conditioning to the sensitive group. See also question Q1.\n\n3. **Evaluation**: Although the paper includes many experiments, additional evaluation—especially studying the effect of different noise levels in the dataset—would be beneficial.\n\nQUESTIONS:\n**Q1.** My first question concerns the entropy term in Equation (4). Specifically, the entropy $H(Y \\mid F(X))$ is not conditioned on the sensitive attribute $G$, so the use of  $T_{g_i}$ is a bit confusing to me. This also appears in the appendix (lines 1049–1055), particularly in line 1052, where the definition of mutual information appears to differ from the one presented in line 202 of the main text. Could you clarify this discrepancy?\n\nThe rest of my questions concern the experiments.  \n**Q2.** Regarding the \"Noise Types,\" I would like to ask why the authors add noise only to one specific sensitive group. Performing additional experiments with the same level of noise applied to different groups seems straightforward and would provide a more comprehensive evaluation.\n\n**Q3.** Regarding the additional experiments, as mentioned before, further evaluation—especially studying the effect of different noise levels in the dataset—would be beneficial. Combining this with experiments that use estimated transition probabilities would provide a more thorough evaluation.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates the challenge of sufficiency in datasets that contain label bias. Similar to previous work that focused on demographic parity and equalized odds the paper tackles the problem first from a theoretical point of view. The authors introduce the sufficiency gap as way of measuring the discrepancy between the sufficiency measure on the biased and unbiased labels. From this they introduce two methodologies to calculate the sufficiency gap without having access to the unbiased labels, which can be used to introduce an additional regularization term to the training loss in order to reduce this sufficiency gap.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper starts out from a strong theoretical basis. In order to come to these theorems, certain assumptions are needed. These assumptions are logically sound and do not effect the usability of the method and are in line with the promises that the paper made in the beginning with relation to the capability of the method. \n\nThe methods are tested on a wide set of datasets with different data types such as tabular, image, and text. All information is provided in the appendix of the paper in order to reproduce the experiments conducted in the paper. \n\nThe value of the novel methods proposed in this paper would have been greatly enhanced if the authors would have used a different method to compare their method against. Fairret is an example of a method that can function for predictive parity (the simplification of sufficiency when dealing with binary classification) and also works as an added loss term. These types of method would allow the authors to show that their specific focus on label bias is warranted as this cannot by done by traditional methods not assuming this level of label bias. \n\nThe paper uses the adult dataset, while still popular in the research area of fairness in AI, it has been shown that this dataset should not be used anymore for showing the capabilities of methods (Fabris et al., Algorithmic Fairness Datasets: the Story so Far). The paper contains datasets with both simulated and real noise with regards to the labels. This is already commendable but the authors could have also used the dataset of Lenders and Calders (Real-life Performance of Fairness Interventions-Introducing A New Benchmarking Dataset for Fair ML) which provides a dataset with these biased and unbiased labels on a dataset where biases are more inherently present. The dataset however is very small. \n\nThe interpretability of figure 2 could be improved; the plots are very small and the legend is not very easy to understand without reading the entire paper.\n\nQUESTIONS:\nIf the camera-ready allows for an additional page then I would ask the authors to increase the size of the plots in Figure 2. Pareto-fronts are still the standard and easy to interpret model capabilities. For the labels I would suggest to replace l_pred with naïve model as I assume this is the model with no bias mitigation method effecting it. It is also quite confusing how this model can exhibit a curve. It is better to plot points that have the same hyperparameters but run for several seeds be 1 point that has a confidence ellipse around it. This shows volatility of a method better. Also if the plots are larger I would suggest having the x-axis to be the same for all plots on the same dataset as this is better visualization for the reader. \n\nThe paper would also improve if the authors could provide results comparing their method with other methods, however I can see how this would fall out of the scope of the rebuttal. While I appreciate it if the authors would do this, I do not expect it.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the group sufficiency fairness metric in the presence of group-dependent label noise. The authors propose a regularization-based fix for such label noise and perform well-designed experiments to demonstrate the effectiveness of their approach.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The problem of studying group sufficiency with label noise surprisingly flew under the radar. Many fairness metrics have been studied with label bias except this one, and thus, this work identifies an important gap.\n2. Theorem 4.1 is interesting as it precisely gives how worse can sufficiency gap can get with label noise.\n3. The experiments are comprehensive (however, there are some doubts, which we will discuss next).\n\nLimitations:\n1. The proposed method has some dominance over others (like vanilla noise correction, since unconstrained clean ERM is sufficient to ensure group sufficiency according to Liu et al. 2019). However, a case where the method can shine out and establish clear dominance is when the transition matrix is estimated and the Brier score itself is noisy. I couldn't find that set of results anywhere (please do point me to them if I have missed those). The supplementary does show real-world label noise results (Table 5), and the CMI-REG doesn't seem to beat vanilla CE, both in Suf-gap and Acc. Because of this, I am skeptical of the empirical advantages\n2. I am also skeptical of the finite sample estimator of the CMI regularizer. Since the regularizer depends on mutual information and conditional entropy, and estimators for both of them have been known to be very tricky and have convergence issues with finite samples,  I am not sure whether the loss function would do its intended job. Had the experimental results outperformed everything else, I could have believed that the estimators are doing their job, but due to the last point, I am not sure.\n\nQUESTIONS:\nI like the problem, but the two severe limitations overpower the theoretical analysis performed by the authors. I would want to ask the authors the following to reconsider the rating on this paper:\n1. In the noisiest of settings where you can only estimate Brier scores and transition matrices with noisy data (perhaps with the help of Clusterability or some anchor points), will the proposed CMI-REG show a lot of significant improvement?\n2. Can the authors comment on the convergence of the proposed mutual information estimators with finite samples? Off the top of the head, they can either argue that it minimizes a relaxed version of the sufficiency gap or assume a parametric distribution on data (Gaussian) to show that it at least works as intended on those kinds of distributions.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a method for ensuring per-sensitive-group calibratedness in the presence of noise. Under the assumption that one knows the per-group transition probabilities for the noisy labels, the paper provides a two-step algorithm to calculate a calibration gap using a mutual information between the ground truth label and the group membership condition on the classifier output. In the first step, the mutual information is estimated, while in the second step, the model is trained using a regularizer informed by the mutual information bound. Empirical results are presented comparing the method with a baseline where the calibration is directly regularized.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, despite the interesting problem, this paper falls below the acceptance threshold for me.  I will enumerate **S**trengths and **W**eaknesses.\n- **S1. The problem of fairness calibration under noise is relevant.** Given how much noise real-world data has and the fact that many risks of systematic discrimination are attributable to noisy data, this is a valuable problem to tackle.\n- **W1. Lack of meaningful baselines.** The baseline discussed in the paper adds its regularizer to prior noise-robust methods to adapt them to the multi-group calibration problem with noisy data. This is useful and does show the versatility of the method, and allows judging the improvement that the use of CMI-REG and CMI-SUF brings to their respective methods. However, these are not competitive baselines for the problem. I understand that the authors claim that no prior work has tackled this exact problem, but it behooves them to construct reasonable baselines for comparison as a result. Here is my suggested baseline: \n  \n  Since the transition matrices are given to CMI-REG, why not transform the noisy data using the transition matrices $T^g$ and $T$ as a pre-processing step and then use any standard multicalibration algorithm?  \n  \n  On the latter note, I find that lack of any discussion or reference to the (now) vast literature of multi-calibration is a miss.  \n- **W2. Novelty and contribution are not clear.** Methodically, the paper's approach can be summarized as adding a calibration regularizer with corrections to prediction rates due to noise, which is completely specified via transition matrices. Furthermore, the robustness to noise seems to be more a function of the base loss $\\ell_{pred}$ in Eq. 2 than a contribution of the presented methods. Unfortunately, the experiments do not convince me otherwise, either due to small relative improvements, many of which fall within error margins.\n- **W3. Uneven presentation.** The level of detail in the presentation is uneven. As an example, I was very surprised to find SUF-REG being presented in the experimental section (Line 254) in 2 lines with minimal details. The authors emphasize that this is a novel contribution. If so, why treat it as an afterthought? Especially given how competitive it can be with the main method, CMI-REG. This left me with more questions than answers, such as: If we can regularize sufficiency directly, what is the point of all the mutual information estimation?\n  Similarly, backward and forward corrections in Lines 206 and 208 are presented with only a reference and no details. \n- **W4. Empirical results are not very convincing.** I touched upon this briefly in W2. Looking at Figure 2, I cannot confirm that CMI-REG and SUF-REG indeed Pareto dominate in every case. In lsac+DMI and lsac+Peer that does not seem to be the case; also, Figure 3 in the appendix seems to be a similarly mixed set of results. Also, overall, the Pareto fronts are rather sparse. I understand that training and hyperparameter tuning may be difficult, but it's really hard to compare a single point to a full curve in a Pareto front setting.\n\nQUESTIONS:\n- Q1. On page 7, I think the transition matrix for Eq. (4) should probably be $T^{-1}$, not $T_{g_{i}}^{-1}$. No? Same for Eq. 6 and onwards, as none of these probability estimates have a conditioning on group $G$.\n- Q2. Please discuss multi-calibration.\n\nLIMITATIONS:\nAs also stated in the paper's limitation section (thanks for making this very clear), the assumption that we have the transition matrices as inputs to the algorithm is a rather strong one that limits the applicability of the algorithm as it is. I think the paper would have benefited greatly from an estimation step using a small, clean dataset (for example).", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-04"}