Title: value: Fair Class-Incremental Learning using Sample Weighting
Authors: value: Jaeyoung Park
Minsu Kim
Steven Euijong Whang
Venue: ICLR 2025
Keywords: value: trustworthy ai
model fairness
continual learning
class-incremental learning

Abstract: value: Model fairness is becoming important in class-incremental learning for Trustworthy AI. While accuracy has been a central focus in class-incremental learning, fairness has been relatively understudied. However, naively using all the samples of the current task for training results in unfair catastrophic forgetting for certain sensitive groups including classes. We theoretically analyze that forgetting occurs if the average gradient vector of the current task data is in an "opposite direction" compared to the average gradient vector of a sensitive group, which means their inner products are negative. We then propose a fair class-incremental learning framework that adjusts the training weights of current task samples to change the direction of the average gradient vector and thus reduce the forgetting of underperforming groups and achieve fairness. For various group fairness measures, we formulate optimization problems to minimize the overall losses of sensitive groups while minimizing the disparities among them. We also show the problems can be solved with linear programming and propose an efficient Fairness-aware Sample Weighting (FSW) algorithm. Experiments show that FSW achieves better accuracy-fairness tradeoff results than state-of-the-art approaches on real datasets.


Review discussion:
- This approach aims to address the issue of unfair catastrophic forgetting in class-incremental learning by modifying the average gradient vector for the current data.
- It provides a theoretical analysis of the unfair forgetting phenomenon within class-incremental learning.
- It also formalizes fairness-focused optimization problems and reformulates them as linear programming problems to identify suitable sample weights.
- The inconsistency between the cross-entropy (CE) loss used and common fairness metrics (e.g., demographic parity, DP): In lines 212–215, the authors use the CE loss to approximate group fairness metrics. A detailed analysis clarifying the relationship between CE loss and these fairness metrics is necessary.
Additionally, in Eqs. (2)–(4), the fairness losses are defined as the loss disparities between demographic groups, which requires further explanation. The reviewer recommends the authors could provide empirical evidence demonstrating how well CE loss approximates these fairness metrics in practice, or discuss any theoretical guarantees or limitations of this approximation.
- The theoretical analysis lacks depth and originality and shows an inconsistency with the fairness loss. The theorems provided are applicable only to the cross-entropy (CE) loss case and not to fairness metrics, leading to a disconnect between the theoretical framework and the algorithms. The optimization objective includes both fairness loss and CE loss;
however, the theorems do not adequately incorporate the fairness losses. The reviewer suggests that the theorems be extended to explicitly incorporate fairness metrics or that a more rigorous justification be provided for how the CE-based theorems relate to fairness in practice. This would help bridge the gap between the theoretical framework and the proposed algorithms.
- The paper does not mention several recent studies in the related work section, which could also adjust new samples' weightings to mitigate fairness disparity.
For example, active learning may be closely related to continual learning, as shown in studies like [1-4].
[1]  Fair Active Learning, SIGKDD 2021.
[2] Fairness Without Harm: An Influence-Guided Active Sampling Approach, arxiv 2024.
[3] Influence selection for active learning, CVPR 2021.
[4] Active finetuning: Exploiting annotation budget in the pretraining-finetuning paradigm, CVPR 2023.
- Why is the loss referred to as the cost function in line 241?
- The fairness definitions used in the experiments appear to differ from those presented in Section 3.2. Are these definitions consistent? If not, why?
- It is unclear what are the sample weight variables appeared in optimization problems?
No ethics review needed.
(Response 1/2)
We appreciate your thoughtful comments. We addressed them below and improved our manuscript.
> W1. Inconsistency between cost function using cross-entropy (CE) and common fairness metrics
A1. We clarify that existing works [1, 2, 3, 4] empirically verified that using other functions like cross-entropy (CE) loss can provide reasonable proxies for common group fairness metrics including equalized odds (EO) and demographic parity (DP). We did make this point in lines 212-215 of our initial submission, but agree there needs to be more clarification. We thus elaborate on how minimizing the cost function for EO using CE loss ensures EO in multi-class classification in Section A.2 of our revised paper by extending the proof in [1].
In addition, we have included our experimental results in the revised paper that compares EO disparity and the cost function for EO, demonstrating that their trends move simultaneously.
[1] Shen et al., ``Optimising equal opportunity fairness in model training'', NAACL 2022.
[2] Roh et al., ``Fairbatch: Batch selection for model fairness'', ICLR 2021.
[3] Roh et al., ``Dr-fairness: Dynamic data ratio adjustment for fair training on real and generated data'', TMLR 2023.
[4] Gupta et al., ``Fairly Accurate: Optimizing Accuracy Parity in Fair Target-Group Detection'', arXiv 2024.
> W2-1. The theorems provided are applicable only to the cross-entropy (CE) loss case and not to fairness metrics
A2. Our method can be applied regardless of the loss definition if (1) the fairness loss promotes fairness (as in A1) and (2) the loss update process can be linearly approximated.
When the loss can be approximated as $\tilde{\ell}(f_{\theta}^{l}, G) = \ell(f_{\theta}^{l-1}, G) - \eta \nabla_{\theta} \ell(f_{\theta}^{l-1}, G)^\top \biggl(\frac{1}{|T_l|} \sum_{d_i \in T_l} \mathbf{w}^i_l \nabla_{\theta} \ell(f_{\theta}^{l-1}, d_i)\biggr)$, all loss and gradient terms with $f_{\theta}^{l-1}$ can be measured from the previous epoch, which means these values are constant.
Consequently, the overall optimization becomes linear programming.
Furthermore, we can apply the theorem regardless of the type of loss, as long as it satisfies the two conditions mentioned above.
We choose to use CE-based loss as it is not only differentiable, allowing the approximated loss to be derived using gradients, but also effectively approximates fairness metrics. This makes CE loss well-suited to promote fairness measures.
> W2-2. The theorems do not adequately incorporate the fairness losses
A3. We would like to clarify that the fairness loss can be transformed into linear programming problems, as demonstrated in Theorem 2 and Section A.4 of the paper. The fairness loss is defined as the absolute difference between the sensitive group losses.
For example, in the EO cases, $L_{EO}$ is defined as $\frac{1}{|\mathbb{Y}| |\mathbb{Z}|} \sum_{y \in \mathbb{Y}, z \in \mathbb{Z}} |\ell(f_{\theta}, G_{y, z}) - \ell(f_{\theta}, G_{y})|$. As $\ell(f_{\theta}^{l}, G)$ can be approximated into $\ell(f_{\theta}^{l-1}, G) - \eta \nabla_{\theta} \ell(f_{\theta}^{l-1}, G)^\top\biggl(\frac{1}{|T_l|} \sum_{d_i \in T_l} \mathbf{w}^i_l \nabla_{\theta} \ell(f_{\theta}^{l-1}, d_i)\biggr)$ and all values in loss and gradient terms with $f_{\theta}^{l-1}$ are measured as constant, $L_{EO}$ becomes $\frac{1}{|\mathbb{Y}| |\mathbb{Z}|} \sum_{y \in \mathbb{Y}, z \in \mathbb{Z}} |a - \mathbf{b}^T\mathbf{w}|$ where the values in $a$ and $\mathbf{b}$ are appropriate constants (Details about $a$ and $\mathbf{b}$ can be found in Section A.
4 - Case 2). The transformation for minimizing the sum of absolute values to linear programming was introduced in [5, 6, 7] and is also included in the Lemma of Section A. 4.
Therefore, minimizing $L_{EO}$ also becomes a linear programming problem. Theorem 2 and Section A.4 of the paper also provide proofs for the DP and EER cases.
[5] Ferguson et al., ``Linear programming, fundamentals and applications'', McGraw-Hill, 1958.
[6] McCarl et al., ``Applied mathematical programming using algebraic systems'', Cambridge, 1997.
[7] Asghari et al., ``Transformation and linearization techniques in optimization: A state-of-the-art survey'', Mathematics, 2022.
(Response 2/2)
> W3. More related work
A4. We thank you for the valuable references and would like to make the following clarifications:
* Adaptive weighting methods for fairness [8, 9] have some relevance to our work in the sense that these methods ensure fairness by adjusting the sample weights of the training dataset.
However, these methods assume access to the entire training dataset, which is unrealistic in continual learning.
* Fairness-aware active learning  [10, 11, 12] adjusts the weight of the sample to label, but is about reweighting the amount of labeling instead of sample weighting for fair training.
We added the above discussions along with the references in Sections 2 and C of our revised paper.
[8] Chai et al., ``Fairness with Adaptive Weights'', ICML 2022.
[9] Jung et al., ``Re-weighting Based Group Fairness Regularization via Classwise Robust Optimization'', ICLR 2023.
[10] Anahideh et al., ``Fair Active Learning'', Expert Systems with Applications 2022.
[11] Pang et al., ``Fairness without Harm: An Influence-Guided Active Sampling Approach'', NeurIPS 2024.
[12] Tae et al., ``Falcon: Fair Active Learning Using Multi-Armed Bandits'', VLDB 2024.
> Q1. Cost function based on cross-entropy
A5. In W1-A1, we clarify that the cost function mentioned in line 238 of the revised paper can approximate the fairness disparity. We have also included this discussion in our revision (See Section 3.1 and Section A.2).
> Q2. The difference of fairness measures used in our method and experiments
A6. In our method, we use cross-entropy loss to implement approximate measures for EER, EO, and DP disparity. The reason is that the fairness measures in our method are now differentiable, enabling the computation of gradients and the updating of losses unlike when using the fairness measures defined in the experiments, which are not differentiable (we clarify this point in Section 3.1 of our initial submission).
In addition, the fairness measures in the experiments and the cost functions presented in Section 3.2 are indeed consistent with each other, as explained in W1-A1.
> Q3. Sample weight variables in optimization problems
A7. We clarify that the sample weight variable is $\mathbf{w}_l^i$ in our optimization problems. We defined $\mathbf{w}_l^i$ and the loss-updating equation for reweighting in lines 234-236 of our initial submission. We realize these details may not have been visible enough and now mention them in lines 228-232 of our revised paper right next to the optimization description.
Dear Reviewer yfqp,
We have carefully revised and improved our paper based on your suggestions. We hope that our response addresses your questions. We greatly appreciate your time and look forward to the opportunity to discuss any further feedback or suggestions you may have.
Thank you!
Authors
I thank the authors for their extensive reply to my questions.
**Inconsistency between CE and fairness group metrics**: I appreciate the additional clarification on this point.
However, the explanation in the main paper still lacks clarity. Could the authors consider condensing this argument and presenting it as a Proposition? This would make it easier for readers to follow and understand the reasoning.
**Related Work**: In Lines 125–126, when discussing active learning references, the claim regarding training data may be incorrect, as the training data in active learning changes with the acquisition of new samples. Please check carefully for your claim.
**Differences in fairness measures used in the method and experiments**: Thank you for the clarification. Most prior work relies on differentiable surrogates to approximate fairness metrics. Could the authors perform additional experiments evaluating the proposed method using such differentiable surrogates? This would provide direct evidences to show the effectiveness of the approach.
Then, one follow-up question is why use CE loss instead of differentiable surrogates for fairness metrics? From the overview of this method, there doesn’t seem to be any technical difficulty to doing so. Using differentiable surrogates would make the approach more intuitive and align naturally with the overall narrative.
We appreciate your additional questions and respond to each point below.
> P1. Inconsistency between CE and fairness group metrics
A1. We appreciate your suggestion to present the argument about using cross-entropy loss for measuring fairness in the form of a proposition for better clarity. We added the proposition in Sec. 3.1 and more details in Sec. A.2 of our revised paper.
> P2. Related Work
A2. Thank you for pointing out that active learning methods also incrementally train a model with newly acquired data.
However, we clarify that active learning methods use all the currently labeled data for training, which is impractical in continual learning settings. We included this discussion in Sec. 2 and Sec. C of our revised paper.
> P3. Differences in fairness measures used in the method and experiments
A3. We thank you for suggesting the application of differentiable surrogate measures to approximate fairness metrics [1, 2, 3, 4], rather than using cross-entropy loss. We perform experiments using differentiable surrogate losses such as hinge loss (i.e., $\sum_{j\neq y_i}\max(0, s_j-s_{y_i} + 1$) where $y_i$ is the true integer label, $s_j$ is the softmax output for label $j$) to approximate fairness metrics in our proposed method.
When employing hinge loss in FSW, the loss also satisfies the conditions in A2 of our initial response: (1) the disparity between losses reflects fairness, and (2) the loss update process can be linearly approximated.
The tables below present the experimental results for the original FSW (using cross-entropy loss) and FSW with hinge loss to approximate fairness metrics.
Overall, both methods show comparable accuracy-fairness results, suggesting that FSW performs well regardless of the type of loss used to approximate fairness metrics, if the loss satisfies the conditions in A2 of our initial response. We would like to note that the cross-entropy loss is widely used and empirically verified as a reasonable proxy for capturing group fairness metrics in [5, 6, 7, 8], which is why we use it, although we could also use surrogate losses. We will include this discussion and experimental results on using additional surrogate measures to approximate fairness metrics in our revised paper.
| MNIST       |   **Accuracy**   | **EER Disparity** |
|-------------|--------------|---------------|
| FSW         |    0.9243    |    0.0322     |
| FSW (hinge) |    **0.9250**    |    **0.0309**     |
| FMNIST      |   **Accuracy**   | **EER Disparity** |
|-------------|--------------|---------------|
| FSW         |    0.8249    |    **0.0373**     |
| FSW (hinge) |    **0.8256**    |    0.0387     |
| Biased MNIST|   **Accuracy**   | **EO Disparity**  |\||   **Accuracy**   | **DP Disparity**  |
| FSW         |    **0.9086**    |    **0.0601**     |\||    0.8895    |    **0.0072**     |
| FSW (hinge) |    0.9076    |    0.0605     |\||    **0.8909**    |    0.0080     |
| DRUG        |   **Accuracy**   | **EO Disparity**  |\||   **Accuracy**   | **DP Disparity**  |
| FSW         |    **0.4288**    |    0.1381     |\||    0.4050    |    **0.0429**     |
| FSW (hinge) |    0.4261    |    **0.1364**     |\||    **0.4068**    |    0.0455     |
| BiasBios    |   **Accuracy**   | **EO Disparity**  |\||   **Accuracy**   | **DP Disparity**  |
| FSW         |    0.7924    |    0.0726     |\||    0.7966    |    **0.0219**     |
| FSW (hinge) |    **0.7933**    |    **0.0721**     |\||    **0.7967**    |    **0.0219**     |
[1] Cruz et al., ``FairGBM: Gradient Boosting with Fairness Constraints'', ICLR 2023.
[2] Rosa et al., ``A Differentiable Distance Approximation for Fairer Image Classification'', ACCV 2022.
[3] Jiang et al., ``Wasserstein Fair Classification'', UAI 2019.
[4] Beutel et al., ``Putting Fairness Principles into Practice: Challenges, Metrics, and Improvements'', AIES 2019.
[5] Shen et al., ``Optimising equal opportunity fairness in model training'', NAACL 2022.
[6] Roh et al., ``Fairbatch: Batch selection for model fairness'', ICLR 2021.
[7] Roh et al., ``Dr-fairness: Dynamic data ratio adjustment for fair training on real and generated data'', TMLR 2023.
[8] Gupta et al., ``Fairly Accurate: Optimizing Accuracy Parity in Fair Target-Group Detection'', arXiv 2024.
We thank you again for the insightful questions, and please let us know if there are any further concerns.
1. The paper presents a new approach to incremental learning by integrating fairness considerations directly into the learning process through sample weighting. This is a creative solution that addresses a gap in the literature, as most prior work has focused on accuracy alone.
2. The theoretical analysis provided is rigorous and well-founded.
1. The convergence analysis is necessary for the proposed method. Due to the sample reweighting, the learning objective is changed, so it is not clear whether the converged point is close to the global optimum or not. Intuitively, adjusting the sample weights will encourage overfitting to a small group of "easily" learned samples.
However, this small group of data may not be a good representation of the whole data distribution.
2. Should $m$ change according to different sample weights $w$? In some cases where $w_i=0$, this sample should not be counted as one in calculating $m$.
3. Theorem 1 assumes overperforming groups have a lower loss than underperforming groups. Where is the loss defined wrt, on training data or validation data? There might also be some overfitting issues.
Please see the weakness.
No ethics review needed.
We thank you for the insightful comments. We addressed them below and improved our manuscript.
> W1. Convergence analysis
A1. According to [1, 2, 3], if the training loss is Lipschitz continuous and strongly convex, then with a proper learning rate, the model parameters are guaranteed to converge to the optimal points with convergence bounds for any adaptive sample reweighting algorithm. We thus clarify that FSW theoretically guarantees convergence under the above assumptions. The reviewer raised concerns about overfitting, but we address this issue by incorporating the average loss for the current task data as a regularization term in Eq. 3, 4, and 5 to prevent assigning biased weights to a small subset of data and ensures a representation of the overall data distribution for the current task data. We think the consistent and comparable accuracy performance of FSW also empirically supports our claims about convergence and overfitting.
[1] Killamsetty et al., ``GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training'', ICML 2021.
[2] Chai et al., ``Fairness with Adaptive Weights'', ICML 2022.
[3] Lu et al., ``Hybrid Block Successive Approximation for One-Sided Non-Convex Min-Max Problems: Algorithms and Applications'', IEEE Transactions on Signal Processing, 2020.
> W2. Varying sensitive group ratio $m$ according to sample weights $w$
A2. We clarify that the number of samples in sensitive groups (e. g. , $m_{\*, z}$ and $m_{y, z}$) used in Equation 5 is derived from the sufficient condition for demographic parity, which is independent of sample weights. In Sec. A. 3, we start from the definition of demographic parity and derive a sufficient condition for satisfying demographic parity. Here we define $m_{\*, z}$ in the equation as $\frac{\Pr(\hat{\text{y}} = y, \text{y}=y, \text{z}=z)}{\Pr(\text{z}=z)} = \frac{1}{m_{\*,z}} \sum_{i:y_i=y,z_i=z} (1 - \textbf{1}(y_i \neq \hat{y}\_i))$ and $m_{y, z}$ in the equation as $\sum_{i:y_i=y,z_i=z} \ell(f_{\theta}, d_i) = m_{y,z} \ell(f_{\theta}, G_{y,z})$, both for counting the number of samples in a sensitive group.
We then define the cost function for demographic parity using the sufficient condition and formalize the optimization problem, similar to those for equal error rate and equalized odds.
> W3. Notation in Theorem 1
A3. In Theorem 1, we clarify that the overperforming and underperforming groups $G_1$ and $G_2$ are validation data, whereas $d_i$ is training data, which means that the disparity of the validation losses between the two groups could become larger after training on the current task sample.
Nonetheless, in a class-incremental learning setting, the limited buffer data of previous tasks necessitates the approximation of validation loss using training loss, and this approach and corresponding notations are commonly employed in related works within continual learning [4, 5, 6, 7].
[4] Lopez-Paz et al., ``Gradient Episodic Memory for Continual Learning'', NIPS 2017.
[5] Chaudhry et al., ``Efficient Lifelong Learning with A-GEM'', ICLR 2019.
[6] Aljundi et al., ``Gradient based sample selection for online continual learning'', NeurIPS 2019.
[7] Yoon et al., ``Online Coreset Selection for Rehearsal-based Continual Learning'', ICLR 2022.
Dear Reviewer bARJ,
We have carefully revised and improved our paper based on your suggestions. We hope that our response addresses your questions. We greatly appreciate your time and look forward to the opportunity to discuss any further feedback or suggestions you may have.
Thank you!
Authors
Thanks for the explanation. My first question is about the change of learning objective during sample reweighting, aka distribution shift, which may lead to a different converged point. The main concern is not overfitting.
Thanks for clarifying your first question. Since we add the average loss of the current task data to our optimization problems as a regularization term (i.e., $\lambda \frac{1}{|\mathbb{Y}\_c|} \sum_{y \in \mathbb{Y}\_c} \tilde{\ell}(f_{\theta}, G_y)$ in Eq. 3 and $\lambda\frac{1}{|\mathbb{Y}\_c| |\mathbb{Z}|} \sum_{y \in \mathbb{Y}\_c, z \in \mathbb{Z}} \tilde{\ell}(f_{\theta}, G_{y, z})$ in Eq. 4 and 5, where $\mathbb{Y}_c$ is the set of classes in the current task, and no weights are used), the distribution shift you mentioned is actually limited. Please understand we intended to make this point in our first response while mentioning overfitting as you mentioned overfitting as an example.
We thank you again for the insightful questions, and please let us know if there are any further concerns.
To supplement the claim from the previous response, we provide the average accuracy of our method on the current task data compared to other class-incremental learning baselines that do not use sample reweighting as shown in the table below. If there were a severe distribution shift, then the accuracy of our method should be clearly lower than the others.
However, the accuracy results are instead comparable, which suggests that while reweighting the training weight may lead to different convergence points, it still effectively preserves the data distribution of the current task.
| Methods \ Datasets | MNIST | FMNIST | Biased MNIST | DRUG  | BiasBios |
|--------------------|-------|--------|--------------|-------|----------|
|       iCaRL        | 0.915 |  0.918 |     0.805    | 0.468 |   0.834  |
|         WA         | 0.985 |  0.940 |     0.973    | 0.578 |   0.577  |
|        CLAD        | 0.975 |  0.951 |     0.897    | 0.510 |   0.872  |
|        GSS         | 0.994 |  0.986 |     0.968    | 0.489 |   0.874  |
|        OCS         | 0.969 |  0.939 |     0.952    | 0.529 |    -     |
|       FaIRL        | 0.963 |  0.976 |     0.583    | 0.366 |   0.672  |
|        FSW         | 0.944 |  0.844 |     0.947    | 0.494 |   0.860  |
1. Theoretical Foundation: This paper gives good theoretical analysis of how unfair catastrophic forgetting can occur in class-incremental learning, and adjusts the training weights to reduce the forgetting of underperforming groups and achieve fairness.
2. Rigorous Experiments: The experiments are conducted across multiple datasets, which adds to the robustness of the findings. The paper demonstrates that FSW achieves a better accuracy-fairness tradeoff, which is a significant result in the context of fair machine learning.
1. Generalization to Multiple Sensitive Attributes
Issue: The paper primarily focuses on a single sensitive attribute, and the performance of the FSW algorithm in scenarios with multiple intersecting sensitive attributes is not fully explored (for ecample, race and gender in Adult dataset).
Recommendation: The authors should conduct experiments or theoretical analysis to evaluate how the FSW algorithm performs when multiple sensitive attributes are present, such as Adult and Compas dataset .
2. Quantify the tradeoff between fairness and accuracy.
Issue: The paper does not quantify the tradeoff between fairness and accuracy. As iCaRL may outperform FSW in accuracy, the authors should quantify this tradeoff, such as the area under the fairness-accuracy curve.
1. Generalization to Multiple Sensitive Attributes
Will the modification in gradient affect another sensitive attribute while imporving the fairness on one sensitive attribute?
No ethics review needed.
We appreciate your insightful comments and valuable suggestions, which have helped us significantly improve our manuscript.
> W1. Generalization to multiple sensitive attributes
A1. Thanks for noting this limitation. FSW can be extended to tasks involving multiple sensitive attributes by defining a sensitive group as a combination of sensitive attributes.
For instance, recall the loss for EO in a single sensitive attribute is $\frac{1}{|\mathbb{Y}| |\mathbb{Z}|} \sum_{y \in \mathbb{Y}, z \in \mathbb{Z}} |\tilde{\ell}(f_{\theta}, G_{y, z}) - \tilde{\ell}(f_{\theta}, G_{y})|$. This definition can be extended to the case of multiple sensitive attributes as $\frac{1}{|\mathbb{Y}| |\mathbb{Z}\_1| |\mathbb{Z}\_2|} \sum_{y \in \mathbb{Y}, z_1 \in \mathbb{Z}\_{1}, z_2 \in \mathbb{Z}\_2} |\tilde{\ell}(f_{\theta}, G_{y, z_1, z_2}) - \tilde{\ell}(f_{\theta}, G_{y})|$. The new definition for multiple sensitive attributes allows the overall optimization problem to optimize both sensitive attributes simultaneously.
The design above can also help prevent `fairness gerrymandering' [1], a situation where fairness is superficially achieved across multiple groups, but specific individuals or subgroups within those groups are systematically disadvantaged. This is because minimizing all combinations of subgroups disrupts the potential for unfair prediction on certain attribute combinations.
However, the experiment cannot be conducted on the Adult and Compas datasets since both of these datasets have multiple sensitive attributes, but only have two classes, which is not suitable for continual learning.
Overall, we discuss such an extension in Section D.1 and leave it for future work.
[1] Kearns et al., ``Preventing fairness gerrymandering: Auditing and learning for subgroup fairness'', ICML 2018.
> W2. Quantify the tradeoff between fairness and accuracy
A2. Thank you for pointing out the need for fairness and accuracy tradeoff. We report the tradeoff comparison between FSW and other baselines in Sections 4.1, B.6, and Figures 8-11 of our revision.
> Q1. Influence of gradient modification in multiple sensitive attribute schema
A3. As demonstrated in W1-A1, we can design a loss function that considers multiple sensitive attributes, enabling us to find a gradient modification that minimizes the loss function for all fairness attributes. Particularly, since LP optimization is convex, we can determine the globally optimal sample weight, which improves fairness across all sensitive attributes.
Dear Reviewer 4cfM,
We have carefully revised and improved our paper based on your suggestions. We hope that our response addresses your questions. We greatly appreciate your time and look forward to the opportunity to discuss any further feedback or suggestions you may have.
Thank you!
Authors
We are pleased to address your questions and appreciate your feedback in helping us improve our paper.
Best,
Authors
- The presentation of the paper is quite clear and well organized. The example given in the introduction and the first figure help readers understand the problem setup and the challenge.
- To my knowledge, the problem formulation is novel and the proposed method is original. It introduces group fairness notions to class-incremental learning where the class itself can even be a sensitive attribute. Based on existing literature on gradient angle analyses for forgetting previous classes, the authors extend this to unfair forgetting by examining the gradient misalignment between the sensitive groups and new data. The authors also show that theoretically how the misaligned gradients could exacerbate the disparity of the loss between underperforming and overperforming groups.
- The proposed sample weighting algorithm is equivalent to linear programming and could be solved in polynomial time.
- The experiments section is comprehensive, the authors consider several different datasets and baselines. The experiments show that FSW has better performance in terms of balancing classification accuracy and fairness.
- Although overall the paper is well organized, I think that several parts could have more clarification in terms of mathematical writing.
For example, in Equation 1 there are two gradient vectors multiplied with a dot $\nabla_{\theta} \ell( f_{\theta}^{l-1}, G ) \cdot\nabla_{\theta} \ell( f_{\theta}^{l-1}, d_{i} )$, using a transpose could improve the clarity and readability. A similar argument can be made for the weighted gradient term as well $w_l\nabla_{\theta} \ell( f_{\theta}^{l-1}, T_{l})$, it would be helpful to introduce this notation explicitly before using it.
- The authors show that the sample weighting problem is equivalent to linear programming. Prior to this conversion, the feasible set is defined as a hypercube, $[0,1]^d$. If the conversion to a linear program does not alter this feasible set, it implies that the optimal solution for the weight vector would lie at a vertex of the hypercube, meaning all weights will be either 0 or 1. Figures 2 and 15-22 appear to support this claim. Can you clarify whether the optimal weights are indeed binary (0 or 1) in practice? If so, how does this affect the interpretation of FSW as a weighting method versus a selection method? I am not sure that it will affect the normalizing factor during the parameter update step(normalizing with the total number of samples instead of normalizing with the number of samples whose weight is 1).
Additionally, if this observation/claim is true, then the visualization of the weights in Figure 2 could be improved. It could show the percentage of the active samples that have weight 1 from each group.
You can see the points raised in the weaknesses section. I have also additional questions:
- Could authors explain why they consider the average absolute difference between the loss of a class and the average loss of all classes for group fairness notions? Have you considered using squared differences instead of absolute differences in your fairness measures? If so, what were the tradeoffs in terms of computational complexity and performance? Would this alternative formulation allow for a closed-form solution with a projection step?
- It’s unclear whether FSW is applied to each batch individually or calculated once at the beginning of each epoch, with the same weights used for all batches within that epoch. A clarification on this would be helpful.
- Although buffer size is not the main focus of the paper, it would be insightful to understand its impact on unfair forgetting. Have you considered running an ablation study varying the buffer size to examine its impact on unfair forgetting and the proportion of new data assigned non-zero weights?
No ethics review needed.
We appreciate your thoughtful comments, which have helped us significantly improve our manuscript.
> W1. Notation
A1. We correct Equation 1 to be $\tilde{\ell}(f_{\theta}, G) = \ell(f_{\theta}^{l-1}, G) - \eta \nabla_{\theta} \ell(f_{\theta}^{l-1}, G)^\top \nabla_{\theta} \ell(f_{\theta}^{l-1}, d_i)$.
In addition, we correct Equation 2 using training weights $\mathbf{w}\_l \in [0, 1]^{|T_l|}$ to be $\tilde{\ell}(f_{\theta}, G) = \ell(f_{\theta}^{l-1}, G) - \eta \nabla_{\theta} \ell(f_{\theta}^{l-1}, G)^\top \bigl(\frac{1}{T_l} \sum_{d_i \in T_l} \mathbf{w}\_l^i \nabla_{\theta} \ell(f_{\theta}^{l-1}, d_i)\bigr)$, where $\mathbf{w}\_l^i$ is a training weight for the current task sample $d_i$. We have reflected this in our revision.
> W2. Sample weighting analysis
A2. We clarify that the optimal sample weights are close to 0 or 1 in practice, but they are not strictly binary (0 or 1). If we change the optimization problem for sample selection, the problem would become a mixed-integer linear programming problem, which is NP-hard and cannot be solved efficiently. We thus propose sample weighting instead of sample selection, where the optimal weights naturally lie close to extreme values at the boundaries (0 or 1), as the reviewer anticipated. We have added this clarification in our revision.
> Q1. Definition of Equal Error Rate (EER) disparity
A3. We define EER disparity using the average absolute difference, following the definition of other group fairness metrics such as EO and DP disparity. If we define EER disparity using squared differences instead of absolute differences, the optimization problems in Equations 3, 4, and 5 would become quadratic programming problems, which generally require more time to solve due to computing the Hessian matrix. If the Hessian matrix is positive semi-definite (i.e., the problem is convex) and there are no inequality constraints, we can derive a closed-form solution, but solving it still has polynomial time complexity. We have addressed these points in our revision.
> Q2. Update frequency of sample weights using FSW
A4. We clarify that sample weights are computed once at the beginning of each epoch using FSW, and the weights are applied to all batches within the epoch.
Although it is more accurate to update sample weights every batch, we perform FSW every epoch for computational efficiency, and the empirical effectiveness of this approximation is clearly shown in gradient-based subset selection and weighting works [1, 2]. We have added this discussion in our revision.
[1] Killamsetty et al., ``GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning'', AAAI 2021.
[2] Killamsetty et al., ``GRAD-MATCH: Gradient Matching based Data Subset Selection for Efficient Deep Model Training'', ICML 2021.
> Q3. Additional experiments -- Varying the buffer size
A5. Thanks, we have new experimental results of FSW on the MNIST and Biased MNIST datasets when varying the buffer size to 16, 32, 64, and 128 per sensitive group. As the buffer size increases, both accuracy and fairness performances improve.
In addition, we compute the number of current task data assigned with non-zero weights (i.e., not close to zero) and there is no clear relationship between buffer size and weights. We have added these additional experimental results and analysis in our revision.
Thanks for the responses.
I think that the current notation is much better. I appreciate the effort in improving its presentation. Your explanation regarding the use of absolute differences instead of squared distances is well-reasoned and aligns with computational efficiency.
Additionally, the clarification about the frequency of weight updates and the newly added results on the impact of buffer size are valuable contributions.
However, I have the following questions:
- I understand that the linear program does not produce strictly binary solutions, I am curious whether this is due to numerical precision issues or properties of the optimization.
Specifically, if you round the solutions to binary (0 or 1) values and evaluate the objective function, does the rounded solution give a lower value? If so, this could suggest that the true optimal solution is effectively binary, and the algorithm may function more as a sample selection.
- I have one more question regarding the normalization coefficient in Equation 2. Currently you normalize by $\frac{1}{|T_l|}$ where $|T_l|$ denotes the number of samples depending on the considered fairness notion. Would it not be more reasonable to normalize by $\frac{1}{\sum w_i}$ which approximates the effective number of samples used for training?
I will maintain my positive rating for this paper.
We are glad our paper has improved based on your great suggestions and questions. We appreciate your additional questions and respond to each point as follows.
P1. Close to binary weights (0 or 1) in FSW
A1. As the optimization solver CPLEX finds the optimal solution to minimize the objective function, rounding the solutions to binary values (0 or 1) results in a higher value for the objective function. We also perform experiments to compare the accuracy and fairness performance of FSW, with and without rounding applied, as shown in the tables below.
Overall, FSW without rounding shows better accuracy and fairness results in most cases.
However, there are some underperforming cases, which yield results opposite to those from the optimization problem, since we use approximated losses in our optimization problems.
Overall, we believe this further analysis suggests that our method still functions as a sample weighting method, rather than as a sample selection method, although the acquired sample weights are close to 0 or 1.
| MNIST        |   Accuracy  | EER Disparity |
|--------------|-------------|---------------|
| FSW          |    **0.9241**   |     **0.0315**    |
| FSW (round)  |    0.9223   |     0.0328    |
| FMNIST       |   Accuracy  | EER Disparity |
|--------------|-------------|---------------|
| FSW          |    **0.8248**   |     **0.0372**    |
| FSW (round)  |    0.8243   |     0.0378    |
| Biased MNIST |   Accuracy  | EO Disparity  |
|--------------|-------------|---------------|
| FSW          |    0.9091   |     0.0604    |
| FSW (round)  |    **0.9101**   |     **0.0585**    |
| Biased MNIST |   Accuracy  | DP Disparity  |
|--------------|-------------|---------------|
| FSW          |    **0.8888**   |     **0.0072**    |
| FSW (round)  |    0.8791   |     0.0094    |
P2. Normalization coefficient
A2. We clarify that $|T_l|$ is the number of samples in the current task data, not depending on the considered fairness notions. If the training data is given as $T_l$ in general training without sample weighting, we update a model using the average gradient of training data as follows: $\theta = \theta^{l-1} - \eta \biggl(\frac{1}{|T_l|} \sum_{d_i \in T_l} \nabla_{\theta} \ell(f_{\theta}^{l-1}, d_i)\biggr)$. According to Lemma 1 and Section A. 1, the approximated loss of a sensitive group $G$ is updated as follows: $\tilde{\ell}(f_{\theta}, G) = \ell(f_{\theta}^{l-1}, G) - \eta \nabla_{\theta} \ell(f_{\theta}^{l-1}, G)^\top \biggl(\frac{1}{|T_l|} \sum_{d_i \in T_l} \nabla_{\theta} \ell(f_{\theta}^{l-1}, d_i)\biggr)$.
In Equation 2, we assign a training weight to each sample for sample weighting as follows: $\tilde{\ell}(f_{\theta}, G) = \ell(f_{\theta}^{l-1}, G) - \eta \nabla_{\theta} \ell(f_{\theta}^{l-1}, G)^\top \biggl(\frac{1}{|T_l|} \sum_{d_i \in T_l} \mathbf{w}\_l^i \nabla\_{\theta} \ell(f\_{\theta}^{l-1}, d_i)\biggr)$. We believe this transformation is natural and valid. If the normalization coefficient $\frac{1}{|T_l|}$ is replaced with $\frac{1}{\sum w_l^i}$, the revised Equation 2 cannot handle the case where all weights are zero, and our revised optimization problems of Equations 3, 4, and 5 cannot be solved as linear programming.
We thank you again for the insightful questions, and please let us know if there are any further questions.
Thanks for your detailed responses. I do not have any other questions and I will maintain my score.
We are pleased to address your questions and appreciate your feedback in helping us improve our paper.
Best,
Authors

Meta-review:
role: AC
data: metareview: value: This paper aims to address unfair catastrophic forgetting in class-incremental learning where the model learns new classes over time without forgetting the previous ones. The paper proposes a fairness-aware sample weighting algorithm which adjusts the training weights of current task samples to mitigate the unfair forgetting problem for the sensitive groups. 

Reviewers raised concerns on the convergence analysis of the algorithm when the learning objective shifts, inconsistency in the use of CE loss and fairness metrics, missing discussions/comparisons with the literature, among others.
additional_comments_on_reviewer_discussion: value: The main concerns of a couple of reviewers were not alleviated after the rebuttal.
role: PC

Decision: Reject