Fair Class-Incremental Learning using Sample Weighting

## Abstract

Abstract Model fairness is becoming important in class-incremental learning for Trustworthy AI. While accuracy has been a central focus in class-incremental learning, fairness has been relatively understudied. However, naïvely using all the samples of the current task for training results in unfair catastrophic forgetting for certain sensitive groups including classes. We theoretically analyze that forgetting occurs if the average gradient vector of the current task data is in an “opposite direction” compared to the average gradient vector of a sensitive group, which means their inner products are negative. We then propose a fair class-incremental learning framework that adjusts the training weights of current task samples to change the direction of the average gradient vector and thus reduce the forgetting of underperforming groups and achieve fairness. For various group fairness measures, we formulate optimization problems to minimize the overall losses of sensitive groups while minimizing the disparities among them. We also show the problems can be solved with linear programming and propose an efficient Fairness-aware Sample Weighting (FSW) algorithm. Experiments show that FSW achieves better accuracy-fairness tradeoff results than state-of-the-art approaches on real datasets.

## 1 Introduction

Trustworthy AI is becoming critical in various continual learning applications including autonomous vehicles, personalized recommendations, healthcare monitoring, and more (Liu et al., ; Kaur et al., ). In particular, it is important to improve model fairness along with accuracy when developing models incrementally in dynamic environments. Unfair model predictions have the potential to undermine the trust and safety in human-related automated systems, especially as observed frequently in the context of continual learning. There are largely three continual learning scenarios (van de Ven & Tolias, ): task-incremental, domain-incremental, and class-incremental learning where the task, domain, or class may change over time, respectively. In this paper, we focus on class-incremental learning, where the objective is to incrementally learn new classes as they appear.

The main challenge of class-incremental learning is to learn new classes of data, while not forgetting previously-learned classes (Belouadah et al., ; Lange et al., ). If we simply fine-tune the model on the new classes only, the model will gradually forget about the previously-learned classes. This phenomenon called catastrophic forgetting (McCloskey & Cohen, ; Kirkpatrick et al., ) may easily occur in real-world scenarios where the model needs to continuously learn new classes. We cannot simply stop learning new classes to avoid this forgetting either. Instead, we need to have a balance between learning new information and retaining previously-learned knowledge, which is called the stability-plasticity dilemma (Abraham & Robins, ; Mermillod et al., ; Kim & Han, ).

In this paper, we solve the problem of fair class-incremental learning where the goal is to satisfy various notions of fairness among sensitive groups including classes in addition to classifying accurately. In some scenarios, the class itself can be considered a sensitive attribute, especially in classification tasks where a model produces biased predictions toward a specific group of classes (Truong et al., ). In continual learning, unfair forgetting may occur if the current task data has similar characteristics to previous data, but belongs to different sensitive groups including classes, which negatively affects the performance on the previous data during training. Despite the importance of the problem, the existing research (Chowdhury & Chaturvedi, ; Truong et al., ) is still nascent and has limitations in terms of technique or scope (see Sec. 2). In comparison, we support fairness more generally in class-incremental learning by satisfying various notions of group fairness for sensitive groups including classes.

Figure: (a)
Refer to caption: x1.png

We demonstrate how unfair forgetting can occur on a synthetic dataset with two attributes $(x_{1},x_{2})$, and one true label $y$ as shown in Fig. 1(a). We sample data for each class from three different normal distributions: $(x_{1},x_{2})|y=0\sim\mathcal{N}([-2;-2],[1;1])$, $(x_{1},x_{2})|y=1\sim\mathcal{N}([2;4],[1;1])$, and $(x_{1},x_{2})|y=2\sim\mathcal{N}([4;2],[1;1])$. Note that each data distribution can also be defined as a sensitive group with a sensitive attribute $z$. To simulate class-incremental learning, we introduce data for Class 0 (blue) and Class 1 (orange) in Task 1, followed by Class 2 (green) data in Task 2, where Class 2’s data is similar to Class 1’s data. We observe that this setting frequently occurs in real datasets, where different classes of data exhibit similar features or characteristics, as shown in Sec. B.1. We assume a data replay setting where only a small amount of previous data from Classes 0 and 1 are stored and utilized together when training on Class 2 data. After training the model for Task 1, we observe how the model accuracies on the three classes change when training for Task 2 in Fig. 1(b). As the accuracy on Class 2 improves, there is a catastrophic forgetting of Class 1 only, which leads to unfairness.

To analytically understand the unfair forgetting, we project the average gradient vector for each class data on a 2-dimensional space in Fig. 1(c). Here $g_{0}$, $g_{1}$, and $g_{2}$ represent the average gradient vectors of the samples of Classes 0, 1, and 2, respectively. We observe that $g_{2}$ is $127^{\circ}$ apart from $g_{1}$, but $88^{\circ}$ from $g_{0}$, which means that the inner products $g_{2}\cdot g_{1}$ and $g_{2}\cdot g_{0}$ are negative and close to 0, respectively. In Sec. 3.1, we theoretically show that a negative inner product between average gradient vectors of current and previous data results in higher loss for the previous data as the model is being updated in an opposite direction and identify a sufficient condition for unfair forgetting. As a result, Class 1’s accuracy decreases, while Class 0’s accuracy remains stable.

Our solution to mitigate unfair forgetting is to adjust the average gradient vector of the current task data by weighting its samples. The light-green vectors in Fig. 1(c) are the gradient vectors of individual samples from Class 2, and by weighting them we can adjust $g_{2}$ to $g_{2}^{*}$ to make the inner product with $g_{1}$ less negative. At the same time, we do not want $g_{2}^{*}$ to be too different from $g_{2}$ and lose accuracy. In Sec. 3.2, we formalize this idea using the weighted average gradient vector of the current task data. We then optimize the sample weights such that unfair forgetting and accuracy reduction over sensitive groups including classes are both minimized. We show this optimization can be solved with linear programming and propose our efficient Fairness-aware Sample Weighting (FSW) algorithm. Fig. 1(d) shows how using FSW mitigates the unfair forgetting between Classes 0 and 1 without harming Class 2’s accuracy much. Our framework supports the group fairness measures equal error rate (Venkatasubramanian, ), equalized odds (Hardt et al., ), and demographic parity (Feldman et al., ) and can be potentially extended to other measures.

In our experiments, we show that FSW achieves better fairness and competitive accuracy compared to state-of-the-art baselines on various domains of image, text, and tabular datasets. The benefits mainly come from assigning different weights to the current task samples during training with accuracy and fairness in mind.

Summary of Contributions: (1) We theoretically analyze how unfair catastrophic forgetting can occur in class-incremental learning; (2) We formulate optimization problems for mitigating the unfairness for various group fairness measures and propose an efficient fairness-aware sample weighting algorithm, FSW; (3) We demonstrate how FSW outperforms state-of-the-art baselines in terms of fairness with comparable accuracy on various datasets.

## 2 Related Work

Class-incremental learning is a challenging type of continual learning where a model continuously learns new tasks, each composed of new disjoint classes, and the goal is to minimize catastrophic forgetting (Mai et al., ; Masana et al., ). Data replay techniques (Lopez-Paz & Ranzato, ; Rebuffi et al., ; Chaudhry et al., ) store a small portion of previous data in a buffer to utilize for training and is widely used with other techniques (Zhou et al., ) including knowledge distillation (Rebuffi et al., ; Buzzega et al., ), model rectification (Wu et al., ; Zhao et al., ), and dynamic networks (Yan et al., ; Wang et al., ; Zhou et al., ). Simple buffer sample selection methods such as random or herding-based approaches (Rebuffi et al., ) are also commonly used as well. There are also more advanced gradient-based sample selection techniques like GSS (Aljundi et al., ) and OCS (Yoon et al., ) that manage buffer data to have samples with diverse and representative gradient vectors. All these works do not consider fairness and simply assume that the entire incoming data is used for model training, which may result in unfair forgetting as we show in our experiments.

Model fairness research mitigates bias by ensuring that a model’s performance is equitable across different sensitive groups, thereby preventing discrimination based on race, gender, age, or other sensitive attributes (Mehrabi et al., ). Existing model fairness techniques can be categorized as pre-processing (Kamiran & Calders, ; Feldman et al., ; Calmon et al., ; Jiang & Nachum, ), in-processing (Agarwal et al., ; Zhang et al., ; Cotter et al., ; Roh et al., ), and post-processing (Hardt et al., ; Pleiss et al., ; Chzhen et al., ). However, most of these techniques assume that the training data is given all at once, which may not be realistic.

A recent line of research addresses model fairness in class-incremental learning where there is a risk of disproportionately forgetting previously-learned sensitive groups including classes, leading to unfairness across different groups. A recent study (He, ) addresses the dual imbalance problem involving both inter-task and intra-task imbalance by reweighting gradients. However, the bias is not only caused by the data imbalance, but also by the inherent or acquired characteristics of the data (Mehrabi et al., ; Angwin et al., ). CLAD (Xu et al., ) first discovers imbalanced forgetting between learned classes caused by conflicts in representation and proposes a class-aware disentanglement technique to improve average accuracy. Among the fairness-aware techniques, FaIRL (Chowdhury & Chaturvedi, ) supports group fairness measures like demographic parity for class-incremental tasks, but proposes a representation learning method that does not directly optimize the given fairness measure and thus has limitations in improving fairness as we show in our experiments. FairCL (Truong et al., ) also addresses fairness in a continual learning setup, but only focuses on resolving the imbalanced class distribution based on the number of pixels of each class in an image for semantic segmentation tasks. In comparison, we support fairness more generally in class-incremental learning by satisfying multiple notions of group fairness for sensitive groups including classes.

## 3 Framework

In this section, we first theoretically analyze unfair forgetting using gradient vectors of sensitive groups and the current task data. Next, we propose sample weighting to mitigate unfairness by adjusting the average gradient vector of the current task data and provide an efficient algorithm. We use the following notations for class-incremental learning and fairness.

#### Notations

In class-incremental learning, a model incrementally learns new current task data along with previous buffer data using data replay. Suppose we train a model to incrementally learn $L$ tasks $\{T_{1},T_{2},\ldots,T_{L}\}$ over time, and there are $N$ new classes in each task as $C^{T_{l}}=\{C_{1}^{T_{l}},C_{2}^{T_{l}},\ldots,C_{N}^{T_{l}}\}$ with no overlapping classes between different tasks (i.e., $C^{T_{l_{1}}}\cap C^{T_{l_{2}}}=\emptyset$ if $l_{1}\neq l_{2}$). After learning the $l^{th}$ task $T_{l}$, we would like the model to remember all $(l-1)\cdot N$ previous task classes and an additional $N$ current task classes. We assume the buffer has a fixed size of $M$ samples. For $L$ tasks, we allocate $m=M/L$ samples of buffer data per task. If each task consists of $N$ classes, then we allocate $m/N=M/(L\cdot N)$ samples of buffer data per class (Chaudhry et al., ; Mirzadeh et al., ; Chaudhry et al., ). Each task $T_{l}=\{d_{i}=(X_{i},y_{i})\}_{i=1}^{k}$ is composed of feature-label pairs where a feature $X_{i}\in\mathbb{R}^{d}$ and a true label $y_{i}\in\mathbb{R}^{c}$. We also use $\mathcal{M}_{l}=\{d_{j}=(X_{j},y_{j})\}_{j=1}^{m}$ to represent the buffer data for each previous $l^{th}$ task $T_{l}$. We assume the buffer data per task is small, i.e., $m\ll k$ (Chaudhry et al., ).

When defining fairness for class-incremental learning, we utilize sensitive groups including classes. According to the fairness literature, sensitive groups are divided by sensitive attributes like gender and race. For example, if the sensitive attribute is gender, the sensitive groups can be Male and Female. The classes of class-incremental learning can also be viewed as sensitive groups where the sensitive attribute is the class. Since we would like to support any sensitive group in a class-incremental setting, we use the following unifying notations: (1) if the sensitive groups are classes, then they form the set $G_{y}=\{(X,\text{y})\in\mathcal{D}:\text{y}=y,y\in\mathbb{Y}\}$ where $\mathcal{D}$ is a dataset, y is a class attribute, and $\mathbb{Y}$ is the set of classes; (2) if we are using sensitive attributes in addition to classes, we can further divide the classes into the set $G_{y,z}=\{(X,\text{y},\text{z})\in\mathcal{D}:\text{y}=y,\text{z}=z,y\in
\mathbb{Y},z\in\mathbb{Z}\}$ where z is a sensitive attribute, and $\mathbb{Z}$ is the set of sensitive attribute values.

### 3.1 Unfair Forgetting

Catastrophic forgetting occurs when a model adapts to a new task and exhibits a drastic decrease in performance on previously-learned tasks (Parisi et al., ). We take inspiration from GEM (Lopez-Paz & Ranzato, ), which theoretically analyzes catastrophic forgetting by utilizing the angle between gradient vectors of data. If the inner products of gradient vectors for previous tasks and the current task are negative (i.e., $90^{\circ}<$ angle $\leq 180^{\circ}$), the loss of previous tasks increases after learning the current task. Catastrophic forgetting thus occurs when the gradient vectors of different tasks point in opposite directions. Intuitively, the opposite gradient vectors update the model parameters in conflicting directions, leading to forgetting while learning.

Using the notion of catastrophic forgetting, we propose theoretical results for unfair forgetting:

###### Lemma 1 .

Denote $G$ as a sensitive group of data composed of features $X$ and true labels $y$. Also, denote $f_{\theta}^{l-1}$ as a previous model and $f_{\theta}$ as the updated model after training on the current task $T_{l}$. Let $\ell$ be any differentiable standard loss function (e.g., cross-entropy loss), and $\eta$ be a learning rate. Then, the loss of the sensitive group of data after training with a current task sample $d_{i}\in T_{l}$ is approximated as follows:

$$ $\tilde{\ell}(f_{\theta},G)=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_ {\theta}^{l-1},G)\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}),$ (1) $$

where $\tilde{\ell}(f_{\theta},G)$ is the approximated average loss between model predictions $f_{\theta}(X)$ and true labels $y$, whereas $\ell(f_{\theta}^{l-1},G)$ is the exact average loss, $\nabla_{\theta}\ell(f_{\theta}^{l-1},G)$ is the average gradient vector for the samples in the group of data $G$, and $\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})$ is the gradient vector for a sample $d_{i}$, each with respect to the previous model $f_{\theta}^{l-1}$.

The proof is in Sec. A.1. We employ first-order Taylor series approximation for the proof, which is widely used in the continual learning literature, by assuming that the loss function is locally linear in small optimization steps and considering the first-order term as the cause of catastrophic forgetting (Lopez-Paz & Ranzato, ; Aljundi et al., ; Lee et al., ). We empirically find that the approximation error is large when a new task begins because new samples with unseen classes are introduced. However, the error gradually becomes quite small as the number of epochs increases while training a model for the task, as shown in Sec. B.2.

To define fairness in class-incremental learning with the approximated loss, we adopt the definition of approximate fairness that considers a model to be fair if it has approximately the same error on the positive class, independent of the group membership (Donini et al., ). In addition, we use a differentiable loss function instead of the widely used 0-1 loss to implement fairness measures because the approximated loss can be derived using gradients. In practice, we can utilize cross-entropy loss as it is known to effectively approximate group fairness metrics (Shen et al., ; Roh et al., ; ). Using Lemma 1, the following theorem suggests a sufficient condition for unfair forgetting. Intuitively, if a training sample’s gradient is in an opposite direction to the average gradient of an underperforming group, but not for an overperforming group, the training causes more unfairness.

###### Theorem 1 .

Denote $G_{1}$ and $G_{2}$ as the overperforming and underperforming groups of data that satisfy the following conditions: $\ell(f_{\theta}^{l-1},G_{1})<\ell(f_{\theta}^{l-1},G_{2})$ while $\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{1})\cdot\nabla_{\theta}\ell(f_{\theta}
^{l-1},d_{i})>0$ and $\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{2})\cdot\nabla_{\theta}\ell(f_{\theta}
^{l-1},d_{i})<0$. Then $|\tilde{\ell}(f_{\theta},G_{1})-\tilde{\ell}(f_{\theta},G_{2})|>|\ell(f_{
\theta}^{l-1},G_{1})-\ell(f_{\theta}^{l-1},G_{2})|$.

The proof is in Sec. A.1. The result shows that the disparity of loss between the two groups could become larger after training on the current task sample, which leads to worse fairness. This theorem can be extended to when we have a set of current task samples $T_{l}=\{d_{i}=(X_{i},y_{i})\}_{i=1}^{k}$ where we can replace $\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})$ with $\frac{1}{|T_{l}|}\sum_{d_{i}\in T_{l}}\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{
i})$.
If the average gradient vector of the current task data satisfies the derived sufficient condition, training with all of the current task samples using equal weights could thus result in unfair catastrophic forgetting.

###### Lemma 1 .

###### Theorem 1 .

### 3.2 Sample Weighting for Unfairness Mitigation

To mitigate unfairness, we propose sample weighting as a way to suppress samples that negatively impact fairness and promote samples that help. Finding the weights is not trivial as there can be many sensitive groups, and even a single sample may improve the fairness of a pair of groups, but worsen the fairness for another pair of groups. Given training weights $\mathbf{w}_{l}$ for the samples in the current task data, the approximated loss of a group $G$ after training is now $\tilde{\ell}(f_{\theta},G)=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_
{\theta}^{l-1},G)\cdot\mathbf{w}_{l}\nabla_{\theta}\ell(f_{\theta}^{l-1},T_{l})$.

We then formulate an optimization problem to find the weights such that both loss and unfairness are minimized. Here we define $\mathbb{Y}$ as the set of all classes and $\mathbb{Y}_{c}$ as the set of classes in the current task. We represent accuracy as the average loss over the current task data and minimize the cost function $L_{acc}=\tilde{\ell}(f_{\theta},G_{\mathbb{Y}_{c}})=\frac{1}{|\mathbb{Y}_{c}|}
\sum_{y\in\mathbb{Y}_{c}}\tilde{\ell}(f_{\theta},G_{y})=\frac{1}{|\mathbb{Y}_{
c}||\mathbb{Z}|}\sum_{y\in\mathbb{Y}_{c},z\in\mathbb{Z}}\tilde{\ell}(f_{\theta
},G_{y,z})$. For fairness, the cost function $L_{fair}$ depends on the group fairness measure as we explain below. We then minimize $L_{fair}+\lambda L_{acc}$ where $\lambda$ is a hyperparameter that balances fairness and accuracy.

#### Equal Error Rate (EER)

This measure (Venkatasubramanian, ) is defined as $\Pr(\hat{\text{y}}\neq y_{1}|\text{y}=y_{1})=\Pr(\hat{\text{y}}\neq y_{2}|
\text{y}=y_{2})$ for $y_{1},y_{2}\in\mathbb{Y}$, where $\hat{\text{y}}$ is the predicted class and y is the true class. We define the cost function for EER as the average absolute difference between the loss of a class and the average loss of all classes: $L_{EER}=\frac{1}{|\mathbb{Y}|}\sum_{y\in\mathbb{Y}}|\tilde{\ell}(f_{\theta},G_
{y})-\tilde{\ell}(f_{\theta},G_{\mathbb{Y}})|$. The entire optimization problem is:

$$ $\displaystyle\min_{\mathbf{w}_{l}}\frac{1}{|\mathbb{Y}|}\sum_{y\in\mathbb{Y}}| \tilde{\ell}(f_{\theta},G_{y})-\tilde{\ell}(f_{\theta},G_{\mathbb{Y}})|+ \lambda\frac{1}{|\mathbb{Y}_{c}|}\sum_{y\in\mathbb{Y}_{c}}\tilde{\ell}(f_{ \theta},G_{y}),$ (2) $\displaystyle\text{where}\ \ \tilde{\ell}(f_{\theta},G_{y})=\ell(f_{\theta}^{l -1},G_{y})-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{y})\cdot\mathbf{w}_{l} \nabla_{\theta}\ell(f_{\theta}^{l-1},T_{l}).$ $$

#### Equalized Odds (EO)

This measure (Hardt et al., ) is satisfied when sensitive groups have the same accuracy, i.e., $\ell(f_{\theta},G_{y,z_{1}})=\ell(f_{\theta},G_{y,z_{2}})$ for $y\in\mathbb{Y}$ and $z_{1},z_{2}\in\mathbb{Z}$. We design the cost function for EO as $L_{EO}=\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in\mathbb{Z}}|
\tilde{\ell}(f_{\theta},G_{y,z})-\tilde{\ell}(f_{\theta},G_{y})|$ to compute the EO disparity, and the entire optimization problem is:

$$ $\displaystyle\min_{\mathbf{w}_{l}}\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in \mathbb{Y},z\in\mathbb{Z}}|\tilde{\ell}(f_{\theta},G_{y,z})-\tilde{\ell}(f_{ \theta},G_{y})|+\lambda\frac{1}{|\mathbb{Y}_{c}||\mathbb{Z}|}\sum_{y\in\mathbb {Y}_{c},z\in\mathbb{Z}}\tilde{\ell}(f_{\theta},G_{y,z}),$ (3) $\displaystyle\text{where}\ \ \tilde{\ell}(f_{\theta},G_{y,z})=\ell(f_{\theta}^ {l-1},G_{y,z})-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{y,z})\cdot\mathbf{w }_{l}\nabla_{\theta}\ell(f_{\theta}^{l-1},T_{l}).$ $$

#### Demographic Parity (DP)

This measure (Feldman et al., ) is satisfied by minimizing the difference in positive prediction rates between sensitive groups. Here, we extend the notion of demographic parity to the multi-class setting (Alabdulmohsin et al., ; Denis et al., ), i.e., $\Pr(\hat{\text{y}}=y|\text{z}=z_{1})=\Pr(\hat{\text{y}}=y|\text{z}=z_{2})$ for $y\in\mathbb{Y}$ and $z_{1},z_{2}\in\mathbb{Z}$. In the binary setting of $\mathbb{Y}=\mathbb{Z}=\{0,1\}$, a sufficient condition for demographic parity is suggested using the loss multiplied by the ratios of sensitive groups (Roh et al., ). By extending the setting to multi-class, we derive a sufficient condition for demographic parity as follows: $\frac{m_{y,z_{1}}}{m_{*,z_{1}}}\ell(f_{\theta},G_{y,z_{1}})=\frac{m_{y,z_{2}}}
{m_{*,z_{2}}}\ell(f_{\theta},G_{y,z_{2}})$ where $m_{y,z}:=|\{i:\text{y}_{i}=y,\text{z}_{i}=z\}|$ and $m_{*,z}:=|\{i:\text{z}_{i}=z\}|$. The proof is in Sec. A.2. Let us define $\ell^{\prime}(f_{\theta},G_{y,z})=\frac{m_{y,z}}{m_{*,z}}\ell(f_{\theta},G_{y,
z})$ and $\ell^{\prime}(f_{\theta},G_{y})=\frac{1}{|\mathbb{Z}|}\sum_{n=1}^{|\mathbb{Z}|
}\frac{m_{y,z_{n}}}{m_{*,z_{n}}}\ell(f_{\theta},G_{y,z_{n}})$. We then define the cost function for DP using the sufficient condition as $L_{DP}=\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in\mathbb{Z}}|
\tilde{\ell}^{\prime}(f_{\theta},G_{y,z})-\tilde{\ell}^{\prime}(f_{\theta},G_{
y})|$. The entire optimization problem is:

$$ $\displaystyle\min_{\mathbf{w}_{l}}\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in \mathbb{Y},z\in\mathbb{Z}}|\tilde{\ell}^{\prime}(f_{\theta},G_{y,z})-\tilde{ \ell}^{\prime}(f_{\theta},G_{y})|+\lambda\frac{1}{|\mathbb{Y}_{c}||\mathbb{Z}| }\sum_{y\in\mathbb{Y}_{c},z\in\mathbb{Z}}\tilde{\ell}(f_{\theta},G_{y,z}),$ (4) $\displaystyle\text{where}\ \ \tilde{\ell}(f_{\theta},G_{y,z})=\ell(f_{\theta}^ {l-1},G_{y,z})-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{y,z})\cdot\mathbf{w }_{l}\nabla_{\theta}\ell(f_{\theta}^{l-1},T_{l}).$ $$

To find the optimal sample weights for the current task data considering both model accuracy and fairness, we first transform the defined optimization problems of Eq. 2, 3, and 4 into the form of linear programming (LP) problems.

###### Theorem 2 .

The fairness-aware optimization problems (Eq. 2, 3, and 4) can be transformed into the form of linear programming (LP) problems.

The loss of each group can be approximated as a linear function, as described in Lemma 1. This implies that the optimization problems, consisting of the loss of each group, can likewise be transformed into LP problems. A comprehensive proof of this assertion can be found in Sec. A.3. We then solve the LP problems using linear optimization solvers (e.g., CPLEX (Cplex, )).

###### Theorem 2 .

### 3.3 Algorithm

We describe the overall process of fair class-incremental learning in Alg. 1. For the recently arrived current task data, we first perform fairness-aware sample weighting (FSW) to assign training weights that can help learn new knowledge of the current task while retaining accurate and fair memories of previous tasks (Step 2). Next, we train the model using the current task data with its corresponding weights and stored buffer data of previous tasks (Step 3), where $\eta$ is a learning rate, and $\tau$ is a hyperparameter to balance between them during training. This procedure is repeated until the model converges (Steps 1–3). Before moving on to the next task, we employ buffer sample selection to store a small data subset for the current task (Steps 4–5). Buffer sample selection can also be done with consideration for fairness, but our experimental observations indicate that selecting representative and diverse samples for the buffer, as previous studies have shown, results in better accuracy and also fairness performance. We thus employ a simple random sampling technique for the buffer sample selection in our framework.

Figure: Algorithm 1 Fair Class-Incremental Learning

Alg. 2 shows the fairness-aware sample weighting (FSW) algorithm for the current task data. We first divide both the previous buffer data and the current task data into groups based on each class and sensitive attribute. Next, we compute the average loss and gradient vectors for each group (Steps 1–2), and individual gradient vectors for the current task data (Step 3). To compute gradient vectors, we use the last layer approximation, which only considers the gradients of the model’s last layer, that is efficient and known to be reasonable (Katharopoulos & Fleuret, ; Ash et al., ; Mirzasoleiman et al., ). We then solve linear programming to find the optimal sample weights for a user-defined target fairness measure such as EER (Step 5), EO (Step 6), and DP (Step 7). We use CPLEX as a linear optimization solver that employs an efficient simplex-based algorithm. Since the gradient norm of the current task data is significantly larger than that of the buffer data, we utilize normalized gradients to update the loss of each group and replace the learning rate parameter $\eta$ with a hyperparameter $\alpha$ in the equations. Finally, we return the weights for the current task samples to be used during training (Step 8). Since the losses and gradients of both the buffer data and the current task data change every epoch, the resulting training weights may also change accordingly.

The computational complexity of FSW is quadratic to the number of current task samples, as CPLEX generally has quadratic complexity with respect to the number of variables when solving LP problems (Bixby, ). However, our empirical results show that for about ten thousand current task samples, the time to solve an LP problem is a few seconds, which leads to a few minutes of overall runtime for MNIST datasets (see Sec. B.3 for details). Since we focus on continual offline training of large batches or separate tasks, rather than online learning, the overhead is manageable enough to deploy updated models in real-world applications. If the task size becomes too large, clustering similar samples and assigning weights to the clusters, rather than samples, could be a solution to reduce the computational overhead.

## 4 Experiments

We implement FSW using Python and PyTorch. All evaluations are performed on separate test sets and repeated with five random seeds. We write the average and standard deviation of performance results and run experiments on Intel Xeon Silver 4114 CPUs and NVIDIA TITAN RTX GPUs.

#### Metrics

We evaluate all methods using accuracy and fairness metrics as in the fair continual learning literature (Chowdhury & Chaturvedi, ; Truong et al., ).

- •
Average Accuracy: We denote $A_{l}=\frac{1}{l}\sum_{t=1}^{l}a_{l,t}$ as the accuracy at the $l^{th}$ task, where $a_{l,t}$ is the accuracy of the $t^{th}$ task after learning the $l^{th}$ task. We measure accuracy for each task and then take the average across all tasks to produce the final average accuracy, denoted as $\overline{A_{l}}=\frac{1}{L}\sum_{l=1}^{L}A_{l}$ where $L$ represents the total number of tasks.
- •
Average Fairness:
We measure fairness for each task and then take the average across all tasks to produce the final average fairness. We use one of three measures for per-task fairness: (1) Equal Error Rate (EER) disparity, which computes the average difference in test error rates among classes: $\frac{1}{|\mathbb{Y}|}\sum_{y\in\mathbb{Y}}|\Pr(\hat{\text{y}}\neq y|\text{y}=
y)-\Pr(\hat{\text{y}}\neq\text{y})|$; (2) Equalized Odds (EO) disparity, which computes the average difference in accuracy among sensitive groups for all classes: $\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in\mathbb{Z}}|\Pr(
\hat{\text{y}}=y|\text{y}=y,\text{z}=z)-\Pr(\hat{\text{y}}=y|\text{y}=y)|$; and (3) Demographic Parity (DP) disparity, which computes the average difference in class prediction ratios among sensitive groups for all classes: $\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in\mathbb{Z}}|\Pr(
\hat{\text{y}}=y|\text{z}=z)-\Pr(\hat{\text{y}}=y)|$. For all measures, low disparity is desirable.

**Table 1: Experimental settings for the five datasets. If the class is used as the sensitive attribute, the number of classes is the same as the number of sensitive groups.**
| Dataset | Size | #Features | #Classes | #Tasks | #Sensitive groups |
| --- | --- | --- | --- | --- | --- |
| MNIST | 60K | 28$\times$28 | 10 | 5 | 10 |
| FMNIST | 60K | 28$\times$28 | 10 | 5 | 10 |
| Biased MNIST | 60K | 3$\times$28$\times$28 | 10 | 5 | 2 |
| DRUG | 1.3K | 12 | 6 | 3 | 2 |
| BiasBios | 253K | 128$\times$768 | 25 | 5 | 2 |

#### Datasets

We use a total of five datasets as shown in Table 1. We first utilize commonly used benchmarks for continual image classification tasks, which include MNIST and Fashion-MNIST (FMNIST). Here we regard the class as the sensitive attribute and evaluate fairness with EER disparity. We also use multi-class fairness benchmark datasets that have sensitive attributes (Xu et al., ; Putzel & Lee, ; Churamani et al., ; Denis et al., ): Biased MNIST, Drug Consumption (DRUG), and BiasBios. We consider background color as the sensitive attribute for Biased MNIST, and gender for DRUG and BiasBios, respectively. We use EO disparity and DP disparity to evaluate fairness on these datasets. We also consider using standard benchmark datasets in the fairness field, but they are unsuitable for class-incremental learning experiments because either there are only two classes, or it is difficult to compute fairness (see Sec. B.4 for more details). For datasets with a total of $\mathcal{C}$ classes, we divide the datasets into $L$ sequences of tasks where each task consists of $\mathcal{C}/L$ classes, and assume that task boundaries are available (van de Ven & Tolias, ).
Table 1 also shows other experimental settings (see Sec. B.4 for more details).

#### Models

Following the experimental setups of Chaudhry et al. (); Mirzadeh et al. (), we use a two-layer MLP with each 256 neurons for the MNIST, FMNIST, Biased MNIST, and DRUG datasets. For BiasBios, we use a pre-trained BERT language model (Devlin et al., ; Xian et al., ). We employ single-head evaluation where a final layer of the model is shared for all the tasks (Farquhar & Gal, ; Chaudhry et al., ). For training, we use an SGD optimizer with momentum 0.9 for all the experiments. We set appropriate learning rates and epochs for each dataset, with detailed experimental settings provided in Sec. B.5.

#### Baselines

In the continual learning literature (Aljundi et al., ; Yoon et al., ), it is natural for all the baselines to be continual learning methods. In particular, we consider FaIRL (Chowdhury & Chaturvedi, ) to be the first fairness paper for continual learning. We thus compare our algorithm with the following baselines categorized into four types:

- •
Naïve methods: Joint Training assumes access to all the data of previous classes for training and thus has an upper-bound performance; Fine Tuning trains a model using only new classes of data without access to previous data and thus has a lower-bound performance.
- •
State-of-the-art methods: iCaRL (Rebuffi et al., ) performs herding-based buffer selection and representation learning using additional knowledge distillation loss; WA (Zhao et al., ) is a model rectification method designed to correct the bias in the last fully-connected layer of the model. WA uses weight aligning techniques to align the norms of the weight vectors over classes; CLAD (Xu et al., ) is a representation learning method that disentangles the representation interference between old and new classes.
- •
Sample selection methods:
GSS (Aljundi et al., ) and OCS (Yoon et al., ) are gradient-based sample selection methods. GSS selects a buffer with diverse gradients of samples; OCS uses gradient-based similarity, diversity, and affinity scores to rank and select samples for both current and buffer data.
- •
Fairness-aware methods: FaIRL (Chowdhury & Chaturvedi, ) performs fair representation learning by controlling the rate-distortion function of representations. FairCL (Truong et al., ) addresses fairness in semantic segmentation tasks arising from the imbalanced class distribution of pixels, but we consider this problem to be unrelated from ours to add the method as a baseline.

#### Hyperparameters

For our buffer storage, we evenly divide the buffer by the sensitive groups including classes. We store 32 samples per sensitive group for all experiments. For the hyperparameters $\alpha$, $\lambda$, and $\tau$ used in our algorithms, we perform cross-validation with a sequential grid search to find their optimal values one by one while freezing the other parameters. See Sec. B.5 for more details.

**Table 2: Accuracy and fairness results on the four datasets with respect to (1) EER disparity, where class is considered the sensitive attribute for the MNIST and FMNIST datasets, and (2) EO disparity, where background color and gender are the sensitive attributes for the Biased MNIST and DRUG datasets, respectively. We compare FSW with four types of baselines: naïve (Joint Training and Fine Tuning), state-of-the-art (iCaRL, WA, and CLAD), sample selection (GSS and OCS), and fairness-aware (FaIRL) methods.**
| Methods | MNIST | FMNIST | Biased MNIST | DRUG |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | EER Disp. | Acc. | EER Disp. | Acc. | EO Disp. | Acc. | EO Disp. |
| Joint Training | .970$\pm$.004 | .014$\pm$.006 | .895$\pm$.010 | .035$\pm$.004 | .945$\pm$.002 | .053$\pm$.002 | .441$\pm$.015 | .179$\pm$.052 |
| Fine Tuning | .453$\pm$.000 | .323$\pm$.000 | .450$\pm$.000 | .324$\pm$.000 | .448$\pm$.001 | .010$\pm$.003 | .357$\pm$.009 | .125$\pm$.034 |
| iCaRL | .934$\pm$.004 | .037$\pm$.003 | .862$\pm$.002 | .053$\pm$.003 | .818$\pm$.011 | .347$\pm$.025 | .458$\pm$.014 | .216$\pm$.056 |
| WA | .911$\pm$.007 | .052$\pm$.006 | .809$\pm$.005 | .088$\pm$.003 | .447$\pm$.001 | .018$\pm$.002 | .358$\pm$.009 | .112$\pm$.038 |
| CLAD | .835$\pm$.015 | .099$\pm$.015 | .775$\pm$.018 | .115$\pm$.019 | .872$\pm$.001 | .195$\pm$.020 | .410$\pm$.026 | .114$\pm$.043 |
| GSS | .886$\pm$.007 | .080$\pm$.009 | .730$\pm$.013 | .150$\pm$.011 | .819$\pm$.009 | .313$\pm$.021 | .433$\pm$.011 | .177$\pm$.045 |
| OCS | .901$\pm$.003 | .061$\pm$.004 | .785$\pm$.012 | .092$\pm$.007 | .833$\pm$.012 | .303$\pm$.024 | .429$\pm$.007 | .169$\pm$.026 |
| FaIRL | .458$\pm$.008 | .306$\pm$.004 | .455$\pm$.005 | .316$\pm$.001 | .759$\pm$.008 | .408$\pm$.018 | .318$\pm$.006 | .015$\pm$.009 |
| FSW | .924$\pm$.003 | .032$\pm$.004 | .825$\pm$.006 | .037$\pm$.007 | .909$\pm$.003 | .060$\pm$.004 | .429$\pm$.020 | .138$\pm$.037 |

### 4.1 Accuracy and Fairness Results

We compare FSW against other baselines on the five datasets with respect to accuracy and corresponding fairness metrics as shown in Table 2 and Tables 5–7 in Sec. B.6. The results for DP disparity and BiasBios dataset are similar and shown in Sec. B.6. We mark the best and second-best results with bold and underline, respectively, excluding the upper-bound results of Joint Training and the lower-bound results of Fine Tuning. For any method, we store a fixed number of samples per task in a buffer, which may not be identical to its original setup, but necessary for a fair comparison. The detailed sequential performance results for each task are shown in Sec. B.6.

Overall, FSW achieves better accuracy-fairness tradeoff results compared to the baselines for all the datasets. For the DRUG dataset, although FSW does not achieve the best performance in either accuracy or fairness, FSW shows the best fairness results among the baselines with similar accuracies (e.g., CLAD, GSS, and OCS) and thus has the best accuracy-fairness tradeoff. We observe that FSW sometimes improves model accuracy while enhancing the performance of underperforming groups for fairness. The state-of-the-art method, iCaRL, generally achieves high accuracy with low EER disparity results. However, since iCaRL uses a nearest-mean-of-exemplars approach for its classification model, the predictions are significantly affected by sensitive attribute values, resulting in high disparities for EO. Although WA also performs well, the method sometimes increases the model weights for the current task classes, which leads to more forgetting of previous tasks and unstable results. The closest work to FSW is CLAD, which disentangles the representations of new classes and a fixed proportion of conflicting old classes to mitigate imbalanced forgetting across classes. However, the proportion of conflicts may vary by task in practice, limiting CLAD’s ability to achieve group fairness. The two sample selection methods GSS and OCS perform worse. While storing diverse and representative samples in the buffer, these methods sometimes result in an imbalance in the number of buffer samples across sensitive groups. The fairness-aware method FaIRL leverages an adversarial debiasing framework combined with a rate-distortion function, but the method loses significant accuracy because training the feature encoder and discriminator together is unstable. In comparison, FSW explicitly utilizes approximated loss and fairness measures to adjust the training weights for the current task samples, which leads to much better model accuracy and fairness.

Figure: (a) Task 2.
Refer to caption: x5.png

### 4.2 Sample Weighting Analysis

We next analyze how our FSW algorithm weights the current task samples at each task using the Biased MNIST dataset results shown in Fig. 2. The results for the other datasets are similar and shown in Sec. B.7. As the acquired sample weights may change with epochs during training, we show the average weight distribution of sensitive groups over all epochs. Since FSW is not applied to the first task, where the model is trained with only the current task data, we present results starting from the second task. Unlike naïve methods that use all the current task data with equal training weights, FSW usually adjusts different training weights between sensitive groups as shown in Fig. 2. For the Biased MNIST dataset, FSW assigns higher weights on average to the underperforming group (Sensitive group 1 in Fig. 2) compared to the overperforming group (Sensitive group 0 in Fig. 2). We also observe that FSW assigns a weight of zero to a considerable number of samples, indicating that relatively less data is used for training. This weighting approach provides an additional advantage in enabling efficient model training while retaining accuracy and fairness.

### 4.3 Ablation Study

To show the effectiveness of FSW on accuracy and fairness performance, we perform an ablation study comparing the performance of using FSW versus using all the current task samples for training with equal weights. Table 3 shows the results for the MNIST, FMNIST, Biased MNIST, and DRUG datasets, while the results for DP disparity and the BiasBios dataset are similar and shown in Sec. B.8. As a result, applying sample weighting to the current task data is necessary to improve fairness while maintaining comparable accuracy.

**Table 3: Ablation study on the MNIST, FMNIST, Biased MNIST, and DRUG datasets with respect to EER and EO disparity when FSW is used or not.**
| Methods | MNIST | FMNIST | Biased MNIST | DRUG |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | EER Disp. | Acc. | EER Disp. | Acc. | EO Disp. | Acc. | EO Disp. |
| W/o FSW | .921$\pm$.004 | .040$\pm$.005 | .836$\pm$.006 | .048$\pm$.005 | .911$\pm$.003 | .063$\pm$.002 | .423$\pm$.013 | .162$\pm$.034 |
| FSW | .924$\pm$.003 | .032$\pm$.004 | .825$\pm$.006 | .037$\pm$.007 | .909$\pm$.003 | .060$\pm$.003 | .429$\pm$.020 | .138$\pm$.037 |

### 4.4 Integrating FSW with a Fair Post-processing Method

In this section, we emphasize the extensibility of FSW by showing how it can be combined with a post-processing method to further improve fairness. We integrate FSW and other existing continual learning methods (iCaRL, CLAD, and OCS) with the state-of-the-art fair post-processing technique in multi-class tasks, $\epsilon$-fair (Denis et al., ), as shown in Table 4 and Table 11 in Sec. B.9. Since $\epsilon$-fair only supports DP, we only show DP results using the Biased MNIST, DRUG, and BiasBios datasets. We mark the best and second-best results with bold and underline, respectively, regardless of the application of post-processing. Overall, combining the fair post-processing technique can further improve fairness without degrading accuracy much. In addition, FSW still shows a better accuracy-fairness tradeoff with the combination of the fair post-processing technique, compared to existing continual learning methods.

**Table 4: Accuracy and fairness results when combining fair post-processing technique ($\epsilon$-fair) with continual learning methods (iCaRL, CLAD, and FSW) with respect to DP disparity.**
| Methods | Biased MNIST | DRUG | BiasBios |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | DP Disp. | Acc. | DP Disp. | Acc. | DP Disp. |
| iCaRL | .818$\pm$.011 | .012$\pm$.001 | .458$\pm$.014 | .098$\pm$.020 | .828$\pm$.002 | .022$\pm$.000 |
| CLAD | .872$\pm$.011 | .013$\pm$.001 | .410$\pm$.026 | .069$\pm$.019 | .785$\pm$.004 | .022$\pm$.001 |
| FSW | .889$\pm$.006 | .007$\pm$.002 | .405$\pm$.013 | .043$\pm$.004 | .797$\pm$.003 | .022$\pm$.000 |
| iCaRL – $\epsilon$-fair | .805$\pm$.014 | .007$\pm$.002 | .460$\pm$.015 | .035$\pm$.013 | .828$\pm$.001 | .016$\pm$.000 |
| CLAD – $\epsilon$-fair | .868$\pm$.015 | .006$\pm$.002 | .411$\pm$.023 | .030$\pm$.010 | .759$\pm$.049 | .017$\pm$.000 |
| FSW – $\epsilon$-fair | .883$\pm$.007 | .005$\pm$.001 | .403$\pm$.010 | .020$\pm$.004 | .796$\pm$.003 | .016$\pm$.000 |

## 5 Conclusion

We proposed FSW, a fairness-aware sample weighting algorithm for fair class-incremental learning. Unlike conventional class-incremental learning, we showed how training with all the current task data using equal weights may result in unfair catastrophic forgetting. We theoretically showed that the average gradient vector of the current task data should not solely be in the opposite direction of the average gradient vector of a sensitive group to avoid unfair forgetting. We then proposed FSW as a solution to adjust the average gradient vector of the current task data such that unfairness is mitigated without harming accuracy much. FSW supports various group fairness measures and is efficient as it solves the optimization by converting it into a linear program. In our experiments, FSW outperformed other baselines in terms of fairness while having comparable accuracy across various datasets with different domains.

#### Ethics Statement

We anticipate our research will have a positive societal impact by improving fairness in continual learning. However, improving fairness may result in a decrease in accuracy, although we try to minimize the tradeoff. In addition, choosing the right fairness measure can be challenging depending on the application and social context.

#### Reproducibility Statement

To ensure the reproducibility of our work, we provide detailed explanations of all the theoretical and experimental results throughout the appendix and supplementary material. For the theoretical results, we include complete proofs of all our theorems in the appendix. For the experimental results, we present a thorough description of the datasets used, as well as the experimental settings of model architectures and hyperparameters, in the appendix. In addition, we submit the source code necessary for reproducing our experimental results as a part of the supplementary material.

## Appendix A Appendix – Theory

### A.1 Theoretical Analysis of Unfairness in Class-Incremental Learning

Continuing from Sec. 3.1, we prove the lemma on the updated loss of a group of data after learning the current task data.

###### Lemma .

Denote $G$ as a sensitive group of data composed of features $X$ and true labels $y$. Also, denote $f_{\theta}^{l-1}$ as a previous model and $f_{\theta}$ as the updated model after training on the current task $T_{l}$. Let $\ell$ be any differentiable standard loss function (e.g., cross-entropy loss), and $\eta$ be a learning rate. Then, the loss of the sensitive group of data after training with a current task sample $d_{i}\in T_{l}$ is approximated as follows:

$$ $\tilde{\ell}(f_{\theta},G)=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_ {\theta}^{l-1},G)\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}),$ $$

where $\tilde{\ell}(f_{\theta},G)$ is the approximated average loss between model predictions $f_{\theta}(X)$ and true labels $y$, whereas $\ell(f_{\theta}^{l-1},G)$ is the exact average loss, $\nabla_{\theta}\ell(f_{\theta}^{l-1},G)$ is the average gradient vector for the samples in the group of data $G$, and $\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})$ is the gradient vector for a sample $d_{i}$, each with respect to the previous model $f_{\theta}^{l-1}$.

###### Proof.

We update the model using gradient descent with the current task sample $d_{i}\in T_{l}$ and learning rate $\eta$ as follows:

$$ $\theta=\theta^{l-1}-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}).$ $$

Using the Taylor series approximation,

$$ $\displaystyle\tilde{\ell}(f_{\theta},G)$ $\displaystyle=\ell(f_{\theta}^{l-1},G)+\nabla_{\theta}\ell(f_{\theta}^{l-1},G) (\theta-\theta^{l-1})$ $\displaystyle=\ell(f_{\theta}^{l-1},G)+\nabla_{\theta}\ell(f_{\theta}^{l-1},G) (-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}))$ $\displaystyle=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1 },G)\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}).$ $$

If we update the model using all the current task data $T_{l}$, the equation is formulated as $\tilde{\ell}(f_{\theta},G)=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_
{\theta}^{l-1},G)\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},T_{l})$. Therefore, if the average gradient vectors of the sensitive group and the current task data have opposite directions, i.e., $\nabla_{\theta}\ell(f_{\theta}^{l-1},G)\cdot\nabla_{\theta}\ell(f_{\theta}^{l-
1},T_{l})<0$, learning the current task data increases the loss of the sensitive group data and finally leads to catastrophic forgetting.
∎

We next derive a sufficient condition for unfair forgetting.

###### Theorem .

Denote $G_{1}$ and $G_{2}$ as the overperforming and underperforming groups of data that satisfy the following conditions: $\ell(f_{\theta}^{l-1},G_{1})<\ell(f_{\theta}^{l-1},G_{2})$ while $\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{1})\cdot\nabla_{\theta}\ell(f_{\theta}
^{l-1},d_{i})>0$ and $\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{2})\cdot\nabla_{\theta}\ell(f_{\theta}
^{l-1},d_{i})<0$. Then $|\tilde{\ell}(f_{\theta},G_{1})-\tilde{\ell}(f_{\theta},G_{2})|>|\ell(f_{
\theta}^{l-1},G_{1})-\ell(f_{\theta}^{l-1},G_{2})|$.

###### Proof.

Using the derived equation in the lemma above $\tilde{\ell}(f_{\theta},G)=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_
{\theta}^{l-1},G)\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})$, we compute the disparity of losses between the two groups $G_{1}$ and $G_{2}$ after the model update as follows:

$$ $\displaystyle|\tilde{\ell}(f_{\theta},G_{1})-\tilde{\ell}(f_{\theta},G_{2})|$ $\displaystyle=|(\ell(f_{\theta}^{l-1},G_{1})-\eta\nabla_{\theta}\ell(f_{\theta }^{l-1},G_{1})\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}))-$ $\displaystyle\quad\,\,(\ell(f_{\theta}^{l-1},G_{2})-\eta\nabla_{\theta}\ell(f_ {\theta}^{l-1},G_{2})\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}))|$ $\displaystyle=|(\ell(f_{\theta}^{l-1},G_{1})-\ell(f_{\theta}^{l-1},G_{2}))-$ $\displaystyle\quad\,\,\eta(\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{1})\cdot \nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})-\nabla_{\theta}\ell(f_{\theta}^{l- 1},G_{2})\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}))|.$ $$

Since $\ell(f_{\theta}^{l-1},G_{1})<\ell(f_{\theta}^{l-1},G_{2})$, it leads to $\ell(f_{\theta}^{l-1},G_{1})-\ell(f_{\theta}^{l-1},G_{2})<0$. Next, the two assumptions of $\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{1})\cdot\nabla_{\theta}\ell(f_{\theta}
^{l-1},d_{i})>0$ and $\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{2})\cdot\nabla_{\theta}\ell(f_{\theta}
^{l-1},d_{i})<0$ make $-\eta(\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{1})\cdot\nabla_{\theta}\ell(f_{
\theta}^{l-1},d_{i})-\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{2})\cdot\nabla_{
\theta}\ell(f_{\theta}^{l-1},d_{i}))<0$. Since the two terms in the absolute value equation are both negative,

$$ $\displaystyle|\tilde{\ell}(f_{\theta},G_{1})-\tilde{\ell}(f_{\theta},G_{2})|$ $\displaystyle=|\ell(f_{\theta}^{l-1},G_{1})-\ell(f_{\theta}^{l-1},G_{2})|+$ $\displaystyle\quad\,\,|-\eta(\nabla_{\theta}\ell(f_{\theta}^{l-1},G_{1})\cdot \nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})-\nabla_{\theta}\ell(f_{\theta}^{l- 1},G_{2})\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i}))|$ $\displaystyle>|\ell(f_{\theta}^{l-1},G_{1})-\ell(f_{\theta}^{l-1},G_{2})|.$ $$

We finally have $|\tilde{\ell}(f_{\theta},G_{1})-\tilde{\ell}(f_{\theta},G_{2})|>|\ell(f_{
\theta}^{l-1},G_{1})-\ell(f_{\theta}^{l-1},G_{2})|$, which implies that fairness deteriorates after training on the current task data.
∎

###### Lemma .

###### Proof.

###### Theorem .

###### Proof.

### A.2 Derivation of a Sufficient Condition for Demographic Parity in the Multi-Class Setting

Continuing from Sec. 3.2, we derive a sufficient condition for satisfying demographic parity in the multi-class setting.

###### Proposition .

In the multi-class setting, $\frac{m_{y,z_{1}}}{m_{*,z_{1}}}\ell(f_{\theta},G_{y,z_{1}})=\frac{m_{y,z_{2}}}
{m_{*,z_{2}}}\ell(f_{\theta},G_{y,z_{2}})$ where $m_{y,z}:=|\{i:\text{y}_{i}=y,\text{z}_{i}=z\}|$ and $m_{*,z}:=|\{i:\text{z}_{i}=z\}|$ for $y\in\mathbb{Y}$ and $z_{1},z_{2}\in\mathbb{Z}$ can serve as a sufficient condition for demographic parity.

###### Proof.

In the multi-class setting, we can extend the definition of demographic parity as $\Pr(\hat{\text{y}}=y|\text{z}=z_{1})=\Pr(\hat{\text{y}}=y|\text{z}=z_{2})$ for $y\in\mathbb{Y}$ and $z_{1},z_{2}\in\mathbb{Z}$. The term $\Pr(\hat{\text{y}}=y|\text{z}=z)$ can be decomposed as follows: $\Pr(\hat{\text{y}}=y|\text{z}=z)=\Pr(\hat{\text{y}}=y,\text{y}=y|\text{z}=z)+
\sum_{y_{n}\neq y}\Pr(\hat{\text{y}}=y,\text{y}=y_{n}|\text{z}=z)$. Without loss of generality, we set $z_{1}=0$ and $z_{2}=1$. Then the definition of demographic parity in the multi-class setting now becomes

$$ $\displaystyle\Pr(\hat{\text{y}}=y,\text{y}=y|\text{z}=0)+\sum_{y_{n}\neq y}\Pr (\hat{\text{y}}=y,\text{y}=y_{n}|\text{z}=0)$ $\displaystyle=\Pr(\hat{\text{y}}=y,\text{y}=y|\text{z}=1)+\sum_{y_{n}\neq y} \Pr(\hat{\text{y}}=y,\text{y}=y_{n}|\text{z}=1).$ $$

The term $\Pr(\hat{\text{y}}=y,\text{y}=y|\text{z}=0)$ can be represented with the 0-1 loss as follows:

$$ $\displaystyle\Pr(\hat{\text{y}}=y,\text{y}=y|\text{z}=0)$ $\displaystyle=\frac{\Pr(\hat{\text{y}}=y,\text{y}=y,\text{z}=0)}{\Pr(\text{z}= 0)}$ $\displaystyle=\frac{\Pr(\hat{\text{y}}=y|\text{y}=y,\text{z}=0)\Pr(\text{y}=y, \text{z}=0)}{\Pr(\text{z}=0)}$ $\displaystyle=\frac{1}{m_{*,0}}\sum_{i:y_{i}=y,z_{i}=0}(1-\mathbbm{1}(y_{i} \neq\hat{y}_{i}))$ $$

Similarly, $\Pr(\hat{\text{y}}=y,\text{y}=y_{n}|\text{z}=0)$ for $y_{n}\neq y$ can be transformed as follows:

$$ $\displaystyle\Pr(\hat{\text{y}}=y,\text{y}=y_{n}|\text{z}=0)$ $\displaystyle=\frac{\Pr(\hat{\text{y}}=y,\text{y}=y_{n},\text{z}=0)}{\Pr(\text {z}=0)}$ $\displaystyle=\frac{\Pr(\hat{\text{y}}=y|\text{y}=y_{n},\text{z}=0)\Pr(\text{y }=y_{n},\text{z}=0)}{\Pr(\text{z}=0)}$ $\displaystyle=\frac{1}{m_{*,0}}\sum_{j:y_{j}=y_{n},z_{j}=0}\mathbbm{1}(y_{j} \neq\hat{y}_{j})$ $$

By applying the same technique to $\Pr(\hat{\text{y}}=y,\text{y}=y|\text{z}=1)$ and $\Pr(\hat{\text{y}}=y,\text{y}=y_{n}|\text{z}=1)$, we have the 0-1 loss-based definition of demographic parity:

$$ $\displaystyle\frac{1}{m_{*,0}}\sum_{i:y_{i}=y,z_{i}=0}(1-\mathbbm{1}(y_{i}\neq \hat{y}_{i}))+\sum_{i:y_{i}\neq y}\frac{1}{m_{*,0}}\sum_{j:y_{j}=y_{i},z_{j}=0 }\mathbbm{1}(y_{j}\neq\hat{y}_{j})$ $\displaystyle=\frac{1}{m_{*,1}}\sum_{i:y_{i}=y,z_{i}=1}(1-\mathbbm{1}(y_{i} \neq\hat{y}_{i}))+\sum_{i:y_{i}\neq y}\frac{1}{m_{*,1}}\sum_{j:y_{j}=y_{i},z_{ j}=1}\mathbbm{1}(y_{j}\neq\hat{y}_{j}).$ $$

Since the 0-1 loss is not differentiable, it is not suitable to approximate the updated loss using gradients as in Eq. 1. We thus approximate the 0-1 loss to a standard loss function $\ell$ (e.g., cross-entropy loss),

$$ $\displaystyle\frac{1}{m_{*,0}}\sum_{i:y_{i}=y,z_{i}=0}-\ell(f_{\theta},d_{i})+ \sum_{i:y_{i}\neq y}\frac{1}{m_{*,0}}\sum_{j:y_{j}=y_{i},z_{j}=0}\ell(f_{ \theta},d_{j})$ $\displaystyle=\frac{1}{m_{*,1}}\sum_{i:y_{i}=y,z_{i}=1}-\ell(f_{\theta},d_{i}) +\sum_{i:y_{i}\neq y}\frac{1}{m_{*,1}}\sum_{j:y_{j}=y_{i},z_{j}=1}\ell(f_{ \theta},d_{j}),$ $$

where $\ell(f_{\theta},d_{j})$ is the loss between the model prediction $f_{\theta}(d_{j})$ and the true label $y_{j}$. By replacing $\sum_{i:y_{i}=y,z_{i}=z}\ell(f_{\theta},d_{i})=m_{y,z}\ell(f_{\theta},G_{y,z})$,

$$ $\frac{m_{y,0}}{m_{*,0}}(-\ell(f_{\theta},G_{y,0}))+\sum_{i:y_{i}\neq y}\frac{m _{y_{i},0}}{m_{*,0}}\ell(f_{\theta},G_{y_{i},0})=\frac{m_{y,1}}{m_{*,1}}(-\ell (f_{\theta},G_{y,1}))+\sum_{i:y_{i}\neq y}\frac{m_{y_{i},1}}{m_{*,1}}\ell(f_{ \theta},G_{y_{i},1}).$ $$

To satisfy the constraint for all $y\in\mathbb{Y}$, the corresponding terms on the left-hand side and the right-hand side of the equation should be equal, i.e., $\frac{m_{y,0}}{m_{*,0}}\ell(f_{\theta},G_{y,0})=\frac{m_{y,1}}{m_{*,1}}\ell(f_
{\theta},G_{y,1})$. In general, we derive a sufficient condition for demographic parity as $\frac{m_{y,z_{1}}}{m_{*,z_{1}}}\ell(f_{\theta},G_{y,z_{1}})=\frac{m_{y,z_{2}}}
{m_{*,z_{2}}}\ell(f_{\theta},G_{y,z_{2}})$.
∎

###### Proposition .

###### Proof.

### A.3 LP Formulation of our Fairness-aware Optimization Problems

Continuing from Sec. 3.2, we prove that minimizing the sum of absolute values with linear terms can be transformed into a linear programming form.

###### Lemma .

The following optimization problem can be reformulated into a linear programming form. Note that in the following equation, y and z refer to arbitrary variables, not to the label or sensitive attribute, respectively.

$$ $\displaystyle\min_{\mathbf{x}}\sum_{i=1}^{n}|y_{i}|+z_{i}$ $\displaystyle\begin{aligned} \text{s.t.}\quad&y_{i}=a_{i}-\mathbf{b}_{i} \mathbf{x},\quad z_{i}=c_{i}-\mathbf{d}_{i}\mathbf{x}\\ &a_{i},c_{i},y_{i},z_{i}\in\mathbb{R},\quad\mathbf{b}_{i},\mathbf{d}_{i}\in \mathbb{R}^{m}\quad\forall i\in\{1,\ldots,n\}\\ &\mathbf{x}\in[0,1]^{m}\end{aligned}$ $$

###### Proof.

The transformation for minimizing the sum of absolute values was introduced in Joksch (); McCarl & Spreen (); Asghari et al. (). Note that considering the additional affine term does not affect the flow of the proof. We first substitute $y_{i}$ for $y_{i}^{+}-y_{i}^{-}$ where both $y_{i}^{+}$ and $y_{i}^{-}$ are nonnegative. Then, the optimization problem becomes

$$ $\displaystyle\min_{\mathbf{x}}\sum_{i=1}^{n}|y_{i}^{+}-y_{i}^{-}|+z_{i}$ $\displaystyle\begin{aligned} \text{s.t.}\quad&y_{i}^{+}-y_{i}^{-}=a_{i}- \mathbf{b}_{i}\mathbf{x},\quad z_{i}=c_{i}-\mathbf{d}_{i}\mathbf{x},\quad y_{i }^{+}-y_{i}^{-}=y_{i}\\ &y_{i}^{+},y_{i}^{-}\in\mathbb{R}^{+},\quad a_{i},c_{i},y_{i},z_{i}\in\mathbb{ R},\quad\mathbf{b}_{i},\mathbf{d}_{i}\in\mathbb{R}^{m}\quad\forall i\in\{1, \ldots,n\}\\ &\mathbf{x}\in[0,1]^{m}.\end{aligned}$ $$

This problem is still nonlinear. However, the absolute value terms can be simplified when either $y_{i}^{+}$ or $y_{i}^{-}$ equals to zero (i.e., $y_{i}^{+}y_{i}^{-}=0$), as the consequent absolute value reduces to zero plus the other term. Then, the absolute value term can be written as the sum of two variables,

$$ $\displaystyle|y_{i}^{+}-y_{i}^{-}|=|y_{i}^{+}|+|y_{i}^{-}|=y_{i}^{+}+y_{i}^{-} \quad\text{if}\quad y_{i}^{+}y_{i}^{-}=0.$ $$

By using the assumption, the formulation becomes

$$ $\displaystyle\min_{\mathbf{x}}\sum_{i=1}^{n}y_{i}^{+}+y_{i}^{-}+z_{i}$ $\displaystyle\begin{aligned} \text{s.t.}\quad&y_{i}^{+}-y_{i}^{-}=a_{i}- \mathbf{b}_{i}\mathbf{x},\quad z_{i}=c_{i}-\mathbf{d}_{i}\mathbf{x},\quad y_{i }^{+}-y_{i}^{-}=y_{i},\quad\underline{y_{i}^{+}y_{i}^{-}=0}\\ &y_{i}^{+},y_{i}^{-}\in\mathbb{R}^{+},\quad a_{i},c_{i},y_{i},z_{i}\in\mathbb{ R},\quad\mathbf{b}_{i},\mathbf{d}_{i}\in\mathbb{R}^{m}\quad\forall i\in\{1, \ldots,n\}\\ &\mathbf{x}\in[0,1]^{m}\end{aligned}$ $$

with the underlined condition added. However, this condition can be dropped. Assume there exist $y_{i}^{+}$ and $y_{i}^{-}$, which do not satisfy $y_{i}^{+}y_{i}^{-}=0$. When $y_{i}^{+}\geq y_{i}^{-}>0$, there exists a better solution ($y_{i}^{+}-y_{i}^{-}$, 0 0) instead of ($y_{i}^{+}$, $y_{i}^{-}$), which satisfies all the conditions, but has a smaller objective function value $y_{i}^{+}-y_{i}^{-}+0+z_{i}<y_{i}^{+}+y_{i}^{-}+z_{i}$. For the case of $y_{i}^{-}>y_{i}^{+}>0$, a solution (0 0, $y_{i}^{-}-y_{i}^{+}$) works as the same manner. Thus, the minimization automatically leads to $y_{i}^{+}y_{i}^{-}=0$, and the underlined nonlinear constraint becomes unnecessary. Consequently, the final formulation becomes the linear problem as follows:

$$ $\displaystyle\min_{\mathbf{x}}\sum_{i=1}^{n}y_{i}^{+}+y_{i}^{-}+z_{i}$ $\displaystyle\begin{aligned} \text{s.t.}\quad&y_{i}^{+}-y_{i}^{-}=a_{i}- \mathbf{b}_{i}\mathbf{x},\quad z_{i}=c_{i}-\mathbf{d}_{i}\mathbf{x},\quad y_{i }^{+}-y_{i}^{-}=y_{i}\\ &y_{i}^{+},y_{i}^{-}\in\mathbb{R}^{+},\quad a_{i},c_{i},y_{i},z_{i}\in\mathbb{ R},\quad\mathbf{b}_{i},\mathbf{d}_{i}\in\mathbb{R}^{m}\quad\forall i\in\{1, \ldots,n\}\\ &\mathbf{x}\in[0,1]^{m}.\end{aligned}$ $$

∎

By applying this lemma, we next prove the transformation of the defined fairness-aware optimization problems in Eq. 2, 3, and 4 to the form of linear programming.

###### Theorem .

The fairness-aware optimization problems (Eq. 2, 3, and 4) can be transformed into the form of linear programming (LP) problems.

###### Proof.

For every update of the model, the corresponding loss of each group can be approximated linearly in the same way as in Sec. A.1: $\tilde{\ell}(f_{\theta},G)=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_
{\theta}^{l-1},G)\cdot\nabla_{\theta}\ell(f_{\theta}^{l-1},T_{l})$. With a technique of sample weighting for the current task data, $\nabla_{\theta}\ell(f_{\theta}^{l-1},T_{l})$ can be changed as $\sum_{d_{i}\in T_{l}}w_{i}\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})$ where $w_{i}$ represents a weight for the current task sample $d_{i}$. Thus, $\tilde{\ell}(f_{\theta},G)$ can be rewritten as follows:

$$ $\displaystyle\tilde{\ell}(f_{\theta},G)$ $\displaystyle=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1 },G)\cdot\sum_{d_{i}\in T_{l}}w_{i}\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})$ $\displaystyle=\ell(f_{\theta}^{l-1},G)-\eta\nabla_{\theta}\ell(f_{\theta}^{l-1 },G)\cdot\begin{bmatrix}&\cdots&\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})^{ \top}&\cdots&\\ \end{bmatrix}\cdot\begin{bmatrix}\\ \vdots\\ w_{i}\\ \vdots\\ \\ \end{bmatrix}$ $\displaystyle=a_{G}-\mathbf{b}_{G}\mathbf{w},$ $$

where $a_{G}:=\ell(f_{\theta}^{l-1},G)$ and $\mathbf{b}_{G}:=\eta\nabla_{\theta}\ell(f_{\theta}^{l-1},G)\cdot\begin{bmatrix
}&\cdots&\nabla_{\theta}\ell(f_{\theta}^{l-1},d_{i})^{\top}&\cdots&\\
\end{bmatrix}$ are constants, and $\mathbf{w}:=\begin{bmatrix}\\
\vdots\\
w_{i}\\
\vdots\\
\\
\end{bmatrix}$ is a variable where $w_{i}\in[0,1]$.

###### Case 1 .

If target fairness measure is EER ($L_{fair}=L_{EER}$),

$$ $\displaystyle L_{EER}+\lambda L_{acc}$ $\displaystyle=\frac{1}{|\mathbb{Y}|}\sum_{y\in\mathbb{Y}}|\tilde{\ell}(f_{ \theta},G_{y})-\tilde{\ell}(f_{\theta},G_{\mathbb{Y}})|+\lambda\frac{1}{| \mathbb{Y}_{c}|}\sum_{y\in\mathbb{Y}_{c}}\tilde{\ell}(f_{\theta},G_{y})$ $\displaystyle=\frac{1}{|\mathbb{Y}|}\sum_{y\in\mathbb{Y}}|(a_{G_{y}}-a_{G_{ \mathbb{Y}}})-(\mathbf{b}_{G_{y}}-\mathbf{b}_{G_{\mathbb{Y}}})\mathbf{w}|+ \lambda\frac{1}{|\mathbb{Y}_{c}|}\sum_{y\in\mathbb{Y}_{c}}(a_{G_{y}}-\mathbf{b }_{G_{y}}\mathbf{w}).$ $$

###### Case 2 .

If target fairness measure is EO ($L_{fair}=L_{EO}$),

$$ $\displaystyle L_{EO}+\lambda L_{acc}$ $\displaystyle=\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in \mathbb{Z}}|\tilde{\ell}(f_{\theta},G_{y,z})-\tilde{\ell}(f_{\theta},G_{y})|+ \lambda\frac{1}{|\mathbb{Y}_{c}||\mathbb{Z}|}\sum_{y\in\mathbb{Y}_{c},z\in \mathbb{Z}}\tilde{\ell}(f_{\theta},G_{y,z})$ $\displaystyle=\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in \mathbb{Z}}|(a_{G_{y,z}}-a_{G_{y}})-(\mathbf{b}_{G_{y,z}}-\mathbf{b}_{G_{y}}) \mathbf{w}|+$ $\displaystyle\quad\,\,\lambda\frac{1}{|\mathbb{Y}_{c}||\mathbb{Z}|}\sum_{y\in \mathbb{Y}_{c},z\in\mathbb{Z}}(a_{G_{y,z}}-\mathbf{b}_{G_{y,z}}\mathbf{w}).$ $$

###### Case 3 .

If target fairness measure is DP ($L_{fair}=L_{DP}$),

$$ $\displaystyle L_{DP}+\lambda L_{acc}$ $\displaystyle=\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in \mathbb{Z}}|\tilde{\ell}^{\prime}(f_{\theta},G_{y,z})-\tilde{\ell}^{\prime}(f_ {\theta},G_{y})|+\lambda\frac{1}{|\mathbb{Y}_{c}||\mathbb{Z}|}\sum_{y\in \mathbb{Y}_{c},z\in\mathbb{Z}}\tilde{\ell}(f_{\theta},G_{y,z})$ $\displaystyle=\frac{1}{|\mathbb{Y}||\mathbb{Z}|}\sum_{y\in\mathbb{Y},z\in \mathbb{Z}}|(a^{\prime}_{G_{y,z}}-a^{\prime}_{G_{y}})-(\mathbf{b}^{\prime}_{G_ {y,z}}-\mathbf{b}^{\prime}_{G_{y}})\mathbf{w}|+$ $\displaystyle\quad\,\,\lambda\frac{1}{|\mathbb{Y}_{c}||\mathbb{Z}|}\sum_{y\in \mathbb{Y}_{c},z\in\mathbb{Z}}(a_{G_{y,z}}-\mathbf{b}_{G_{y,z}}\mathbf{w}),$ $$

where $a^{\prime}_{G_{y,z}}:=\frac{m_{y,z}}{m_{*,z}}a_{G_{y,z}}$, $a^{\prime}_{G_{y}}:=\sum\limits_{z\in\mathbb{Z}}\frac{m_{y,z}}{m_{*,z}}a_{G_{y
,z}}$, $\mathbf{b}^{\prime}_{G_{y,z}}:=\frac{m_{y,z}}{m_{*,z}}\mathbf{b}_{G_{y,z}}$, $\mathbf{b}^{\prime}_{G_{y}}:=\sum\limits_{z\in\mathbb{Z}}\frac{m_{y,z}}{m_{*,z
}}\mathbf{b}_{G_{y,z}}$.

Since $a_{G}$ and $\mathbf{b}_{G}$ are composed of constant values, each equation above can be reformulated to a linear programming form by applying the above lemma.
∎

###### Lemma .

###### Proof.

###### Theorem .

###### Proof.

###### Case 1 .

###### Case 2 .

###### Case 3 .

## Appendix B Appendix – Experiments

### B.1 T-SNE Results for Real Datasets

Figure: (a) MNIST.
Refer to caption: x9.png

Continuing from Sec. 1, we provide t-SNE results for real datasets to show that data overlapping between different classes also occurs in real scenarios, similar to the synthetic dataset results depicted in Fig. 1(a). Using t-SNE, we project the high-dimensional data of the MNIST, FMNIST, Biased MNIST, and DRUG datasets into a lower-dimensional 2D space with $x_{1}$ and $x_{2}$, as shown in Fig. 3. Since BiasBios is a text dataset that requires pre-trained embeddings to represent the data, we do not include the t-SNE results for it. In the MNIST dataset, the images with labels of 3 (red), 5 (brown), and 8 (yellow) exhibit similar characteristics and overlap, but belong to different classes. As another example, in the FMNIST dataset, the images of the classes ‘Sandal’ (brown), ‘Sneaker’ (gray), and ‘Ankel boot’ (sky-blue) also have similar characteristics and overlap.

### B.2 Approximation Error of Taylor Series

Continuing from Sec. 3.1, we provide empirical approximation errors between true losses and approximated losses derived from first-order Taylor series on the MNIST and Biased MNIST datasets as shown in Fig. 4. For each task, we train the model for 5 epochs and 15 epochs on the MNIST and Biased MNIST datasets, respectively. The approximation error is large when a new task begins because new samples with unseen classes are introduced. However, the error gradually decreases as the number of epochs increases while training a model for the task.

Figure: (a) MNIST dataset
Refer to caption: x13.png

### B.3 Computational Complexity and Runtime Results of FSW

Figure: Figure 5: Runtime results of solving a single LP problem in FSW using CPLEX for the MNIST.
Refer to caption: x15.png

Continuing from Sec. 3.3, we provide computational complexity and overall runtime results of FSW using the MNIST and Biased MNIST datasets as shown in Fig. 6 and Fig. 6. Our empirical results show that for about ten thousand current-task samples, the time to solve an LP problem is a few seconds for the MNIST dataset as shown in Fig. 6. By applying the log-log regression model to the results in Fig. 6, the computational complexity of solving LP at each epoch is $\mathcal{O}(|T_{l}|^{1.642})$ where $|T_{l}|$ denotes the number of current task samples. We note that this complexity can be quadratic in the worst case. If the task size becomes too large, we believe that clustering similar samples and assigning weights to the clusters, rather than samples, could be a solution to reduce the computational overhead. In Fig. 6, we compute the overall runtime of FSW divided into three components: Gradient Computation, CPLEX Computation, and Model Training.

### B.4 Dataset Descriptions

Continuing from Sec. 4, we provide more details of the two datasets using the class as the sensitive attribute and the three datasets with separate sensitive attributes. We also consider using standard benchmark datasets in the fairness field, but they are unsuitable for class-incremental learning experiments because either there are only two classes (e.g., COMPAS (Angwin et al., ), AdultCensus (Kohavi, ), and Jigsaw (cjadams et al., )), or it is difficult to compute fairness (e.g., for CelebA (Liu et al., ), each person is a class).

- •
MNIST (LeCun et al., ): The MNIST dataset is a standard benchmark for evaluating the performance of machine learning models, especially in image classification tasks. The dataset is a collection of grayscale images of handwritten digits ranging from 0 to 9, each measuring 28 pixels in width and 28 pixels in height. The dataset consists of 60,000 training images and 10,000 test images. We configure a class-incremental learning setup, where a total of 10 classes are evenly distributed across 5 tasks, with 2 classes per task. We assume the class itself is the sensitive attribute.
- •
Fashion-MNIST (FMNIST) (Xiao et al., ): The Fashion-MNIST dataset is a specialized variant of the original MNIST dataset, designed for the classification of various clothing items into 10 distinct classes. The classes include ‘T-shirt/top’, ‘Trouser’, ‘Pullover’, ‘Dress’, ‘Coat’, ‘Sandal’, ‘Shirt’, ‘Sneaker’, ‘Bag’, and ‘Ankle boot’. The dataset consists of grayscale images with dimensions of 28 pixels by 28 pixels including 60,000 training images and 10,000 test images. We configure a class-incremental learning setup, where a total of 10 classes are evenly distributed across 5 tasks, with 2 classes per task. We assume the class itself is the sensitive attribute.
- •
Biased MNIST (Bahng et al., ): The Biased MNIST dataset is a modified version of the MNIST dataset that introduces bias by incorporating background colors highly correlated with the digits. We select 10 distinct background colors and assign one to each digit from 0 to 9. For the training images, each digit is assigned the selected background color with a probability of 0.95, or one of the other colors at random with a probability of 0.05. For the test images, the background color of each digit is assigned from the selected color or other random colors with equal probability of 0.5. The dataset consists of 60,000 training images and 10,000 test images. We configure a class-incremental learning setup, where a total of 10 classes are evenly distributed across 5 tasks, with 2 classes per task. We set the background color as the sensitive attribute and consider two sensitive groups: the origin color and other random colors for each digit.
- •
Drug Consumption (DRUG) (Fehrman et al., ): The Drug Consumption dataset contains information about the usage of various drugs by individuals and correlates it with different demographic and personality traits. The dataset includes records for 1,885 respondents, each with 12 attributes including NEO-FFI-R, BIS-11, ImpSS, level of education, age, gender, country of residence, and ethnicity. We split the dataset into the ratio of 70/30 for training and testing. All input attributes are originally categorical, but we quantify them as real values for training. Participants were questioned about their use of 18 drugs, and our task is to predict cannabis usage. The label variable contains six classes: ‘Never Used’, ‘Used over a Decade Ago’, ‘Used in Last Decade’, ‘Used in Last Year’, ‘Used in Last Month’, and ‘Used in Last Day’. We configure a class-incremental learning setup, where a total of 6 classes are distributed across 3 tasks, with 2 classes per task. We set gender as the sensitive attribute and consider two sensitive groups: male and female.
- •
BiasBios (De-Arteaga et al., ): The BiasBios dataset is a benchmark designed to explore and evaluate bias in natural language processing models, particularly in the context of profession classification from bios. The dataset consists of short textual biographies collected from online sources, labeled with one of the 28 profession classes, such as ‘professor’, ‘nurse’, or ‘software engineer’. The dataset includes gender annotations, which makes it suitable for studying biases related to gender. The dataset contains approximately 350k biographies where 253k are for training and 97k for testing. We configure a class-incremental learning setup using the 25 most-frequent professions, where a total of 25 classes are distributed across 5 tasks, with 5 classes per task. As the number of samples for each class varies significantly, we arrange the classes in descending order based on their size (Chowdhury & Chaturvedi, ). We set gender as the sensitive attribute and consider two sensitive groups: male and female.

### B.5 More Details on Experimental Settings

Continuing from Sec. 4, we provide more details on experimental settings. We use a batch size of 64 for all the experiments. We set the initial learning rate and the total epochs for each dataset. For the MNIST, FMNIST, and DRUG datasets, we train both our model and baselines with initial learning rates of [0.001, 0.01, 0.1], for 5, 5, and 25 epochs, respectively. For Biased MNIST, we use learning rates of [0.001, 0.01, 0.1] for 15 epochs. For the BiasBios dataset, we use learning rates of [0.00002, 0.0001, 0.001] for 10 epochs and set the maximum token length to 128. For hyperparameters, we perform cross-validation with a grid search for $\alpha\in\{0.0005,0.001,0.002,0.01\}$, $\lambda\in\{0.1,0.5,1\}$, and $\tau\in\{1,2,5,10\}$. To solve the fairness-aware optimization problems and find optimal sample weights, we use CPLEX, a high-performance optimization solver developed by IBM that specializes in solving linear programming (LP) problems.

### B.6 More Results on Accuracy and Fairness

Continuing from Sec. 4.1, we compare FSW with other baselines with respect to EER, EO, and DP disparity as shown in Tables 5, 6, and 7, respectively, on the next page. In addition, we present the sequential performance results for each task as shown in Fig. 7–Fig. 14, after the next page. Due to the excessive time required to run OCS on BiasBios, we are not able to measure the results.

**Table 5: Accuracy and fairness results on the MNIST and FMNIST datasets with respect to EER disparity, where the class is the sensitive attribute. We compare FSW with four types of baselines: naïve (Joint Training and Fine Tuning), state-of-the-art (iCaRL, WA, and CLAD), sample selection (GSS and OCS), and fairness-aware (FaIRL) methods. We mark the best and second best results with bold and underline, respectively.**
| Methods | MNIST | FMNIST |  |  |
| --- | --- | --- | --- | --- |
|  | Acc. | EER Disp. | Acc. | EER Disp. |
| Joint Training | .970$\pm$.004 | .014$\pm$.006 | .895$\pm$.010 | .035$\pm$.004 |
| Fine Tuning | .453$\pm$.000 | .323$\pm$.000 | .450$\pm$.000 | .324$\pm$.000 |
| iCaRL | .934$\pm$.004 | .037$\pm$.003 | .862$\pm$.002 | .053$\pm$.003 |
| WA | .911$\pm$.007 | .052$\pm$.006 | .809$\pm$.005 | .088$\pm$.003 |
| CLAD | .835$\pm$.015 | .099$\pm$.015 | .775$\pm$.018 | .115$\pm$.019 |
| GSS | .886$\pm$.007 | .080$\pm$.009 | .730$\pm$.013 | .150$\pm$.011 |
| OCS | .901$\pm$.003 | .061$\pm$.004 | .785$\pm$.012 | .092$\pm$.007 |
| FaIRL | .458$\pm$.008 | .306$\pm$.004 | .455$\pm$.005 | .316$\pm$.001 |
| FSW | .924$\pm$.003 | .032$\pm$.004 | .825$\pm$.006 | .037$\pm$.007 |

**Table 6: Accuracy and fairness results on the Biased MNIST, DRUG, and BiasBios datasets with respect to EO disparity, where background color is the sensitive attribute for Biased MNIST, and gender for DRUG and BiasBios, respectively. Due to the excessive time required to run OCS on BiasBios, we are not able to measure the results and mark them as ‘–’. The other settings are same as in Table 5.**
| Methods | Biased MNIST | DRUG | BiasBios |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | EO Disp. | Acc. | EO Disp. | Acc. | EO Disp. |
| Joint Training | .945$\pm$.002 | .053$\pm$.002 | .441$\pm$.015 | .179$\pm$.052 | .823$\pm$.003 | .075$\pm$.001 |
| Fine Tuning | .448$\pm$.001 | .010$\pm$.003 | .357$\pm$.009 | .125$\pm$.034 | .425$\pm$.006 | .029$\pm$.002 |
| iCaRL | .818$\pm$.011 | .347$\pm$.025 | .458$\pm$.014 | .216$\pm$.056 | .828$\pm$.002 | .083$\pm$.003 |
| WA | .447$\pm$.001 | .018$\pm$.002 | .358$\pm$.009 | .112$\pm$.038 | .732$\pm$.008 | .069$\pm$.002 |
| CLAD | .872$\pm$.011 | .195$\pm$.020 | .410$\pm$.026 | .114$\pm$.043 | .785$\pm$.004 | .075$\pm$.001 |
| GSS | .819$\pm$.009 | .313$\pm$.021 | .433$\pm$.011 | .177$\pm$.045 | .774$\pm$.007 | .086$\pm$.005 |
| OCS | .833$\pm$.012 | .303$\pm$.024 | .429$\pm$.007 | .169$\pm$.026 | – | – |
| FaIRL | .759$\pm$.008 | .408$\pm$.018 | .318$\pm$.006 | .015$\pm$.009 | .332$\pm$.009 | .039$\pm$.003 |
| FSW | .909$\pm$.003 | .060$\pm$.004 | .429$\pm$.020 | .138$\pm$.037 | .792$\pm$.005 | .073$\pm$.003 |

**Table 7: Accuracy and fairness results on the Biased MNIST, DRUG, and BiasBios datasets with respect to DP disparity. The other settings are the same as in Table 6.**
| Methods | Biased MNIST | DRUG | BiasBios |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | DP Disp. | Acc. | DP Disp. | Acc. | DP Disp. |
| Joint Training | .945$\pm$.002 | .005$\pm$.001 | .441$\pm$.015 | .091$\pm$.020 | .823$\pm$.003 | .021$\pm$.000 |
| Fine Tuning | .448$\pm$.001 | .016$\pm$.008 | .357$\pm$.009 | .102$\pm$.013 | .425$\pm$.006 | .028$\pm$.001 |
| iCaRL | .818$\pm$.011 | .012$\pm$.001 | .458$\pm$.014 | .098$\pm$.020 | .828$\pm$.002 | .022$\pm$.000 |
| WA | .447$\pm$.001 | .016$\pm$.004 | .358$\pm$.009 | .076$\pm$.019 | .732$\pm$.008 | .022$\pm$.001 |
| CLAD | .872$\pm$.011 | .013$\pm$.001 | .410$\pm$.026 | .069$\pm$.019 | .785$\pm$.004 | .022$\pm$.000 |
| GSS | .819$\pm$.009 | .038$\pm$.005 | .433$\pm$.011 | .083$\pm$.018 | .774$\pm$.007 | .022$\pm$.001 |
| OCS | .816$\pm$.012 | .030$\pm$.003 | .429$\pm$.007 | .079$\pm$.020 | – | – |
| FaIRL | .759$\pm$.008 | .033$\pm$.001 | .318$\pm$.006 | .015$\pm$.007 | .332$\pm$.009 | .026$\pm$.002 |
| FSW | .889$\pm$.006 | .007$\pm$.002 | .405$\pm$.013 | .043$\pm$.004 | .797$\pm$.003 | .022$\pm$.000 |

Figure: (a) Accuracy.
Refer to caption: x17.png

Figure: (a) Accuracy.
Refer to caption: x20.png

Figure: (a) Accuracy.
Refer to caption: x23.png

Figure: (a) Accuracy.
Refer to caption: x26.png

Figure: (a) Accuracy.
Refer to caption: x29.png

Figure: (a) Accuracy.
Refer to caption: x32.png

Figure: (a) Accuracy.
Refer to caption: x35.png

Figure: (a) Accuracy.
Refer to caption: x38.png

### B.7 More Results on Sample Weighting Analysis

Continuing from Sec. 4.2, we show more results from the sample weighting analysis for all sequential tasks of each dataset, as shown in the figures below (Fig. 15–Fig. 22). We compute the number of samples for weights in sensitive groups including classes. For each task, we show the average weight distribution over all epochs, as sample weights may change during each epoch of training. Since FSW is not applied to the first task, where the model is trained with only the current task data, we present the results starting from the second task.

Figure: (a) Task 2.
Refer to caption: x41.png

Figure: (a) Task 2.
Refer to caption: x45.png

Figure: (a) Task 2.
Refer to caption: x49.png

Figure: (a) Task 2.
Refer to caption: x53.png

Figure: (a) Task 2.
Refer to caption: x57.png

Figure: (a) Task 2.
Refer to caption: x59.png

Figure: (a) Task 2.
Refer to caption: x61.png

Figure: (a) Task 2.
Refer to caption: x65.png

### B.8 More Results on Ablation Study

Continuing from Sec. 4.3, we present additional results of the ablation study to demonstrate the contribution of our proposed fairness-aware sample weighting (FSW) to the overall accuracy and fairness performance. The results are shown in Tables 8, 9, and 10.

**Table 8: Accuracy and fairness results on the MNIST and FMNIST datasets with respect to EER disparity when FSW is used or not.**
| Methods | MNIST | FMNIST |  |  |
| --- | --- | --- | --- | --- |
|  | Acc. | EER Disp. | Acc. | EER Disp. |
| W/o FSW | .921$\pm$.004 | .040$\pm$.005 | .836$\pm$.006 | .048$\pm$.005 |
| FSW | .924$\pm$.003 | .032$\pm$.004 | .825$\pm$.006 | .037$\pm$.007 |

**Table 9: Accuracy and fairness results on the Biased MNIST, DRUG, and BiasBios datasets with respect to EO disparity when FSW is used or not.**
| Methods | Biased MNIST | DRUG | BiasBios |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | EO Disp. | Acc. | EO Disp. | Acc. | EO Disp. |
| W/o FSW | .911$\pm$.003 | .063$\pm$.002 | .423$\pm$.013 | .162$\pm$.034 | .790$\pm$.003 | .076$\pm$.001 |
| FSW | .909$\pm$.003 | .060$\pm$.004 | .429$\pm$.020 | .138$\pm$.037 | .792$\pm$.005 | .073$\pm$.003 |

**Table 10: Accuracy and fairness results on the Biased MNIST, DRUG, and BiasBios datasets with respect to DP disparity when FSW is used or not.**
| Methods | Biased MNIST | DRUG | BiasBios |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | DP Disp. | Acc. | DP Disp. | Acc. | DP Disp. |
| W/o FSW | .911$\pm$.003 | .009$\pm$.001 | .423$\pm$.013 | .080$\pm$.015 | .790$\pm$.003 | .022$\pm$.000 |
| FSW | .889$\pm$.006 | .007$\pm$.002 | .405$\pm$.013 | .043$\pm$.004 | .797$\pm$.003 | .022$\pm$.000 |

### B.9 More Results on Integrating FSW with a Fair Post-processing Method

Continuing from Sec. 4.4, we provide additional results on integrating continual learning methods with fair post-processing, including OCS and OCS – $\epsilon$-fair performances as shown in Table 11.

**Table 11: Accuracy and fairness results when combining fair post-processing ($\epsilon$-fair) with continual learning methods (iCaRL, CLAD, OCS, and FSW) with respect to DP disparity. Due to the excessive time required to run OCS on BiasBios, we are not able to measure the results and mark them as ‘–’.**
| Methods | Biased MNIST | DRUG | BiasBios |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
|  | Acc. | DP Disp. | Acc. | DP Disp. | Acc. | DP Disp. |
| iCaRL | .818$\pm$.011 | .012$\pm$.001 | .458$\pm$.014 | .098$\pm$.020 | .828$\pm$.002 | .022$\pm$.000 |
| CLAD | .872$\pm$.011 | .013$\pm$.001 | .410$\pm$.026 | .069$\pm$.019 | .785$\pm$.004 | .022$\pm$.001 |
| OCS | .816$\pm$.012 | .030$\pm$.003 | .429$\pm$.007 | .079$\pm$.020 | – | – |
| FSW | .889$\pm$.006 | .007$\pm$.002 | .405$\pm$.013 | .043$\pm$.004 | .797$\pm$.003 | .022$\pm$.000 |
| iCaRL – $\epsilon$-fair | .805$\pm$.014 | .007$\pm$.002 | .460$\pm$.015 | .035$\pm$.013 | .828$\pm$.001 | .016$\pm$.000 |
| CLAD – $\epsilon$-fair | .868$\pm$.015 | .006$\pm$.002 | .411$\pm$.023 | .030$\pm$.010 | .759$\pm$.049 | .017$\pm$.000 |
| OCS – $\epsilon$-fair | .825$\pm$.016 | .005$\pm$.001 | .431$\pm$.021 | .033$\pm$.007 | – | – |
| FSW – $\epsilon$-fair | .883$\pm$.007 | .005$\pm$.001 | .403$\pm$.010 | .020$\pm$.004 | .796$\pm$.003 | .016$\pm$.000 |

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: You can see the points raised in the weaknesses section. I have also additional questions:
- Could authors explain why they consider the average absolute difference between the loss of a class and the average loss of all classes for group fairness notions? Have you considered using squared differences instead of absolute differences in your fairness measures? If so, what were the tradeoffs in terms of computational complexity and performance? Would this alternative formulation allow for a closed-form solution with a projection step?
- It’s unclear whether FSW is applied to each batch individually or calculated once at the beginning of each epoch, with the same weights used for all batches within that epoch. A clarification on this would be helpful.
- Although buffer size is not the main focus of the paper, it would be insightful to understand its impact on unfair forgetting. Have you considered running an ablation study varying the buffer size to examine its impact on unfair forgetting and the proportion of new data assigned non-zero weights?
soundness: 3
strengths: - The presentation of the paper is quite clear and well organized. The example given in the introduction and the first figure help readers understand the problem setup and the challenge. 
- To my knowledge, the problem formulation is novel and the proposed method is original. It introduces group fairness notions to class-incremental learning where the class itself can even be a sensitive attribute. Based on existing literature on gradient angle analyses for forgetting previous classes, the authors extend this to unfair forgetting by examining the gradient misalignment between the sensitive groups and new data. The authors also show that theoretically how the misaligned gradients could exacerbate the disparity of the loss between underperforming and overperforming groups.
- The proposed sample weighting algorithm is equivalent to linear programming and could be solved in polynomial time.
- The experiments section is comprehensive, the authors consider several different datasets and baselines. The experiments show that FSW has better performance in terms of balancing classification accuracy and fairness.
summary: The paper introduces Fairness-aware Sample Weighting (FSW), a novel approach for addressing unfair catastrophic forgetting in class-incremental learning. In class-incremental learning, the goal is to learn new classes of data without forgetting previously learned ones. The authors extend this problem to fair class-incremental learning where the goal is to satisfy fairness notions among sensitive groups and have a high classification accuracy. This method learns sample weights for current task samples based on a given fairness metric to mitigate the "unfair forgetting" problem, which arises when a model disproportionately forgets specific groups. The authors theoretically show that if the gradient direction of new task samples is misaligned with that of an underperforming group, the loss disparity between underperforming and overperforming groups could increase. To overcome this, FSW adjusts the average gradient vector by assigning optimized sample weights to current task samples, calculated based on the selected fairness metric. The proposed framework supports multiple group fairness notions, equal error rate(EOO), equalized odds(EO), and demographic parity(DP) and it can be extended to other notions. They also show that the proposed sample weighting method is equivalent to a linear program and could be solved efficiently. The authors validate FSW on several datasets and show that it achieves a favorable accuracy-fairness balance compared to existing baselines.
weaknesses: - Although overall the paper is well organized, I think that several parts could have more clarification in terms of mathematical writing. For example, in Equation 1 there are two gradient vectors multiplied with a dot $\nabla_{\theta} \ell( f_{\theta}^{l-1}, G ) \cdot\nabla_{\theta} \ell( f_{\theta}^{l-1}, d_{i} )$, using a transpose could improve the clarity and readability. A similar argument can be made for the weighted gradient term as well $w_l\nabla_{\theta} \ell( f_{\theta}^{l-1}, T_{l})$, it would be helpful to introduce this notation explicitly before using it.
- The authors show that the sample weighting problem is equivalent to linear programming. Prior to this conversion, the feasible set is defined as a hypercube, $[0,1]^d$. If the conversion to a linear program does not alter this feasible set, it implies that the optimal solution for the weight vector would lie at a vertex of the hypercube, meaning all weights will be either 0 or 1. Figures 2 and 15-22 appear to support this claim. Can you clarify whether the optimal weights are indeed binary (0 or 1) in practice? If so, how does this affect the interpretation of FSW as a weighting method versus a selection method? I am not sure that it will affect the normalizing factor during the parameter update step(normalizing with the total number of samples instead of normalizing with the number of samples whose weight is 1). Additionally, if this observation/claim is true, then the visualization of the weights in Figure 2 could be improved. It could show the percentage of the active samples that have weight 1 from each group.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Generalization to Multiple Sensitive Attributes
Will the modification in gradient affect another sensitive attribute while imporving the fairness on one sensitive attribute?
soundness: 3
strengths: 1. Theoretical Foundation: This paper gives good theoretical analysis of how unfair catastrophic forgetting can occur in class-incremental learning, and adjusts the training weights to reduce the forgetting of underperforming groups and achieve fairness.
2. Rigorous Experiments: The experiments are conducted across multiple datasets, which adds to the robustness of the findings. The paper demonstrates that FSW achieves a better accuracy-fairness tradeoff, which is a significant result in the context of fair machine learning.
summary: This paper presents a novel approach to address the challenge of fairness in class-incremental learning, a subfield of continual learning where models must learn new classes over time without forgetting previously learned ones. The core contribution is the Fairness-aware Sample Weighting (FSW) algorithm, which adjusts the training weights of current task samples to mitigate unfair catastrophic forgetting for sensitive groups, including classes. The authors propose a theoretical framework to analyze the conditions under which forgetting occurs due to opposite gradient directions between current and sensitive group data. They formulate optimization problems to find sample weights that minimize loss and unfairness, supporting fairness measures like Equal Error Rate (EER), Equalized Odds (EO), and Demographic Parity (DP).
weaknesses: 1. Generalization to Multiple Sensitive Attributes
Issue: The paper primarily focuses on a single sensitive attribute, and the performance of the FSW algorithm in scenarios with multiple intersecting sensitive attributes is not fully explored (for ecample, race and gender in Adult dataset).
Recommendation: The authors should conduct experiments or theoretical analysis to evaluate how the FSW algorithm performs when multiple sensitive attributes are present, such as Adult and Compas dataset . 
2. Quantify the tradeoff between fairness and accuracy.
Issue: The paper does not quantify the tradeoff between fairness and accuracy. As iCaRL may outperform FSW in accuracy, the authors should quantify this tradeoff, such as the area under the fairness-accuracy curve.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: Please see the weakness.
soundness: 2
strengths: 1. The paper presents a new approach to incremental learning by integrating fairness considerations directly into the learning process through sample weighting. This is a creative solution that addresses a gap in the literature, as most prior work has focused on accuracy alone.

2. The theoretical analysis provided is rigorous and well-founded.
summary: This paper addresses fairness in class-incremental learning, proposing a Fairness-aware Sample Weighting (FSW) algorithm. In class-incremental learning, where models incrementally learn new classes, unfair catastrophic forgetting can lead to bias, especially in sensitive groups. The paper theoretically analyzes the phenomenon, identifying that forgetting occurs when gradients for new and sensitive groups diverge. FSW adjusts sample weights to align gradient directions, thus minimizing forgetting in sensitive groups. Experimental results indicate that FSW provides a better accuracy-fairness tradeoff than existing methods on various datasets.
weaknesses: 1. The convergence analysis is necessary for the proposed method. Due to the sample reweighting, the learning objective is changed, so it is not clear whether the converged point is close to the global optimum or not. Intuitively, adjusting the sample weights will encourage overfitting to a small group of "easily" learned samples. However, this small group of data may not be a good representation of the whole data distribution.

2. Should $m$ change according to different sample weights $w$? In some cases where $w_i=0$, this sample should not be counted as one in calculating $m$.

3. Theorem 1 assumes overperforming groups have a lower loss than underperforming groups. Where is the loss defined wrt, on training data or validation data? There might also be some overfitting issues.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: - Why is the loss referred to as the cost function in line 241?

- The fairness definitions used in the experiments appear to differ from those presented in Section 3.2. Are these definitions consistent? If not, why?

- It is unclear what are the sample weight variables appeared in optimization problems?
soundness: 2
strengths: - This approach aims to address the issue of unfair catastrophic forgetting in class-incremental learning by modifying the average gradient vector for the current data.

- It provides a theoretical analysis of the unfair forgetting phenomenon within class-incremental learning.

- It also formalizes fairness-focused optimization problems and reformulates them as linear programming problems to identify suitable sample weights.
summary: This paper tackles the challenge of unfair catastrophic forgetting in class-incremental learning by refining the training weights for samples from the current task. The core method involves adjusting the average gradient vector of the current task’s data by applying specific weights, aiming to minimize both unfairness and loss through the inner product of the gradient vectors. The proposed algorithm is evaluated against multiple baseline methods across various datasets and fairness metrics.
weaknesses: - The inconsistency between the cross-entropy (CE) loss used and common fairness metrics (e.g., demographic parity, DP): In lines 212–215, the authors use the CE loss to approximate group fairness metrics. A detailed analysis clarifying the relationship between CE loss and these fairness metrics is necessary. Additionally, in Eqs. (2)–(4), the fairness losses are defined as the loss disparities between demographic groups, which requires further explanation. The reviewer recommends the authors could provide empirical evidence demonstrating how well CE loss approximates these fairness metrics in practice, or discuss any theoretical guarantees or limitations of this approximation.


- The theoretical analysis lacks depth and originality and shows an inconsistency with the fairness loss. The theorems provided are applicable only to the cross-entropy (CE) loss case and not to fairness metrics, leading to a disconnect between the theoretical framework and the algorithms. The optimization objective includes both fairness loss and CE loss; however, the theorems do not adequately incorporate the fairness losses. The reviewer suggests that the theorems be extended to explicitly incorporate fairness metrics or that a more rigorous justification be provided for how the CE-based theorems relate to fairness in practice. This would help bridge the gap between the theoretical framework and the proposed algorithms.


- The paper does not mention several recent studies in the related work section, which could also adjust new samples' weightings to mitigate fairness disparity. For example, active learning may be closely related to continual learning, as shown in studies like [1-4].


[1]  Fair Active Learning, SIGKDD 2021.

[2] Fairness Without Harm: An Influence-Guided Active Sampling Approach, arxiv 2024.

[3] Influence selection for active learning, CVPR 2021.

[4] Active finetuning: Exploiting annotation budget in the pretraining-finetuning paradigm, CVPR 2023.

Meta-review: This paper aims to address unfair catastrophic forgetting in class-incremental learning where the model learns new classes over time without forgetting the previous ones. The paper proposes a fairness-aware sample weighting algorithm which adjusts the training weights of current task samples to mitigate the unfair forgetting problem for the sensitive groups. 

Reviewers raised concerns on the convergence analysis of the algorithm when the learning objective shifts, inconsistency in the use of CE loss and fairness metrics, missing discussions/comparisons with the literature, among others.

Decision: Reject