Paper: Small Variance, Big Fairness: A Path to Harmless Fairness without Demographics (ICLR-2024)

Reviewer comment (Writing):
I found the motivation for why they want to minimize the variance of loss weak a little weak in the main body. Moving Proposition 2 to the main body and adding more explanation could help.

Author response:
+ Regarding the motivation of minimizing the variance of losses, we suggest readers also refer to the **Idea**​ paragraph in the Introduction section, where variance is also viewed as the second moment of loss distribution. We thought it is straightforward and useful to character accuracy disparities when demographics are unknown. 
+ Note that Proposition 2 is derived by using MAD metric, and it also expects that loss should be (approximately) independent of input, which is used to help understand the problem only. Thus, we left it to the appendix as a supplementary.

Reviewer comment (Theory):
I found Proposition 1 unpersuasive because the squareroot of the variance is a *very* loose upper bound on the maximum accuracy difference loss. At one point in the proof, they blow up one side by a factor of n. I would only find a result like this persuasive if there was a (somewhat) tight upper and lower bound.

Author response:
+ We will add a remark under Proposition 1 to explain that although being a bit loose, $\hat{\sigma}$ essentially takes advantage of global information (via using mean, compared to pairwise loss $\pi$) and is not as loose as the naïve variance $\hat{\sigma}^2$. We will also refer readers to the experimental justification (Table 3 in Appendix) wherein $\hat{\sigma}$ is demonstrated as the most powerful loss for achieving fairness. 
   + We authors have also discussed about other possible losses which could be tighter and more useful as commented by the reviewer. To the best of our knowledge, however we are not able to provide feasible solutions to this concern. We agreed this is important and would make it as a future research direction.

Reviewer comment (Theory):
Is my assessment of Proposition 1 as loose correct?

Author response:
Thank you for your detailed reading.

+ In Appendix B, we presented that Eq. (10) formulated as a pairwise loss which is tighter than variance without blowing up one side by a factor of $n$. However, shown as in Table 3 on Page 13, the pairwise loss has not achieved better results because the global information (e.g., mean loss) is hard to be incorporated for a minibatch update in practical implementation. 
+ Although loose, experimental results in Table 3 demonstrated that variance minimizing is useful to characterize accuracy disparity without demographics. We understand tight bound is a pursuit in most cases, while it is not the only criterion in this scenario.

Reviewer comment (Novelty):
Can the problem be sold in the context of federated learning? It seems like a natural fit there where you're training on a private distribution and you want your model to perform well on other distributions that you don't have access to.

Author response:
We understand that learning on data without demographics is closely related to the data privacy community. However, please note that our problem setting does not introduce any prior about demographics, which indicates the solution should adapt to any demographic distribution. Thus, there is no need to set distributed groups that are based on federated learning. If each local group is exactly a group divided by interested attributes, then the central model at least knows how many local groups there exist (when it executes gradient aggregation), which can be some side information to demographics and thus beyond our problem setting.

Reviewer comment (Evaluation):
Do you have results for the time it took to train each model (sorry if I missed this)? It seems like your training method uses more information which might be an unfair advantage over other methods if it takes significantly longer.

Author response:
We monitored the training time for each method and reported the results in Table 1. All of these experiments were conducted on one Nvidia RTX 3090.

 **Table 1: Average training time of each method on four datasets.**

|             | ERM      | DRO      | ARL      | FairRF    | VFair     |
| ----------- | -------- | -------- | -------- | --------- | --------- |
| UCI  Adult  | 5.469sec | 3.947sec | 8.436sec | 10.686sec | 11.238sec |
| Law  School | 6.978sec | 5.695sec | 6.066sec | 7.255sec  | 7.204sec  |
| COMPAS      | 1.534sec | 1.681sec | 2.069sec | 2.946sec  | 2.838sec  |
| CelebA      | 46.71min | 47.44min | 49.57min | -         | 58.63min  |

The results showed that training time of our method, VFair, is slightly higher than that of ERM and DRO. This is because the derivation of dynamic $\lambda$ needs to **compare** the gradients of the primary and secondary objectives before updating the model, necessitating twice the gradient calculations. ARL is time-intensive due to the introduction of an additional adversary network for computing instance weights. The training time of FairRF is comparable to ours and it uses a candidate descent approach and may not be scalable to large datasets. For a more direct comparison, all methods on CelebA were trained for 20 epochs, and we do not leverage more knowledge but use longer training times.

Reviewer comment (Theory):
How does this paper relate with works in machine learning that argue that a flat loss landscape implies better generalization [1]? Does variance minimization imply a flat minimum in the loss landscape?

Author response:
+ We understand variance regularization on ERM aims at improving the model reliability but may not necessarily imply a flat minimum in the loss landscape [1]. A flat minimum like sharpness-aware minimization [4] expects that average loss should not change by much if a model is perturbed within a small region. However mathematically, variance minimization cannot ensure such a property.

Reviewer comment (Novelty):
The problem of minimizing some model performance constrained to the loss being small enough has been studied in the Rashomon effect literature. The paper may benefit from related work in the field, such as [2]. Moreover, the authors argue that if $\delta$ (Eq. 1) is small enough, then the "fair" model and the ERM are equivalent. A recent paper showed how small $\delta$ needs to be to ensure that the models are equivalent [3].

Author response:
+ Based on the concept of Rashomon [3], research [2] also proposed a method that achieves fairness from good-utility models, which is quite similar to the branch of “Harmless Fairness” in Related Work. Our work differs from theirs by two key differences. (1) We further focused on “Harmless fairness without demographic” and thus paid much attention to how to describe fairness under such a restrictive scenario. (2) Research [2] solves a constrained optimization problem, which means we need a proper upper bound for the average loss. One may obtain it by running an ERM model beforehand and then setting it, which is not needed as we treat it as a bi-objective problem and the constraint is satisfied by adjusting the weight factor $\lambda$.

Reviewer comment (Novelty):
Although this is the first time introducing the idea of reducing the variance of the loss function into the fairness domain, similar concepts have been explored in earlier works [2, 3, 4]. It may be advisable to include a more comprehensive discussion in the related work section.

Author response:
Thank you for suggesting the related studies. As we discussed in Section 5, we agreed that penalizing the variance of losses can be motivated by different applications, and we would add its advantage of robustness [1]. Particularly, in the submitted manuscript, we have pointed out the relation between variance regularization and instance-reweight learning inspired by the work [2], which is not a fairness study though. TERM [3] reformulates the standard ERM which improves the worst-performance fairness and thus would be included as a worst-case fairness method.

Reviewer comment (Reproducibility):
It is advisable to supplement additional experimental details. For example, a more comprehensive elaboration on the configuration of the ERM model for various datasets, particularly those not discussed in the DRO, would be beneficial.

Author response:
Your suggestion regarding additional experimental details is well-received. We apologize for the omission of the model structure and will include the necessary details in the paper.

+ All the models, excluding FairRF, which operates within a distinct problem setting, conform to a shared neural network framework. Specifically, for binary classification tasks, the core neural network architecture consists of an embedding layer followed by two hidden layers, with 64 and 32 neurons, respectively. In the ARL model, an additional adversarial component is integrated, detailed in its respective paper, featuring one hidden layer with 32 neurons. For multi-classification tasks, the primary neural network transforms into resnet18, and the embedding layer transitions to a Conv2d-based frontend. Throughout these experiments, the Adagrad optimizer was employed. FairRF, utilizing its officially published code implementation, maintains the same backbone network with nuanced variations in specific details. 
+ To address the ambiguity in the explanation of the hyperparameter epsilon, we will provide a more detailed clarification. When the angle of the gradient reaches an orthogonal angle, Formula 4 can be simplified as $\lambda \ge \epsilon$, indicating that in the absence of conflicts in gradient updating, to what extent do we still want the primary objective to be updated? **We designate this as a hyperparameter because we have observed variations in the quality of data across different datasets.** For instance, on the COMPAS dataset, known for its numerous noisy samples [4], a larger epsilon is needed to control the model to maintain utility. In this case, the regularization of the second moment of the distribution is constrained, preventing the model from focusing more on outlier samples. For reproducibility, it is recommended to set the value of epsilon as 0, 0.5, 0.5, and 3 for Law School, UCI Adult, CelebA, and COMPAS, respectively.

Reviewer comment (Experiments):
While the paper briefly mentions the setting of the hyperparameter $\epsilon$, it may appear somewhat ambiguous. It is also necessary to clarify the robustness of the hyperparameter $\epsilon$.

Author response:
To address the ambiguity in the explanation of the hyperparameter epsilon, we will provide a more detailed clarification. When the angle of the gradient reaches an orthogonal angle, Formula 4 can be simplified as $\lambda \ge \epsilon$, indicating that in the absence of conflicts in gradient updating, to what extent do we still want the primary objective to be updated? **We designate this as a hyperparameter because we have observed variations in the quality of data across different datasets.** For instance, on the COMPAS dataset, known for its numerous noisy samples [4], a larger epsilon is needed to control the model to maintain utility. In this case, the regularization of the second moment of the distribution is constrained, preventing the model from focusing more on outlier samples. For reproducibility, it is recommended to set the value of epsilon as 0, 0.5, 0.5, and 3 for Law School, UCI Adult, CelebA, and COMPAS, respectively.

Reviewer comment (Writing):
The explanation of the optimization objective in Appendix B.2 seems to contradict the main body of the paper. The $\hat{\sigma}$ in the main body refers to a different object than the $\hat{\sigma}$ mentioned in the appendix.

Author response:
We apologize for the ambiguity of objectives. To be clearer, we used $\pi, \hat{\sigma}, \hat{\sigma}^2$ to denote the 3 objectives derived in B.1 and Table 3: 

+ $\pi=\sum_{i=1}^{N-1}|\ell_i - \ell_{i+1}|$
+  $\hat{\sigma} = \frac{1}{\sqrt{N}}\sqrt{\sum_{i=1}^N(\ell_i - \hat{\mu})^2}$
+ $\hat{\sigma}^2 = \frac{1}{N}\sum_{i=1}^N(\ell_i - \hat{\mu})^2$

Reviewer comment (Writing):
Is the "the sample will be filtered out from updating in this iteration." in the second-to-last paragraph on page five meaning not using this portion of the sample for training?

Author response:
Your interpretation is accurate. This subset of samples will not be utilized for training during the current iteration. Nevertheless, in the subsequent iteration, we will reevaluate the selection of samples

Reviewer comment (Reproducibility):
Code or experimental artifacts are not shared. Properly reviewing the paper would require access to code and empirical results, since the main claims are quite empirical in nature (e.g., lower loss variance equates to higher fairness).

Author response:
2. **Code and Experimental Artifacts**

Following your suggestion, we have included our code in the supplementary materials, which showcases a simple yet effective approach to achieving fairness. We will release the full code once this paper is accepted.

Reviewer comment (Experiments):
It would be important to have a comparison with methods from the constrained optimization literature (standard variance minimization subject to constraint on the empirical risk) or from the standard fairness literature (e.g., with access to sensitive attributes). Of course, it is not expected for VFair to surpass methods that have access to sensitive attributes, but it would be useful to properly contextualize VFair's results.

Author response:
3. **Comparison with Constrained Optimization Approaches**

+ We reminded that utilizing constrained optimization essentially requires a proper $\delta$ (an upper bound of the minimum of loss in Eq. 1&2) which is not accessible unless minimizing the average losses on the entire training set beforehand. 
+ We clarified that the targeted problem is treated as a bi-objective problem in this paper, with two objectives having different priorities. In this sense, the proposed algorithm does not have to handle $\delta$ separately, while “the constraint” is implicitly satisfied. While the suggested [1] and TFCO focus on solving constrained optimization problems. To avoid confusion, we will cite these works and offer a reminder in the main body of the paper. 
+ Using a fixed λ, which is simply set to 1 in Table 2, we linearly combined two objectives, which serve as an ablation study in our experiments. The results showed that the model performance may significantly drop, e.g., on COMPAS. 
+ We apologize for the description of "a fixed multiplier". Our intended meaning is that the advanced Lagrange method aims at the optimal multiplier while it is dynamic along with the optimization process in our method.

Reviewer comment (Experiments):
Was a standard constrained optimization approach to the problem attempted before constructing the proposed algorithm? Although the intuition behind VFair is well explained, it seems that to motivate a new algorithm it would make sense to compare to standard CO algorithms or justify in some other way why they are not used. Why is the standard Lagrangian dual ascent formulation not used? e.g., see [A], and the TensorFlow CO framework.

Author response:
3. **Comparison with Constrained Optimization Approaches**

+ We reminded that utilizing constrained optimization essentially requires a proper $\delta$ (an upper bound of the minimum of loss in Eq. 1&2) which is not accessible unless minimizing the average losses on the entire training set beforehand. 
+ We clarified that the targeted problem is treated as a bi-objective problem in this paper, with two objectives having different priorities. In this sense, the proposed algorithm does not have to handle $\delta$ separately, while “the constraint” is implicitly satisfied. While the suggested [1] and TFCO focus on solving constrained optimization problems. To avoid confusion, we will cite these works and offer a reminder in the main body of the paper. 
+ Using a fixed λ, which is simply set to 1 in Table 2, we linearly combined two objectives, which serve as an ablation study in our experiments. The results showed that the model performance may significantly drop, e.g., on COMPAS. 
+ We apologize for the description of "a fixed multiplier". Our intended meaning is that the advanced Lagrange method aims at the optimal multiplier while it is dynamic along with the optimization process in our method.

Reviewer comment (Theory):
The statement in the text — "Lagrangian function, which however employs a fixed and finite multiplier and thus may not fully satisfy the constraint" — is incorrect. The Lagrangian function does not employ a fixed multiplier; the $\lambda$ would have to be optimized jointly with the $\theta$ until a stable saddle point is reached.

Author response:
3. **Comparison with Constrained Optimization Approaches**

+ We reminded that utilizing constrained optimization essentially requires a proper $\delta$ (an upper bound of the minimum of loss in Eq. 1&2) which is not accessible unless minimizing the average losses on the entire training set beforehand. 
+ We clarified that the targeted problem is treated as a bi-objective problem in this paper, with two objectives having different priorities. In this sense, the proposed algorithm does not have to handle $\delta$ separately, while “the constraint” is implicitly satisfied. While the suggested [1] and TFCO focus on solving constrained optimization problems. To avoid confusion, we will cite these works and offer a reminder in the main body of the paper. 
+ Using a fixed λ, which is simply set to 1 in Table 2, we linearly combined two objectives, which serve as an ablation study in our experiments. The results showed that the model performance may significantly drop, e.g., on COMPAS. 
+ We apologize for the description of "a fixed multiplier". Our intended meaning is that the advanced Lagrange method aims at the optimal multiplier while it is dynamic along with the optimization process in our method.

Reviewer comment (Reproducibility):
What ML algorithm is actually used to fit the data? The paper is written with respect to generic model parameters $\theta$, which is fine, but what actual model was used in the experiments? Was it a neural network?

Author response:
4. **Details on the model structure**

Following your advice, we have included the necessary description in the paper.

All the models, excluding FairRF, which operates within a distinct problem setting, conform to a shared neural network framework. Specifically, for binary classification tasks, the core neural network architecture consists of an embedding layer followed by two hidden layers, with 64 and 32 neurons, respectively. In the ARL model, an additional adversarial component is integrated, detailed in its respective paper, featuring one hidden layer with 32 neurons. For multi-classification tasks, the primary neural network transforms into resnet18, and the embedding layer transitions to a Conv2d-based frontend. Throughout these experiments, the Adagrad optimizer was employed. FairRF, utilizing its officially published code implementation, maintains the same backbone network with nuanced variations in specific details.

Reviewer comment (Reproducibility):
There appears to be no explicit definition of the loss function $\ell$ used throughout the paper. Is it BCE? Multiple plots show the distribution of sample-wise loss for different models; are all algorithms evaluated on the same loss function?

Author response:
5. **Details about Loss Function**

Thank you once again for your detailed reading. Throughout the experiments, we implemented binary cross-entropy (BCE) for binary classification and categorical cross-entropy (CE) for multi-class classification. However, we would like to underscore that our method is general and is compatible with other forms of loss as well. We will incorporate this information into our main paper.

Reviewer comment (Evaluation):
Is there any explanation for why some (greyed out) rows of Table 2 always result in a constant/uniform classifier over the 10 runs? Perhaps plotting $\lambda_1$ and $\lambda_2$ as well as loss as training progresses could shed some light.

Author response:
6. **Grayed Out Rows in Table 2**

Table 2 shows that applying a fixed $\lambda$ or using $\lambda_1$ only might result in uniform classifiers. This is because without adjustment of $\lambda_2$, the model is at risk of providing very uncertain predictions to training samples, that is, most of the data lies near the decision boundary. Fig. 3 observes the same phenomenon by monitoring the distance between samples to the decision boundary, which demonstrates the necessity of $\lambda_2$. Mathematically, $\lambda_2$ lifts up the importance of the average loss from the “loss perspective” even if the gradients of two objectives do not conflict with each other by much. Following your advice, we added the plot of $\lambda$ in Appendix G.