Paper: f-FERM: A  Scalable Framework for  Robust Fair Empirical Risk Minimization (ICLR-2024)

Reviewer comment (Novelty):
This paper is missing a related reference called TERM (Tilted Empirical Risk Minimization) which also studies fairness.

Author response:
This paper was cited and discussed in the original submission. The proposed fairness notion in that paper is **based on equal performance for different sub-groups**, which is different from our fairness notions. In contrast to our work, **their paper focus is not on finding convergent stochastic algorithms or distributionally robust fair learning**.

Reviewer comment (Novelty):
The main formulation (1) could be simplified. The f-divergence between the joint distribution and the product of marginals is called f-mutual information. See https://openreview.net/forum?id=ZD03VUZmRx for example and the references therein.

Author response:
Thank you for bringing up this work, which we cited in our revision. While f-mutual information is proposed in the contrastive learning context in the paper you mentioned, **their first formulation is exactly equal to our regularization term (for the case of demographic parity).** Therefore, though we do not use the same terminology for our regularizer, they are equivalent, and none is simpler than the other. The other developments in the paper are based on assuming the joint feature distribution is proportional to the Gaussian kernel, which we do not rely on. As we explained in our general response, our contribution is the reformulation leading to a stochastic optimization algorithm and its extension to the distributionally robust settings.

Reviewer comment (Theory):
SGDA for minimax optimization is highly inefficient and Theorem 2.2 has a really weak convergence result. How is it related to the experiments?

Author response:
Good question! We agree that SGDA can be improved under some additional assumptions (see our general response, the added discussion in the paper after Theorem 2.2, and Appendix G in the revised manuscript). However, SGDA is widely used for minmax optimization due to its simplicity and ease of tuning. There are **other double-loop and triple-loop algorithms with overall less iteration complexity, as we included in the paper**. However, **the underlying condition for using all these algorithms is access to an unbiased gradient estimator with a bounded variance.** This condition is satisfied in our formulation and will allow users to choose their own favorite algorithm. In our experiments, we found SGDA dependable because of its **single-loop nature** and the **small number of parameters for tuning.**

Reviewer comment (Experiments):
The paper didn't discuss which $f$ (in f-div) is the best choice in detail.

Author response:
This is an important and natural question. As can be seen in our experiments, **there is no single divergence that is always better than others in all settings.** In the case of binary-sensitive attributes and full batch algorithms, their performance is almost the same. In the case of non-binary sensitive attributes, if the batch size is small, we observe that KL divergence works impressively well (we mentioned this in the analysis of Figure 1). We believe in applications, the choice of the $f$-divergence can be viewed as a hyperparameter that can be tuned by cross-validation. We have **clarified this point in our revised manuscript at the beginning of section 4**. An interesting theoretical future research direction could be why and when each of these $f$ divergence measures can outperform the others.

Reviewer comment (Experiments):
All the experiments are conducted on small-scale datasets.

Author response:
We have used all popular benchmark datasets, including Adult, German Credit, and COMPAS datasets. For the DRO section, there are indeed not many available real-world datasets containing natural distribution shifts. The New Adult dataset (which we used in our experiments) is one of the newest and largest datasets available (50 states, with each having its own distribution). We believe our experiments on a wide range of datasets are the reason that some other reviewers found our experiments extensive and comprehensive. Having said that, we would love to include additional experiments in the GitHub repository of our paper if you have any specific dataset suggestions.

Reviewer comment (Novelty):
The proposed unified formulation and its minimax reformulation seem incremental as similar minimax forms have appeared in many existing works of fair machine learning

Author response:
You are correct that there exist other min-max formulations in the literature, such as Renyi Fair Inference (Baharlouei et al., 2020) and Fairness-Aware Learning (https://proceedings.mlr.press/v97/mary19a/mary19a.pdf). However, the algorithms they provided only work for **full-batch** settings.  The reason is that it is not straightforward for their formulation to derive an unbiased estimator of the gradient (as a necessary condition for SGDA and other first-order algorithms) for small batches. In addition, there are two other aspects that completely distinguish our work from existing works. First, our framework covers a **wide range of divergences (and it is not a single divergence)**. This will give the practitioner the option to use the most suitable/favorite divergence given their application. Second, our framework **can handle settings involving distribution shifts**. We believe this step is another major contribution of our work which sets us apart from previous works and should not be overlooked.

Reviewer comment (Novelty):
The convergence analysis is straightforward from the paper by Lin et al. (2020).

Author response:
We agree that our main contribution is not the convergence analysis of the algorithm. Our primary contribution is that **we provide a formulation leading to an unbiased estimator of the gradient with a bounded variance** that can be naturally obtained. Having access to such an estimator enables us to use existing first-order stochastic optimization methods with convergence guarantees to solve the problem. Indeed, **for both the iid setting and the distribution shift setting**, our novel formulation can be solved with existing optimization algorithms. To clarify this point, we added a discussion in our revised paper explaining existing algorithms that can be used in our problem. Please see the paragraph after Theorem 2.2 and the new Appendix G.

Reviewer comment (Novelty):
The only novel theoretical part is the convergence analysis of the SGDA for solving the ERM formulation. The corresponding sample complexity for finding $\epsilon$-statariony solution of (f-FERM) is $O(\epsilon^{-8})$. However, I wondered if the authors use the state-of-the-art optimization algorithm for solving nonconvex-concave min-max games and if the sample complexity is currently the optimal one in the literature. If so, the authors should highlight it in the literature.

Author response:
We agree with you that faster algorithms can be used to solve our problem (under some additional assumptions). As explained in our general response above, the rates can be improved under certain additional assumptions and via using more sophisticated algorithms in the literature. However, we chose the simple SGDA algorithm due to its simple implementation (single loop with much fewer tuning parameters compared to other existing methods). In addition, SGDA may have generalizability benefits as pointed out in the general response. Nevertheless, this is not the major contribution of our work. We clarified these points in our revised manuscript. However, for the sake of completeness and **based on your suggestion, we added a detailed and comprehensive discussion on what rates can be achieved using existing algorithms on page 5, Appendix F, and Appendix G**. Having said that, we do think that most of these algorithms are **less practical than SGDA** due to their heavy hyperparameter tuning needs. This has also been observed in the past for much simpler training tasks (such as regular minimization problems), as SGD is still the workhorse for training convex and nonconvex models, while there are theoretically faster available algorithms out there. 
Regarding the novelty, we agree with your earlier assessment that the formulation of the problem is novel. Our formulation can cover a wide range of divergences and lead to efficient algorithms. Indeed, as explained in our general response, our main contribution to non-robust fair learning is the formulation and the reformulation of it as a **min-max optimization with unbiased (and bounded variance) gradient estimator**. Once we have this step, the rest would mostly rely on existing literature. Similarly, in the robust fair setting, our main contribution is to reformulate the problem in a form that can be efficiently solved. These are non-trivial contributions, in our opinion, and should not be overlooked.

Reviewer comment (Novelty):
For the distributionally robust formulation (8), the authors used $\ell_p$-norm to model the ambiguity set, but I am afraid that this usage may not be a novel choice because it is not a flexible choice as Wasserstein, $f$-divergece, or MMD, to quantify the difference between distributions. I encourage the authors consider the extension to other choices of ambiguity sets.

Author response:
Thank you for your suggestion. This is a theoretically interesting future research direction to pursue. Having said that, we believe this is much less practically relevant in our context. For example, the Wasserstein ball relies on an underlying distance between the random variables. However, the fairness features in our case are **mostly categorical** (e.g., male vs female) and **not easy to define a natural distance on**. More specifically, Wasserstein distance is natural for **continuous variables or ordinal discrete variables** (e.g. movie ratings or age groups). However, it is not natural to consider an order or distance between different races or genders as sensitive attributes. Further, the Fair ERM is studied under the Wasserstein ball in the past for the special case of logistic regression (convex loss) (https://arxiv.org/pdf/2007.09530.pdf). However, even for this simple case, no scalable stochastic algorithm is offered, even in the convex fair ERM. In contrast, our methodology offers scalable algorithms for even nonconvex models. In addition, our extensive experiments show the effectiveness of the proposed method, which is also supported by our theory.
Regarding the novelty of using $L_p$ ball, to the best of our knowledge, our work **is the first to use $L_p$ uncertainty balls for DRO fair empirical risk minimization.** It is shown to be practically effective in the experiments and is backed by a reasonable amount of theory. Hence, we still believe there is novelty in our work.

Reviewer comment (Theory):
When the ambiguity set of formulation (8) is small, the authors propose a first-order approximation formula to solve it (see Eq.(10)). Although the authors present the approximation error between formula (8) and (10), the complexity for solving Eq.(10) is not presented in this paper.

Author response:
Thanks for bringing up this concern.  At the end of the day, we apply the SGDA algorithm to Problem (10). We showed how to compute an unbiased estimator of the gradient with respect to each set of parameters. The problem is nonconvex-concave, and thus, all the discussions we had in our general response apply here. In particular, the algorithm converges in $O(\epsilon^{-8})$ iterations with a very small batch size requirement (even batch size of $O(1)$). However, one can obtain faster algorithms under additional assumptions. For example, if the set for $\theta$ is assumed to be compact (e.g., we restrict the norm of the weight of the gradient), then we can accelerate the algorithm to $O(\epsilon^{-6})$, see https://arxiv.org/pdf/1810.02060.pdf. Moreover, if we consider full batch sizes, we can utilize Algorithm 2 in the cited work (Ostrovskii et al 2021). This will give you the rate of convergence of $O(\epsilon^{-2})$; see  (Theorem 5.2, Ostrovskii et al 2021). We added these discussions to the paper in Appendix H. Thank you for helping us detect this miscommunicated point and improve the clarity of the paper.

Reviewer comment (Theory):
When the ambiguity set of formulation (8) is potentially large, the first-order approximation formula may not achieve good performance. To this end, the authors consider another formulation in Section 3.2. However, I have several concerns of the deviation: Can the authors add more details regarding the sentence "One can easily see that the optimal $p_j=\min(\hat{p}_j+\delta, 1)$ and $q_j=\max(\hat{q}_j-\delta,0)$"? It is not obvious for readers to check this point.

Author response:
Great point! Since the problem is a constrained convex maximization problem (not minimization), the solution will be on the boundary. Since $f$ is a non-decreasing function for the choice of KL-divergence and chi-square, the maximum value with respect to $P$ occurs when $P$ takes the maximum possible value, which is either $p + \delta$ or $1$. Similarly, we can see that the maximum value with respect to $Q$ happens when $Q$ takes the minimum possible value, which is $q - \delta$ or $0$. We added a detailed step-by-step proof of the claim as an Appendix (Appendix I) to the paper to clarify this point.

Reviewer comment (Theory):
When the ambiguity set of formulation (8) is potentially large, the first-order approximation formula may not achieve good performance. To this end, the authors consider another formulation in Section 3.2. However, I have several concerns of the deviation: What is the meaning of the scalar $\delta>0$? It is not introduced. If I remember correctly, in standard f-divergence DRO it is the Lagrangian multiplier corresponding to the probability simplex constraint that needs to be optimized.

Author response:
$\delta$ represents the uncertainty ball radius (it is usually represented by $\epsilon$. Since we already use $\epsilon$ as the optimality approximation, we use $\delta$ to represent the size of the $L_p$ norm uncertainty set. We analyze the DRO problem for small $\delta$ in Section 3.1. Alternatively, when $\delta$ is large (more robustness), we analyze the problem for the special case of $L_{ \infty}$ norm.