From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: Based on the collected information from all of reviewers and my personal judgment, I can make the recommendation on this paper, **rejection**. In sum, this paper is in good shape. Here are the comments that I summarized, which include my opinion and evidence.

**Research Problem**

The authors argue that the generated data, also known as antidote data, might play a positive role in alleviating the unfairness issue. However, the ratio between the generated data and the real training data is under-explored. In the paper, the authors aim to propose an adaptive sampling approach to find the optimal data sampling ratios among protected groups and between real and generated data. In general, this research problem is interesting to me. Bringing in the extra generated data might break the utility-and-fairness trade-off on a fixed training set. 

**Motivation**

From the introduction part, I am aware of two motivations, (1) one authors’ conjecture that a similar amount of data across groups might not be optimal to improve fairness and (2) the empirical result in Figure 1(b) that demonstrates there exists better solutions than the one with 1:1 ratio. To be frank, these two motivations are not very strong to me. I suggest the authors dig deeper and explore the drawback of the one with a 1:1 ratio. 

**Philosophy**

The authors cast the research problem into an optimization problem to seek the optimal data sampling ratios among protected groups and between real and generated data. 

**Technique**

A bilevel optimization is proposed, where an outer optimization adjusts data sampling ratios considering both fairness and accuracy and an inner optimization minimizes the standard empirical risk on both real and generated data. Although the authors propose an approximate algorithm to avoid the expensive inverse Hessian matrix, the technique is standard to me. 

**Information Usage**

Although the authors only use real data to generate new data, downstream fairness learning involves both real and generated data. It would be nice to disentangle the performance gain from the generated data or the optimal ratio. This is my major concern. 

**Experiments**

In general, the experimental results are good. I have some minor issues.

(1) Hyperparameter. The authors mention $k$ is chosen to have the best fairness score while minimizing the accuracy degradation in the validation set. This is unclear to me. More specifics are needed. Other competitive methods also follow the same procedure to select hyperparameters? 

(2) Several reviewers mention the recent reweigh methods. It is suggested to include them for comparisons. 

(3) Reviewer ZWfo thought the proposed bilevel optimization is too complicated. Instead, the optimal ratio can be chosen in a simple way. For example, based on the performance of the validation set. It is also suggested to add this baseline. 

(4) Dr-Fairness achieves the best performance in Bias Amp. of the *y:age and z:gender* setting. The -0.026 should be bold. No matter positive or negative, the number close to zero, the better. 

(5) The legend is missing in Figure 3(b). 

(6) Why are the numbers of competitive methods not consistent in Tables 1&5?

*PS: I suggest the authors first understand the reviewers' concerns and then respond. Several responses did not get the points.*

No objection was raised from the reviewers on the rejection recommendation.
justification_for_why_not_higher_score: In general, this paper considers a specific research problem in fairness learning on the optimal ratio of generated data. Although all the reviewers including me agree this is a decent paper, we all hold the opinion with the more negative side. One reviewer made a note that if there is room for acceptance, the reviewer would support this paper only in terms of its practical value.
justification_for_why_not_lower_score: N/A
summary_of_AC-reviewer_meeting: All the reviewers gave 5 score, which makes this paper below borderline.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: To Reviewer geXT
comment: Thank you again for your constructive feedback. Since the discussion period is ending soon, we would really appreciate it if you can reconsider an increase in the score as you mentioned above.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: To Reviewer SsCr
comment: Thank you again for your constructive feedback. Since the discussion period is ending soon, we wanted to check in and ask if the rebuttal sufficiently answered your queries. We are happy to address if there are any further questions.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: To Reviewer niP4
comment: Thank you again for your constructive feedback. Since the discussion period is ending soon, we wanted to check in and ask if the rebuttal sufficiently answered your queries. We are happy to address if there are any further questions.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Additional Response to Reviewer ZWfo
comment: We really appreciate the response. Also, thank you for requesting further clarification on two valuable questions. We are happy to address them below and in our revision.

-----------------
**Comment 1. Difference from the previous work**
 
We appreciate your additional explanation on the Fair-Path paper and would like to clarify the difference from our work.

As you mentioned, both our work and Fair-Path [1] first utilize bilevel optimization to formulate each paper’s goal and then solve it via the implicit function theorem. Here, this theorem allows us to find the solution without investigating the entire gradient history. However, our work has several clear differences in the problem formulation and algorithmic aspect:
- First, although several previous works have used the implicit function theory in fairness [1] or other fields [2, 3], we believe that our novelty especially lies in *formulating a bilevel optimization problem for learning sampling group ratios dynamically for both real and generated data*. Specifically, we design the inner objective by explicitly separating the group-wise terms and real/generated data terms to adequately apply the data weights. We note that this inner structure is different from Fair-Path, where Fair-Path applies the common outputs of the outer objective to all terms in the inner objective that only considers real data. 
- In addition, the key algorithmic difference lies in the process of solving the problem *after* applying the implicit function theorem. After applying the theorem, we need to evaluate the inverse Hessian matrix (e.g., Eq. 2 in our paper), which is another computational bottleneck. To solve this issue, the Fair-Path paper uses the conjugate gradient (CG) method [2] to approximate the Hessian-related terms. In comparison, we utilize the identity-matrix approximation [4, 5] to handle the inverse-Hessian, where the identity-matrix approximation is known to be much more efficient and may achieve on-par or sometimes better performances compared to the CG method [3]. 

As per your important comment, we now further clarified the similarities and differences between our work and FairPath in our revision (Section C, highlighted in red).

[1] Shui et al., “Fair Representation Learning through Implicit Path Alignment”, ICML’22

[2] Rajeswaran et al., “Meta-Learning with Implicit Gradients”, NeurIPS’19

[3] Lorraine et al., “Optimizing millions of hyperparameters by implicit differentiation”, AISTATS’20

[4] Luketina et al., “Scalable gradient-based tuning of continuous regularization hyperparameters”, ICML’16

[5] Geng et al., “Is attention better than matrix decomposition?”, ICLR’21

-----------------
**Comment 2. Using DRO in our setting**

We thank you for the suggestion on the possible distributionally robust optimization (DRO) formulation. We agree that one could use DRO to adjust the sampling ratios between groups and even real/generated data given a specific prediction loss. However, we would like to clarify that real data and generated data play very different roles in our bilevel objectives. At the test-time evaluation, we only care about the fairness and accuracy losses (in our outer objective) on the real data distribution. Therefore, the empirical risk for generated data (L^gen) only appears in the inner objective for model parameters update. Directly applying DRO to empirical risks of real and generated data does not lead to the same effect as our bilevel objective because this ignores the EO-based loss in our outer objective, and more importantly, it is unclear what is the benefit of optimizing the real/generated data sampling ratio to maximize the empirical risks. If we consider an example where we have a high loss on generated data and a low loss on real data, then DRO should increase the sampling ratio for the generated data to increase the overall loss. However, the high loss on generated data could be the result of the low quality of generated data, and increasing its sampling ratio might instead hurt the performance and fairness. On the other hand, as shown in Section 3.1.4, Dr-Fairness would decrease the ratio of generated data when its quality is low. 

We added this comparison with DRO in our revision (Sections 2.1 and A.6, highlighted in red).

---------------------
We hope that our new responses address your remaining concerns. Please let us know if you have any further questions.

From: ICLR.cc/2023/Conference/Paper2270/Reviewer_ZWfo
title: Post Rebuttal
comment: Thanks for your effort in the rebuttal. After checking responses, I have updated my score. 

There are still two important things to be clarified. (1) I agree that bi-level objective could be applied in different contexts such as your paper and fair-path (ref[2]). However, the key point is that both papers solved bi-level objectives through implicit theorems. That means fair learning with bi-level optimization **through implicit theorems**. I would think it should be clearly illustrated to differentiate the differences between previous work. (2) For adaptation DRO is not hard, given a specific prediction loss on real/fake data and class (y). We could use DRO to balance these loss weights instead of bi-level objective.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: To Reviewer geXT
comment: We really appreciate your response, and we are very happy that our responses have addressed your comments. Also, as per your great suggestion, we will add a motivating example in our final version. Please let us know if you have any further questions.

From: ICLR.cc/2023/Conference/Paper2270/Reviewer_geXT
title: Thanks for providing the response
comment: I am glad to receive the response from the authors and I appreciate the effort in writing the rebuttal.

Most of your explanations make sense to me, especially comment#2. Regarding comment#3, probably a motivating example that explains why using generated data instead of real data can help readers understand the core motivation of this paper (some edge case maybe?). Just a suggestion, not a request in the rebuttal :)

I would reconsider my score after the discussion period. Thanks!

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer SsCr (Part 2) 
comment: **Comment 3: Comparison with FairBatch**

We appreciate the question and further explain how Dr-Fairness outperforms FairBatch. 
- As you mentioned, FairBatch has a theoretical guarantee to find the optimal data group ratio for fairness, but such a guarantee only holds for 1) specific fairness definitions, e.g., equal opportunity (i.e., a relaxed version of equalized odds) with binary settings, and 2) only using real data. 
- Thus, in the last part of Section 2 (i.e., when we empirically verify the approximation performance of Dr-Fairness by comparing it with FairBatch), we follow the same setting of FairBatch to ensure its theoretical guarantee. In this setting, we observe that Dr-Fairness and FairBatch converge to similar optimal group ratios (Figure 2) and thus verify the correctness of the approximations used in Dr-Fairness.
- However, in our main experiments in Section 3, all the experimental settings are more complex (e.g., non-binary group and labels, different fairness metrics, adding generated data), so the theoretical guarantee of FairBatch does not hold anymore. Thus, we now have room for improvement in fairness compared to FairBatch, and Dr-Fairness indeed shows similar or better fairness than FairBatch.
- Moreover, FairBatch may suffer from an accuracy degradation caused by oversampling minority groups from real data. In comparison, Dr-Fairness can avoid such accuracy degradation by using generated data instead of oversampled real data.

We clarified the comparison between Dr-Fairness and FairBatch in our revision (Sections 2.2 and 4, highlighted in blue).

-----------------------
**Comment 4: The behaviors of Dr-Fairness on complex data**

We would like to clarify that 1) the performance differences between CelebA and ImageNet People Subtree are due to the labeling noise in the generated data and 2) the performance of Dr-Fairness is orthogonal to the complexity of the datasets without the noise. 
- In our experiment, we observed that the generated data in ImageNet People Subtree have more label noises compared to CelebA (e.g., some male images are indicated as female). The reason is that the controllable data generation in ImageNet People Subtree is an extremely challenging scenario: 1) the original data in ImageNet People Subtree already contains many labeling noises and also only 10% of the data has annotated group attributes, and 2) the large number of (y, z)-classes (112 x 36 = 4032 classes) makes it difficult to perfectly control the generative models to match the input labels. Thus, the fact that the generated data of ImageNet contains more label noises results in the lower performance of Dr-Fairness in ImageNet compared to CelebA. 
- Although Dr-Fairness still outperforms the baselines in accuracy, fairness, or both in ImageNet People Subtree, we can expect that Dr-Fairness will show better results by reducing the label noises in the generated data. Similarly, on the other complex datasets, we believe that Dr-Fairness will achieve good performances with clean labels and outperform the baselines even if the labels are noisy.

We added this clarification in our revision (Section 3.2, highlighted in blue).

------------------------
We hope that our responses address your concerns. If we have successfully addressed your questions, we would really appreciate it if you consider raising your score. If you have any further concerns, we are happy to discuss and address them.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer SsCr (Part 1)
comment: We appreciate your thoughtful comments and useful suggestions. We addressed them below and improved our manuscript.

------------------------------
**Comment 1: Using the µ ratio**

We would like to clarify that 1) the two types of data ratios have different roles in Dr-Fairness, 2) we can use all real data, and 3) Dr-Fairness can automatically reduce the size of low-quality generated data. 
- The two data ratios λ and µ are designed to find optimal ratios among groups and between real and generated data. Here, we expect that λ mitigates the unfairness issues from the biased data, and µ handles different data qualities in the generated data.
- Also, we clarify that the λ and µ values are the data ratios within one mini-batch, which means that Dr-Fairness actually uses all samples in the real data and dynamically adjusts the λ and µ values across mini-batches during training. For example, if the µ value of a specific (y, z)-class is 0.7, then we select (minibatch_size * 0.7) samples from real data and (minibatch_size * 0.3) samples from generated data to construct one minibatch.
- As you expected, it is important to reduce the impact of low-quality generated data during the training. One possible way is to pre-process the generated data to get rid of low-quality images, but this process requires additional costs. Instead, Dr-Fairness is designed to automatically reduce the ratio of generated data when its quality is low. We show this scenario in Section 3.1.4, where we vary the quality of generated data by adding random Gaussian noise to the original images. As a result, Dr-Fairness indeed reduces the ratio of low-quality generated data (2nd column in Table 4), which results in robust performances even with such low-quality data (3rd–5th columns in Table 4).

We clarified the above points in our revision (Section 2.1, highlighted in blue).

------------------------------
**Comment 2: Using generated data**

We thank you for this comment and would like to explain why we believe using generative data is practical. Recently, obtaining high-quality generated data has become dramatically easier than a few years ago, as many pre-trained generative models are now open to the public (e.g., [1]). We thus believe that obtaining high-quality generated data is increasingly becoming feasible with the rapid growth of generative models. We added an in-depth discussion regarding generated data in the "common answer for all reviewers" above and would appreciate it if you can take a look.

We also note that when we cannot utilize any generated data, Dr-Fairness can still work with real data, as shown in the ablation study of the paper (Section 3.1.3). In Table 3, the row marked by A2 shows the accuracy ablation study if we do not use the generated data. We see that it results in lower accuracy than the original Dr-Fairness, but the overall performances are still comparable to many baselines.

We added the above discussion in our revision (Sections 1 and 3.1.3, highlighted in blue).

[1] Stable Diffusion, https://github.com/CompVis/stable-diffusion

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer geXT (Part 2) 
comment: **Comment 4: Related work**

We thank you for the valuable references. Indeed, these data reweighing techniques [1, 2, 3] are relevant to our data sampling framework, as they keep finding data weights to improve group fairness. Compared to our work, 1) the papers [1, 2] require multiple re-trainings of the model, and 2) the paper [3] uses additional assumptions, including the loss function being twice differentiable and strictly convex in the model parameters. Therefore, applying these reweighing techniques on large-scale models and data may lead to significant training times due to multiple re-trainings or performance degradation due to violations of the assumptions. In comparison, Dr-Fairness works well in large-scale scenarios as shown in our experiments.

We added the above discussion with references in our revision (Sections 4 and C, highlighted in blue).

[1] Krasanakis et al., “Adaptive Sensitive Reweighting to Mitigate Bias in Fairness-aware Classification”, WWW’18 

[2] Jiang et al., “Identifying and Correcting Label Bias in Machine Learning”, AISTATS’20 

[3] Li and Liu, “Achieving Fairness at No Utility Cost via Data Reweighing”, ICML’22

-------------------------
We hope that our responses address your concerns. If we have successfully addressed your questions, we would really appreciate it if you consider raising your score. If you have any further concerns, we are happy to discuss and address them.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer geXT (Part 1)
comment: We appreciate your thoughtful comments and useful suggestions. We addressed them below and improved our manuscript.

--------------------------
**Comment 1: Additional explanation on generated data**

We thank you for this important question and answer as follows. 
- First, we believe that any synthetic data, including data from deep generative models, can be the “generated data” in fair training. The key role of the generated data in algorithmic fairness is supporting the limited subset of the real data. Although our work mainly uses the generated data from generative models, any kind of synthetic data can be used in our framework. We will give a more detailed explanation of the role of the generated data in our answer to Comment 3 below. 
- Also, we implicitly assume that the domain of the generated data and real data are the same, but the distributions of the two datasets do not need to be the same. For example, if the real data represent human faces, then we assume the generated data also contain human faces. However, generated data may have a fairer distribution than real data because we can synthesize the same amount of data for each group.

As per your great comment together with other Reviewers’ comments, we added more explanations on the generated data in our revision (Sections 1 and 2, highlighted in blue).

------------------------
**Comment 2: Clarification on some claims**

Based on your comment, we better clarified our claims in our revision. Here we clarify our claim “In particular, proper usage of generated data can reduce the accuracy degradation caused by over-sampling minority groups from real data”. Suppose the training data has minority groups that only represent a small part of the entire data distribution. Then, if an algorithm like FairBatch over-samples these minority groups too much for fairness, the model may overfit to the small group distribution and thus not generalize well for the data in other groups. As a result, the model may suffer from overall accuracy degradation. In comparison, if we properly use generated data to enrich the distributions of such minority groups, we can lessen the accuracy degradation.

We added the above discussion in our revision (Section 2.1, highlighted in blue).

------------------------
**Comment 3: The motivation for utilizing generated data**

We appreciate this comment and further explain how generated data plays a key role in our framework. Although our framework can be used for only reweighing the original training (real) data without using any external data, we believe that generated data can complement the limitation of real data, resulting in better performances. For example, if a specific group's training data is collected from a smaller region compared to other groups, then the specific group may be discriminated against in model training due to the less-diverse data [1]. In this case, simply reweighing the real data cannot address the diversity issue, but well-generated data can supplement the discriminated group's data. On the other hand, generated data may also have disproportionate qualities among groups or labels [2]. Therefore, it is essential to carefully combine real and generated data, rather than simply using similar-sized generated data for all groups and labels.

We added this discussion in our revision (Section 1, highlighted in blue). In addition, we gave detailed explanations regarding generated data in the "common answer for all reviewers." We would appreciate it if you can check the common answer above.

[1] Mehrabi et al., “A Survey on Bias and Fairness in Machine Learning”, ACM Computing Surveys’21

[2] Grover et al., “Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting”, NeurIPS’19

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer ZWfo (Part 3)
comment: **Comment 6: Using loss-based constraints for equalized odds (EO)**

We clarify how the loss-based constraints can capture equalized odds. We recall our notation: the empirical risk over samples in the set ($y, z$) is $L_{y, z}(w) = 1/m_{y,z} \sum_{i:y,z} ℓ(y_i, \hat{y_i})$, where $ℓ(y_i, \hat{y_i})$ is the loss function, and $m_{y,z}$ be the number of samples in the set ($y, z$). Here, when the loss function $ℓ(y_i, \hat{y_i})$ is 1/0-loss (i.e., $ℓ(y_i, \hat{y_i}) = 1(y_i \neq \hat{y_i})$, where $1(\cdot)$ is an indicator function), the loss-based constraint can perfectly express the equalized odds disparity. Specifically, $L_{y, z}(w)$ with 1/0-loss is equivalent to the probability of the correct predictions in each (y, z)-class (i.e., $\Pr(\hat{y}=y|y, z))$. Therefore, our fairness loss constraint (i.e., $\max_{y, z_1, z_2}{|L_{y, z_1}(w) - L_{y, z_2}(w)|}$) becomes the equalized odds metric, which describes the class-conditioned accuracy disparity among groups (i.e., $max_{y, z_1, z_2} |\Pr(\hat{y}=y|y, z_1) - \Pr(\hat{y}=y|y, z_2)|$). In practice, we can also use other loss functions like cross-entropy loss instead of the 1/0-loss, as other loss functions have been empirically verified as reasonable proxies for capturing group fairness metrics [1, 2].

We clarified the connection between the loss-based constraint and the original equalized odds in our revision (Section A.2, highlighted in blue).

[1] Roh et al., “FairBatch: Batch Selection for Model Fairness”, ICLR’21

[2] Shen et al., “Optimising Equal Opportunity Fairness in Model Training”, ACL’22

----------------------
**Comment 7: How both fairness and accuracy can be improved**

We appreciate this important comment and explain when Dr-Fairness can improve both fairness and accuracy compared to other fairness baselines. We believe this phenomenon is related to the optimal accuracy-fairness tradeoff, which is known to be determined by the data distribution [1]. When the performance of a fair algorithm lies on the optimal accuracy-fairness tradeoff, any other algorithm can only achieve either better fairness or better accuracy, but cannot improve both. However, when the fairness algorithms do not achieve the optimal accuracy-fairness tradeoff in the given data, there is an opportunity to improve the model’s performances toward the optimal tradeoff line. Our framework takes the opportunity of achieving better accuracy-fairness tradeoffs by utilizing the generated data together so as to mitigate the problems of the original training (real) data. In our experiments, we indeed observe that Dr-Fairness sometimes improves both accuracy and fairness compared to other fairness baselines, implying that the baselines do not achieve the optimal accuracy-fairness tradeoff in the first place.

We added the above discussion in our revision (Section B.7, highlighted in blue).

[1] Menon and Williamson, “The cost of fairness in binary classification”, FAccT’18

----------------------
We hope that our responses address your concerns. If we have successfully addressed your questions, we would really appreciate it if you consider raising your score. If you have any further concerns, we are happy to discuss and address them.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer ZWfo (Part 2)
comment: **Comments 4, 5: Real-world fairness impact**

We agree that societal impact is essential and we want to emphasize how our work has real-world fairness impacts. Our work reveals the importance of using synthetic (generated) data together with real data in improving model fairness. Specifically, our framework shows that the generated data can compensate for the unfairness issues in the real data (e.g., size bias and diversity difference) and thus helps achieve better accuracy-fairness tradeoffs, which would not have been achieved when only using real data. Thus, our work provides an opportunity to apply better-performing models to real-world applications that struggle with fair, but less accurate models trained only with real data.

In addition, our experiments focus on more large-scale datasets compared to other fairness tabular benchmarks, as we aim to develop a framework that can work in large-scale real-world applications. Although many valuable fair algorithms have been proposed for the last few years, most of them have focused on relatively small-size models and datasets, including COMPAS [1] and Adult Income [2]. Of course, fairness tabular datasets like COMPAS and Adult Income are important benchmarks that capture the historical discriminations in high-stakes applications like criminal prediction and loan decisions. However, since machine learning is becoming indispensable in various large-scale applications like face recognition and text translation, we believe that it is essential to develop and evaluate fair algorithms for large models and datasets. In this paper, we chose the ImageNet People-subtree classification task as a prominent problem in the computer vision community to study. We do believe our work can make a real-world impact in the long-term.

We added these discussions in our revision (Sections 1 & 3, and the Ethics Statement, highlighted in blue).

[1] Angwin et al., “Machine bias: There’s software used across the country to predict future criminals. And its biased against blacks”, ProPublica’16

[2] Kohavi, “Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid”, SIGKDD’96

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer ZWfo (Part 1)
comment: We appreciate your thoughtful comments and useful suggestions. We addressed them below and improved our manuscript.

--------------------------

**Comment 1: Importance of utilizing generated data in fair training**

We thank you for this comment, and we gave detailed explanations regarding generated data in the "common answer for all reviewers." We would appreciate it if you can also check the common answer above.

------------------------------
**Comment 2: Using bilevel optimization**

We appreciate your viewpoint and would like to clarify why bilevel optimization is a suitable choice for our problem compared to distributionally robust optimization (DRO). Since DRO can solve a target objective in a min-max formulation, various fairness works [1, 2] utilize DRO to improve the *worst-case group accuracy*, one of the fairness criteria. Although these fairness papers using DRO have shown promising results, it is unclear how to extend DRO to 1) support other prominent group fairness metrics (e.g., equalized odds and demographic parity), 2) minimize multiple objectives, and 3) utilize additional (generated) data for fairness. In comparison, our bilevel optimization formulation allows us to handle all these three issues by keeping the standard model training process with real and generated data in the inner objective and adding desired properties like fairness in the outer objective. 

We clarified why we utilize the bilevel formulation in our revision (Section 2.1, highlighted in blue).

[1] Hashimoto et al., “​​Fairness Without Demographics in Repeated Loss Minimization”, ICML’18

[2] Goel et al., “Model Patching: Closing the Subgroup Performance Gap with Data Augmentation”, ICLR’21

-----------------------------
**Comment 3: Related work**

We thank you for the valuable references, which we now cite in our revision. Here we compare the references with our work:
- First, FairMixup [1], a fair augmentation technique, is relevant to our work, but the key difference is that FairMixup only augments the data within the original training data distribution. FairMixup also cannot dynamically adjust sampling ratios from different groups explicitly. On the other hand, Dr-Fairness can utilize any additional data, which is not limited to the original training (real) data distribution, and also find the optimal sampling ratios among groups and between real and generated data.
- Second, the recent fair representation paper [2] indeed uses the bilevel optimization formulation, but this paper targets a different problem from ours: It aims to map the input feature $X$ into the latent variable $X’$ for fairness, but we use the bilevel formulation in adjusting the ratio among groups and between real and generated data to improve fairness.

We added the above discussion with references in our revision (Sections 4 and C, highlighted in blue).

[1] Chuang et al., “Fair Mixup: Fairness via Interpolation”, ICLR’21 

[2] Shui et al., “Fair Representation Learning through Implicit Path Alignment”, ICML’22

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer niP4 (Part 2)
comment: **Comment 2: Using generated data**

We thank you for this comment. We now explain 1) why using generative data becomes a promising and practical direction in fair training and 2) why we believe that Dr-Fairness is robust to the quality of generated data. 

Recently, obtaining high-quality generated data has become dramatically easier than a few years ago, as many pre-trained generative models are now open to the public (e.g., [1]). The quality of generated data we can obtain will keep increasing with the rapid growth of generative models. We added an in-depth discussion regarding generated data in the "common answer for all reviewers." We would appreciate it if you can also check the common answer above.

We believe that Dr-Fairness is robust to the quality of generated data for both CelebA and ImageNet People Subtree. In CelebA, we clearly show that Dr-Fairness is effective even with low-quality generated data (Section 3.1.4). Here, when the generated data quality decreases, Dr-Fairness automatically reduces the usage of the generated data, which is in fact one of our method’s strengths. In ImageNet People Subtree, although the performance improvement seems less significant compared to those in CelebA, Dr-Fairness still outperforms the baselines in accuracy, fairness, or both. We would like to note that the controllable data generation in ImageNet People Subtree is an extremely challenging scenario where the large number of (y, z)-classes (112 x 36 = 4032 classes) makes it difficult to perfectly label images.

We added the above discussion in our revision (Sections 1 and 3.2, highlighted in blue).

[1] Stable Diffusion, https://github.com/CompVis/stable-diffusion

-------------------------
**Comment 3: Experimental results of Dr-Fairness**

We apologize for the confusion and would like to clarify that Dr-Fairness indeed achieves state-of-the-art performances in terms of the accuracy-fairness tradeoff. As discussed in various fairness papers, we believe the fair algorithms should be compared via the accuracy-fairness tradeoff because fairness and accuracy performances cannot always be improved together [1]. For better clarity, we now added new graphs to demonstrate the accuracy-fairness tradeoffs of the algorithms in CelebA and ImageNet People Subtree (Section B.7, highlighted in blue). Here we observe that Dr-Fairness achieves the best tradeoffs in both datasets. 

[1] Menon and Williamson, “The cost of fairness in binary classification”, FAccT’18

-------------------------

We hope that our responses address your concerns. If we have successfully addressed your questions, we would really appreciate it if you consider raising your score. If you have any further concerns, we are happy to discuss and address them.

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: Response to Reviewer niP4 (Part 1)
comment: We appreciate your thoughtful comments and useful suggestions. We addressed them below and improved our manuscript.

--------------------------
**Comment 1: The novelty of our framework**

We respect your viewpoint and would like to clarify that 1) our framework cannot be easily extended via FairBatch, and 2) utilizing various approximations to solve our bilevel optimization is non-trivial. 

We explain why FairBatch cannot be easily extended to handle our problem. First, FairBatch is designed to adjust only the group-wise ratios for fairness **without** considering the generated data. FairBatch’s algorithm is not applicable to our problem setting with generated data because FairBatch training is derived with an assumption that the outer objective of the bilevel problem is quasi-convex [1]. However, when introducing the real and generated data ratio, our bilevel optimization contains non-convex objectives, making FairBatch’s algorithm not applicable anymore. Instead, we need a better algorithm to efficiently solve the non-convex bilevel optimization problem, which motivates the development of our method. Note that even introducing the accuracy term in the objective of FairBatch without considering any generated data cripples FairBatch’s assumptions.

Also, although we are inspired by the approximation techniques from previous works, we believe that adapting them for fair training is one of our novelties. Solving our optimization problem is especially challenging due to its bilevel nature and the non-convex objectives [2]. Thus, we carefully selected reasonable approximations based on theoretical and empirical evidence. For example, among various methods of solving bilevel problems, we chose to use the implicit function theorem to improve the algorithm efficiency under theoretical understanding and converted the original theorem into our setting in Corollary 2 (Section A.3). In addition, we applied the identity matrix approximation based on empirical evidence, i.e., comparing our approximation solutions with the exact inverse Hessian computation (Section A.5) and FairBatch (Section 2) in a well-controlled synthetic setting. 

We clarified these points in our revision (Sections 2.2, 4, and A.3, highlighted in blue).

[1] Boyd et al., “Convex optimization”, 2004

[2] Liu et al., “Investigating bi-level optimization for learning and vision from a unified perspective: A survey and beyond”, IEEE TPAMI 2021

From: ICLR.cc/2023/Conference/Paper2270/Authors
title: The common answer for all reviewers
comment: We appreciate your thoughtful comments and valuable suggestions, which helped us improve the manuscript. We would like to first address the reviewers’ common question and answer the other comments in each individual response.

--------------------------
**The motivation for using generative data in fair training**

As per this important comment, we clarify 1) why generated data is needed in fair training now, 2) the challenges of using generated data from a fairness perspective, and 3) how our framework is effective in utilizing the generated data.

We believe it is very timely to use generated data in fair training, as there are many recent breakthroughs in generative learning in both academia and industry, where many of the trained models have also been made public.
- Until just a few years ago, generative models could not guarantee high-quality outputs despite their learning difficulties. However, recent generative learning paradigms, such as diffusion models [1] and advanced GANs [2], reveal the great potentials of data generation and open the way to utilize the generated data in real-world applications. For example, DALLE2 (a diffusion-based generative model from OpenAI) [3] shows incredible performances in converting any input text into a creative set of images.
- Furthermore, many high-quality pre-trained generative models are now open to the public, which obviates the need to train such models ourselves. Thus, we can now obtain high-quality generated images with only a small adaptation using publicly released generators like Stable Diffusion [4] or StyleGAN [2].
- As a result, generated data is increasingly used to improve ML performance and fairness, where the key challenge is how exactly it should be used along with real data. For example, a recent study in robust training [5] has shown that finding the sweet spot ratio between real and generated data can significantly increase the robustness performance of the classifier. Likewise, we see the same opportunity for fair machine learning [6], which we address in our paper.

We clarify the advantages and risks of using generated data from a fairness perspective. The most important advantage is that generated data can complement the less-diverse subset of real data. For example, if a specific group's training data was collected from a limited data source that does not have the full data distribution,  then the specific group may be discriminated against in model training due to the less-diverse data [7]. In this case, well-generated data can supplement that group's data. However, at the same time, generated data may also have disproportionate qualities among groups or labels [8]. Thus, simply using similar-sized generated data for all groups and labels is not the best approach and may lead to suboptimal accuracy and fairness performances.

Therefore, it is essential to find the right mix of generated and real data for the best accuracy and fairness. And here, we would like to highlight that Dr-Fairness is specially designed to automatically adjust data ratios among groups and between real and generated data to maximize the potential of generated data. Indeed, we empirically observe that the careful combination of real and generated data improves the model's fairness and accuracy. We believe such empirical results demonstrate the importance of utilizing generated data in fair training.

We clarified the above motivation and explanation related to generated data in our revision (Section 1, highlighted in blue).

--------------------------
[1] Dhariwal and Nichol, “Diffusion models beat gans on image synthesis” NeurIPS’21

[2] Karras et al., “Analyzing and improving the image quality of stylegan”, IEEE CVPR’20

[3] OpenAI, https://openai.com/dall-e-2/

[4] Stable Diffusion, https://github.com/CompVis/stable-diffusion 

[5] Gowal et al., “Improving Robustness using Generated Data”, NeurIPS’21

[6] Ramaswamy et al., “Fair Attribute Classification Through Latent Space De-Biasing”, CVPR’21

[7] Mehrabi et al., “A Survey on Bias and Fairness in Machine Learning”, ACM Computing Surveys’21

[8] Grover et al., “Bias Correction of Learned Generative Models using Likelihood-Free Importance Weighting”, NeurIPS’19

From: ICLR.cc/2023/Conference/Paper2270/Reviewer_niP4
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: This submission develops a novel adaptive sampling approach that leverages both real and generated data for fairness. The authors design a bilevel optimization that finds the optimal data sampling ratios among groups and between real and generated data while training a model. The proposed method exhibits desired trade-off between accuracy and fairness on the CelebA and ImageNet People Subtree datasets.
strength_and_weaknesses: Strength: Different from Fairbatch, which finds the optimal group ratio for fairness only using real data, Dr-Fairness optimizes  not only group ratio but also sampling ratios between real and generated data. Therefore, Dr-Fairness can enjoy the potential of both real and generated data. It design a bilevel optimization to perform adaptive sampling.


Weakness: The main novelty of the method lies in extending Fairbatch to Dr-Fairness which seems marginal. Specifically, Dr-Fairness adopts a conditional generator to geerate samples and adopt the to-be-learned sampling ration for real and generated data, which is similar to the group ration. The authors stated that they propose a simple approximation. I am not sure which part is new given the implicit function theorem and its corresponding work. Besides, this work needs more auxiliary information from pre-trained generator. And, from the experiments on ImageNet People Subtree test set, we find that the performance of Dr-Fairness is highly dependent on the quality of the generated samples. Also, it is unclear why the authors claimed that Dr-Fairness achieves the state-of-the-art accuracy and fairness, which is not.

clarity,_quality,_novelty_and_reproducibility: The paper is easy to follow. It considers an interesting problem in fairness, and has originality. But the proposed method relys on an additional generator  which might be not easy to reproduce, especially for image datasets that are difficult to generate.




summary_of_the_review: Although the proposed method is reasonable and achieves desired trade-off between acurracy and fairness, the proposed method only combines existing components. 


correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper2270/Reviewer_ZWfo
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper studied the unbalancing problem in data generation based fairness, i.e, the generated data should be balanced or adjusted to control the prediction/fair performance with raw data. Based on this idea, an bi-level optimization (through implicit function) is proposed to learn the balancing coefficient $\lambda$, $\mu$. Empirical results on CelebA/ImageNet demonstrated its effectiveness. 

strength_and_weaknesses: ### Pros
- This paper considered a meaningful problem in data-augmentation based fairness, i.e, rebalancing the training loss of generated and real data. Furthermore a bi-level objective is proposed to automatically balance the sensitive attribute and label balancing weights. The idea sounds reasonable and empirically evaluations are conducted. 

### Cons:
- This paper focused on a very narrow improvement in technical fair learning without proper motivation in social aspects. Thus I would say the paper scope and significance are quite modest. 
- The bi-level objective is not well-motivated. The framework could also be realized by DRO (distribution robust optimization).  
- Lack of clear discussion on previous work.

Overall I would say, this paper has merits, while they are rather modest in terms of technical part and understanding fairness.

### Comments on cons

1. [Paper scope] This paper focused on the specific aspect in data augmentation based fairness. I.e, data generation to improve unfair predictions. However, data generation through generative models is generally difficult. Thus I would be a bit unsure about the importance of fair generative models in data-augmented fair learning or general fair learning. 
2. [Technical motivation] The key contribution is to introduce the label and sensitive attribute reweighted loss to balance different factors. While I think this framework could also be achieved by the DRO framework by properly adjusting the weights of different groups/labels. Why not DRO? Is it necessarily to formulate the problem as a bi-level objective?
3. [Related work/Technical novelty] The bi-level objective in fair learning or data-augmentation are not entirely new. Discussions on related work seem quite lacking. For example, fair-mixup [1] as a typical data augmentation approach. As for the bi-level in learning fairness is not new, for example fair-path [2] indeed introduced bi-level objective and implicit function to learn fair representation. 
4. [Impact on real-world fairness] This paper focused on a nich technical point in learning the balancing coefficient. However, this paper does not consider the real-world fairness impact. Please take note that algorithmic fairness is not merely a method to improve state-of-the-art. An important aspect is to understand the social impact of proposed fair analysis. Unfortunately, this part is completely missing in the paper, making the contribution in fairness (non technical) quite modest. 
5. [Experiments, Impact on real-world fairness] The experiments are fine to validate the improved performance on benchmark but are not well-motivated to support the real-world fairness. We could always have better results on some benchmarks while the real-world discrimination still is not addressed. 
6. [Technical correctness] In Sec 2.1 and Sec 2.2 the loss constraint is not equivalent to enforce EO. They could be equivalent only in certains specific assumptions such as the loss is calibrated. This part is not correct for me. 
7. [Claims on achieving fair and accuracy] In many parts, this paper claimed the fair and accuracy could be both improved, this is quite a big claim. In fact, this paper did not provide rigorous justifications on why/when/how these could satisfy. I would think there is an over-claim of the contribution.  

Reference 
[1] Fair mixup: Fairness via interpolation. ICLR 2021
[2] Fair Representation Learning through Implicit Path Alignment. ICML 2022

clarity,_quality,_novelty_and_reproducibility: In general this paper is clearly written and easy to follow. The reproducibility seems high. 

summary_of_the_review: [ I would recommend that *the paper is generally fine, but not good enough* (4/10). While it seems that no such a score is provided. Thus I would currently recommend 3.]

This paper studied the unbalancing problem in data generation based fairness, i.e, the generated data should be balanced or adjusted to control the prediction/fair performance with raw data. Based on this idea, an bi-level optimization (through implicit function) is proposed. 

This paper has merits, while they are rather modest in technical parts (see 1-3, 6 in detailed comments) and understanding fairness (see 4-5, 7 in detailed comments).

correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper2270/Reviewer_geXT
confidence: 5: You are absolutely certain about your assessment. You are very familiar with the related work and checked the math/other details carefully.
summary_of_the_paper: This paper studies machine learning fairness in visual recognition. The unifying idea of this paper is to use both real and generated data to improve fairness via adaptive sampling. The iterative procedure adaptively adjusts data ratios between real and generated data based on feedback from a classifier. The authors cast this problem into a bi-level optimization and accelerate the solving process with some tricks. Experiments on image data demonstrate the effectiveness.
strength_and_weaknesses: Pros:

1. The high-level idea makes sense to me. The authors want to precisely sample data from different groups and generated data to achieve better fairness and lower accuracy cost. Sample data more carefully should work for this problem because all biases inherently come from data.

2. The paper is technically sound.

3. The experiments are comprehensive and solid. Compared to FairBatch, the proposed method shows a better fairness as well as good tradeoffs between fairness and accuracy.

Cons:

1. The authors give limited explanations on 'generated data.' I would query what kinds of data are 'generated data' for algorithmic fairness, and what is the distributional assumption of such kinds of data.

2. Some clarifications are needed in the current version. For example, the statement 'In particular, proper usage of generated data can reduce the accuracy degradation caused by over-sampling minority groups from real data' lacks corresponding explanations.

3. The motivation for using generated data can be further elaborated. In my opinion, the proposed method can also be adapted to reweighing training data only without using any external data. Why there are different treatments between the original data and generated data in this case?

4. Since this paper works on data resampling and reweighing, probably the authors want to focus on similar techniques in the related work part. I list three relevant works on data reweighing for authors' reference [1,2,3].

[1] Adaptive Sensitive Reweighting to Mitigate Bias in Fairness-aware Classification, WWW'18
[2] Identifying and Correcting Label Bias in Machine Learning, AISTATS‘20
[3] Achieving Fairness at No Utility Cost via Data Reweighing, ICML'22
clarity,_quality,_novelty_and_reproducibility: 1. The paper is well-written and the clarity is good.

2. The proposed techniques are somehow novel.

3. Good reproducibility.
summary_of_the_review: In sum, I think the paper brings some interesting ideas to this field. However, I hold some concerns about the generated data part and the later usage. I am looking forward to the authors' response.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
empirical_novelty_and_significance: 3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold

From: ICLR.cc/2023/Conference/Paper2270/Reviewer_SsCr
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: This paper formulates the fairness setting with a bi-level optimization on finding good group ratios. Some approximation techniques are proposed to overcome the difficulty of bi-level optimization.

strength_and_weaknesses: Strength

+ This paper formulate the fairness setting with a bi-level optimization on finding good group ratios.

Weaknesses

- "λy,z is the ratio for group z in class y, µy,z is the ratio for real data in the (y, z)-class, and λ and µ are the sets of all λy,z and µy,z". It is reasonable to adjust λ due to different generated data qualities, but why is adjustment of µ beneficial (not using all available real data)? simply due to class imbalance? For different generated data qualities, the ideal case is to weed out generated data with low qualities? Is this tried in the proposed algorithm? Group ratio is a quite coarse measure.

- One limitation of this paper is that it applies only when synthetic data samples are generated.

- If FairBatch has a theoretical guarantee to find the optimal group ratios, why does "Dr-Fairness achieves much higher accuracy with similar or better fairness than FairBatch" if they converge to similar group ratios?

- Experiment results are better in CelebA than ImageNet People Subtree. So will more complex data hurt Dr Fairness even more? Any insight?
clarity,_quality,_novelty_and_reproducibility: Clarity

This paper is well organized.

Quality

The paper is technically solid in general.

Novelty

The novelty of this paper is reasonable as a new fairness setting is formulated with a bi-level optimization on finding good group ratios.

Reproducibility

Good.
summary_of_the_review: This paper formulates a new fairness setting with a bi-level optimization on finding good group ratios. However, to overcome the difficulty of bi-level optimization approximation techniques are proposed, which may hurt the quality of found group ratios.
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 5: marginally below the acceptance threshold