Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors propose a technique for improving distributionally robust techniques for individual fairness by generating antidote samples. These adversarial samples are constrained to be on the manifold of the data distribution. The antidote data generator is a generative adversarial network (GAN) model that takes samples from data distribution and generates comparable samples, where their non-sensitive attributes differ within some thresholded values, whereas their sensitive attributes could differ arbitrarily. The authors then incorporate the generated antidote data into a DRO optimization by forcing the optimization objective not only to optimize the loss w.r.t. original data but also a wors-case loss w.r.t. generated antidote data. Finally, the authors demonstrate the benefit of the model in several public fairness datasets.
strength_and_weaknesses: Strengths:
- The technique makes sure that the adversarial samples do not fall far from the data manifold.
- Interesting GAN-based data generators.
- The antidote technique is applicable to both DRO and non-DRO models.
- The experiments demonstrate the benefit of the model compared to baselines.

Weakness:
- The proposed method looks like a variation of data augmentation methods, whereby the samples can be perturbed freely within a boundary without changing the label. The perturbation scheme makes sure that non-sensitive attributes are kept in check, i.e., they should not fall too far from the original data. However, the sensitive attributes can change without any restriction. 
To me, this construction looks more similar to a group fairness constraint rather than individual fairness. The groups are represented by the discrete values in the sensitive attributes. The data augmentation scheme suggests the model outputs the same prediction regardless what the value of the sensitive attributes. This is very similar to the concept of demographic parity in group fairness, where the probability of positive prediction should be equalized across multiple groups. 
- The metrics chosen for the comparison are different than the metrics used in previous works (e.g., Yurochkin et al. (2020)). For example, the authors use AP rather than the standard accuracy metric for the performance metrics and the positive/negative comparable samples for the fairness metrics. As the 'positive/negative comparable samples' are specifically designed for the 'comparable' concept presented in the paper, this may give an unfair advantage to the proposed model compared to the baselines. I suggest the authors use standard metrics for the evaluation, such as the one used by Yurochkin et al. (2020), for a better comparison with baselines.
- In Table 3, DRO-Anti achieves the worst ROC result. Why is it marked as bold?
clarity,_quality,_novelty_and_reproducibility: Clarity: overall, the writing of the paper is clear.

Quality and novelty: overall, I think the novelty of the paper is moderate.

Reproducibility: Source code is provided.
summary_of_the_review: It's an interesting paper overall. But I have some concerns about novelty and quality. Therefore, I recommend weak rejection.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper presents a method for learning a GAN for tabular data that performs conditional generation with semantic constraints for the structural properties in the tabular dataset. This model is used to generate perturbed samples that are, it is argued, intuitively performing a similar role to the perturbations in distributionally robust optimization. These models are created for three standard fairness datasets (Adult, COMPAS, Law School) and empirical results are given.
strength_and_weaknesses: 
## Major Comments

* The paper is missing discussion of, and comparison against, many of the more canonical works in the DRO literature. It seems to me that all of these should be discussed in the paper (they are not) and more importantly, compared against. These papers present formal approaches to solving the DRO problem and would be important baselines for the current work (which claims to be "similar to DRO" but is in fact not DRO, since there is no notion of worst-case distributional shift being solved for here -- I think the connection to DRO here is not entirely clear, it seems more like adversarial training). For example:

- Sagawa, Shiori, et al. "Distributionally Robust Neural Networks." International Conference on Learning Representations. 2019.

- Levy, Daniel, et al. "Large-scale methods for distributionally robust optimization." Advances in Neural Information Processing Systems 33 (2020): 8847-8860. (This paper has a nice git repo with implementations.)

- Namkoong, Hongseok, and John C. Duchi. "Stochastic gradient methods for distributionally robust optimization with f-divergences." Advances in neural information processing systems 29 (2016).

* Since the paper is explicitly focused on tabular datasets, it should compare against *strong* baselines for tabular data. This is widely considered to be methods such as XGBoost and LightGBM (see e.g. below):

- Shwartz-Ziv, Ravid, and Amitai Armon. "Tabular data: Deep learning is not all you need." Information Fusion 81 (2022): 84-90.

- Grinsztajn, Léo, Edouard Oyallon, and Gaël Varoquaux. "Why do tree-based models still outperform deep learning on tabular data?." arXiv preprint arXiv:2207.08815 (2022).

I will note that *any* of these tabular methods, it seems, could be used alongside "antidote"-augmented datasets, and an augmented dataset should ideally be able to achieve improved fairness without degrading the accuracy of the best possible model.

* A major issue with GAN-style models, and DRO methods, is both are extremely sensitive to hyperparameters. Combining them seems an even more fragile exercise -- please clearly describe the process for hyperparameter tuning/selection (not just the final values, given in B) and what checks were performed to ensure that the results were not sensitive to these parameterizations. This will also be important when the canonical DRO methods described above are used.

* Similarly, it is not clear at all how the percentages of antidote data were chosen: Why do you use 45.25% antitode data for Anti, 225.97% for DRO-Anti on Adult, and 56.18% Angi/338.50% on Law School? How sensitive are the results to these values?


## Minor Comments

* It seems that different sets of results are presented for each dataset (besides the tables). For example, Fig. 1 cocntains extra results for COMPAS, Fig. 2 shows a different set of results for Adult. It would be really nice to have identical plots, for all three datasets, side by side to compare.

* From Figure 1, it looks like none of the methods have significant differences. 

* I found the introduction very difficult to read, I would suggest revising for clarity.

## Typos etc.

There are several typos in the paper. Here are a few.

P2: "impracticable"

P2: "may correlated with"

P2: "are come to be comparable"

P3: "by given an original training sample" -> given an original training sample

P5: "censual" - I am not sure this is a word

* The shading in the tables is very difficult to see.
clarity,_quality,_novelty_and_reproducibility: I have concerns about the clarity and reproducibility of the work. See comments above.
summary_of_the_review: Tabular data in particular needs more attention in the robustness literature, and I believe in the generative modeling literature as well, and it is good to see the authors' effort in this direction. I have several concerns about the paper in its current form. It seems that only a narrow slice of the DRO literature is discussed, leaving some more mainstream DRO methods out of the work entirely. Well-known effective baselines for tabular data are also not discussed in the work or compared against in the experiments. It is also not clear whether the proposed approach is really equivalent to "robust optimization" at all. The experimental results are a bit difficult to parse due to differences across datasets, but seem inconclusive. Finally, I have some reproducibility concerns. Detailed comments above.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a method to learn and generate "antidote data", which are comparable samples, to resist individual unfairness at minimal cost to model predictivity. The antidote data is generated using a GAN type generator. The paper discusses applications to various datasets and observes the effect of antidote data on fairness for specific sensitive data.
strength_and_weaknesses: Some comments on strengths of the paper:
1. The paper addresses an important topic of fairness within machine learning.
2. The paper contains good examples of the importance of fairness, as well as good comments on why certain practices in the literature may fail to generate adversarial samples that remain on the data manifold.

Some comments on weaknesses of the paper:
1. Overall, the clarity of the paper can be improved. Many technical details in Section 3 were vague or unclearly defined, see points listed below in 3. for some comments. 
2. How are the thresholds Tc and Td chosen for the experiments? Are the results consistent for a wide selection of different threshold values? The numbers shown in Appendix A seem to be quite arbitrary --- are the reported numbers a consequence of choosing a "better" set of thresholds?
3. Some definitions and wordings are unclear and may benefit from a revision. For example:
  - definition 2.1 might benefit from a rephrasing emphasizing "comparable samples given thresholds Td and Tc".
  - In Equation (3), the notions of $\bar{s}$ and $s_{\hat{x}}$ are not immediately clear, although one can trace back to it after reading page 
    4 to have a better idea. It may be helpful to include clarifications of definitions early on to aid clarity.
  - The $\oplus$ operator used in (4), (5) and (6) is not explained and causes confusion when understanding the algorithms in page 4.
  - It would also be helpful to be consistent in notation, for example, variations such as "DRO-Anti" and "Dro-Anti" are repeatedly seen.
  - The small term $\delta$ is not clearly defined in (8).
4. What do the experiment numbers look like for other sensitive features? Are there any intuitive explanations to why LCIFR is better in terms of Neg. Comp. on the Law School dataset?
5. It would be helpful to add discussions in the main text to clarify how the generated data using the algorithms of Section 3.1 are "on-manifold": from the paragraph right above Section 3.1, a vague logic/goal of "the generated data should fit into existing data manifold and obey the inherent feature correlations or innate data constraints" is stated. Can this be elaborated? More precisely, why would "comparable samples" fit into "existing data manifold" for general datasets?
clarity,_quality,_novelty_and_reproducibility: Overall, the paper is interesting, but quality and clarity can be improved; please find comments in the above section. There are no new theoretical proofs / techniques.
summary_of_the_review: Overall, the paper is interesting but still has shortcomings in its quality and clarity; please find detailed comments in the above sections.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors proposed a data-augmentation method to improve a measure of individual fairness in classification. They propose to learn an "antidote data generator" using GANs, which is intended to learn "comparable" samples to any given sample from another sensitive class. This data could then be added directly to the training data or incorporated using a form of DRO. Experimentally, they show that this provides better results in some individual fairness metrics, without losing too much predictive utility.
strength_and_weaknesses: Strengths:
- generative data approaches to individual fairness questions are sensible and promising
- experimental evaluation is pretty extensive and thorough

Weaknesses:
- The notion of "comparable samples" feels a little bit off to me, and may not capture some important cases. For instance, there could be some non-sensitive feature which nonetheless correlates strong with a sensitive feature, and when trying to find a comparable sample for a given sample, we may have to change that feature by a large absolute amount. However, this type of change is disallowed by Def 2.1, and in my opinion, really restricts the cases that we could apply this model to.
- The setup around the notation and "comparable samples" definition feels a little bit ad-hoc, and doesn't totally define some cases. what if we have ordinal features? are they considered continuous or discrete in this framework? are sensitive attributes s continuous or discrete? In Def 2.1: not sure why the discrete features are constrained by a sum, but the continuous features are constrained by a max
- I don't quite understand all the choices made in the data generating setup. For instance, if we're using GANs I don't understand why we first need to model the continuous data as a mixture of Gaussians - there are some implicit assumptions here that I think should be stated. I'm also not sure why you can't guarantee that your outputs will be comparable according to Def 2.1 by, for instance, clipping the output to the required range.
- some experimental details are a little bit unclear to me - I can't quite follow exactly where only real samples are used, where only generated samples are used, and where a mixture is used. It would be good to have clear delineation of these regimes. Also, it seems that some detail/hyperparameters are missing around the antidote generation - for instance how many comparable samples are generated at each DRO step.
- some confusion in the additional experiments: 1. I can't find the comparability ratio defined anywhere 2. I'm not sure why you would need an additional Post(x) step


Notes:
- the paper discusses IF only in terms of sensitive attributes (see the abstract), but there is a large literature which discusses it without reference to sensitive attributes (e.g. https://arxiv.org/pdf/1905.10607.pdf) - would be good to note this early on, in the introduction
- bottom of p1: "primary use case of DRO in model robustness is to adversarially perturb a sample by a small degree" - don't agree with this statement, there is a long line of work in statistics, recently popular in ML (particularly see John Duchi + collaborators' work eg. https://arxiv.org/abs/1810.08750) which uses a different approach to DRO based on perturbing distributions rather than samples
- the inequality in the last line of Eq 8 loses me, I think a little bit more explanation would help
- in general, I recommend against using the COMPAS dataset - it's not a great dataset for demonstrating points about algorithmic fairness - see https://openreview.net/forum?id=qeM58whnpXM for some commentary
- it would be good to list the hyperparameters which are "default for other methods" in your paper itself, just so it's self contained.
- a graphical representation of Table 1 might be helpful
- Figure 1 and 2 would benefit from having a black-and-white friendly visualization
- a group fairness type baseline - e.g. post-processing, would be interesting here, just to note that it isnt' helpful for your objective
- bottom of p8 - total number of pertubed features should be in (0, T_d)
- bottom of p8 - adding noise from [-Tc, Tc] seems a lot, wouldn't this result in a lot of extreme values?
- the "Learning Efficacy" experiment is interesting, but I'd be very interested to see what happens as you add antidote data to the original dataset
clarity,_quality,_novelty_and_reproducibility: Clarity: pieces of exposition around comparable data and the generation process aren't clear
Quality: it's a primarily empirical paper and the experiments seem of good quality, and thorough
Novelty: there is some novelty here with respect to the IF+DRO literature I believe, but a fair amount of similar work in a related vein which should at least be conceptually compared against, see FairGAN (https://ieeexplore.ieee.org/abstract/document/8622525) or http://krvarshney.github.io/pubs/SharmaZRBMV_aies2020.pdf
Reproducibility: pretty reproducible, although some hyperparameters not given
summary_of_the_review: I think this paper is probably not quite there in terms of the clarity of the ideas and novelty. Due to these issues, the experimental section, which is pretty thorough, doesn't quite have the impact that I think it could. As such, I'm recommending rejection.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper proposes methodology for achieving individual fairness, based on creating so-called 'antidote' data. Unlikely approaches that aim to achieve insensitivity to sensitive features, which the authors claim results in a focus of the performance of the classifier off-manifold, the approach proposed here attempts to use GANs to generate comparable samples as an antidote to unfairness.

The abstract presents an exciting sounding idea. The reviews were, however, unanimously against publication. I suspect that the writing is mostly to blame in the sense that, had the authors anticipated these points of confusions / counter-arguments, they could have sharpened their paper / experimental work. 

Chief among these points of confusions / concerns are:

1. Concern that the approach is actually a group fairness approach. These seems like it should be a straightforward one to address. The reviewer is concerned that the approach here actually introduces group fairness. Of course, that does not imply that it does not also achieve individual fairness. The authors could demonstrate that the approach does not achieve group fairness for a specific reason. But really the authors should focus on rigorous / formal notions of individual fairness and they should provide guarantees or at least evidence that, irrespective of whether group fairness is also achieved, individual fairness is achieved.

2. Concern that the baselines for tabular data need to be refined. The authors quickly produced tabular results for more SOTA methods, but there were precious few details on how these numbers were achieved. The gap also shrank and so this opens up the possibility that, after appropriate tuning of these methods, the gap might vanish. Would that mean the approach is doomed? Not necessarily, but the authors have to at least understand what's going on to tell a different story.

3. Concern that the connection with the proposed method and DRO is not clear. One question here is, what is the formal connection with DRO? Can this approach be understood as precisely a form of DRO? This might be easy to answer if the question is "yes". It might be tricky to argue that the answer is "no". So I'm not certain this is a fair question to demand an answer to before publication. If the connection were better understood, then appropriate baselines / comparisons would be easier to identify. Even if the method is an example of DRO, in practice, there are only a handful of DRO methods and this one seems to be new.  It also seems to be similar to adversarial training. At the end of the day, I think having benchmarks comparing it to the "standard" DRO methods and "standard" adversarial training can serve as initial evidence that there's something new here. If the authors cannot make a formal connection or cannot prove that their method is not an example of DRO / adversarial training, then they might anticipate this and raise this as an open question. (Sometimes the best defense is a strong offense / conjecture.)

In summary, this paper seems to be describing something interesting, but reviewers got too caught up in multiple details that "weren't quite right" in their eyes. I'd recommend a careful revision, anticipating what issues will snag reviewers and distract them from seeing the merits of the work. 
justification_for_why_not_higher_score: Reviewers remained unconvinced that their chief concerns were addressed, despite author rebuttals.
Decision: Reject