Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes "fair mixup" for training fair classifiers.  Inspired by the mixup algorithm, which was presented to improve the generalization performance in Zhang et al., 2018b, fair mixup pick two samples from two different sensitive groups.  Instead of regularizing the gap (e.g., \delta DP), the authors regularized the derivative along the path between two samples.  This algorithm can be applied either to the input space or to the feature space.  The idea of using mixup for training fair classifiers is interesting, and the paper is well written and easy to read.  However, I have a few critical concerns.
 
Concerns: 
- E_{x from P0} f(x) = E_{x from P1} f(x) does not imply fairness:  My first concern is the choice of this relaxed metric, which the authors borrowed from Madras et al. (2018).  Most surrogate conditions are sufficient conditions for the target fairness condition.  For instance, zero mutual information between the sensitive attribute and the classification output implies demographic parity.  The empirical covariance, used in Zafar et al., is also similar.  However, this one is neither necessary nor sufficient.  A classifier can be completely unfair while satisfying this condition: f(x) = 1 w.p. 60%, 0 w.p. 40% on P0, and f(x) = 0.6 w.p. 100% on P1.  This satisfies this expectation-based condition.  However, it is highly unfair against P0 as P(Y=1|P0) = 0.6 and P(Y=1|P1) = 1, assuming the threshold is 0.5.  I would make sense if this metric helps impose the actual fairness conditions we care about. Still, the authors only reported \Delta DP and \Delta EO, without reporting the actual DP/EO differences or multiplicative gaps.  

- Does fairness condition not generalize?:  First of all, I haven't seen large gaps between the level of fairness measured in the train set and the test set.  I believe that it really depends on the choice of training algorithm, and most of the methods I have seen in the literature do not exhibit huge gaps.  Even when there is such a gap, one can choose the best model based on the validation performance (accuracy/fairness).  The authors may want to add more evidence on the lack of fairness generalization of existing algorithms to justify the considered problem.

- Other baseline algorithms with early stopping:  The authors seem not to use any validation or early-stoping to maximize the test performance.  This is commonly done in most work in the literature, so please clarify this or do so.  Most importantly, related to the above concern, GapReg's performance might be improved if used with proper validation and early stopping.  Also, please report the train performance of GapReg.  Its train performance should be better than Fair mixup in every aspect.  AdvDebias is not a good baseline algorithm as it's not optimized for a target task.  Please add proper baseline algorithms.  

===

post-rebuttal:  The authors have addressed some of my concerns, but the experimental results are still missing several important baselines.   Raising my score from 4 to 5.
review_title: Review

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper provided a fair mixup strategy to improve the generalizability of fair classifiers. The authors also provide theoretical understanding of the proposed methods. Extensive experimental results are presented to verify the effectiveness of proposed method. Mainly, I have two questions that need to be answered.

1) The authors claimed that "the proposed fair mixup can improve the generalization of group fairness metrics", and they "provide a theoretical analysis to deepen the understanding of the proposed method". However, it can not be seen from either proposition 1 or proposition 2 to show the improvement of generalization. Either proposition 1 or proposition 2 does not show the generalization bound. Could you please give some explanations? 

2) In the theoretical analysis, the authors considered $f(x)$ as a linear function of $\Phi(x)$ where $\Phi$ is an embedding function. However, this is not true in many applications. Even for the experiments in this paper, this is not true. For example, two-layer ReLu networks (this is not a linear function) is used in section 6.1. If so, the theoretical results can not fully deepen the understanding of the proposed method, which makes the results less important. On the other hand, if general non-convex function $f(x)$ (for example, NN with ReLu activation function) is considered, then the optimal $v^*$ is very hard to obtain since this is a NP-hard for general non-convex minimization problem.

Finally, I am wondering if the proposed methods can be extended to multi-class classification since this paper is studying binary classification problem?
review_title: An interesting topic but the theoretical results are not strong enough

final_score: rating: 7: Good paper, accept
review_content: Summary: This paper proposed a data augmentation method for training a classifier which is intended to have predictive parity between two identified groups. It is based on the “mixup” idea – samples from the two groups are interpolated between, and the smoothness of this path is encouraged. The authors recommend doing the interpolation in latent space. An optimal solution in a constrained setting is derived, and experiments show the empirical success of the model at this task on 3 datasets.

Recommendation: I recommend acceptance of this paper. The application of mixup to this task is sensible, the paper is clearly written, and the theoretical and empirical work both seem solid.

Strengths:
-	Using mixup for a task like this seems reasonable. There is a literature on data augmentation methods for this task
-	Explanation of the mixup method is clear and the theoretical work seems good
-	The empirical results are pretty strong, beating each method at each task. Showing the tradeoff is good, this is the right type of diagram to use here
-	I like that this method is non-adversarial – easier to train and more reliable imo

Weaknesses + Clarifications:
-	The question of the latent variable model seems relevant and interesting. It seems that the mixup method is only as good as the model, and also the trained model might add its own biases to the classification task. It would be nice to see some discussion of this in the paper
-	I am surprised that mixup improves precision on the adult task. It would be good to see some exploration of this
-	For experiments, are all runs shown? Or just the Pareto fronts.
-	A number of hyperparameters (e.g. regularization) are not given 
-	For all the latent path figures (eg Fig 3) why is the y value at x= 0 always 0? Is it normalized to this? Be clear in your description (or maybe I missed it)
-	I would be interested in seeing some further analysis on this model, perhaps using the interpolations themselves

review_title: Nice, clear paper, setup & experiments

final_score: rating: 7: Good paper, accept
review_content: Brief Summary:

This paper introduces a method to improve fairness generalization.  The authors utilize a modified version of mixup where they regularize in favor of invariance of classifier prediction for interpolations between sensitive groups.  The idea is that we expect changes to sensitive attributes not to affect classification decisions.  They introduce a number of regularization terms for both demographic parity and equal opportunity.  Because the mixup interpolations are initially taken as linear interpolations between data --- and thus may not be naturally occurring --- they also introduce a version where samples are taken from a latent space.  They perform assessments on a number of datasets and modalities including Adult, CelebA, and  Jigsaw toxic comments.  They find their methods improve fairness generalization compared to a few baselines, including directly regularizing the model for fairness.

Questions + Comments:

Section 1:
- Figure 1 is a nice example. It could be good if Expected Output was written out a bit more explicitly or make it so that $\Delta DP$ is clearly the difference between the two points.  When I glanced at this graph at first, I thought $\Delta DP$ was being plotted --- consider that readers haven't reached the methods section at this point.  I was somewhat confused about what was being shown at first. 

Section 4:
- When introducing lemma 1, I'd recommend stating explicitly that $T(x_0, x_1, t)$ corresponds to a function that outputs an interpolated sample between $x_0$ and $x_1$ at step $t$. This is clarified immediately below the lemma, but currently, the reader is left guessing when they first read the lemma as to what $T(x_0, x_1, t)$.  I found it slightly confusing while reading and ask the authors to consider stating it upfront. 
- Why is  $P_0 (x_0)$ what we're integrating over? What is this notation meant to say? I notice that this is equivalent to the expected value of drawing $x_0 \sim P_0$, so is this meant to indicate the likelihood of each $x_0$ as well? 

Section 5:
- When describing manifold mixup, there's an assumption that interpolations in the latent space $\mathcal{Z}$ will map nicely onto the set of legitimate data instances. If the encoder-decoder pair is an autoencoder trained without any specific regularization, it could be the case that many instances for linear interpolations could be off manifold.  I don't think this assumption is poor by any means, but I think describing the desired properties of the latent space could be a nice addition. 

Section 6:

- Subsection 6.1: The two layer ReLU model with hidden size of $200$ is very large for the adult data set.  Is it necessary to use such a large model for this method and train for $1000$ iterations, which also feels like a long time for such a small data set? Its surprising that mixup improves overall performance on the data so noticeably, and I'm wondering if the ERM model has overfit the data at this point --- though such training regimes may be needed for fair mixup given that only one $t$ is sampled iteration. Could the authors try something like (1) decreasing the training time or size of the model or (2) just add another baseline such as a random forest or logistic regression to this graph? I think this would help us understand the effects of fair mixup a bit better. 

- Subsections 6.2: For selecting the models, did you always choose the model with best validation accuracy for both ERM and fairmixup? Further how long did you train? 

- Subsection 6.3: Was model selection done in this experiment?

Overall Experimental questions:

- I'm curious as to the training requirements imposed by the use of this method. The authors compare ERM models trained at as many iterations as the fairmixup models.  However, given that only one $t$ is sampled per batch, it might take quite a few iterations to get a good fairmixup model.  Is this the correct intuition? Will the ERM models produce optimal solutions long before fairmixup? Further, if we increase the number of $t$'s, will the fairmixup models converge quicker at the cost of computation time? I'd appreciate if the authors add a bit of discussion related to this.  If the question is unfounded, I'd appreciate clarification. 

For the experimental selection, some model selection for the adult data sets and toxicity classification would add the experiments and help disentangle the effects of overfitting (unless I'm missing something) -- particularly in the case of the adult data set because right now, I'm inclined to think the model is overfitting.

Overall, I'm fairly convinced this method is working well and appreciate the breadth of data modalities and experiments the authors performed performed. 

Overall thoughts:

I think this is a nice paper with a useful contribution to fairness generalization.  If it is really the case that this method is flexible enough to improve both fairness and classification generalization, this method could be quite useful in general. I have a couple clarification questions from the body of the paper, and I'd appreciate answers from the authors.  Further, I'd also appreciate it if the authors provided clarification to the experimental decisions I asked about above. 

One last point for the authors to consider is that for CelebA, the authors build classifiers to preform attractiveness classification. Though they do so from the perspective of discouraging classifiers to exhibit biases, it still could be worthwhile to include a brief discussion in the appendix on the ethical considerations of such models.

 


review_title: Review

Meta-review: # Paper Summary

The goal of this paper is to improve generalization of fairness metrics by borrowing ideas from "mixup", which attempts to improve generalization in the non-fairness setting by introducing convex combinations of training examples as virtual examples.

They adapt this idea by interpolating between protected *groups*, and adding a regularizer that forces the classifier to vary smoothly along this interpolation path. To this end, they show that, for a particular interpolation function, the (empirical) disparity in the fairness metric is upper bounded by their proposed regularizer (which depends both on the fairness metric, and the interpolation function). They consider two fairness metrics (disparate impact and equalized odds) and two interpolation functions (convex combinations in the feature space, or in a latent space).

As Reviewer 4 points out, the above is not a complete explanation for why their regularizer works: they've only really shown that it upper bounds the empirical disparity in the fairness metric (and we could have regularized this empirical disparity directly, and indeed they do so, as a baseline, in their experiments). Presumably the intuition is that their regularizer is improving generalization by (implicitly) depending on virtual examples, but this isn't made explicit.

In a "theoretical analysis" section, they give closed form solutions using classification loss, along with L2 regularization and either (i) a regularizer penalizing the true disparity of impact or (ii) their proposed regularizer (which upper bounds the former). Both reviewer 4 and I seem to doubt if this adds much insight (the other reviewers didn't discuss this section).

They close with experiments on Adult, CelebA, and Jigsaw Toxicity, all of which show dramatic performance gains using their regularizer. However, they only compare to one external baseline (adversarial debiasing).

# Pros

1. Reviewers agreed that the paper was well-written
1. The derivation of their regularizer is somewhat complex, but is described step-by-step, and very clearly
1. Adapting mixup to the problem of improving fairness generalization seems natural and intuitive, but this intuition is maybe given short shrift in the later sections
1. Experiments show impressive results

# Cons

1. Reviewer 1 notes that having the expected value of the classification function be equal for both protected groups does not imply fairness, since the classification function would presumably be thresholded to make hard classification decisions
1. Reviewer 4 points out that they do not actually explain why their regularizer will improve generalization better than the "usual" disparity regularizer. Instead, they only show that it upper-bounds the empirical disparity in the fairness metric. Presumably, the intuition is that their mixup regularizer is doing something like adding "virtual samples"
1. I would like to see a more detailed explanation of how their regularizer is implemented, in the main text (they only say that it "can be easily optimized by computing the Jacobian of f on mixup samples")
1. Reviewers 1 and 2 would like more external baselines (there is only one at the moment, "adversarial robustness"), with reviewer 1 suggesting early stopping. The authors added a new early stopping experiment on CelebA to the appendix, but it would be nice to have this baseline included in all experiments in the main text

# Conclusion

Three of the four reviewers recommended acceptance, with the "reject" reviewer scoring it "5: weak reject". This reviewer had three main criticisms: (i) matching expected classification functions is not the same as matching classification *decisions*, (ii) fairness problems might not have a generalization problem to begin with, and (iii) the experiments don't include enough external baselines. I disagree with the second point, but agree with the other two. I think the third is the most critical, since the first could be solved in many cases by e.g. sampling instead of making hard deterministic decisions.

Overall, my opinion is that this is a borderline paper, but that it falls on the "accept" side of the boundary. The idea is intuitive, and exposition is clear, the derivation is quite interesting, and the experimental results are (aside from not having enough baselines) impressive.
Decision: Accept (Poster)