Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper is motivated by a common problem in real world applications of deep models, distributional shifts, which can cause unreliable behavior in the deployed models. In particular,  state-of-the-art fairness algorithms would be affected by such distributional shifts in the test data. This poses the following question studied by the paper: how can one achieve fairness when there exist unseen distributional shifts?

Towards this end, the paper proposes a new objective: Equalized Robustness (ER), that imposes equalized model robustness against distributions shifts across majority and minority. Fruthermore, the paper develops a new algorithm called Curvature Matching (CUMA) that imposes ER during training, and tests it through experiments.


main_review: Comments: 
* Although the ideal model would indeed be robust against any unforeseen distributional shift, in certain applications (e.g., demographic changes in a student population applying for college), these changes are less dramatic over time (altough not necessarily predefined as described on page 4, Section 3). Furthermore, the decision-maker (e.g., school) might have some external information about the type of distributional shift. How can we get better guarantees in this case? Would the algorithm still be applicable with some modification to take into account this knowledge (and how)?
* In terms of writing, the paper is well written and precise but remains less accessible to readers who are familiar with the fairness literature but not other related parts (e.g., robustness and smoothness). 
* In particular, this makes it harder for me to evaluate the novelty of the methodological contribution. Does the algorithm bring new ideas (of potential general interest) or just adapt existing robustness techniques to this particular fairness-related application? If it is the latter, what potential difficulties and special conditions one needs to address (that make the problem challenging)?
* I found the experiments extensive enough. However, a major question I have is how the distributional shift is modeled in those static datasets? On page 7, the paper mentions that Gaussian and uniform noise is used. First, I think that these shifts are not necessarily representable of the true shifts in a population (if not, are there some real cases that justify this assumption?). Second, this approach essentially introduces a “synthetic” distributional shift. It think it would make more sense to perform the same experiments on a dataset with observations over multiple years or after a policy change. The idea is to test how a model trained on old data performs on the most recent ones. I suspect that the changes might be not be very dramatic (since we are talking mostly about demographic changes that are slower).
* Given the focus on fairness, the choice of the CelebA dataset (with sensitive features “eyeglasses” and “chubby”) is odd and this case study could potentially be omitted.

Minor comments: 
* the bright green color in links is difficult for reading
* I am not very familiar with the ICLR format, but Section 2 reads like a mix of math preliminaries and related literature. I think it’s better to shorten and remove unnecessary mathematical definitions.
* Typo: “the the stability”, p.1


summary_of_the_review: The paper tackles an emerging problem (fairness under distributional shifts) and proposes a new objective and algorithm. While both the metric and the algorithm seem novel, I am not entirely convinced about the methodological contribution overall, given that I am not familiar with the general literature on robustness against distributional shifts. I also have concerns about the value of the experimental evaluation as explained in my comments above.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The paper proposed a new penalty term which is expected to make the final prediction model
robust to distribution shift of test data. The proposed penalty is based on the
distributions of local curvatures of two sensitive groups and enforces these two distributions to be similar.
The proposed idea is motivated by robust learning methods where the prediction model is robust to distributional shift. 
main_review: Strength
  - The idea is new and interesting.
  - The paper is well written.

Weakness
  - I do not understand the fundamental claim that the similarity of the distributions of local curvatures
  ensure the fairness robustness. Even there is no explicit definition of fairness robust to distribution shift.

- I agree that local curvatures affects the robustness of a prediction model but I do not understand why and how  local curvatures affect the degree of fairness robustness. 

- Numerical results do not  seem to support the main claim. For standard fair learning algorithms (not
robust to distribution shift) including AdvDebias and LAFTR, the degree of unfairness (e.g. $\Delta E_{OPP}$) even decreases as noises are added in the test data.   

- Adding a small noise would not be sufficient to see the effect of robustness. As  (Ensuring Fairness Beyond the Training Data, NIPS2020) is done, the worst case distribution shift could be considered for numerical comparison.
summary_of_the_review: I do not understand why making the prediction model robust results in fairness robustness.
For this reason, I recommend "rejection". However, it the authors explain successively why and how the two concepts - prediction robustness and fairness robustness, are related and how the proposed method utilizes this relation. Instead of intuitive explanations, some equations with rigorous definitions
would be helpful for understanding the value of the proposed method.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper studies fairness under distribution shifts. The model claims to be fair in robustness against distribution shifts with several experiments being conducted. 
main_review: Points in favor
+ Fairness is a hot and timely topic.
+ Selection of problem domain, i.e., fairness under distribution shifts. 


Points against
- The authors overstate several of their contributions while the actual contribution is unknown. This paper does not propose a new fairness goal and the first fairness metric under distribution shifts. Rather it extends equalized model performance to distribution shift settings. Similarity, the 'new' learning algorithm is basically a combination of existing techniques such as Hessian matrix and adversarial learning frameworks. The authors should tone down these claims. It is also a stretch to argue that the proposed method does not achieve "much more robust fairness" and is insignificant instead. The statistic test is therefore suggested to verify the importance of these results. 

- The paper does not discuss and compare proper state of the art. The related work discusses pre-, in-, and post-processing approaches while studies focus on fairness under distribution shifts are not cited and compared. To give concrete examples, fairness drift are considered in "FARF: A Fair and Adaptive Random Forests Classifier" as well as reference therein. I urge the authors to compare their method with recent works that consider fairness drift and discuss at least incomparability otherwise.   

- Format violations such as the deceased margins before and after equations.    
summary_of_the_review: Limited contribution, inappropriate related work and format violations.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes a new fairness metric, Equalized Robustness (ER), to assess model robustness for sensitive subgroups. ER measures the maximum mean discrepancy (MMD) between the loss curvature of the two sensitive groups. The paper proposes using gradient direction to approximate the spectral norm and using finite differences to approximate the Hessian. The MMD is calculated using the RBF kernel.

 The paper shows that existing in-distribution fairness promoting methods do not achieve parity with respect to the new metric. A new method, curvature matching (CUMA), is proposed to achieve both in-distribution and robust fairness. CUMA adds the MMD distance as a regularizer to the standard adversarially fair learning approach (which includes a cross entropy loss term and an adversarially fair loss term for in-distribution fairness).

The paper performs ablation studies for the hyper parameter used in the RBF used to compute MMD.





main_review: + The paper is very well-written and well-motivated. The paper provides a clear motivating example in the context of self-driving cars: a model may be fair with respect to skin tone for optimal light conditions, but disparities may arise when there is rain, poor lighting, etc. Figure 1 also provides a nice visual illustration of the key concepts.
+ The paper is very accessible. The notation is clear and the authors provide intuition about the key concepts.

- Does matching curvatures only guarantee that the groups will have similar level robustness which may be low or high robustness? Or does it specifically encourage high robustness?
- Would be helpful to be more specific about distributional shifts. It appears these are covariate shifts—i.e. shifts in the distribution of features/predictors, but assuming there is no shift in p(y |x).
- Do we expect the simulated distribution shifts (Gaussian and uniform noise) to be representative of real-world distribution shifts? Is it possible to empirically analyze a real world distribution shift?
- It is important to acknowledge the problems of a task like predicting "attractiveness".

Minor
- Would be useful to discuss the choice of kernel for MMD and what that may impact
- Empirical results should include uncertainty estimates
- I found the explanation of how hyper parameter S impacts the MMD to be confusing. I did not follow the sentences starting with: “A small sigma will make ….” Does small refer to the value of the spectral norm or to the number of sigmas in the sampled set?
- Would be helpful to include dimensions in introduction of the setting in section 3. E.g. x is a p-dimensional, Hessian is p x p matrix, etc


summary_of_the_review: The ability for fairness properties to generalize is an important problem tackled by this paper. This paper makes a couple novel contributions: 1) a fairness metric that captures robustness to distributional shifts; 2)  learning method to achieve this notion of “out-of-distribution” fairness as well as standard in-distribution fairness. The main technical areas of improvement are 1) include analysis on how well this method works for real-world (as opposed to simulated) distributional shifts and 2) provide error bars to support the empirical claims. I would also encourage the authors to exercise more care/caution around a task like predicting "attractiveness".
review_title: null

Meta-review: The paper considers learning a fair classifier under distribution shift. The proposal involves an additional MMD penalty between the model curvatures on the data subgroups defined by the sensitive attribute. Reviewers generally found the problem setting to be well motivated, and the paper to have interesting ideas. Some concerns were raised in the initial set of reviews:

(1) _Relation between local curvatures and fairness robustness_. The concern was that the paper does not make sufficiently clear how similarity of the distributions of local curvatures ensures fairness robustness, and that there is no explicit definition of fairness robust to distribution shift.

(2) _Comparison to related work_. The concern was that works such as FARF as also considering the issue of distribution shift.

(3) _Technical novelty_. The concern was that technical depth of the proposal may be limited, as it builds on existing ideas (e.g., adversarial learning, Hessian to measure curvature).

(4) _Significance of results_. The concern was that the improvements of the proposed method are not significant, statistically and/or practically.

For point (1), the response clarified that the proposal is to ensure that the local curvature (and hence robustness) across data subgroups is similar. The relevant reviewer was still unclear as to whether this ensures what one might intuitively consider "robust fairness". On my review of the paper, I do concur that from the Introduction, and the para preceding Eqn 4, it appears that one natural notion is

$ \sup_{\mathbb{Q} \in \mathcal{U}( \mathbb{P} )} \Delta( \mathbb{Q}( \hat{Y}, Y \mid A = 0 ), \mathbb{Q}( \hat{Y}, Y \mid A = 1 ) ) $

where $\mathbb{P}$ is the observed data distribution, $\mathcal{U}$ is some uncertainty set, and $\Delta$ is some fairness measure (e.g., DP). Assuming this is indeed the ideal, it would be useful to mathematically contrast it to the proposal adopted in the present paper. The para preceding Eqn 4 correctly notes that the above notion would require specifying $\mathcal{U}$. This may be challenging, but an apparently reasonable strategy that follows the distributionally-robust optimization literature would be to use a specific ball around the training distribution (e.g., all distributions with bounded KL divergence against $\mathbb{P}$). Further, it is of interest to ask whether the proposed objective in any way approximate this one; put another way, is there any implicit assumption made as to which class of distributions one is likely to encounter?

Further discussion would also be useful on the following alternative to the objective presented in the paper: rather than match the curvatures for the subgroups, simply minimise their unweighted average. This ought also to ensure robustness under the two different distributions; page 2 hints that this might not work owing to the different scales of these terms (i.e., the minority subgroup being much less robust), but the point does not seem to be discussed very explicitly subsequently.

For point (2), the response noted that FARF is designed for online learning, whereas the present paper involves a single, static training set drawn iid from a single distribution. In the present paper, the drift happens at test time, and the learner has no access to samples from this distribution. The authors argued that FARF can be applied as-is to this setting. From my reading of this and the FARF paper, I agree that while the latter should be cited, it is not clearly applicable to the present setting.

This said, the present paper primarily focusses on the covariate shift setting, for which there have been some relevant recent works; see:

Singh et al., "Fairness Violations and Mitigation under Covariate Shift", FAccT '21.

Rezaei et al., "Robust Fairness under Covariate Shift", AAAI '21.

The former uses tools from joint causal graphs, while the latter assumes access to an unlabelled sample for the target distribution. The present work is certainly different in technical details, but at a minimum it seems prudent to acknowledge that there are relevant works on ensuring fairness outside the observed training distribution, and thus tone down statements such as "As a pioneer work...". There also seems scope to compare against the latter, e.g., to see how valuable having a few samples from the target domain are.

Another work relevant to the spirit of ensuring fairness beyond the observed data is

Mandal et al., "Ensuring Fairness Beyond the Training Data", NeurIPS 2020.

This is in line with the distributionally-robust objective suggested in point (1), where one considers test distributions that can be arbitrary re-weightings of the training distribution.

For point (3), from my reading, the technical content is reasonable. I would however have liked more mathematical discussions on point (1) above, which is important as it is the foundation of the strategy followed.

For point (4), the response asserts their improvements are significant practically and statistically. From my reading, I am inclined to agree with this claim. I would however note that another reviewer raised the question of whether Gaussian and uniform noise are reflective of real-world distribution shifts. I concur with this concern; this part of the paper seems a little disappointing. The response mentioned results on a new setting with more realistic shift, which we suggest is incorporated into future versions of the paper.

Overall, the paper has some interesting ideas for a topical and important problem. At the same time, there is scope for tightening the work per the comments above, particularly on points (1) and (2), and to some extent (4). We believe that addressing these would help properly situate the work, and thus increase its clarity and potential impact. We thus encourage the authors to consider incorporating these for a future submission.
Decision: Reject