Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents an empirical study to better understand the ability to fairwash unfair black-box models. They illustrate trade-offs between fidelity and fairness in explanation models. They also demonstrate that fairwashing can generalize beyond the suing group and transfer across models.
main_review: The paper explores a topic that is broadly relevant and significant. The approach is a natural and useful extension of prior work. The methods are not novel, but they produce interesting results and a better understanding of fairwashing. I like the clarity and presentation of key takeaways.

Some kind of theoretical analysis would strengthen the paper. Since the paper lacks novel methods (for understandable and appropriate reasons) as well as theoretical analysis, the main contributions are the questions it chooses to address and the experiments. I like the questions a lot. The experiments are good, but could be improved.

The datasets used are far from ideal and this undermines the results, but they are the standards in the field. So it could be argued that these datasets are important due to the likelihood of their use in fairness auditing. The fact that experiments are repeated on different samples of the data helps too. However, I do have some questions about the experiments below and additional datasets would strengthen the paper.



Major questions and comments:

It is difficult to tell from the experiment descriptions whether the experiments have been set up correctly or have some common pitfalls. For example, it appears that the equal opportunity model is backwards for some datasets, Default Credit and COMPAS. Are your fair models giving men and women equal opportunity to be labeled as a default risk and people of different races equal opportunity to be label high-risk of reoffending?

I like the takeaway that fidelity is a poor proxy for quality of explanation. However, I wonder if the following more nuanced takeaway is also valid. Since there does seem to be a relationship between fidelity and fairness, do these results imply that fidelity could be used, but only with strict thresholds? In other words, does this work show evidence that very minor discrepancies in fidelity could still be used as a filtering step to identify potential fairwashing instances that should be given higher scrutiny?

Some results for the black box models are left to Table 4 in the appendix as mentioned in the quote below. Can you add some additional discussion of these to the main paper to make it clear that the baseline performance of your black box models is reasonable?
“The performances (i.e., accuracy and unfairness) of the four black-box models on the training set, the suing group set as well as the test set of the four datasets are provided in Table 4 in the appendix“



Minor comments/typos:

The term “suing group” should be defined early in the introduction at least informally. It would also help to have a more prominent formal definition than what is on lines 131-132.

Line 20: high stake -> high stakes
Line 177: one-hot -> shot

Please fix all instances of commas in latex math mode. 
E.g., line 155: $48,842$ should be $48{,}842$.

limitations_and_societal_impact: I’m not sure this was adequately addressed, but maybe I’ve missed something. Can the authors point to where they believe they’ve addressed these points?
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper addresses the problem of fairwashing, i.e., the process in which an adversary builds a manipulated explanation of the unfair black-box model. The manipulated explanations should ideally hide the fact that the block-box model is unfair. The authors conduct a series of experimental evaluations and conclude:
-	Manipulated explanations can be up to 99.2 % as accurate as the original model, while being much less (up to 50% less) unfair
-	Fairwashed explanations can generalize beyond the data points they were trained on.

main_review: The paper addresses a timely and relevant topic in the area of model accountability. However, there are several concerns regarding this work:
1. The authors conduct their experimental evaluation for one fair-washing attack; yet the title suggests that a general approach has been identified.
2. The experimental results in the main paper are shown for one data set only. Yet, there is plenty of space dedicated to a table summarizing related work. I suggest presenting some of the experimental results from the appendix in the main paper.
3. I do not believe that the way fidelity is described in the paper provides a useful measure to understand how faithful the explanation model is to the black-box model. As an example, an explanation model with a probability of 0.51 would receive the same fidelity score as a model with a probability of 1. Moreover, even when the probability is 1 (or 0), the pre-sigmoid scores (i.e., logits) can still be substantially different, essentially indicating that the explanation model reasons differently. In principle one could imagine that the former explanation model, when outputting a constant class, would be much fairer than a more discriminative model. 
3. At times, the paper is not clear! I will give examples below.

Clarifications & Questions:
-	The authors have not really made clear what their paper is trying to achieve: sometimes it sounds as if developing more stable methods is a concern?
-	Question: Figure 1 - EOpp is essentially constant across different unfairness thresholds. In contrast to that, the fidelity under PE and SP constraints are more responsive to the fairness thresholds. Why is that? Has it to do with EOpp not considering individuals with true label y=0?
-	How does the proposed method generalize for the attacks by Dombrowski et al (2020, ICML) and Slack et al (2020, AIES)? Can the authors comment on this?
-	At no point, the suing group X_sg was formally defined. What kind of data points are in there? Are those random samples from the data set?
-	Equation 1 contains very little to no intuition. It would have been nice to include some sentences describing the objective and why we should care about it.
-	There are many claims being made which I feel are unsubstantiated. At least, there is no theory supporting such strong claims, e.g.,
 in the Abstract: ‘[…] show that fairwashed explanation models can generalize beyond the suing group, which will only worsen as more stable fairness methods get developed.’ To me this seems like speculation.
-	In the figures, the vertical lines are thin and hard to distinguish.
-	In Section 3.1, you first describe your experimental results in Figures 1 and 2. Then, in a separate paragraph, you describe the conclusions without referring to the figures these conclusions could be based on. 

limitations_and_societal_impact: The limitations of the proposed fidelity measure could be better discussed and evaluated (see comments above).
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: This paper studies "fairwashed" explanation of black-box ML models -- interpretable models that achieve high fidelity with respect to the black-box predictions but substantially better scores on various fairness metrics. The authors empirically demonstrate how fairwashing can be achieved with very small decreases in fidelity, and how a fairwashed explanation of one type of model (say, random forests) can be successfully repurposed to explain a different model (eg AdaBoost). Because of this, they argue, auditors need to be very careful about how they use interpretable models in fairness assessments.
main_review: This is a well-written, interesting paper about an area of growing concern for the field. Its goals -- to characterize what is possible with fairwashing attacks -- are reasonably modest, but it lands its point clearly. There's room for expanding on this work, but I think it would be a worthy addition to NeurIPS. 

### Questions

Why not also test calibration on the continuous predictions, since this is very often what is used to measure fairness in real-world settings?

Intuitively, it seems like these attacks would be easy to identify because to decrease some unfairness measure by 50% while only changing <1% of predictions would require obvious structure in the changed predictions (eg all the changes affect the same group). Is this the case?

### Notes

The x axes in Fig 1 would be more readable if they didn't have the same scale.
limitations_and_societal_impact: This paper is focused on the bad outcomes that could occur when interpretable models are used to assess fairness claims. It is clearly concerned with averting the potential negative societal impact of such assessments.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper attempts to empirically characterize the risk of "fair washing" attacks, in which an unfair model can be described with high fidelity by a global explainer that exhibits substantially less unfairness. The authors determine the fidelity-unfairness Pareto curve for several black box models, explanation models, datasets, and fairness criteria and find that across all tested groups, it is generally possible to reduce apparent unfairness by 50% with a minimal reduction in fidelity to the black box model. The authors additionally show that fair washing attacks trained on a specific subgroup (the suing group) extrapolate well to an unseen test group and that fairwashed explanations trained on one black box model may still have high fidelity on other black box models. Finally, the authors determined the range of unfairness metric values that can be achieved at various levels of fidelity to the black box in order to quantify how susceptible different black box-dataset-explainer combinations may be to fair washing. 
main_review: The paper is well-written and begins to address the important issue of identifying and preventing fair washing attacks. My main concern is that the analysis involved mostly involves repeating tasks in the original fair washing paper [Aivodji 2019] on a wider range of black box/explainer/dataset/metric combinations, without attempting to rigorously quantify the results and how they may relate to the characteristics of the black box/explainer/dataset/metric combination. 

In particular, experiment 1 generally seems to repeat the primary result of [Aivodji 2019], but on a larger combination of black box/explainer/dataset/metrics. I would find this more interesting (and more reflective of the title "characterizing" the risk of fair washing) if it were possible to determine some properties or trends across these results: e.g. how and why are the results for COMPAS different than those for default credit? How and why are the results for LogReg different than those for rule lists?  

Again, experiment 2 repeats a result from [Aivodji 2019], but on a larger combination of black box/explainer/dataset/metrics.

Experiment 3 is new relative to [Aivodji 2019], but I find the motivation rather unconvincing, and perhaps the result is unsurprising - if the signal in the dataset is simple, or if black box accuracy is high across several black boxes, it mostly seems natural that the explanation for one will be a reasonably good match for another, with some variation around decision boundaries.

Experiment 4 again uses a method from another work [Coston 2021] in a very straightforward way, without attempting to characterize the variation in results.

Overall, I think the work is well-presented, but I find that it is lacking with respect to some significant contribution toward the understanding of when and why fair washing attacks are possible. 

Clarity-wise, my only complaint is that the variation in black box/explainer/dataset/metric combinations presented in the main paper images is somewhat hard to follow, e.g. Figures 1, 2, and 3 concern Decision Tree, Figure 4 concerns Logistic Regression. Perhaps either remain consistent or provide some rationale for the switch.
limitations_and_societal_impact: I don't have any outstanding concerns about negative social impact after reading this paper.
review_title: null

Meta-review: This is an experimental study on the phenomenon of "fairwashing", namely an explanation model making an unfair blackbox model more fair than it is. In particular, the authors show that fidelity of an explanation model is not necessarily an indication of "fairness-fidelity", that is an explanation model may be misleading with respect to a given fairness measure, while scoring high no a fidelity metric. The study demonstrates this phenomenon with respect to various fairness metrics and explanation models. 

The reviewers appreciated the clarity of the experimental setup and presentation. Moreover, the phenomenon studied here is clearly important and of interest to the ML community.

On the negative side, the submission does not attempt a formal/theoretical analysis of the the phenomenon. Section 4 seems of speculative nature, empirically exploring the range of unfairness among explanation models of a set fidelity. It is shown that the relationship can differ among different datasets, but no attempt of formally analyzing these seems to have been made.


Decision: Accept (Poster)