Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper focuses on the problem of fairwashing detection in black-box models. To be specific, fairwashing is a technique used to fool model explanation methods to produce deceptive outputs for unfair models. In this paper, the authors argue the importance of studying the faiwashing issue, and propose a novel framework FRAUD-Detect to detect fairwashed models, and present empirical results. In addition, theoretical analysis is provided to show that fairwashing is unlikely to be avoided. And the robustness of the proposed detection method towards an informed adversary is also discussed.
strengths_and_weaknesses: Strengths:
1. This paper investigates a novel and important problem. Model explanation is a popular and promising approach for auditing a model with critical usage to avoid unfairness in automated decision-making. But such model explanation approaches are also vulnerable to being attacked, which has been overlooked. The problem is also challenging, a thorough study on it is needed.

2. The paper provides a simple but effective method for detecting dishonest interpretation models that take advantage of the fairwashing technique.

3. Theoretical conclusion is provided to show that fairwashing is unlikely to be completely avoided. Empirical results are provided to validate the effectiveness of the proposed detection method and also its robustness towards an informed attack.

Weaknesses:
1. The logic of the paper is clear but the expressions in the paper are kind of obscure and not easy for readers to follow. The motivation and the algorithm in the proposed detection model are simple and clear, but the authors use a lot of long sentences to describe them, which makes the simple things complicated. It would be better if the authors could use more short and succinct sentences, which will improve the presentation and readability of the paper.

2. The hyperparameter \delta (the fairwashing detection threshold) is a key parameter in this proposed detection algorithm. More discussions on how to choose this hyperparameter in practical use are needed.
questions: 1. How should we choose the hyperparameter \delta in different scenarios (e.g. different tasks, different data)?
limitations: No potential negative societal impact.
review_title: null

final_score: rating: 3: Reject: For instance, a paper with technical flaws, weak evaluation, inadequate reproducibility and incompletely addressed ethical considerations.
review_content: summary: The authors deal with the problem of detecting fairwashing, a maliciously generated explanation such that the model looks fair even if it is actually unfair. They investigated fairwashing in the global explanation and found that elimination of fairwashing is difficult, and fairwashing can be detected using false- and true- positive rates. Then, their proposed detection algorithm for fairwashing is to check the deviation of the false/true positive/negative rates between the original and explanation models using KL divergence. To evaluate the robustness of their algorithm, they introduce a fairwashing algorithm with a mechanism evading the proposed detection method. The empirical evaluations demonstrate the high detectability and robustness of their proposed detection algorithm.
strengths_and_weaknesses: (Strengths)
- The fairwashing is an urgent risk. Hence, the detection of the fairwashing is well motivated.
- To the best of my knowledge, this is the first to propose a detection method for fairwashing.

(Weakness)
- The authors make an impractical assumption about the defender's knowledge.
- Lacks some important references.

The authors deal with the interesting and demanding task of detecting fairwashing, and most of the results are technically sound. However, I found a fundamental flaw in the problem setting, by which they deal with the situation where there is no risk of fairwashing. Due to such a considerable flaw, I recommend the rejection of this paper.

The problem setting looks weird. The authors assume the detector can query the original model $B$ in a black-box way. However, if we can query $B$, we can also calculate the unfairness score of $B$, such as the difference of the conditional probabilities in the demographic parity, by which we can confirm the fairness of the original model $B$. Hence, under the assumption of having accessibility to $B$, the defender easily detects the unfairness of the original model $B$. In other words, there is no risk of fairwashing in this context. While this paper adequately validates the fact that their method can detect fairwashing in this context,  the detectability is obvious due to accessibility to $B$.

The statement of Theorem 1 is not rigorously defined. What mean by "completely eliminating fairwashing"? Also, what mean by "sufficient"? 

This paper lacks the comparison with the following studies:
- K. Fukuchi, et al. "Faking fairness via stealthily biased sampling."  AAAI2020.
- D. Slack, et al. "Fooling lime and shap: Adversarial attacks on post hoc explanation methods." AIES2020.

Both papers investigate the risk of deceiving fairness. Remarkably, the first paper discusses the detectability of malicious modification for deceiving unfairness under situations where the detector can access the labeled benchmark dataset. 
questions: See strengths and weaknesses.
limitations: The accessibility to $B$ makes the situation no fairwashing risk. Hence, I recommend the authors reconsider the situation to be meaningful.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper presents a theoretical analysis of fair washing and demonstrates that completely reducing fair washing in certain cases is not possible. In addition, they present a simple technique that looks at the KL between surrogate model confusing matrices to detect fairwashing. They evaluate their technique and find it helps detect fairwashing issues.
strengths_and_weaknesses: Strengths:
- Interesting problem 
- The empirical evaluation of fraud detect is comprehensive, considering many different models and settings.

Weaknesses:
The main weakness is that the theory is missing explicitness in several places and there are gaps in what the exact setting is. Specifically, here are the main gaps I see:
- The initial problem setup is not defined explicitly enough. It seems like the setting is a classifier with a sensitive attribute, but the specifics here are lacking. For example, in equation (1), is this a multiclass or binary class setting? Is the protected attributed discrete or continuous? If it's continuous, is it allowed to take on multiple values? These details are a bit hazy right now making it hard to figure out where the claims apply exactly.
- What are the specifications of the interpretable model? Does this model mirror the predictions across multiple classes or just a single class?
- The construction of interpretable model $I$ is a bit hard to follow as well, and $I$ is introduced as just "a simple interpretable model I". I think what is exactly meant by the interpretable model needs to be stated more explicitly, because right now, it is a bit hazy and hard to follow. This makes it difficult to follow the statement of Theorem 1, where the theorem specifices and interpretable model, without this really being defined. 

While the empirical results about the detection algorithm are pretty complete, it seems tricky to choose this threshold $\Delta$ that determines whether fair washing is going on. It would useful to provide more guidance here.

Overall, I think this paper needs to much more rigorously provide the problem setting and explicit definitions of what is going on.

 
questions: - Figure 3 is a bit hard to follow. Could you help clarify this figure? Where does the dotted line come from? Why are there multiple fidelity values for every $\Delta$?
- Could you help clarify the problem setup a bit more, per the weaknesses section?
- Could you clarify what you mean by interpretable model?

limitations: The authors have done a sufficient job.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper characterizes the problem of fairwashing for interpretable models. It proposes a fairwashed model detection method based on measuring the difference over subpopulations in true-positive and false-positive rates of the interpretable model w.r.t. the black-box model. It shows the method’s sufficiency theoretically and effectiveness empirically even under the assumption of an informed attacker. The evaluation is conducted on two interpretable models of four classification models on three widely used fairness datasets.
strengths_and_weaknesses: ## Strengths
1.The problem is well-motivated. Fairwashing can be a practical threat in reality.

2.The theoretical analyses look correct.

3.The proposed method is comprehensively evaluated and shows decent performance for practical usage. In particular, the setting of an informed adversary strengthens the generalizability of the proposed method.

4.The paper is well-written, well-organized, and easy to follow.

## Weaknesses
1.The method currently only applies to one fairness definition. However, in practice, other definitions like equality of opportunity are also commonly used.

questions: 1.For other fairness criteria such as equality of opportunity, how easy is it to adapt the current method?
limitations: They are well addressed.
review_title: null

Meta-review: metareview: The reviewers were split about this paper: on one hand they appreciated the motivation and the comprehensive experiments in the paper, on the other they were concerned about the clarity of the paper, even worried about a potential flaw. I have decided to vote to accept given the clear and convincing author response. I urge the authors to take all of the reviewers changes into account (if not already done so). Once done this paper will be a nice addition to the conference!
confidence: Less certain
Decision: Accept