Original: Washing The Unwashable : On The (Im)possibility of Fairwashing Detection
Revised: Washing The Unwashable : On The (Im)possibility of Fairwashing Detection

Original: Anonymous Author(s)

Original: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: The use of black-box models ( e.g. , deep neural networks) in high-stakes decisionmaking systems, whose internal logic is complex, raises the need for providingexplanations about their decisions.
Revised: The use of black-box models ( e.g. , deep neural networks) in high-stakes decisionmaking systems, whose internal logic is complex, raises the need for providing explanations about their decisions.

Original: Model explanation techniques mitigate thisproblem by generating an interpretable and high-fidelity surrogate model ( e.g. , alogistic regressor or decision tree) to explain the logic of black-box models.
Revised: Model explanation techniques mitigate this problem by generating an interpretable and high-fidelity surrogate model ( e.g. , a logistic regressor or decision tree) to explain the logic of black-box models.

Original: Inthis work, we investigate the issue of fairwashing, in which model explanationtechniques are manipulated to rationalize decisions taken by an unfair black-boxmodel using deceptive surrogate models.
Revised: In this work, we investigate the issue of fairwashing, in which model explanation techniques are manipulated to rationalize decisions taken by an unfair black-box model using deceptive surrogate models.

Original: More precisely, we theoretically characterize and analyze fairwashing, proving that this phenomenon is difficult to avoiddue to an irreducible factor—the unfairness of the black-box model.
Revised: More precisely, we theoretically characterize and analyze fairwashing, proving that this phenomenon is difficult to avoid due to an irreducible factor—the unfairness of the black-box model.

Original: Based on thetheory developed, we propose a novel technique, called FRAUD-Detect (FaiRness AUDit Detection), to detect fairwashed models by measuring a divergence oversubpopulation-wise fidelity measures of the interpretable model.
Revised: Based on the theory developed, we propose a novel technique, called FRAUD-Detect (FaiRness AUDit Detection), to detect fairwashed models by measuring a divergence over subpopulation-wise fidelity measures of the interpretable model.

Original: We empiricallydemonstrate that this divergence is significantly larger in purposefully fairwashedinterpretable models than in honest ones.
Revised: We empirically demonstrate that this divergence is significantly larger in purposefully fairwashed interpretable models than in honest ones.

Original: Furthermore, we show that our detectoris robust to an informed adversary trying to bypass our detector.
Revised: Furthermore, we show that our detector is robust to an informed adversary trying to bypass our detector.

Revised: The code implementing FRAUD-Detect is available at https://github.com/cleverhans-lab/FRAUD-Detect .

Original:  Introduction
Revised:  Introduction

Original: The wide applicability of machine learning models has recently increased their usage in high-stakesdecision systems such as credit scoring [43], insurance risk [10] and predictive justice [30].
Revised: The wide applicability of machine learning models has recently increased their usage in high-stakes decision systems such as credit scoring [43], insurance risk [10] and predictive justice [30].

Original: Theconsequences of erroneous decisions that are based on predictions of machine learning models( e.g. , people being wrongly denied parole [46]) have increased the demand—from both the publicand government—to provide an explanation to humans about model decisions.
Revised: The consequences of erroneous decisions that are based on predictions of machine learning models ( e.g. , people being wrongly denied parole [46]) have increased the demand—from both the public and government—to provide an explanation to humans about model decisions.

Original: For instance, thisappears as an explanation requirement in the European General Data Protection Regulation [26].
Revised: For instance, this appears as an explanation requirement in the European General Data Protection Regulation [26].

Original: However, various widely-used model architectures, such as deep neural networks, are consideredblack-boxes due to their complex and hidden internal logic, impeding the ability to explain theirdecisions in terms that are understandable by a human.
Revised: However, various widely-used model architectures, such as deep neural networks, are considered black-boxes due to their complex and hidden internal logic, impeding the ability to explain their decisions in terms that are understandable by a human.

Original: To address this issue, black-boxmodelexplanation techniques [27, 6] aim to provide reasons for predictions of black-box models in humanunderstandable terms. One possible way to realize this is to approximate the black-box modelusingan inherently interpretable model ( e.g. , logistic regression or decision tree) whose decisions can beeasily explained by design [27].
Revised: To address this issue, black-box model

Original: For instance, in logistic regression, the coefficients of the modeldirectly represent the importance of each input feature, while a decision tree is composed of a set ofexplainable if-then decision rules related to the input features. In this black-box model explanation setting, the interpretable model is trained with the objective thatits predictions agree with the predictions of the black-box model ( i.e. , high fidelity [20]). However,black-box model explanations can be manipulated to provide deceiving explanations, even if these explanations display a high fidelity with respect to the original black-box model. An example of such ma-
Revised: [Figure]

Original: [Figure]
Revised: [Figure]

Original: T −
Revised: T −

Original: T +
Revised: T +

Original: F +
Revised: F +

Original: F −.2.
Revised: F −.2.

Original: R a t e
Revised: R a t e

Original: Honest interpretable model
Revised: Honest interpretable model

Original: T −
Revised: T −

Original: T +
Revised: T +

Original: F +
Revised: F +

Original: F −.2.
Revised: F −.2.

Original: R a t e
Revised: R a t e

Original: Fairwashed interpretable model nipulation occurs when the explanations provided cover up the unfairness of the underlying black-
Revised: [Figure] explanation techniques [27, 6] aim to provide reasons for predictions of black-box models in humanunderstandable terms.

Revised: One possible way to realize this is to approximate the black-box model using an inherently interpretable model ( e g. , logistic regression or decision tree) whose decisions can be easily explained by design [27].

Original:  model, resulting in fairwashing [2, 3].
Revised: For instance, in logistic regression, the coefficients of the model directly represent the importance of each input feature, while a decision tree is composed of a set of explainable if-then decision rules related to the input features. In this black-box model explanation setting, the interpretable model is trained with the objective that its predictions agree with the predictions of the black-box model ( i.e. , high fidelity [20]). However, black-box model explanations can be manipulated to provide deceiving explanations, even if these explanations display a high fidelity with respect to the original black-box model. An example of such manipulation occurs when the explanations provided cover up the unfairness of the underlying black-box model, resulting in fairwashing [2, 3].

Original: As a concrete example, a dishonest bank [4] could use a blackbox model that denies a loan to a customer in a discriminative manner—based on a sensitive attributesuch as race or gender identity—while providing an interpretable surrogate model promoting the falseimpression that the decisions are fair and based solely on non-sensitive features.
Revised: As a concrete example, a dishonest bank [4] could use a blackbox model that denies a loan to a customer in a discriminative manner—based on a sensitive attribute such as race or gender identity—while providing an interpretable surrogate model promoting the false impression that the decisions are fair and based solely on non-sensitive features.

Original: These manipulationsare made possible in part by the ambiguity in current regulations [26] when it comes to describingwhat constitutes a valid explanation [45].
Revised: These manipulations are made possible in part by the ambiguity in current regulations [26] when it comes to describing what constitutes a valid explanation [45].

Original: Fairwashing is clearly an ethical issue as individuals whohave unfairly received unfavorable decisions are deprived of a chance to contest the decision [3].
Revised: Fairwashing is clearly an ethical issue as individuals who have unfairly received unfavorable decisions are deprived of a chance to contest the decision [3].

Original: Due to the growing importance of fairness in machine learning [7], we focus on a scenario in whicha dishonest entity attempts to perform fairwashing [2, 3] when providing model explanations to anauditor ( e.g. , an external dedicated party).
Revised: Due to the growing importance of fairness in machine learning [7], we focus on a scenario in which a dishonest entity attempts to perform fairwashing [2, 3] when providing model explanations to an auditor ( e.g. , an external dedicated party).

Original: For example, if a company’s users expect that the decisionsthey received were a result of model bias, they may send their data and received decisions to anauditor.
Revised: For example, if a company’s users expect that the decisions they received were a result of model bias, they may send their data and received decisions to an auditor.

Original: The auditor may make a legal demand [8] on the company to provide an interpretable modelto explain these users’ received decisions (see Figure 1).
Revised: The auditor may make a legal demand [8] on the company to provide an interpretable model to explain these users’ received decisions (see Figure 1).

Original: In this case, a dishonest company hasan incentive to manipulate the black-box model explanation technique, to under-report and occludethe unfairness of the black-box model under scrutiny to evade the potential consequences of unfairdecision-making.
Revised: In this case, a dishonest company has an incentive to manipulate the black-box model explanation technique, to under-report and occlude the unfairness of the black-box model under scrutiny to evade the potential consequences of unfair decision-making.

Original: The dishonest entity could thus mislead the auditor by rationalizing its decisionsusing the corresponding fairwashed interpretable model.
Revised: The dishonest entity could thus mislead the auditor by rationalizing its decisions using the corresponding fairwashed interpretable model.

Original: In this paper, we are the first to propose amethod for detecting such dishonest model explanations.
Revised: In this paper, we are the first to propose a method for detecting such dishonest model explanations.

Original: This is a challenging problem.
Revised: This is a challenging problem.

Original: Fairwashingcannot be detected through computing the explanation violation of the explainable model, such asfidelity of explainable models, as we demonstrate that the explainable model cannot achieve perfectfidelity with respect to the black-box model.
Revised: Fairwashing cannot be detected through computing the explanation violation of the explainable model, such as fidelity of explainable models, as we demonstrate that the explainable model cannot achieve perfect fidelity with respect to the black-box model.

Original: In addition to this, measuring the differences betweenthe fairness of the black-box model and the explainable model cannot help to detect fairwashing asthere are several design choices (including fairwashing) that could lead to different fairness, thusnon-intentional fairwashing.
Revised: In addition to this, measuring the differences between the fairness of the black-box model and the explainable model cannot help to detect fairwashing as there are several design choices (including fairwashing) that could lead to different fairness, thus non-intentional fairwashing.

Original: Our primary contributions include (1) an extensive theoretical analysis of fairwashing and the fairnesslimitations in explanation techniques and (2) a novel approach for detecting, and thus deterring,fairwashing.
Revised: Our primary contributions include (1) an extensive theoretical analysis of fairwashing and the fairness limitations in explanation techniques and (2) a novel approach for detecting, and thus deterring, fairwashing.

Original: Our proposed method, FRAUD-Detect (FaiRness AUDit Detection), formulates theproblem of detecting fairwashing as a non-cooperative test; both to overcome possible dishonestbehaviors in entities that are actively being audited and for communication efficiency.
Revised: Our proposed method, FRAUD-Detect (FaiRness AUDit Detection), formulates the problem of detecting fairwashing as a non-cooperative test; both to overcome possible dishonest behaviors in entities that are actively being audited and for communication efficiency.

Original: FRAUDDetect operates in a realistic scenario: FRAUD-Detect does not require access to the black-boxmodel provided the users calling for the audit have provided their data and received decisions, which isa realistic setting as often the model is not provided by the entity due to intellectual property and tradesecret concerns [39, 37].
Revised: FRAUDDetect operates in a realistic scenario: FRAUD-Detect does not require access to the black-box model provided the users calling for the audit have provided their data and received decisions, which is a realistic setting as often the model is not provided by the entity due to intellectual property and tradesecret concerns [39, 37].

Original: Thus, FRAUD-Detect only relies on the predictions of the interpretable and black-box models.
Revised: Thus, FRAUD-Detect only relies on the predictions of the interpretable and black-box models.

Original: Both our theoretical and empirical analysis quantify the per-subpopulation,per-label fidelity of the interpretable model with respect to the black-box model.
Revised: Both our theoretical and empirical analysis quantify the per-subpopulation, per-label fidelity of the interpretable model with respect to the black-box model.

Original: Figure 1 shows confusion matrix distributions for two sensitive subpopulations before and afterfairwashing.
Revised: Figure 1 shows confusion matrix distributions for two sensitive subpopulations before and after fairwashing.

Original: These matrices are constructed by comparing the interpretable to the black-box modelpredictions.
Revised: These matrices are constructed by comparing the interpretable to the black-box model predictions.

Original: Fairwashing induces a divergence between subpopulation values to conceal the unfairness of the black-box model.
Revised: Fairwashing induces a divergence between subpopulation values to conceal the unfairness of the black-box model.

Original: FRAUD-Detect leverages the Kullback–Leibler (KL) divergencebetween subpopulation confusion matrices to distinguish between honest and fairwashed interpretablemodels.
Revised: FRAUD-Detect leverages the Kullback–Leibler (KL) divergence between subpopulation confusion matrices to distinguish between honest and fairwashed interpretable models.

Original: For a comprehensive empirical study, we examine logistic regressors and decision treesas interpretable models with respect to black-box Deep Neural Networks, AdaBoost [24], Gradient
Revised: For a comprehensive empirical study, we examine logistic regressors and decision trees as interpretable models with respect to black-box Deep Neural Networks, AdaBoost [24], Gradient

Original: Boosted Decision Trees [17] and Random Forests [13] on three benchmark datasets: COMPAS [5],
Revised: Boosted Decision Trees [17] and Random Forests [13] on three benchmark datasets: COMPAS [5],

Original: Adult Income [22] and Bank Marketing [34].
Revised: Adult Income [22] and Bank Marketing [34].

Original: We evaluate the strength of our detector in the presence of an informed adversary representing aninformed dishonest entity with knowledge of FRAUD-Detect that seeks to fairwash and bypassdetection by constraining the divergence over subpopulation-wise confusion matrices.
Revised: We evaluate the strength of our detector in the presence of an informed adversary representing an informed dishonest entity with knowledge of FRAUD-Detect that seeks to fairwash and bypass detection by constraining the divergence over subpopulation-wise confusion matrices.

Original: Our empiricalexperiments quantify the capacity of this informed adversary by computing the achievable fairnessgap and fidelity under the additional detection constraint.
Revised: Our empirical experiments quantify the capacity of this informed adversary by computing the achievable fairness gap and fidelity under the additional detection constraint.

Original: We provide additional theoretical supportfor the challenge of jointly satisfying fairwashing and evasion in appendices.
Revised: We provide additional theoretical support for the challenge of jointly satisfying fairwashing and evasion in appendices.

Original: In summary, our main contributions are as follows:
Revised: In summary, our main contributions are as follows:

Original: • We characterize fairwashing via a theoretical analysis on the difference in subpopulation gapbetween the black-box model and the interpretable model, fairwashing violation (Section 3).
Revised: • We characterize fairwashing via a theoretical analysis on the difference in subpopulation gap between the black-box model and the interpretable model, fairwashing violation (Section 3).

Original: • Based on our fairwashing theory, we establish requirements for a sufficient fairwashingdetection method and prove that fairwashing is impossible to avoid completely due to anirreducible component in the violation term.
Revised: • Based on our fairwashing theory, we establish requirements for a sufficient fairwashing detection method and prove that fairwashing is impossible to avoid completely due to an irreducible component in the violation term.

Original: • We introduce the first method for fairwashing detection (Section 4.2).
Revised: • We introduce the first method for fairwashing detection (Section 4.2).

Original: Informed by our theoretical results, we observe that fairwashing causes disparate effect on the subpopulation-wisefidelity distributions of the interpretable model with respect to the black-box predictions.
Revised: Informed by our theoretical results, we observe that fairwashing causes disparate effect on the subpopulation-wise fidelity distributions of the interpretable model with respect to the black-box predictions.

Original: Weleverage this observation to propose a non-cooperative black-box access method, FRAUDDetect , which utilizes KL divergence on per-subpopulation confusion matrices to distinguishbetween fairwashed and honest interpretable 
Revised: We leverage this observation to propose a non-cooperative black-box access method, FRAUDDetect , which utilizes KL divergence on per-subpopulation confusion matrices to distinguish between fairwashed and honest interpretable models.

Original: models.• Our empirical results demonstrate that FRAUD-Detect successfully detects fairwashingbased on the KL divergence between subpopulation-wise confusion matrices (Section 6).We show that the divergence over subpopulation confusion matrices can vary by over 0 . 6between an honest and a fairwashed interpretable models.•
Revised: • Our empirical results demonstrate that FRAUD-Detect successfully detects fairwashing based on the KL divergence between subpopulation-wise confusion matrices (Section 6). We show that the divergence over subpopulation confusion matrices can vary by over 0 .between an honest and a fairwashed interpretable models.

Original:  We illustrate the robustness of FRAUD-Detect against an informed adversary ( i
Revised: • We illustrate the robustness of FRAUD-Detect against an informed adversary ( i

Original: e. , adishonest entity who attempts to fairwash while evading detection).
Revised: e. , a dishonest entity who attempts to fairwash while evading detection).

Original: Our empirical resultsshow that evading our detector comes at the cost of a significant increase in subpopulationgap, negating fairwashing.
Revised: Our empirical results show that evading our detector comes at the cost of a significant increase in subpopulation gap, negating fairwashing.

Original: Specifically, a dishonest entity that jointly attempts to fairwashwhile evading detection only achieves parity gaps greater than 10 %.
Revised: Specifically, a dishonest entity that jointly attempts to fairwash while evading detection only achieves parity gaps greater than 10 %.

Original: Preliminaries and ProblemFormulation
Revised: Preliminaries and Problem

Original: 2.1 Related Work
Revised: 2.1 Related Work

Original: Fairness.
Revised: Fairness.

Original: Many different formal definitions of algorithmic fairness have been proposed in the machinelearning community [35].
Revised: Many different formal definitions of algorithmic fairness have been proposed in the machine learning community [35].

Original: One of the main challenges of algorithmic fairness is the lack of consensuson a universally applicable fairness definition.
Revised: One of the main challenges of algorithmic fairness is the lack of consensus on a universally applicable fairness definition.

Original: More precisely, it has been formally demonstrated thatmany such measures are incompatible with each other ( i.e. , they cannot be achieved jointly in somesituations) [31].
Revised: More precisely, it has been formally demonstrated that many such measures are incompatible with each other ( i.e. , they cannot be achieved jointly in some situations) [31].

Original: In addition, their differences are also rooted in their underlying philosophical andmoral assumptions [28].
Revised: In addition, their differences are also rooted in their underlying philosophical and moral assumptions [28].

Original: Thus, the choice of a particular fairness metric is usually context dependent.
Revised: Thus, the choice of a particular fairness metric is usually context dependent.

Original: Nonetheless broadly speaking, there are two main families of fairness definitions: individual fairnessand group fairness [23].
Revised: Nonetheless broadly speaking, there are two main families of fairness definitions: individual fairness and group fairness [23].

Original: Individual fairness posits that “similar individuals should be treated similarly”.
Revised: Individual fairness posits that “similar individuals should be treated similarly”.

Original: In contrast, group fairness strives to equalize statistical properties of classification outcomes acrosssubpopulations created by partitioning the population based on a sensitive attribute A such as gender.
Revised: In contrast, group fairness strives to equalize statistical properties of classification outcomes across subpopulations created by partitioning the population based on a sensitive attribute A such as gender.

Original: In our work, we focus on a particular notion of group fairness, demographic parity, also calledstatistical group parity, which refers to observing equal probability of positive label prediction over subpopulations.
Revised: In our work, we focus on a particular notion of group fairness, demographic parity, also called statistical group parity, which refers to observing equal probability of positive label prediction over subpopulations.

Original: One of the reasons we rely on demographic parity as the fairness metric is thattrue labels will not be available in a realistic scenario (especially in fairwashing scenario describedin Section 2.3).
Revised: One of the reasons we rely on demographic parity as the fairness metric is that true labels will not be available in a realistic scenario (especially in fairwashing scenario described in Section 2.3).

Original: In general, the existence of true labels conflicts with the assumption that whenrequesting decisions (or explanations), users (or auditors) do not have access to these true labels [42].
Revised: In general, the existence of true labels conflicts with the assumption that when requesting decisions (or explanations), users (or auditors) do not have access to these true labels [42].

Original: Furthermore, using the predictions of a black-box model as the true labels is paradoxical in thesense that the black-box model is assumed to be unfair.
Revised: Furthermore, using the predictions of a black-box model as the true labels is paradoxical in the sense that the black-box model is assumed to be unfair.

Original: Demographic parity enforces independencebetween the class predicted by the model and inclusion in a particular subpopulation.
Revised: Demographic parity enforces independence between the class predicted by the model and inclusion in a particular subpopulation.

Original: Definition 1 (Demographic parity [15]) .
Revised: Definition 1 (Demographic parity [15]) .

Original: A classifier ˆ Y satisfies demographic parity with respect tosensitive attribute A
Revised: A classifier ˆ Y satisfies demographic parity with respect to sensitive attribute A

Original: if
Revised: if

Original: [Equation]
Revised: [Equation]

Original: Global black-box explanation methods focus on explaining the whole logic of black-box modelsby training an inherently interpretable surrogate model.
Revised: Global black-box explanation methods focus on explaining the whole logic of black-box models by training an inherently interpretable surrogate model.

Original: We refer the interested reader to Appendix Afor the description of other explanation methods.
Revised: We refer the interested reader to Appendix A for the description of other explanation methods.

Original: In terms of abusing explanations via fairwashing,
Revised: In terms of abusing explanations via fairwashing,

Original: Fukuchi et al., similarly to us, try to detect fairwashing but with a different setting [25].
Revised: Fukuchi et al., similarly to us, try to detect fairwashing but with a different setting [25].

Original: Slack et al.shows an alternate method of fairwashing that evades local model explanation techniques in a settingthat is comparable to ours but they do not attempt to detect fairwashing [44].
Revised: Slack et al. shows an alternate method of fairwashing that evades local model explanation techniques in a setting that is comparable to ours but they do not attempt to detect fairwashing [44].

Original: We explore theseadditional fairwashing settings in Appendix B.
Revised: We explore these additional fairwashing settings in Appendix B.

Original: 2.2 Desiderata for Global Black-box Explanations
Revised: 2.2 Desiderata for Global Black-box Explanations

Original: We consider the setting in which an entity learns a complex black-box model B ( · ) on a trainingset ( X Tr , Y Tr , A ) , in which A represents a sensitive attribute.
Revised: We consider the setting in which an entity learns a complex black-box model B ( · ) on a training set ( X Tr , Y Tr , A ) , in which A represents a sensitive attribute.

Original: We maintain the setup from recentfairwashing literature [3, 2] and consider binary classifiers mapping M features into a binary label
Revised: We maintain the setup from recent fairwashing literature [3, 2] and consider binary classifiers mapping M features into a binary label

Original: B : R M (cid:55)→ { 0 , 1 } .
Revised: B : R M (cid:55)→ { 0 , 1 } .

Original: We also consider binary-valued sensitive attributes, A ∈ { 0 , 1 } .
Revised: We also consider binary-valued sensitive attributes, A ∈ { 0 , 1 } .

Original: Note that we do143 not assume that A is necessarily used during the training of B ( · ) .
Revised: Note that we do not assume that A is necessarily used during the training of B ( · ) .

Original: Given user queries, the entity usesthe predictions of B ( · ) as part of a high-stakes decision system.
Revised: Given user queries, the entity uses the predictions of B ( · ) as part of a high-stakes decision system.

Original: Later, a group of these users mayrequest an explicit explanation from the entity due to concerns over improper use of the sensitiveattribute.
Revised: Later, a group of these users may request an explicit explanation from the entity due to concerns over improper use of the sensitive attribute.

Original: Such a scenario frequently arises in high-impact domains, such as bank loans and creditscoring.
Revised: Such a scenario frequently arises in high-impact domains, such as bank loans and credit scoring.

Original: Let X sg be a suing set comprising the unlabeled data of users demanding an explanationfor their particular outcomes (see Figure 1).
Revised: Let X sg be a suing set comprising the unlabeled data of users demanding an explanation for their particular outcomes (see Figure 1).

Original: Consequently, an external auditing entity requires thecompany to provide explanations in terms that are understandable to humans about the predictions ofthe black-box.
Revised: Consequently, an external auditing entity requires the company to provide explanations in terms that are understandable to humans about the predictions of the black-box.

Original: Here, we will assume that the explanation will be provided in the form of a globalexplanation.
Revised: Here, we will assume that the explanation will be provided in the form of a global explanation.

Original: This means that a simple interpretable model I ( · ) will be trained on a dataset X labeledby querying the black-box model B ( · ) trained on X Tr , such that I ( · ) accurately reflects and explainsthe logic of B ( · ) in terms that are understandable to humans.
Revised: This means that a simple interpretable model I ( · ) will be trained on a dataset X labeled by querying the black-box model B ( · ) trained on X Tr , such that I ( · ) accurately reflects and explains the logic of B ( · ) in terms that are understandable to humans.

Original: Formalizing further the auditing desiderata for black-box model explanations, the interpretable model
Revised: Formalizing further the auditing desiderata for black-box model explanations, the interpretable model

Original: I ( · ) must accurately mirror: 1) the output predictions of the black-box model ( i.e. , fidelity) and 2) thefairness (according to a pre-defined metric) of the black-box model.
Revised: I ( · ) must accurately mirror: 1) the output predictions of the black-box model ( i.e. , fidelity) and 2) the fairness (according to a pre

Original: Fidelity.
Revised: Fidelity.

Original: The interpretable model should satisfy the fidelity criterion for any X ∼ D .
Revised: The interpretable model should satisfy the fidelity criterion for any X ∼ D .

Original: Fidelity(defined in Appendix C) is difficult to perfectly achieve and challenging to measure for an auditorover all X ∼ D .
Revised: Fidelity (defined in Appendix C) is difficult to perfectly achieve and challenging to measure for an auditor over all X ∼ D .

Original: Thus, we additionally introduce the notion of empirical fidelity.
Revised: Thus, we additionally introduce the notion of empirical fidelity.

Original: Definition 2 (Empirical Fidelity) .
Revised: Definition 2 (Empirical Fidelity) .

Original: The empirical fidelity on a dataset X = { x i } Ni =1 including N data161 points x
Revised: The empirical fidelity on a dataset X = { x i } Ni =1 including N data points x

Original: i is defined as the relative accuracy of I ( · ) with respect to B ( · ) on X :
Revised: i is defined as the relative accuracy of I ( · ) with respect to B ( · ) on X :

Original: EmpiricalFidelity
Revised: [Equation] B

Original: [Equation]
Revised: [Equation]

Original: (1) where the indicator 1 ( · , · ) outputs 1 if the output label of I ( · ) and B ( · ) are the same and 0 otherwise.
Revised: where the indicator 1 ( · , · ) outputs 1 if the output label of I ( · ) and B ( · ) are the same and 0 otherwise.

Original: Fairness.
Revised: Fairness.

Original: As the audit of B ( · ) is expressly requested due to fairness concerns with respect to A , theinterpretable model should reflect the fairness violations or adherences of the black-box model.
Revised: As the audit of B ( · ) is expressly requested due to fairness concerns with respect to A , the interpretable model should reflect the fairness violations or adherences of the black-box model.

Original: 2.3 Fairwashing Definitions
Revised: 2.3 Fairwashing Definitions

Original: To evade legal consequences of decision-making based on improper use of a sensitive attribute, adishonest company may perform fairwashing .
Revised: To evade legal consequences of decision-making based on improper use of a sensitive attribute, a dishonest company may perform fairwashing .

Original: Fairwashing with respect to a pre-defined fairness metric permits the dishonest company to learn an interpretable model that hides the unfair behaviourof their black-box model.
Revised: Fairwashing with respect to a pre-defined fairness metric permits the dishonest company to learn an interpretable model that hides the unfair behaviour of their black-box model.

Original: In practice, fairwashing could also occur due to the fact that currentregulations [26] do not define what constitutes a valid explanation , thus leaving the possibility for themodel provider to choose the interpretable model that meets their needs [45].
Revised: In practice, fairwashing could also occur due to the fact that current regulations [26] do not define what constitutes a valid explanation , thus leaving the possibility for the model provider to choose the interpretable model that meets their needs [45].

Original: Fairwashing can bequantified with respect to a particular fairness measure.
Revised: Fairwashing can be quantified with respect to a particular fairness measure.

Original: Definition 3 (Fairwashing) .
Revised: Definition 3 (Fairwashing) .

Original: Let Γ I and Γ B define the fairness gaps of the interpretable model andblack-box model, respectively, with respect to a pre-defined fairness metric.
Revised: Let Γ I and Γ B define the fairness gaps of the interpretable model and black-box model, respectively, with respect to a pre-defined fairness metric.

Original: For example, in terms ofdemographic parity (Definition 1):
Revised: For example, in terms of demographic parity (Definition 1):

Original: [Equation]
Revised: [Equation]

Original: We define the fairwashing violation γ > 0 as the difference between Γ I and Γ B :
Revised: We define the fairwashing violation γ > 0 as the difference between Γ I and Γ B :

Original: [Equation]
Revised: [Equation]

Original: Assumption 1.
Revised: Assumption 1.

Original: The fairwashing violation γ is non-negative.
Revised: The fairwashing violation γ is non-negative.

Original: Remark that the case of γ < 0 is theopposite of fairwashing since the interpretable model is displaying a bigger fairness gap than that ofthe black-box model, thus a model owner has no incentive to employ such an interpretable model.
Revised: Remark that the case of γ < 0 is the opposite of fairwashing since the interpretable model is displaying a bigger fairness gap than that of the black-box model, thus a model owner has no incentive to employ such an interpretable model.

Original:  Fairness Gap of the Black-Box Breeds Fairwashing
Revised:  Fairness Gap of the Black-Box Breeds Fairwashing

Original: We analyze whether it is possible to eliminate the risk of fairwashing altogether.
Revised: We analyze whether it is possible to eliminate the risk of fairwashing altogether.

Original: We characterizethe fairwashing violation γ and show that completely eliminating fairwashing is impossible: aninterpretable model explaining an unfair black-box model always has a non-zero fairwashing violation.
Revised: We characterize the fairwashing violation γ and show that completely eliminating fairwashing is impossible: an interpretable model explaining an unfair black-box model always has a non-zero fairwashing violation.

Original: Theorem 1.
Revised: Theorem 1.

Original: Assume ˆ Y B are the black-box model B ( · ) predictions, ˆ Y
Revised: Assume ˆ Y B are the black-box model B ( · ) predictions, ˆ Y

Original: I are the predictions of aninterpretable model I ( · ) explaining B ( · )  and A is a sensitive attribute: 1) If B ( · ) does not satisfy186 demographic parity, completely eliminating fairwashing is impossible (i.e., γ > 0 ); 2) A detectorfor fairwashing measuring false-positive rates and true-positive rates of I ( · ) with respect to B ( · ) issufficient.
Revised: I are the predictions of a surrogate (interpretable) model I ( · ) trained on B ( · ) outputs and A is a sensitive attribute: 1) If B ( · ) does not satisfy demographic parity, completely eliminating fairwashing is impossible (i.e., γ > 0 ); 2) A detector for fairwashing measuring false-positive rates and true-positive rates of I ( · ) with respect to B ( · ) is sufficient.

Original: We provide a proof sketch here (see Appendix D for full proof).
Revised: We provide a proof sketch here (see Appendix D for full proof).

Original: Proof Sketch.
Revised: Proof Sketch.

Original: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which191 D a denotes the distribution over data with attribute a ∈ A .
Revised: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which D a denotes the distribution over data with attribute a ∈ A .

Original: Define F + a , the false-positive rate of I ( · )with respect to B ( · ) on X ∼ D a .
Revised: Define F + a , the false-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a .

Original: Denote the differences between T +0 and T +1 , and F +0 and F +1 as˜ δ, δ ′ , respectively.
Revised: Denote the differences between T +0 and T +1 , and F +0 and F +1 as ˜ δ, δ ′ , respectively.

Original: We can express Γ I in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B | A by expanding termsand using Bayes’ formula:
Revised: We can express Γ I in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B | A by expanding terms and using Bayes’ formula:

Original: [Equation]
Revised: [Equation]

Original: Now that we have derived Γ I in terms of our desired terms, we can eliminate Γ I from the equationfor γ .
Revised: Now that we have derived Γ I in terms of our desired terms, we can eliminate Γ I from the equation for γ .

Original: The resulting formula for γ in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B |
Revised: The resulting formula for γ in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B |

Original: A allows us to demonstrate197 correctness , irreducibility and sufficiency .
Revised: A allows us to demonstrate correctness , irreducibility and sufficiency .

Original: [Equation]
Revised: [Equation]

Original: Correctness.
Revised: Correctness.

Original: Note that if Γ B = 0 ( i.e. , black-box model is fair) then no-fairwashing is present, andno detector should mistakenly report fairwashing.
Revised: Note that if Γ B = 0 ( i.e. , black-box model is fair) then no-fairwashing is present, and no detector should mistakenly report fairwashing.

Original: We can verify this with Equation (13).
Revised: We can verify this with Equation (15).

Original: Under
Revised: Under

Original: Assumption 1, we know that γ ≥ 0 and as below terms are, by definition non-negative, thus:
Revised: Assumption 1, we know that γ ≥ 0 and as below terms are, by definition non-negative, thus:

Original: [Equation]
Revised: [Equation]

Original: Irreducibility.
Revised: Irreducibility.

Original: If Γ B ̸ = 0 , even if ˜ δ = δ ′ = 0 , i.e. when true-positive and false-positive rates are equalover subpopulations ( T [-+ 
-]
Revised: If Γ B ̸ = 0 , even if ˜ δ = δ ′ = 0 , i.e. when true-positive and false-positive rates are equal over subpopulations ( T [-+ 
-]+0 =

Original: = T + 1 and F + 0 = 
Revised: T +1 and F +0 = F +1 )

Original: F + 1 ), there exists an irreducible fairwashing violation:
Revised: , there exists an irreducible fairwashing violation:

Original: [Equation]
Revised: [Equation]

Original:  since 1 + F +1 − T +1 > 0 except in trivial cases (see Appendix D for the proof).
Revised: ) since 1 + F +1 − T +1 > 0 except in trivial cases (see Appendix D for the proof).

Original: We further pointout that 1 + F +1 − T +1 is not a function of disparity in the interpretable model as the value does not measure disparity between subpopulations.
Revised: We further point out that 1 + F + 1 − T + 1 is not a function of disparity in the interpretable model as the value does not measure disparity between subpopulations.

Original: Rather, 1 + F +1 − T +1 is a function of the differencein true-positive and false-positive rates ( T +1 and F +1 , respectively) within a single subpopulation.
Revised: Rather, 1 + F +1 − T +1 is a function of the difference in true-positive and false-positive rates ( T +1 and F +1 , respectively) within a single subpopulation.

Original: The sole measure of disparity in the gap above is black-box Γ B , which is a given constant.
Revised: The sole measure of disparity in the gap above is black-box Γ B , which is a given constant.

Original: Sufficiency.
Revised: Sufficiency.

Original: The remaining factors of the gap are only function of ˜ δ and δ ′ as Pr (cid:104)
Revised: The remaining factors of the gap are only function of ˜ δ and δ ′ as Pr (cid:104)

Original: ˆ Y B =
Revised: ˆ Y B =

Original: 1 | A = 0 (cid:105)and Pr (cid:104)
Revised: 1 | A = 0 (cid:105) and Pr (cid:104)

Original: ˆ Y B =
Revised: ˆ Y B =

Original: 1 | A = 0 (cid:105) are also given constants.
Revised: 1 | A = 0 (cid:105) are also given constants.

Original: Therefore, a detector that measures ˜ δ and δ ′ issufficient to detect fairwashing.
Revised: Therefore, a detector that measures ˜ δ and δ ′ is sufficient to detect fairwashing.

Original:  Detecting Fairwashing 
Revised:  Detecting Fairwashing 4.1 Fairwashing In Practice

Original: 4.1 Fairwashing In Practice

Original: Eliminating the fairness gap of the black-box model would be the ideal outcome as it would alsoremove the possibility for a dishonest model provider to perform fairwashing.
Revised: Eliminating the fairness gap of the black-box model would be the ideal outcome as it would also remove the possibility for a dishonest model provider to perform fairwashing.

Original: However, in practiceabsolute fairness is often unrealistic and even regulatory bodies tolerate small violations of fairnesslaws [18].
Revised: However, in practice absolute fairness is often unrealistic and even regulatory bodies tolerate small violations of fairness laws [18].

Original: In Section 3, we showed that fairness violations result in fairwashing violations unless theinterpretable model has a higher fairness gap than the black-box model, which would not be desirablefrom the point of view of the model provider.
Revised: In Section 3, we showed that fairness violations result in fairwashing violations unless the interpretable model has a higher fairness gap than the black-box model, which would not be desirable from the point of view of the model provider.

Original: Therefore, not all fairwashing violations may have beenintended by the model owners.
Revised: Therefore, not all fairwashing violations may have been intended by the model owners.

Original: These involuntary violations are in fact side-effects of the imperfectfidelity, which is inherent to optimizing the empirical fidelity (Definition 2) of the interpretable modelon a training set
Revised: These involuntary violations are in fact side-effects of the imperfect fidelity, which is inherent to optimizing the empirical fidelity (Definition 2) of the interpretable model on a training set

Original: X tr ∼ D ,max
Revised: X tr ∼ D ,

Original: EmpiricalFidelity ( I, B ; X tr ) .
Revised: [Equation]

Original: In contrast, deliberate fairwashing is the result of a similar optimization problem purposefullyconstrained to provide better fairness on a target suing set X sg ∼ D :
Revised: In contrast, deliberate fairwashing is the result of a similar optimization problem purposefully constrained to provide better fairness on a target suing set X sg ∼ D :

Original: Definition 4 (Fairwashing Optimization [2]) .
Revised: Definition 4 (Fairwashing Optimization [2]) .

Original: Given a black-box model B ( · ) and a suing set X sg ,fairwashing optimization is defined as learning an interpretable model I ( · ) from B ( · ) on X sg such226 that the interpretable model has 1) high fidelity with respect to the black-box model and 2) is lessunfair than this black-box model:[Equation] subject
Revised: Given a black-box model B ( · ) and a suing set X sg , fairwashing optimization is defined as learning an interpretable model I ( · ) from B ( · ) on X sg such that the interpretable model has 1) high fidelity with respect to the black-box model and 2) is less unfair than this black-box model:

Original: [Equation]
Revised: [Equation]

Original: in which ϵ is an upper bound on the fairness gap of the interpretable model on the suing set
Revised: in which ϵ is an upper bound on the fairness gap of the interpretable model on the suing set

Original: FairnessGap ( I ; X sg ) , and FairnessGap ( · , · ) constitutes a measure of fairness gap according to230 some fairness metric – generically measured by the subpopulation gap in a fairness metric.
Revised: FairnessGap ( I ; X sg ) , and FairnessGap ( · , · ) constitutes a measure of fairness gap according to some fairness metric – generically measured by the subpopulation gap in a fairness metric.

Original: Note thatϵ <
Revised: Note that ϵ <

Original: FairnessGap ( B ; X sg ) must be less than the fairness gap of the black-box model on the suingset FairnessGap ( B ; X sg ) to cause fairwashing violation γ > 0{+
+} (see Definition 3).
Revised: FairnessGap ( B ; X sg ) must be less than the fairness gap of the black-box model on the suing set FairnessGap ( B ; X sg ) to cause fairwashing violation γ > {+
+} (see Definition 3).

Original: 4.2 Proposed method: FRAUD-Detect
Revised: 4.2 Proposed method: FRAUD-Detect

Original: As demonstrated in Section 3, a fairwashing detection method that relies on the difference oversubpopulations in true-positive and false-positive rates of the interpretable model w.r.t the blackbox model ( ˜ δ and δ ′ respectively)
Revised: As demonstrated in Section 3, a fairwashing detection method that relies on the difference over subpopulations in true-positive and false-positive rates of the interpretable model w.r.t the blackbox model ( ˜ δ and δ ′ respectively)

Original: is sufficient.
Revised: is sufficient.

Original: Inspired by this theoretical analysis, we proposeour fairwashing detector, FRAUD-Detect , which locates fairwashing violations.
Revised: Inspired by this theoretical analysis, we propose our fairwashing detector, FRAUD-Detect , which locates fairwashing violations.

Original: FRAUD-Detectaims to distinguish between honest and fairwashed interpretable models approximating the unfairblack-box model B ( · ) as described in Section 2.2 under the additional following constraints: 1)without white-box access to the black-box and interpretable models as well as; 2) independently ofthe cooperation of the model provider being audited.
Revised: FRAUD-Detect aims to distinguish between honest and fairwashed interpretable models approximating the unfair black-box model B ( · ) as described in Section 2.2 under the additional following constraints: 1) without white-box access to the black-box and interpretable models as well as; 2) independently of the cooperation of the model provider being audited.

Original: These constraints are introduced to conform torealistic scenarios in which entities desire to retain the confidentiality of their black-box model due tointellectual property rights and because they considered it as a company’s asset.
Revised: These constraints are introduced to conform to realistic scenarios in which entities desire to retain the confidentiality of their black-box model due to intellectual property rights and because they considered it as a company’s asset.

Original: Similarly to definitions of true-positive T + a and false-positive F + a rates of I ( · ) w.r.t B ( · ) on datawith attribute a ∈ A= { 0 , 1 } , we define the true-negative rate, T − a , and false-negative rate, F − a as: T − [Equation] =
Revised: Similarly to definitions of true-positive T + a and false-positive F + a rates of I ( · ) w.r.t B ( · ) on data with attribute a ∈ A

Original: Pr[

Original: [Equation]

Original: |

Original: [Equation]

Original: Y

Original: [Equation] =  0 , A = a] ,  F [Equation] =
Revised: = { 0 , 1 } , we define the true-negative rate, T − a , and false-negative rate, F − a as:

Original: Pr

Original: [Equation]
Revised: [Equation]

Original: ˆ Y

Original: [Equation] = [Equation] | ˆ Y B = 1 , A = a ] for a ∈ A = { 0 , 1 } .
Revised: (8) for a ∈ A = { 0 , 1 } .

Original: Note247 that T − a = 1 − F + a and F − a = 1 − T + a .
Revised: Note that T − a = 1 − F + a and F − a = 1 − T + a .

Original: FRAUD-Detect computes the T + a , F + a , T − a , F − a for each subpopulation
Revised: FRAUD-Detect computes the T + a , F + a , T − a , F − a for each subpopulation

Original: a ∈ A = { 0 , 1 } using the predictions of the interpretable model and black-box model on the suing set
Revised: a ∈ A = { 0 , 1 } using the predictions of the interpretable model and black-box model on the suing set

Original: X sg .
Revised: X sg .

Original: Let
Revised: Let

Original: [-C 
 = [ T +0 , F +0 , T − 0 , F − 0 ] and C 1 = [ T +1 , F +1 , T − 1 , F − 1 ] be the (flattened) confusion matrices of the-]
Revised: [-C 
 = [ T +0 , F +0 , T − 0 , F − 0 ] and C 1 = [ T +1 , F +1 , T − 1 , F − 1 ] be the (flattened) confusion matrices of the-][Equation]

Original: Input: Query access to the interpretable model

Original: I ( · ) , suing dataset X sg , Black-box model predictions on the suing set B ( X sg ) , sensitive attributea ∈ A and a threshold ∆ > 0 . Output: (cid:26) T fairwashing is detected F fairwashing is not detected 1: ˆ Y I ← I ( X sg )
Revised: be the (flattened) confusion matrices of the interpretable model w.r.t the black-box model on X sg with attribute 0 and 1 , respectively.

Original: ▷ Query interpretable model 2: for i ← 0 , 1 do 3:

Original: T + i ← |{ ˆ y B =1 , ˆ y I

Original: =1 ,a = i }| |{ ˆ y B

Original: =1 ,a = i }| ▷ TPR 4:

Original: F + i ← |{ ˆ y B =0 ,

Original: ˆ y I =1 ,a = i }| |{ ˆ y B

Original: =0 ,a = i }| ▷ FPR 5:

Original: F −

Original: [Table]

Original: ← |{

Original: [Table]

Original: y

Original: [Table]

Original: =

Original: [Table]

Original: [Table]

Original: =

Original: [Table]

Original: [Table]

Original: = i }| |{

Original: [Table]

Original: y

Original: [Table]

Original: =

Original: [Table]

Original: [Table]

Original: =

Original: [Table]

Original: | ▷ FNR 6:

Original: T

Original: [Table]

Original: ← |{

Original: [Table]

Original: =

Original: [Table]

Original: [Table]

Original: =

Original: [Table]

Original: [Table]

Original: =

Original: [Table]

Original: }| |{

Original: [Table]

Original: y

Original: [Table]

Original: =

Original: [Table]

Original: ,a = i }

Original: [Table]

Original: ▷ TNR 7: C 0 ← [ T +0 , F +0 , T − 0 , F − 0 ] 8: C 1 ←

Original: [ T +1 , F +1 , T − 1 , F − 1 ] 9: C KL ← KL ( C 0 , C 1 ) ▷ Kullback–Leibler 10: if C KL > ∆ then 11: return T 12: else 13: return F interpretable model w t on

Original:  FRAUD-Detect computes the divergence between C  and C 1 using Kullback Leibler ( ) as:[Equation]
Revised: Then, FRAUD-Detect computes the divergence between C 0 and Cusing Kullback–Leibler (KL) as:

Original: [Equation]
Revised: [Equation]

Original: Recall that we demonstrated in Section 3 that the divergence in T +0 and T +1 ( ˜ δ ) and F +0 and F +1 ( δ ′ )were sufficient for a detection method.
Revised: Recall that we demonstrated in Section 3 that the divergence in T +0 and T +1 ( ˜ δ ) and F +0 and F +1 ( δ ′ ) were sufficient for a detection method.

Original: We can complement 1 our detection method with additionaldivergences in T − a and F − a and measure the dissimilarity via KL divergence.
Revised: We can complement 2 our detection method with additional divergences in T − a and F − a and measure the dissimilarity via KL divergence.

Original: This choice is naturalas the KL divergence is commonly used to quantify the divergence over probability distributions—
Revised: This choice is natural as the KL divergence is commonly used to quantify the divergence over probability distributions—

Original: C a functions as a simplified representation of the probability distribution Y I | Y B , A = a .
Revised: C a functions as a simplified representation of the probability distribution Y I | Y B , A = a .

Original: For anhonest interpretable model, C KL is generally relatively low ( ˜ δ, δ ′ ≈ 0 ) as the only divergence arisesfrom general error in fidelity optimization and the fairness gap of the black-box model (as shownin Section 3).
Revised: For an honest interpretable model, C KL is generally relatively low ( ˜ δ, δ ′ ≈ 0 ) as the only divergence arises from general error in fidelity optimization and the fairness gap of the black-box model (as shown in Section 3).

Original: In other words, the honest interpretable model approximates the black-box modelequivalently across subpopulations.
Revised: In other words, the honest interpretable model approximates the black-box model equivalently across subpopulations.

Original: However, for a fairwashed interpretable model, C KL growssignificantly, as the interpretable model is explicitly manipulated to optimize the fairness acrosssubpopulations and improve over the black-box model.
Revised: However, for a fairwashed interpretable model, C KL grows significantly, as the interpretable model is explicitly manipulated to optimize the fairness across subpopulations and improve over the black-box model.

Original: We quantify the distinction between honestand fairwashed interpretable models via a threshold ∆ > 0 on C KL such that if C KL > ∆ , the265 interpretable model is considered fairwashed.
Revised: We quantify the distinction between honest and fairwashed interpretable models via a threshold ∆ > 0 on C KL such that if C KL > ∆ , the interpretable model is considered fairwashed.

Original: Note here that due to the irreducibility result statedin Section 3, ∆ must be chosen such that ∆ is tightly greater than the irreducible term in order toproperly distinguish between accidental fairwashing that occurs as a result of the irreducibility andmalicious fairwashing.
Revised: Note here that due to the irreducibility result stated in Section 3, ∆ must be chosen such that ∆ is tightly greater than the irreducible term in order to properly distinguish between accidental fairwashing that occurs as a result of the irreducibility and malicious fairwashing.

Original: This procedure is detailed in Algorithm 1.
Revised: This procedure is detailed in Algorithm 1.

Original:  Evading FRAUD-Detect
Revised:  Evading FRAUD-Detect

Original: In this section, we investigate on whether a dishonest entity could evade FRAUD-Detect whileperforming fairwashing.
Revised: In this section, we investigate on whether a dishonest entity could evade FRAUD-Detect while performing fairwashing.

Original: We assume that this dishonest entity, which we call the adversary , isinformed about FRAUD-Detect .
Revised: We assume that this dishonest entity, which we call the adversary , is informed about FRAUD-Detect .

Original: Namely, the adversary is aware of the fairwashing detection methodand desires to fairwash while evading detection by the auditor.
Revised: Namely, the adversary is aware of the fairwashing detection method and desires to fairwash while evading detection by the auditor.

Original: To achieve this goal, the informedadversary has the objective of finding a fairwashed interpretable model that satisfies an additionalconstraint on the Kullback–Leibler (KL) divergence of the confusion matrices among subpopulations,
Revised: To achieve this goal, the informed adversary has the objective of finding a fairwashed interpretable model that satisfies an additional constraint on the Kullback–Leibler (KL) divergence of the confusion matrices among subpopulations,

Original: C KL in the previous section.
Revised: C KL in the previous section.

Original: To probe the robustness of our detector to an informed adversaryempirically, we explore the range of fairness gap given a fixed value of fidelity and a fixed value of
Revised: To probe the robustness of our detector to an informed adversary empirically, we explore the range of fairness gap given a fixed value of fidelity and a fixed value of

Original: C KL via solving the informed adversary optimization problem.
Revised: C KL via solving the informed adversary optimization problem.

Revised: Input: Query access to the interpretable model

Revised: I ( · ) , suing dataset X sg , Black-box model predictions on the suing set B ( X sg ) , sensitive attribute a ∈ A and a threshold ∆ > 0 .

Revised: Output: (cid:26) T fairwashing is detected

Revised: F fairwashing is not detected 1: ˆ Y I ← I

Revised: ( X sg )

Original: Definition 5 (Informed Adversary Optimization) .
Revised: ▷ Query interpretable model 2: for i ← 0 , 1 do 3: T + i ← |{ ˆ y B =1 , ˆ y I =1 ,a = i }| |{ ˆ y B =1 ,a = i }| ▷ TPR 4:

Original: Given a black-box model B ( · ) , a suing set X sg ,a sensitive attribute A , a loss threshold v and a fairwashing detection threshold ∆ , an informedadversary aims to learn an interpretable I ( · ) such that
Revised: F + i ← |{ ˆ y B =0 , ˆ y I =1 ,a = i }| |{ ˆ y B =0 ,a = i }| ▷ FPR 5: F − i ← |{ ˆ y B =1 , ˆ y I =0 ,a = i }| |{ ˆ y B

Original: [Equation]
Revised: =1 ,a = i }| ▷ FNR 6:

Original: to
Revised: T − i ← |{ ˆ y B =0 , ˆ y I =0 ,a = i }| |{ ˆ y B

Original: [Equation] where C KL is the KL divergence between the subpopulation confusion matrices of X sg .
Revised: =0 ,a = i }| ▷ TNR 7: C 0 ← [ T +0 , F +0 , T − 0 , F − 0 ] 8: C 1 ← [ T +1 , F +1 , T − 1 , F − 1 ] 9: C KL ← KL ( C 0 , C 1 ) ▷ Kullback–Leibler 10: if C KL > ∆ then 11: return T 12: else 13: return F

Original: 1 1 − (cid:15) . C K L AB & DT .
Revised: Definition 5 (Informed Adversary Optimization) .

Original: 1 1 − (cid:15) XGB & DT . 1 1 − (cid:15) DNN & DT .
Revised: Given a black-box model B ( · ) , a suing set X sg , a sensitive attribute A , a loss threshold v and a fairwashing detection threshold ∆ , an informed adversary aims to learn an interpretable I ( · ) such that minimize

Original: 1 1 − (cid:15) RF & DT . 1 1 − (cid:15)
Revised: FairnessGap ( I ; X sg ) subject to

Original: AB & LR . 1 1 − (cid:15) XGB & LR .
Revised: L ( I ; X sg ) ≤

Original: 1 1 − (cid:15) DNN & LR . 1 1 − (cid:15) RF & LR .
Revised: [Equation]

Original: P a r i t y ga p o f I ( · )
Revised: where C KL is the KL divergence between the subpopulation confusion matrices of X sg .

Original: The differences between the fairwashing optimization in Definition 4, and the informed adversaryoptimization of Definition 5 are two fold.
Revised: The differences between the fairwashing optimization in Definition 4, and the informed adversary optimization of Definition 5 are two fold.

Original: First, the order of the maximand and the constraint isreversed.
Revised: First, the order of the maximand and the constraint is reversed.

Original: Both formulations are valid since each indicates priorities of the adversary, namely, whetherto put a hard limit on the fidelity loss or on the fairness gap of the interpretable model.
Revised: Both formulations are valid since each indicates priorities of the adversary, namely, whether to put a hard limit on the fidelity loss or on the fairness gap of the interpretable model.

Original: Second, in
Revised: Second, in

Original: Definition 5 we replaced the non-differentiable EmpiricalFidelity ( I, B ; X tr ) with a differentiablemeasure of loss L ( I ; X sg ) .
Revised: Definition 5 we replaced the non-differentiable EmpiricalFidelity ( I, B ; X tr ) with a differentiable measure of loss L ( I ; X sg ) .

Original: In our empirical evaluations, we use the logistic regression loss.
Revised: In our empirical evaluations, we use the logistic regression loss.

Original: To solve the above optimisation problem, we consider the Rashomon set of high-fidelity interpretablemodels and compute the range of the fairness gap of interpretable models.
Revised: To solve the above optimisation problem, we consider the Rashomon set of high-fidelity interpretable models and compute the range of the fairness gap of interpretable models.

Original: In short, given a classification task, the Rashomon Set [41] is defined as the set of almost-equally-accurate models (interpretablemodels in our case).
Revised: In short, given a classification task, the Rashomon Set [41] is defined as the set of almost-equally-accurate models (interpretable models in our case).

Original: More precisely, given a model class F , a loss function L D ( · ) over a dataset D293 of interest, a reference model f ∗ ( e.g. , optimal model) and a performance threshold τ ∈ [0 , 1] , the294 Rashomonset R s (
Revised: More precisely, given a model class F , a loss function L D ( · ) over a dataset D of interest, a reference model f ∗ ( e.g. , optimal model) and a performance threshold τ ∈ [0 , 1] , the Rashomon

Original: F , f ∗ , τ ) = { f ∈ F | L D ( f ) ≤ L D ( f ∗ ) + τ } .
Revised: [Equation]

Original: We extend the Fairness in the Rashomon Set (FaiRS) algorithm [19] to compute the range of thefairness gap of interpretable models that can be generated over the set of high-fidelity modelssatisfying a constraint on the KL divergence.
Revised: We extend the Fairness in the Rashomon Set (FaiRS) algorithm [19] to compute the range of the fairness gap of interpretable models that can be generated over the set of high-fidelity models satisfying a constraint on the KL divergence.

Original: FaiRS exploits the so-called Rashomon Effect [14],which is an empirical phenomenon resulting in multiple models displaying the same performanceoverall ( e.g. , w.r.t their global accuracy) but have significant differences in terms of their individualpredictions [19].
Revised: FaiRS exploits the so-called Rashomon Effect [14], which is an empirical phenomenon resulting in multiple models displaying the same performance overall ( e.g. , w.r.t their global accuracy) but have significant differences in terms of their individual predictions [19].

Original: In our setting, the Rashomon effect implies that several interpretable models canachieve the same high fidelity w.r.t the black-box model while displaying different values of fairnessgap.
Revised: In our setting, the Rashomon effect implies that several interpretable models can achieve the same high fidelity w.r.t the black-box model while displaying different values of fairness gap.

Original: Thus, computing the range of the fairness gap of high-fidelity interpretable models that satisfy aconstraint on C KL quantifies the robustness of FRAUD-Detect to the informed adversary.
Revised: Thus, computing the range of the fairness gap of high-fidelity interpretable models that satisfy a constraint on C KL quantifies the robustness of FRAUD-Detect to the informed adversary.

Original: To the best of our knowledge, the FaiRS algorithm of [19] is the only work proposing a practicalsolution to efficiently characterize the range of the fairness gap over the Rashomon set.
Revised: To the best of our knowledge, the FaiRS algorithm of [19] is the only work proposing a practical solution to efficiently characterize the range of the fairness gap over the Rashomon set.

Original: Alternativeapproaches would require measuring the range of the fairness gap over approximations of the
Revised: Alternative approaches would require measuring the range of the fairness gap over approximations of the

Original: Rashomon set obtained by either generating models using several hyperparameter values ( e.g. ,seeds [33, 21]) or brute-forcing over a particular class of model ( e.g. , depth seven decision trees [41]).
Revised: Rashomon set obtained by either generating models using several hyperparameter values ( e.g. , seeds [33, 21]) or brute-forcing over a particular class of model ( e.g. , depth seven decision trees [41]).

Original: This is not practical due to the high computational cost and may lead to sub-optimal results.
Revised: This is not practical due to the high computational cost and may lead to sub-optimal results.

Original:  Empirical validation
Revised:  Empirical validation

Original: FRAUD-Detect aims to detect fairwashing by distinguishing between a fairwashed interpretablemodel and honest interpretable one during an audit.
Revised: FRAUD-Detect aims to detect fairwashing by distinguishing between a fairwashed interpretable model and honest interpretable one during an audit.

Original: Additionally, FRAUD-Detect seeks to be robusteven against an informed adversary, i.e. , demonstrate the ability to detect fairwashing even when adishonest entity is aware of FRAUD-Detect and attempts evasion.
Revised: Additionally, FRAUD-Detect seeks to be robust 1 1 − (cid:15) . C K L AB & DT .

Revised: 1 1 − (cid:15)

Revised: XGB & DT .

Revised: 1 1 − (cid:15)

Revised: DNN & DT .

Revised: 1 1 − (cid:15)

Revised: RF & DT .

Revised: 1 1 − (cid:15)

Revised: AB & LR .

Revised: 1 1 − (cid:15)

Revised: XGB & LR .

Revised: 1 1 − (cid:15)

Revised: DNN & LR .

Revised: 1 1 − (cid:15)

Revised: RF & LR .

Revised: P a r i t

Revised: y ga p

Original:  Therefore, we empirically validatethe performance of FRAUD-Detect in: 1) capturing a relationship between C KL and demographicparity gap of the interpretable model I ( · ) : C KL increases as the demographic parity gap of theinterpretable model decreases ( i.e. as fairwashing becomes more severe); 2) restricting the fairwashingcapabilities of the informed adversary: incorporation of C KL constraint into fairwashing problemprevents favorable demographic parity gaps for the interpretable model, precluding fairwashing.
Revised: o f I ( · ) even against an informed adversary, i.e. , demonstrate the ability to detect fairwashing even when a dishonest entity is aware of FRAUD-Detect and attempts evasion. Therefore, we empirically validate the performance of FRAUD-Detect in: 1) capturing a relationship between C KL and demographic parity gap of the interpretable model I ( · ) : C KL increases as the demographic parity gap of the interpretable model decreases ( i.e. as fairwashing becomes more severe); 2) restricting the fairwashing capabilities of the informed adversary: incorporation of C KL constraint into fairwashing problem prevents favorable demographic parity gaps for the interpretable model, precluding fairwashing.

Original: We assess the performance of FRAUD-Detect using a diverse set of black-box architectures, interpretable models and datasets.
Revised: We assess the performance of FRAUD-Detect using a diverse set of black-box architectures, interpretable models and datasets.

Original: More precisely, we consider four architectures of black-box models:
Revised: More precisely, we consider four architectures of black-box models:

Original: Deep Neural Networks (DNN), AdaBoost (AB) [24], Gradient Boosted Decision Trees (XGBs) [17]and Random Forests (RFs) [13].
Revised: Deep Neural Networks (DNN), AdaBoost (AB) [24], Gradient Boosted Decision Trees (XGBs) [17] and Random Forests (RFs) [13].

Original: We evaluate the approach on three real-world datasets correspondingto critical decision systems: Adult Income [22], Bank Marketing [34] and COMPAS [5].
Revised: We evaluate the approach on three real-world datasets corresponding to critical decision systems: Adult Income [22], Bank Marketing [34] and COMPAS [5].

Original: We relyon Logistic Regression (LR) and Decision Trees (DTs) as our interpretable models due to theirhigh performance and ease of explainability.
Revised: We rely on Logistic Regression (LR) and Decision Trees (DTs) as our interpretable models due to their high performance and ease of explainability.

Original: We refer to Appendix E for details on these datasets,black-box models, interpretable models and implementation of FRAUD-Detect .[-AB .65 .69 .
 XGB .7 .
 DNN .75 .78 .84 .86 .-]
Revised: We refer to Appendix E for details on these datasets, black-box models, interpretable models and implementation of FRAUD-Detect .[-AB .65 .69 .
 XGB .7 .
 DNN .75 .78 .84 .86 .-]

Original: RF 0 .0 .0 .0 .0 .1 .

Original: Fidelity 0 .0 .0 .0 .1 .

Original: P a r i t

Original: y ga p

Original: o f I ( · ) ∆: .

Original: FRAUD-Detect successfully detects fairwashing.
Revised: FRAUD-Detect successfully detects fairwashing.

Original: Figure 2 illustrates both the demographic paritygap and C KL of interpretable models as a function of ϵ .
Revised: Figure 2 illustrates both the demographic parity gap and C KL of interpretable models as a function of ϵ .

Original: The bigger the value of ϵ , the stronger thefairwashing (see Definition 4).
Revised: The bigger the value of ϵ , the stronger the fairwashing (see Definition 4).

Original: Any graph within the figures may be read from left to right as thedegree of fairwashing increases.
Revised: Any graph within the figures may be read from left to right as the degree of fairwashing increases.

Original: As fairwashing becomes more severe, the demographic parity gapdecreases while C KL increases such that we observe a relationship between the demographic paritygap of interpretable models and the associated C KL values.
Revised: As fairwashing becomes more severe, the demographic parity gap decreases while C KL increases such that we observe a relationship between the demographic parity gap of interpretable models and the associated C KL values.

Original: For example, decreasing the demographicparity gap of Decision Trees trained on a COMPAS dataset labelled by a RF black-box model byhalf, increases C KL by a factor of 4.
Revised: For example, decreasing the demographic parity gap of Decision Trees trained on a COMPAS dataset labelled by a RF black-box model by half, increases C KL by a factor of 4.

Original: The amount of change in C KL and in the demographic paritygap of fairwashed interpretable models differ across the dataset and black-box models due to thedifferent original fairness of the black-box models.
Revised: The amount of change in C KL and in the demographic parity gap of fairwashed interpretable models differ across the dataset and black-box models due to the different original fairness of the black-box models.

Original: As C KL is sensitive to changes in fairwashing,
Revised: As C KL is sensitive to changes in fairwashing,

Original: FRAUD-Detect can be used to detect fairwashing by not permitting C KL to pass a certain thresholdvalue.
Revised: FRAUD-Detect can be used to detect fairwashing by not permitting C KL to pass a certain threshold value.

Original: The C KL thresholds corresponding to 5% and 50% fairwashing are reported in Appendix G.
Revised: The C KL thresholds corresponding to 5% and 50% fairwashing are reported in Appendix G.

Original: Per-seed results are provided in Appendix H.
Revised: Per-seed results are provided in Appendix H.

Original: FRAUD-Detect is robust to the informed adversary.
Revised: FRAUD-Detect is robust to the informed adversary.

Original: Figure 3 shows the range of the demographicparity gap of high-fidelity fairwashed interpretable models subjected to a constraint on the C KL bysolving the optimization problem of Equation (9).
Revised: Figure 3 shows the range of the demographic parity gap of high-fidelity fairwashed interpretable models subjected to a constraint on the C KL by solving the optimization problem of Equation (11).

Original: We use four types of black-box models (AB, XGB, DNN and RF).
Revised: We use four types of black-box models (AB, XGB, DNN and RF).

Original: However, for simplicity, we performed the experiment using logistic regression asthe interpretable model.
Revised: However, for simplicity, we performed the experiment using logistic regression as the interpretable model.

Original: The fairwashing detection thresholds were chosen from a wide range, i.e.∆ ∈ { 0 . 01 , 0 .
Revised: The fairwashing detection thresholds were chosen from a wide range, i.e. ∆ ∈ { 0 . 01 , 0 .

Original: 10 , 0 . 20 } (the constraint on C KL in Equation (9)).
Revised: 10 , 0 . 20 } (the constraint on C KL in Equation (11)).

Original: A direct implication347 for FRAUD-Detect is that using low fairwashing detection thresholds makes fairwashing difficult.
Revised: A direct implication for FRAUD-Detect is that using low fairwashing detection thresholds makes fairwashing difficult.

Original: Consistently over all these results, imposing a constraint on the C KL significantly narrows the rangeof demographic parity gap of fairwashed interpretable models.
Revised: Consistently over all these results, imposing a constraint on the C KL significantly narrows the range of demographic parity gap of fairwashed interpretable models.

Original: For instance, on COMPAS with∆ = 0 . 2 , for an AB model that has demographic parity gap of 0 . 214 , fairwashing can producea completely fair ( i.e. , demographic parity gap of 0 . 0 ) high-fidelity interpretable model whileremaining undetected ( i.e. , C KL < 0 . 2 ).
Revised: For instance, on COMPAS with ∆ = 0 . 2 , for an AB model that has demographic parity gap of 0 . 214 , fairwashing can produce a completely fair ( i.e. , demographic parity gap of 0 . 0 ) high-fidelity interpretable model while remaining undetected ( i.e. , C KL < 0 . 2 ).

Original: However, when C KL decreases to 0 .
Revised: However, when C KL decreases to 0 .

Original: 1 , the fairest high-fidelityinterpretable model obtained by the adversary exhibits a demographic parity gap of 0 . 107 . {+AB .65 .69 .
 XGB .7 .
+}
Revised: 1 , the fairest high-fidelity interpretable model obtained by the adversary exhibits a demographic parity gap of 0 . 107 . {+AB .65 .69 .
 XGB .7 .
+}

Revised: DNN .75 .78 .84 .86 .

Revised: RF 0 .0 .0 .0 .0 .1 .

Revised: Fidelity 0 .0 .0 .0 .1 .

Original:   Conclusion and Future Work
Revised: P a r i t y ga p o f I ( · ) ∆: .  Conclusion and Future Work

Original: Incorporating fairness and explainability in decision systems can help foster trust [29] in them bycharacterizing failure modes ( e.g. , unfairness) and providing assurances ( e.g. , fairness constraints)
Revised: Incorporating fairness and explainability in decision systems can help foster trust [29] in them by characterizing failure modes ( e g. , unfairness) and providing assurances ( e.g. , fairness constraints)

Original: Auditing provides an oversight component and helps avoid first-order failures such as unfairness.
Revised: Auditing provides an oversight component and helps avoid first-order failures such as unfairness.

Original: Inthis sense, fairwashing can be seen as failure of auditing—a second-order failure that reduces trustin both fairness and explainability.
Revised: In this sense, fairwashing can be seen as failure of auditing—a second-order failure that reduces trust in both fairness and explainability.

Original: Second-order failures help characterize the limits of techniquesdesigned to reduce the risks of first-order failures.
Revised: Second-order failures help characterize the limits of techniques designed to reduce the risks of first-order failures.

Original: Our work is the first to delineate the theoreticallimits of what is possible in auditing fairness and in containing the risk of fairwashing.
Revised: Our work is the first to delineate the theoretical limits of what is possible in auditing fairness and in containing the risk of fairwashing.

Original: In practice,we demonstrated the possibility of performing fairwashing through joint optimization of fidelity andfairness constraints when providing explanations.
Revised: In practice, we demonstrated the possibility of performing fairwashing through joint optimization of fidelity and fairness constraints when providing explanations.

Original: This could thwart auditing processes that are solelybased on an analysis of the delivered interpretable model.
Revised: This could thwart auditing processes that are solely based on an analysis of the delivered interpretable model.

Original: We address this issue by proposing anadditional auditing protocol that queries the interpretable model.
Revised: We address this issue by proposing an additional auditing protocol that queries the interpretable model.

Original: Finally, through both a theoreticalframework and an experimental evaluation, we demonstrate that our fairwashing detector cannot beevaded by an attacker, informed of our detector, without significantly degrading the fidelity of modelexplanations.
Revised: Finally, through both a theoretical framework and an experimental evaluation, we demonstrate that our fairwashing detector cannot be evaded by an attacker, informed of our detector, without significantly degrading the fidelity of model explanations.

Original: Future directions include the extension of the detector to other fairness measures ( e.g. ,measures incorporating ground-truth information) as well as extending the analysis to multi-classsetups.
Revised: Future directions include the extension of the detector to other fairness measures ( e g. , measures incorporating ground-truth information) as well as extending the analysis to multi-class setups.

Original: However, these would both require additional assumptions on the data manifold (to accountfor class correlations) to establish similar results as in Section3.
Revised: However, these would both require additional assumptions on the data manifold (to account for class correlations) to establish similar results as in Section

Revised: Acknowledgments

Revised: This work was supported by CIFAR (through a Canada CIFAR AI Chair and a Catalyst grant), by

Revised: NSERC (under the Discovery Program and COHESA strategic research network), by the Ontario

Revised: Early Researcher Award, and by gifts from Intel and Meta.

Revised: We also thank the Vector Institute’s sponsors.

Original: Broader Impact Statement
Revised: Ali Shahin Shamsabadi was also partially supported by The Alan Turing Institute. Sierra Wyllie was also partially supported by Engineering Science at University of Toronto through ESROP grant. Ulrich Aïvodji was supported by the NSERC Discovery Grants program (2022-04006).

Revised: Sébastien Gambs is supported by the Canada Research Chair program, a Discovery Grant from

Revised: NSERC as well as the NSERC-RDC DEEL project.

Original: FRAUD-Detect may be used to provide evaluations of the fairness of model explanations, resulting inincreased trust in explanations, public expectation of model fairness, and model owner accountability.
Revised: FRAUD-Detect may be used to provide evaluations of the fairness of model explanations, resulting in increased trust in explanations, public expectation of model fairness, and model owner accountability.

Original: Since FRAUD-Detect is non-cooperative and resists evasion even against an informed adversary, itempowers auditors, protects the intellectual property and ownership of the model owners, and mayresist gamification into a new arms-race system between fairwashing and auditor detection methods.
Revised: Since FRAUD-Detect is non-cooperative and resists evasion even against an informed adversary, it empowers auditors, protects the intellectual property and ownership of the model owners, and may resist gamification into a new arms-race system between fairwashing and auditor detection methods.

Original: By empowering auditors, FRAUD-Detect also enforces democratic ideals of equality by providingan avenue for users to receive fairness-vetted model explanations for high-stakes decisions, furtherincreasing public trust in AI and model auditing.
Revised: By empowering auditors, FRAUD-Detect also enforces democratic ideals of equality by providing an avenue for users to receive fairness-vetted model explanations for high-stakes decisions, further increasing public trust in AI and model auditing.

Original: The potential risks of this work revolve around the fairness definition and datasets used in thispaper.
Revised: The potential risks of this work revolve around the fairness definition and datasets used in this paper.

Original: Demographic parity, while befitting our setting (as discussed in Section 2.1) is not alwaysan appropriate fairness definition, depending on the model’s domain and application.
Revised: Demographic parity, while befitting our setting (as discussed in Section 2.1) is not always an appropriate fairness definition, depending on the model’s domain and application.

Original: We recognisethat continual focus on a concept reinforces it and it’s ideology’s normalcy, and that our focuson demographic parity also reinforces the political and philosophical ideologies underlying it anddeprives alternate definitions of due attention.
Revised: We recognise that continual focus on a concept reinforces it and it’s ideology’s normalcy, and that our focus on demographic parity also reinforces the political and philosophical ideologies underlying it and deprives alternate definitions of due attention.

Original: Likewise, as noted in Appendix E, our evaluation of
Revised: Likewise, as noted in Appendix E, our evaluation of

Original: FRAUD-Detect on the current popular fairness benchmark datasets normalizes these applications of
Revised: FRAUD-Detect on the current popular fairness benchmark datasets normalizes these applications of

Original: AI.
Revised: AI.

Original: We draw specific attention to the use of the COMPAS recidivism prediction dataset [5], and notethat we do not necessarily condone the use of AI models for this application.
Revised: We draw specific attention to the use of the COMPAS recidivism prediction dataset [5], and note that we do not necessarily condone the use of AI models for this application.

Original: To mitigate these societal risks, we advocate for similar auditing measures for second-order failureswith respect to other fairness definitions (see Section 7).
Revised: To mitigate these societal risks, we advocate for similar auditing measures for second-order failures with respect to other fairness definitions (see Section 7).

Original: We also acknowledge the importance ofinvestigating the impact of FRAUD-Detect in real-world scenarios, especially where the group ofusers suing for model explanations represents minority or sensitive demographics.
Revised: We also acknowledge the importance of investigating the impact of FRAUD-Detect in real-world scenarios, especially where the group of users suing for model explanations represents minority or sensitive demographics.

Original: Checklist
Revised: Checklist

Original: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?
Revised: For all authors... (a) Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?

Original: [Yes]
Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 7.
Revised: [Yes] See Section 7.

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes] In the Societal Impact Statement (d) Have you read the ethics review guidelines and ensured that your paper conforms to 
Revised: [Yes] In the Societal Impact Statement (d) Have you read the ethics review guidelines and ensured that your paper conforms to them?

Original: them? [Yes] 
Revised: [Yes] 2.

Original: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?
Revised: If you are including theoretical results... (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Assump- tion 1.
Revised: [Yes] See Assumption 1.

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix D 
Revised: [Yes] See Appendix D 3.

Original: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?
Revised: If you ran experiments... (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?
Revised: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: (c) Did you report error bars (e g., with respect to the random seed after running experi- ments multiple times)?
Revised: (c) Did you report error bars (e.g., with respect to the random seed after running experiments multiple times)?

Original: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
Revised: [Yes] (d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)?

Original: [Yes] 
Revised: [Yes] 4.

Original: If you are using existing assets (e g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?
Revised: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets... (a) If your work uses existing assets, did you cite the creators?

Original: [Yes] (b) Did you mention the license of the assets?
Revised: [Yes] (b) Did you mention the license of the assets?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re using/curating?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?
Revised: If you used crowdsourcing or conducted research with human subjects... (a) Did you include the full text of instructions given to participants and screenshots, if applicable?

Original: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?
Revised: [N/A] (b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable?

Original: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?
Revised: [N/A] (c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Explanation methods
Revised: A Explanation methods

Original: Methods to facilitate the understanding of machine learning models are usually classified into twomain categories: transparent-box design and black-box model explanation [27].
Revised: Methods to facilitate the understanding of machine learning models are usually classified into two main categories: transparent-box design and black-box model explanation [27].

Original: Transparent-boxdesign concerns methods that produce inherently interpretable models such as logistic regressors anddecision trees [12, 32].
Revised: Transparent-box design concerns methods that produce inherently interpretable models such as logistic regressors and decision trees [12, 32].

Original: Black-box model explanation corresponds to approaches designed to explainhow black-box models produce their outcomes.
Revised: Black-box model explanation corresponds to approaches designed to explain how black-box models produce their outcomes.

Original: For instance, global explanations techniques ( e.g. ,
Revised: For instance, global explanations techniques ( e.g. ,

Original: TREPAN [20] and DECTEXT [11]) focus on explaining the whole logic of black-box models bytraining an inherently interpretable surrogate model.
Revised: TREPAN [20] and DECTEXT [11]) focus on explaining the whole logic of black-box models by training an inherently interpretable surrogate model.

Original: On the other hand, local explanations aim atexplaining black-box models decisions on single data points ( e.g. , LIME [38], and GRADCAM [40]).
Revised: On the other hand, local explanations aim at explaining black-box models decisions on single data points ( e.g. , LIME [38], and GRADCAM [40]).

Original: For simplicity, in this paper, we focus on fairwashing performed with global explanation models.
Revised: For simplicity, in this paper, we focus on fairwashing performed with global explanation models.

Original: B Related Work
Revised: B Related Work

Original: We highlight some of the differences in settings, assumptions and solutions between our work and[44, 25]:
Revised: We highlight some of the differences in settings, assumptions and solutions between our work and [44, 25]:

Original: Setting We rely on explainer models while they use a published data subset for fairness auditing.
Revised: Setting We rely on explainer models while they use a published data subset for fairness auditing.

Original: Inparticular, they reveal some subset of the training data and their predictions.
Revised: In particular, they reveal some subset of the training data and their predictions.

Original: We follow the settingconsidered in the literature introducing fairwashing [2, 3].
Revised: We follow the setting considered in the literature introducing fairwashing [2, 3].

Original: Slack et al.
Revised: Slack et al.

Original: explanations require inputperturbations while ours require the model owner to provide an interpretable model.
Revised: explanations require input perturbations while ours require the model owner to provide an interpretable model.

Original: Assumptions Slack et al.
Revised: Assumptions Slack et al.

Original: assume that fairness auditing is performed via model-agnostic localexplanations (e.g., LIME and SHAP) [44].
Revised: assume that fairness auditing is performed via model-agnostic local explanations (e.g., LIME and SHAP) [44].

Original: Both [44] and [25] assume an ideal detector withknowledge of the underlying training distribution of the model; and query access to the black-boxmodel.
Revised: Both [44] and [25] assume an ideal detector with knowledge of the underlying training distribution of the model; and query access to the black-box model.

Original: However, we only assume access to black-box model predictions on the suing set.
Revised: However, we only assume access to black-box model predictions on the suing set.

Original: We stressthat this dataset is available before any formal audit takes place (in the form of asking for modelexplanations).
Revised: We stress that this dataset is available before any formal audit takes place (in the form of asking for model explanations).

Original: This does not constitute query access to the black-box model; in fact it is only datasetaccess, which here is the predictions on the suing set.
Revised: This does not constitute query access to the black-box model; in fact it is only dataset access, which here is the predictions on the suing set.

Original: Solutions [44, 25] attempt to detect fairwashing using a Kolmogorov-Smirnov (KS) test over theunderlying and company-provided subset distribution to determine if the data subset was honestlyprovided.
Revised: Solutions [44, 25] attempt to detect fairwashing using a Kolmogorov-Smirnov (KS) test over the underlying and company-provided subset distribution to determine if the data subset was honestly provided.

Original: More precisely, they show that detection is difficult when fairwashing is conductedby minimizing the Wasserstein distance between the distributions while subject to fairness; thisoptimization also minimizes the upper-bound of the advantage (i.e., distinguishability) of the KS test.
Revised: More precisely, they show that detection is difficult when fairwashing is conducted by minimizing the Wasserstein distance between the distributions while subject to fairness; this optimization also minimizes the upper-bound of the advantage (i.e., distinguishability) of the KS test.

Original: This differs from our setting in which an informed malicious company must optimize the fairnesssubject to both fidelity (loss) and the detection threshold.
Revised: This differs from our setting in which an informed malicious company must optimize the fairness subject to both fidelity (loss) and the detection threshold.

Original: Explanation methods like LIME and SHAPperturb inputs to gauge feature relevance, inadvertently querying with synthetic data that may bedetected with out-of-distribution detection.
Revised: Explanation methods like LIME and SHAP perturb inputs to gauge feature relevance, inadvertently querying with synthetic data that may be detected with out-of-distribution detection.

Original: Queries determined to originate from explainers arefed to a fair model whereas in-distribution points are given to the biased model.
Revised: Queries determined to originate from explainers are fed to a fair model whereas in-distribution points are given to the biased model.

Original: In contrast, ourdetection method is non-cooperative (we require no additional information from the black-box modelas the auditor already has the predictions on the suing set) and therefore does not rely on inputperturbations.
Revised: In contrast, our detection method is non-cooperative (we require no additional information from the black-box model as the auditor already has the predictions on the suing set) and therefore does not rely on input perturbations.

Original: C Fidelity
Revised: C Fidelity

Original: Definition 6 (Fidelity [20]) .
Revised: Definition 6 (Fidelity [20]) .

Original: The fidelity of an interpretable model I ( · ) with respect to a black-boxmodel B ( · ) is defined as the relative accuracy of I ( · ) with respect to B ( · ) :
Revised: The fidelity of an interpretable model I ( · ) with respect to a black-box model B ( · ) is defined as the relative accuracy of I ( · ) with respect to B ( · ) :

Original: [Equation]
Revised: [Equation]

Original: where D is a data distribution, ˆ Y I is the prediction of the interpretable model and ˆ Y B is the predictionof black-box model.
Revised: where D is a data distribution, ˆ Y I is the prediction of the interpretable model and ˆ Y B is the prediction of black-box model.

Original: ˆ Y I and ˆ Y B are assumed to be both binary-valued random variables.
Revised: ˆ Y I and ˆ Y B are assumed to be both binary-valued random variables.

Original: D Additional Proofs
Revised: D Additional Proofs

Original: Below we iterate the full proof for Theorem 1, the main theorem in the paper.
Revised: Below we iterate the full proof for Theorem 1, the main theorem in the paper.

Original: Proof.
Revised: Proof.

Original: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which D adenotes the distribution over data with attribute a ∈ A .
Revised: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which D a denotes the distribution over data with attribute a ∈ A .

Original: Define F + a , the false-positive rate of I ( · ) withrespect to B ( · ) on X ∼ D a :
Revised: Define F + a , the false-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a :

Original: [Equation]
Revised: [Equation]

Original: Note that by definition, T + a , F + a ∈
Revised: Note that by definition, T + a , F + a ∈

Original: We denote ˜ δ, δ ′ > 0 as the difference in value between
Revised: We denote ˜ δ, δ ′ > 0 as the difference in value between

Original: T +0 and T +1 , and F +0 and F +1 , respectively.
Revised: T +0 and T +1 , and F +0 and F +1 , respectively.

Original: Without loss of generality, let T +

Revised: Without loss of generality, let T +

Original: =
Revised: =

Original: T +0 + ˜
Revised: T +0 + ˜

Original: δ
Revised: δ

Original: and
Revised: and

Original: F +1 = F +0 + δ ′ where ˜ δ, δ ′ ∈ [0 , 1] .
Revised: F + 1 = F + 0 + δ ′ where ˜ δ, δ ′ ∈ [0 , 1] .

Original: To characterize γ
Revised: To characterize γ

Original: = Γ B − Γ I , we first express Γ I and Γ B in terms of ˜ δ , δ ′ , T +0 , T +1 , F +0 , 
Revised: = Γ B − Γ I , we first express Γ I and Γ B in terms of ˜ δ , δ ′ , T +0 , T +1 , F +0 , and F +1 , and Y B | A . Recall from Definition 3 that we define Γ I :=

Revised: Pr (cid:104)

Original: {+ˆ Y I = 
+}
Revised: {+ˆ Y I = 
+}

Original: {+| A = 0 (cid:105) − Pr (cid:104) ˆ Y I = 
 | A = 1 (cid:105) .+} We use Bayes’ formula to re-write Pr (cid:104)
Revised: {+| A = 0 (cid:105) − Pr (cid:104) ˆ Y I = 
 | A = 1 (cid:105) .+} We use Bayes’ formula to re-write Pr (cid:104)

Original: ˆ Y I = 

Revised: ˆ Y I =

Original: | A = a (cid:105) , a ∈ A = { 0 , 1 } in terms of the aforementioned variables.
Revised: | A = a (cid:105) , a ∈ A = { 0 , 1 } in terms of the aforemen- tioned variables.

Original: For a ∈ { 0 , 1 } :
Revised: For a ∈ { 0 , 1 } :

Original: [Equation]
Revised: [Equation]

Original: Given our derivation above [Equation] we continue by substituting terms into Γ I :
Revised: Given our derivation above, we continue by substituting terms into Γ I :

Original: [Equation]
Revised: [Equation]

Original: Now that we have derived Γ I in terms of our desired terms and Γ B
Revised: Now that we have derived Γ I in terms of our desired terms and Γ B

Original: [Equation]
Revised: [Equation]

Original: we eliminate Γ I from the equation632 for γ :
Revised: we eliminate Γ I from the equation for γ :

Original: [Equation]
Revised: [Equation]

Original: Corollary 1.
Revised: Corollary 1.

Original: Let T +1 , F +1 as defined in Section 3.
Revised: Let T +1 , F +1 as defined in Section 3.

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Revised: 1 > 0 .

Original: Proof.
Revised: Proof.

Original: Note that F +1 , T +1 ∈ [0 , 1] , therefore 1 + F +

Revised: Note that F +1 , T +1 ∈ [0 , 1] , therefore 1 + F +

Original: − T +1 ≥ 0 .
Revised: − T +1 ≥ 0 .

Original: We will show 1 + F +1 − T +

Revised: We will show 1 + F +1 − T +

Original: = 0if and only if T + = 1 and F + = 0 , i.e. if I ( · ) has perfect fidelity w.r.t.
Revised: = if and only if T + = 1 and F + = 0 , i.e. if I ( · ) has perfect fidelity w.r.t.

Original: B ( · ) .
Revised: B ( · ) .

Original: We re-arrange1 + F +1 − T +

Revised: We re-arrange 1 + F +1 − T +

Original: = 0 as:
Revised: = 0 as:

Original: [Equation]
Revised: [Equation]

Original: Since F +1 , T +1 ∈ [0 , 1] and the size of interval [0 , 1] is 1 − 0 =
Revised: Since F +1 , T +1 ∈ [0 , 1] and the size of interval [0 , 1] is 1 − 0 =

Original: 1 = T +1 − F +1 , T +1 and F +1 must beendpoints 1 and 0 , respectively.
Revised: 1 = T +1 − F +1 , T +1 and F +1 must be endpoints 1 and 0 , respectively.

Original: E Details on Experimental Setup of FRAUD-Detect
Revised: E Details on Experimental Setup of FRAUD-Detect

Original: In this section, we provide additional details on our experimental setup.
Revised: In this section, we provide additional details on our experimental setup.

Original: [Table]

Original: Datasets.
Revised: Datasets.

Original: COMPAS [5] is a recidivism dataset released by ProPublica that includes data on 6 , 167offenders in prison from Broward County (Florida), with the classification task consisting in predicting whether an inmate will re-offend in the two years after their release.
Revised: COMPAS [5] is a recidivism dataset released by ProPublica that includes data on 6 ,offenders in prison from Broward County (Florida), with the classification task consisting in predicting whether an inmate will re-offend in the two years after their release.

Original: 2 We consider race( African-American , Caucasian ) as the sensitive attribute in COMPAS.
Revised: 3 We consider race [Table] ( African-American , Caucasian ) as the sensitive attribute in COMPAS

Original: Adult Income [22] includes48,842 profiles of individuals, each characterized by 13 attributes, drawn from theU.S census.
Revised: Adult Income [22] includes the

Original: The Adult Income binary classification task is to predict whether an individual’s income will exceed $50,000 a year.
Revised: The Adult Income binary classification task is to predict whether an individual’s income will exceed $50,000 a year.

Original: We use gender ( Female , Male ) as the sensitive attribute in Adult Income.
Revised: We use gender ( Female , Male ) as the sensitive attribute in Adult Income.

Original: Finally, Bank Marketing [34] constitutes information collected from a direct marketing campaignof a Portuguese banking institution between 2008 to 2013.
Revised: Finally, Bank Marketing [34] constitutes information collected from a direct marketing campaign of a Portuguese banking institution between 2008 to 2013.

Original: This dataset contains 41,188 profilesof individuals, each described by 20 attributes with the classification task being the prediction ofsubscription to a term deposit.
Revised: This dataset contains 41,188 profiles of individuals, each described by 20 attributes with the classification task being the prediction of subscription to a term deposit.

Original: We consider age ( between 30-60 or not between 30-60 ) as thesensitive attribute in Bank Marketing.
Revised: We consider age ( between 30-60 or not between 30-60 ) as the sensitive attribute in Bank Marketing.

Original: Training of black-box and interpretable models.
Revised: Training of black-box and interpretable models.

Original: For each experiment, the dataset is split randomlyinto three partitions: the training set (67%), the suing set (16.5%) and the testing set (16.5%),following the same procedure as in [3].
Revised: For each experiment, the dataset is split randomly into three partitions: the training set (67%), the suing set (16.5%) and the testing set (16.5%), following the same procedure as in [3].

Original: Each experiment is repeated with 10 separate random seeds.
Revised: Each experiment is repeated with 10 separate random seeds.

Original: The black-box models are learned from the training data points with their associated true labels.
Revised: The black-box models are learned from the training data points with their associated true labels.

Original: However, the interpretable models are trained from the suing set, which has been labelled by theblack-box model.
Revised: However, the interpretable models are trained from the suing set, which has been labelled by the black-box model.

Original: More precisely, we use the exponentiated gradient algorithm [1] subject to thefairwashing constraint from the FairLearn Library [9] to learn the fairwashed interpretable model.
Revised: More precisely, we use the exponentiated gradient algorithm [1] subject to the fairwashing constraint from the FairLearn Library [9] to learn the fairwashed interpretable model.

Original: The exponentiated gradient algorithm when applied for fairness identifies the saddle point solutionjointly maximizing the fairness and minimizing the loss.
Revised: The exponentiated gradient algorithm when applied for fairness identifies the saddle point solution jointly maximizing the fairness and minimizing the loss.

Original: To learn the fairwashed interpretablemodel of the informed adversary, we extend the Fairness In The Rashomon Set (FaiRS) [19]framework by adding an additional constraint on C KL .
Revised: To learn the fairwashed interpretable model of the informed adversary, we extend the Fairness In The Rashomon Set (FaiRS) [19] framework by adding an additional constraint on C KL .

Original: FaiRS reduces the problem of minimizing theunfairness under two constraints on the fidelity and C KL to a saddle point problem, which is solvedwith the exponentiated gradient technique [1].
Revised: FaiRS reduces the problem of minimizing the unfairness under two constraints on the fidelity and C KL to a saddle point problem, which is solved with the exponentiated gradient technique [1].

Original: Table 1 summarizes the performance of the consideredblack-box models on each suing set.
Revised: Table 1 summarizes the performance of the considered black-box models on each suing set.

Original: Implementation.
Revised: Implementation.

Original: FRAUD-Detect solely requires the confusion matrices from the interpretablemodel for both the majority and minority group.
Revised: FRAUD-Detect solely requires the confusion matrices from the interpretable model for both the majority and minority group.

Original: These matrices are flattened, then compared via KLas outlined in Equation 8.
Revised: These matrices are flattened, then compared via KL as outlined in Equation 10.

Original: Experiments were conducted on Intel Xeon Silver 4110 CPUs (2.10 GHz,8GB RAM) and the KL divergence detector implemented in Python 3.8 using scikit-learn [36].
Revised: Experiments were conducted on Intel Xeon Silver 4110 CPUs (2.10 GHz, 8GB RAM) and the KL divergence detector implemented in Python 3.8 using scikit-learn [36].

Original: Giventhat the KL divergence constraint is not a linear constraint on the confusion matrices, the extendedversion of FaiRS may fail to obtain the global minimum of the demographic parity gap on the setof high-fidelity interpretable models.
Revised: Given that the KL divergence constraint is not a linear constraint on the confusion matrices, the extended version of FaiRS may fail to obtain the global minimum of the demographic parity gap on the set of high-fidelity interpretable models.

Original: In our experiments, our approach converges for two datasets,namely COMPAS and Marketing, but does not achieve the global minimum of the demographicparity gap for Adult Income.
Revised: In our experiments, our approach converges for two datasets, namely COMPAS and Marketing, but does not achieve the global minimum of the demographic parity gap for Adult Income.

Original: We are planning to address this in future work by focusing on detectingfairwashing using linear constraints.
Revised: We are planning to address this in future work by focusing on detecting fairwashing using linear constraints.

Original: [Figure]
Revised: FRAUD-Detect successfully detects fairwashing. Figures 4 and 5 illustrate both the demographic parity gap and C KL of interpretable models as a function of ϵ for two datasets (Adult income and Bank marketing), respectively.

Original: [Figure]
Revised: FRAUD-Detect is robust to the informed adversary. Figures 6 and 7 show the range of the demographic parity gap of high-fidelity fairwashed interpretable models subjected to a constraint on the C KL by solving the optimization problem of Equation (11). We use four types of black-box models (AdaBoost, XgBoost, DNN and Random Forest).

Original: [Figure] 
Revised: [Figure] [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: C K L
Revised: C K L

Original: AdaBoost and DT
Revised: AdaBoost and DT

Original: XgBoost and DT
Revised: XgBoost and DT

Original: DNN and DT
Revised: DNN and DT

Original: RF and DT
Revised: RF and DT

Original: [Figure]
Revised: [Figure]

Original: C
Revised: C

Original: [Figure]
Revised: [Figure]

Original: RF
Revised: RF

Original: [Figure]
Revised: [Figure]

Original: LR
Revised: LR

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: P a r i t y ga
Revised: P a r i t y ga

Original: [Figure]
Revised: [Figure]

Original: I ( [Figure] [Figure] [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: P a
Revised: P a

Original: [Figure]
Revised: [Figure]

Original: i t

Original: [Figure]

Original: [Figure]

Original: [Figure]
Revised: [Figure]

Original: F Additional results

Original: FRAUD-Detect successfully detects fairwashing.

Original: Figures 4 and 5 illustrate both the demographicparity gap and C KL of interpretable models as a function of ϵ for two datasets (Adult income and

Original: Bank marketing), respectively.

Original: FRAUD-Detect is robust to the informed adversary.

Original: Figures 6 and 7 show the range of thedemographic parity gap of high-fidelity fairwashed interpretable models subjected to a constraint onthe C KL by solving the optimization problem of Equation (9).

Original: We use four types of black-box models(AdaBoost, XgBoost, DNN and Random Forest).

Original: G Fairwashing Detection Thresholds
Revised: G Fairwashing Detection Thresholds

Original: Table 2 reports several fairwashing detection thresholds observed in Figures 2, 4 and 5.[Table]
Revised: Table 2 reports several fairwashing detection thresholds observed in Figures 2, 4 and 5.

Original: H Per seed fine grain C KL versus fairness
Revised: H Per seed fine grain C KL versus fairness

Original: Figures 8, 9, and 10 show the per-seed results aggregated to form figures 2, 4, and 5, respectively.
Revised: Figures 8, 9, and 10 show the per-seed results aggregated to form figures 2, 4, and 5, respectively.

Original: In these figures (especially in 8 and 10), the order of increasing black-box fairnesses across the10 seeds is often replicated in the order of decreasing interpretable model C KL values across thesame seeds; thus a seed with a low black-box fairness has higher C KL values than a seed with highblack-box fairness. 
Revised: In these figures (especially in 8 and 10), the order of increasing black-box fairnesses across the 10 seeds is often replicated in the order of decreasing interpretable model C KL values across the same seeds; thus a seed with a low black-box fairness has higher C KL values than a seed with high black-box fairness. [Table]

Original: Some seeds from the logistic regression interpretable model (e.g. Figure 9 AdaBoost and logisticregression green and grey lines) show discontinuities at high epsilons where fairwashing is significant.
Revised: Some seeds from the logistic regression interpretable model (e.g. Figure 9 AdaBoost and logistic regression green and grey lines) show discontinuities at high epsilons where fairwashing is significant.

Original: There, the C KL values jump to infinity due to a division by zero in the KL log term when there are 0sin the minority subpopulation confusion matrix.
Revised: There, the C KL values jump to infinity due to a division by zero in the KL log term when there are 0s in the minority subpopulation confusion matrix.

Original: We also observe that for several seeds across all datasets and models, the fairness of the honestinterpretable model is imperfectly aligned and usually higher than the black-box fairness.
Revised: We also observe that for several seeds across all datasets and models, the fairness of the honest interpretable model is imperfectly aligned and usually higher than the black-box fairness.

Original: This isunintentional fairwashing, which may arise from the relatively small capacity of the interpretablemodel compared to the black-box model.
Revised: This is unintentional fairwashing, which may arise from the relatively small capacity of the interpretable model compared to the black-box model.

Original: I Bounds for KL Divergence of the Confusion Matrices
Revised: I Bounds for KL Divergence of the Confusion Matrices

Original: First, denote γ 1 = Γ B (1 + F +1 − T +1 )
Revised: First, denote γ 1 = Γ B (1 + F +1 − T +1 )

Original: =
Revised: =

Original: Γ B ( F − 1 + F +1 ) and similarly γ 0 =
Revised: Γ B ( F − 1 + F +1 ) and similarly γ 0 =

Original: Γ B ( F − 0 + F +0 ) .
Revised: Γ B ( F − 0 + F +0 ) .

Original: Depending on the context of the problem either subgroup could be the more sensitive subgroup, soin Equation (6) we implicitly assumed that subgroup is z = 1 : γ = γ 1 .
Revised: Depending on the context of the problem either subgroup could be the more sensitive subgroup, so in Equation (6) we implicitly assumed that subgroup is z = 1 : γ = γ 1 .

Original: Equation (8) defines thedetector based on the flattened confusion matrix, but since for a binary classification problem, therank of the confusion matrix is 2, it is enough to consider KL 
Revised: Equation (10) defines the detector based on the flattened confusion matrix, but since for a binary classification problem, the rank of the confusion matrix is 2, it is enough to consider KL (cid:18)(cid:20) F + F −(cid:21) , (cid:20) F +

Original: [Equation]
Revised: F −(cid:21)(cid:19) .

Original: 3 We will deriveupper and lower bounds on this divergence measure in terms of γ 0 , γ 1 , F + i , F − i ; i ∈ { 0 , 1 } .
Revised: 4 We will derive upper and lower bounds on this divergence measure in terms of γ 0 , γ 1 , F + i , F − i ; i ∈ { 0 , 1 } .

Original: First, note that γ 1 is the sum of type I F − 1 and type II F +1 detection errors; scaled by the unfairness of theblack-box model Γ B .
Revised: First, note that γ 1 is the sum of type I F − 1 and type II F +1 detection errors; scaled by the unfairness of the black-box model Γ B .

Original: In a detection problem, typically, we bound the type I error and minimize thetype II error.
Revised: In a detection problem, typically, we bound the type I error and minimize the type II error.

Original: The particular choice of type I and type II error is context-dependent (are false positivesa bigger concern than false negatives or not?).
Revised: The particular choice of type I and type II error is context-dependent (are false positives a bigger concern than false negatives or not?).

Original: In the context of fairwashing, a reasonable choice isthat
Revised: In the context of fairwashing, a reasonable choice is that

Original: [Equation]
Revised: [Equation]

Original: That is, the cost of falsely detecting a fairwashed model is larger than missing fairwashing—due tothe legal and liability considerations, for instance.
Revised: That is, the cost of falsely detecting a fairwashed model is larger than missing fairwashing—due to the legal and liability considerations, for instance.

Original: We can bound the irreducible fairwashing gap γ i :
Revised: We can bound the irreducible fairwashing gap γ i :

Original: [Equation]
Revised: [Equation]

Original: [Equation]

Original: Using this we bound the aforementioned KL divergence:
Revised: Using this we bound the aforementioned KL divergence:

Original: [Equation]
Revised: [Equation]

Original: Similarly,
Revised: Similarly,

Revised: KL

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Therefore, κ := KL (cid:18)(cid:20)
Revised: Therefore

Original: [Equation]
Revised: [Equation]

Original: +
Revised: κ := KL

Original: [Equation]
Revised: [Equation]

Original: F
Revised: is bounded from below and above by:

Original: [Equation]
Revised: [Equation]

Original: (cid:21)(cid:19) is bounded from below and above by:
Revised: Equivalently

Original: [Equation]
Revised: [Equation]

Original: Equivalently, we can re-write these bounds in terms of F + and T + to match Theorem 1 (using thefact that F − = 1 − T + ):
Revised: we can re-write these bounds in terms of F + and T + to match Theorem 1 (using the fact that F − = 1 − T + ):

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Therefore any choice between κ min := F +0 log (cid:16) F +0 F − 1 · γ 0 γ 1 (cid:17) and κ max :=
Revised: Therefore any choice between κ min := F +0 log (cid:16) F +0 F − 1 · γ 0 γ [Equation] and κ max :

Original:  F − 0 log (cid:16) 
Revised: = F − 0 log (cid:16) γ

Original: [Table]
Revised: [Equation]

Revised: 1 · F − 0 F +1 (cid:17) is valid.

Original: is valid.κ = κ min is the most conservative choice which increases the risk of false discovery (of fairwashing), while κ = κ max is a permissive choice that minimizes the risk of false discovery at the expense ofincrease in the false negatives (not detecting fairwashing).
Revised: κ = κ min is the most conservative choice which increases the risk of false discovery (of fairwashing), while κ = κ max is a permissive choice that minimizes the risk of false discovery at the expense of increase in the false negatives (not detecting fairwashing).

Original: A balanced choice for threshold could bethe intermediate point:
Revised: A balanced choice for threshold could be the intermediate point:

Original: [Equation]
Revised: [Equation]

Revised: [Equation]

Original: where ∆( R ) =
Revised: where ∆( R ) =

Original: R 0 /R 1 and ∆ l ( R ) = log ∆( R ) .
Revised: R 0 /R 1 and ∆ l ( R ) = log ∆( R ) .

Original: In the last equation, we have used the taylorexpansion of log(1 − x ) at x = 0 .
Revised: In the last equation, we have used the taylor expansion of log(1 − x ) at x = 0 .

Original: J Experimental results with two independent variables
Revised: J Experimental results with two independent variables

Original: In this section, we repeat experiments shown in Section 6 with the following adjustment:
Revised: In this section, we repeat experiments shown in Section 6 with the following adjustment:

Original: [Equation]
Revised: [Equation]

Original: in which we use only two independent variables of true-positive rate T + and false-positive rate F + .
Revised: in which we use only two independent variables of true-positive rate T + and false-positive rate F + .

Original: Figures 11, 12 and 13 show similar results as Figures 2, 4 and 5.
Revised: Figures 11, 12 and 13 show similar results as Figures 2, 4 and 5.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: K Perfect Fairwashing is impossible
Revised: K Perfect Fairwashing is impossible

Original: Theorem 2.
Revised: Theorem 2.

Original: Consider binary-valued random variables ˆ Y B black-box model predictions, ˆ Y I proxy(interpretable) model predictions, and A a sensitive attribute; then these conditions cannot be satisfiedsimultaneously:
Revised: Consider binary-valued random variables ˆ Y B black-box model predictions, ˆ Y I proxy (interpretable) model predictions and A a sensitive attribute; then these conditions cannot be satisfied simultaneously:

Original: Interpretable Model Demographic Parity
Revised: Interpretable Model Demographic Parity

Original: [Equation]
Revised: [Equation]

Original: Non-detectable
Revised: Non-detectable

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Let Equation (39) be equal to T + , and Equation (40) to F + .
Revised: Let Equation (41) be equal to T + , and Equation (42) to F + .

Original: Define the base rates of theblack-box model on subpopulations 0 and 1 as:
Revised: Define the base rates of the black-box model on subpopulations 0 and 1 as:

Original: [Equation]
Revised: [Equation]

Original: Using the rule of total probabilities, for a ∈ [0 , 1] ,
Revised: Using the rule of total probabilities, for a ∈ [0 , 1] ,

Original: [Equation]
Revised: [Equation]

Original: Fairness according to parity requires that
Revised: Fairness according to parity requires that

Original: [Equation]
Revised: [Equation]

Original: [Equation] Substituting Equation (41) and Equation (43) and re-organizing we have: 
Revised: Substituting Equation (43) and Equation (45) and re-organizing we have: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Case 1: If T + = F + , by eliminating both and recognizing that (1 − µ 0 ) =
Revised: Case 1: If T + = F + , by eliminating both and recognizing that (1 − µ 0 ) =

Original: 1 − µ 0 , we see that theequality Equation (43) reduces to 0 = 0 which is trivially correct.
Revised: 1 − µ 0 , we see that the equality Equation (45) reduces to 0 = 0 which is trivially correct.

Original: Case 2: T + ̸ = F + Simplifying Equation (44) further and eliminating T [Equation]
Revised: Case 2: T + ̸ = F + Simplifying Equation (46) further and eliminating T + − F + ̸ =

Original: [Equation]
Revised: [Equation]

Original: But these are probabilities, for Equation (43) to hold we must have that
Revised: But these are probabilities, for Equation (45) to hold we must have that

Original: [Equation]
Revised: [Equation]

Original: which means that the black-box predictions exclusively depend on the sensitive attribute.
Revised: which means that the black-box predictions exclusively depend on the sensitive attribute.

Original: In otherwords, for demographic parity to hold for the interpretable model, the black-box models predictionsall samples of one subpopulation A = 0 and reject the others A = 0 .
Revised: In other words, for demographic parity to hold for the interpretable model, the black-box models predictions all samples of one subpopulation A = 0 and reject the others A = 0 .

Original: Clearly, this is unfair to onesubpopulation.
Revised: Clearly, this is unfair to one subpopulation.

Original: To establish the impossibility result, we further show that such an assumption is equivalent to requiringcompletely inaccurate black-box models with maximum error rates.
Revised: To establish the impossibility result, we further show that such an assumption is equivalent to requiring completely inaccurate black-box models with maximum error rates.

Original: Consider the “true” base rates for each subpopulation:
Revised: Consider the “true” base rates for each subpopulation:

Original: [Equation]
Revised: [Equation]

Original: If interpretable demographic parity Equation (45) holds then, these base rates reduce to:
Revised: If interpretable demographic parity Equation (47) holds then, these base rates reduce to:

Original: [Equation]
Revised: [Equation]

Original: In other words, the black-box model predictions is completely independent of the true label andexclusively dependent on the sensitive attribute A .
Revised: In other words, the black-box model predictions is completely independent of the true label and exclusively dependent on the sensitive attribute A .

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: i t
Revised: i t

Original: [Figure]
Revised: [Figure]

Original: ga
Revised: ga

Original: [Figure]
Revised: [Figure]

Original: o f
Revised: o f

Original: I ( · )
Revised: I ( · )

Original: [Figure]
Revised: [Figure]

Original: 1 Fairness .
Revised: 1 Fairness .

Original: C K L
Revised: C K L

Original: AdaBoost and LR .
Revised: AdaBoost and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: DNN and LR .
Revised: DNN and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: RF and LR .
Revised: RF and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: XgBoost and LR .
Revised: XgBoost and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: C K L
Revised: C K L

Original: AdaBoost and DT .
Revised: AdaBoost and DT .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: DNN and DT .
Revised: DNN and DT .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: RF and DT .
Revised: RF and DT .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: XgBoost and DT
Revised: XgBoost and DT

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: DNN and LR .
Revised: DNN and LR .

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Revised: consider of the

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]