Original: Washing The Unwashable : On The (Im)possibility of Fairwashing Detection
Revised: Washing The Unwashable : On The (Im)possibility of Fairwashing Detection

Original: Anonymous Author(s)
Revised: Anonymous Author(s)

Original: AffiliationAddress email
Revised: AffiliationAddress email

Original: Abstract
Revised: Abstract

Original: The use of black-box models ( e.g. , deep neural networks) in high-stakes decisionmaking systems, whose internal logic is complex, raises the need for providingexplanations about their decisions.
Revised: The use of black-box models ( e.g. , deep neural networks) in high-stakes decisionmaking systems, whose internal logic is complex, raises the need for providingexplanations about their decisions.

Original: Model explanation techniques mitigate thisproblem by generating an interpretable and high-fidelity surrogate model ( e.g. , alogistic regressor or decision tree) to explain the logic of black-box models.
Revised: Model explanation techniques mitigate thisproblem by generating an interpretable and high-fidelity surrogate model ( e.g. , alogistic regressor or decision tree) to explain the logic of black-box models.

Original: Inthis work, we investigate the issue of fairwashing, in which model explanationtechniques are manipulated to rationalize decisions taken by an unfair black-boxmodel using deceptive surrogate models.
Revised: Inthis work, we investigate the issue of fairwashing, in which model explanationtechniques are manipulated to rationalize decisions taken by an unfair black-boxmodel using deceptive surrogate models.

Original: More precisely, we theoretically characterize and analyze fairwashing, proving that this phenomenon is difficult to avoiddue to an irreducible factor—the unfairness of the black-box model.
Revised: More precisely, we theoretically characterize and analyze fairwashing, proving that this phenomenon is difficult to avoiddue to an irreducible factor—the unfairness of the black-box model.

Original: Based on thetheory developed, we propose a novel technique, called FRAUD-Detect (FaiRness
Revised: Based on thetheory developed, we propose a novel technique, called FRAUD-Detect (FaiRness

Original: AUDit Detection), to detect fairwashed models by measuring a divergence oversubpopulation-wise fidelity measures of the interpretable model.
Revised: AUDit Detection), to detect fairwashed models by measuring a divergence oversubpopulation-wise fidelity measures of the interpretable model.

Original: We empiricallydemonstrate that this divergence is significantly larger in purposefully fairwashedinterpretable models than in honest ones.
Revised: We empiricallydemonstrate that this divergence is significantly larger in purposefully fairwashedinterpretable models than in honest ones.

Original: Furthermore, we show that our detectoris robust to an informed adversary trying to bypass our detector.
Revised: Furthermore, we show that our detectoris robust to an informed adversary trying to bypass our detector.

Original:  Introduction
Revised:  Introduction

Original: The wide applicability of machine learning models has recently increased their usage in high-stakesdecision systems such as credit scoring [43], insurance risk [10] and predictive justice [29].
Revised: The wide applicability of machine learning models has recently increased their usage in high-stakesdecision systems such as credit scoring [43], insurance risk [10] and predictive justice [30].

Original: Theconsequences of erroneous decisions that are based on predictions of machine learning models( e.g. , people being wrongly denied parole [45]) have increased the demand—from both the publicand government—to provide an explanation to humans about model decisions.
Revised: Theconsequences of erroneous decisions that are based on predictions of machine learning models( e.g. , people being wrongly denied parole [46]) have increased the demand—from both the publicand government—to provide an explanation to humans about model decisions.

Original: For instance, thisappears as an explanation requirement in the European General Data Protection Regulation [25].
Revised: For instance, thisappears as an explanation requirement in the European General Data Protection Regulation [26].

Original: However, various widely-used model architectures, such as deep neural networks, are consideredblack-boxes due to their complex and hidden internal logic, impeding the ability to explain theirdecisions in terms that are understandable by a human.
Revised: However, various widely-used model architectures, such as deep neural networks, are consideredblack-boxes due to their complex and hidden internal logic, impeding the ability to explain theirdecisions in terms that are understandable by a human.

Original: To address this issue, black-box modelexplanation techniques [26, 6] aim to provide reasons for predictions of black-box models in humanunderstandable terms.
Revised: To address this issue, black-box modelexplanation techniques [27, 6] aim to provide reasons for predictions of black-box models in humanunderstandable terms.

Original: One possible way to realize this is to approximate the black-box model usingan inherently interpretable model ( e.g. , logistic regression or decision tree) whose decisions can beeasily explained by design [26].
Revised: One possible way to realize this is to approximate the black-box model usingan inherently interpretable model ( e.g. , logistic regression or decision tree) whose decisions can beeasily explained by design [27].

Original: For instance, in logistic regression, the coefficients of the modeldirectly represent the importance of each input feature, while a decision tree is composed of a set ofexplainable if-then decision rules related to the input features.
Revised: For instance, in logistic regression, the coefficients of the modeldirectly represent the importance of each input feature, while a decision tree is composed of a set ofexplainable if-then decision rules related to the input features.

Original: In this black-box model explanation setting, the interpretable model is trained with the objective thatits predictions agree with the predictions of the black-box model ( i.e. , high fidelity [20]).
Revised: In this black-box model explanation setting, the interpretable model is trained with the objective thatits predictions agree with the predictions of the black-box model ( i.e. , high fidelity [20]).

Original: However,black-box model explanations can be manipulated to provide deceiving explanations, even if these explanations display a high fidelity with respect to the original black-box model.
Revised: However,black-box model explanations can be manipulated to provide deceiving explanations, even if these explanations display a high fidelity with respect to the original black-box model.

Original: An example of such ma-
Revised: An example of such ma-

Original: [Figure]
Revised: [Figure]

Original: [Figure]

Original: T −
Revised: T −

Original: T +
Revised: T +

Original: F +
Revised: F +

Original: F −.2.
Revised: F −.2.

Original: R a t e
Revised: R a t e

Original: Honest interpretable model
Revised: Honest interpretable model

Original: T −
Revised: T −

Original: T +
Revised: T +

Original: F +
Revised: F +

Original: F −.2.
Revised: F −.2.

Original: R a t e
Revised: R a t e

Original: [Figure]

Original:  interpretable [Figure] nipulation occurs when the explanations provided cover up the unfairness of the underlying black-boxmodel, resulting in fairwashing [2, 3].
Revised: Fairwashed interpretable model nipulation occurs when the explanations provided cover up the unfairness of the underlying black- model, resulting in fairwashing [2, 3].

Original: As a concrete example, a dishonest bank [4] could use a blackbox model that denies a loan to a customer in a discriminative manner—based on a sensitive attributesuch as race or gender identity—while providing an interpretable surrogate model promoting the falseimpression that the decisions are fair and based solely on non-sensitive features.
Revised: As a concrete example, a dishonest bank [4] could use a blackbox model that denies a loan to a customer in a discriminative manner—based on a sensitive attributesuch as race or gender identity—while providing an interpretable surrogate model promoting the falseimpression that the decisions are fair and based solely on non-sensitive features.

Original: These manipulationsare made possible in part by the ambiguity in current regulations [25] when it comes to describingwhat constitutes a valid explanation [44].
Revised: These manipulationsare made possible in part by the ambiguity in current regulations [26] when it comes to describingwhat constitutes a valid explanation [45].

Original: Fairwashing is clearly an ethical issue as individuals whohave unfairly received unfavorable decisions are deprived of a chance to contest the decision [3].
Revised: Fairwashing is clearly an ethical issue as individuals whohave unfairly received unfavorable decisions are deprived of a chance to contest the decision [3].

Original: Due to the growing importance of fairness in machine learning [7], we focus on a scenario in whicha dishonest entity attempts to perform fairwashing [2, 3] when providing model explanations to anauditor ( e.g. , an external dedicated party).
Revised: Due to the growing importance of fairness in machine learning [7], we focus on a scenario in whicha dishonest entity attempts to perform fairwashing [2, 3] when providing model explanations to anauditor ( e.g. , an external dedicated party).

Revised: For example, if a company’s users expect that the decisionsthey received were a result of model bias, they may send their data and received decisions to anauditor.

Original: In particular, such an audit can take place in responseto a legal demand [8] by service users who ask for explanations related to the  received decisions(see Figure 1).
Revised: The auditor may make a legal demand [8] on the company to provide an interpretable modelto explain these users’ received decisions (see Figure 1).

Original: In this case, the dishonest entity has an incentive to manipulate the black-box modelexplanation technique, to under-report and occlude the unfairness of the black-box model underscrutiny to evade the potential consequences of unfair decision-making.
Revised: In this case, a dishonest company hasan incentive to manipulate the black-box model explanation technique, to under-report and occludethe unfairness of the black-box model under scrutiny to evade the potential consequences of unfairdecision-making.

Original: The dishonest entity couldthus mislead the auditor by rationalizing its decisions using the corresponding fairwashed interpretablemodel.
Revised: The dishonest entity could thus mislead the auditor by rationalizing its decisionsusing the corresponding fairwashed interpretable model.

Original: In this paper, we are the first to propose a method for detecting such dishonest modelexplanations.
Revised: In this paper, we are the first to propose amethod for detecting such dishonest model explanations.

Original: This is a challenging problem.
Revised: This is a challenging problem.

Original: Fairwashing cannot be detected through computingthe explanation violation of the explainable model such as fidelity of explainable models as wedemonstrate that the explainable model cannot achieve perfect fidelity with respect to the black-boxmodel.
Revised: Fairwashingcannot be detected through computing the explanation violation of the explainable model, such asfidelity of explainable models, as we demonstrate that the explainable model cannot achieve perfectfidelity with respect to the black-box model.

Original: In addition to this, measuring the differences between the fairness of the black-box model andthe explainable model cannot help to detect fairwashing as there are several design choices (includingfairwashing) that could lead to different fairness, thus non-intentional fairwashing.
Revised: In addition to this, measuring the differences betweenthe fairness of the black-box model and the explainable model cannot help to detect fairwashing asthere are several design choices (including fairwashing) that could lead to different fairness, thusnon-intentional fairwashing.

Original: Our primary contributions include (1) an extensive theoretical analysis of fairwashing and the fairnesslimitations in explanation techniques and (2) a novel approach for detecting, and thus deterring,fairwashing.
Revised: Our primary contributions include (1) an extensive theoretical analysis of fairwashing and the fairnesslimitations in explanation techniques and (2) a novel approach for detecting, and thus deterring,fairwashing.

Original: Our proposed method, FRAUD-Detect (FaiRness AUDit Detection), formulates theproblem of detecting fairwashing as a non-cooperative test, both to overcome possible dishonestbehaviors in entities that are actively being audited and for communication efficiency.
Revised: Our proposed method, FRAUD-Detect (FaiRness AUDit Detection), formulates theproblem of detecting fairwashing as a non-cooperative test; both to overcome possible dishonestbehaviors in entities that are actively being audited and for communication efficiency.

Original: FRAUDDetect operates in a realistic scenario: FRAUD-Detect assumes only query-access to the black-boxmodel, which is a realistic setting as often the model is not provided by the entity due to intellectualproperty and trade-secret concerns [39, 37].
Revised: FRAUDDetect operates in a realistic scenario: FRAUD-Detect does not require access to the black-boxmodel provided the users calling for the audit have provided their data and received decisions, which isa realistic setting as often the model is not provided by the entity due to intellectual property and tradesecret concerns [39, 37].

Original: Thus, FRAUD-Detect only relies on the predictions 
Revised: Thus, FRAUD-Detect only relies on the predictions of the interpretable and black-box models.

Original: As shown in Figure 1 from our  empirical study, subpopulation-wise confusion matrix distributions ofthe honest interpretable model with respect to the black-box model predictions are strongly similar,while thefairwashed interpretable model induces  divergence between subpopulation-wise confusionmatrix distributions to conceal the unfairness of the black-box model.
Revised: Both our theoretical and empirical analysis quantify the per-subpopulation,per-label fidelity of the interpretable model with respect to the black-box model. Figure 1 shows confusion matrix distributions for two sensitive subpopulations before and afterfairwashing. These matrices are constructed by comparing the interpretable to the black-box modelpredictions. Fairwashing induces a divergence between subpopulation values to conceal the unfairness of the black-box model.

Original: FRAUD-Detect leveragesthe Kullback–Leibler (KL) divergence measure on the subpopulation-wise confusion matrices todistinguish between honest and fairwashed interpretable models.
Revised: FRAUD-Detect leverages the Kullback–Leibler (KL) divergencebetween subpopulation confusion matrices to distinguish between honest and fairwashed interpretablemodels.

Original: For a comprehensive empiricalstudy, we examine logistic regressors and decision trees as interpretable models with respect to blackbox Deep Neural Networks, AdaBoost [24], GradientBoosted Decision Trees [17] and Random
Revised: For a comprehensive empirical study, we examine logistic regressors and decision treesas interpretable models with respect to black-box Deep Neural Networks, AdaBoost [24], Gradient

Original:  Forests [13] on three benchmark datasets: COMPAS [5], Adult Income [22] and Bank Marketing [33].
Revised: Boosted Decision Trees [17] and Random Forests [13] on three benchmark datasets: COMPAS [5], Adult Income [22] and Bank Marketing [34].

Original: We evaluate the strength of our detector in the presence of an informed adversary representing aninformed dishonest entity with knowledge of FRAUD-Detect that seeks to fairwash and bypassdetection by constraining the divergence over subpopulation-wise confusion matrices.
Revised: We evaluate the strength of our detector in the presence of an informed adversary representing aninformed dishonest entity with knowledge of FRAUD-Detect that seeks to fairwash and bypassdetection by constraining the divergence over subpopulation-wise confusion matrices.

Original: Our empiricalexperiments quantify the capacity of this informed adversary by computing the achievable fairnessgap and fidelity under the additional detection constraint.
Revised: Our empiricalexperiments quantify the capacity of this informed adversary by computing the achievable fairnessgap and fidelity under the additional detection constraint.

Original: We provide additional theoretical supportfor the challenge of jointly satisfying fairwashing and evasion in appendices.
Revised: We provide additional theoretical supportfor the challenge of jointly satisfying fairwashing and evasion in appendices.

Original: In summary, our main contributions are as follows:
Revised: In summary, our main contributions are as follows:

Original: • We characterize fairwashing via a theoretical analysis on the difference in subpopulation gap between the black-box model and the interpretable model, fairwashing violation (Section 3).
Revised: • We characterize fairwashing via a theoretical analysis on the difference in subpopulation gapbetween the black-box model and the interpretable model, fairwashing violation (Section 3).

Original: • Based on our fairwashing theory, we establish requirements for a sufficient fairwashing detection method and prove that fairwashing is impossible to avoid completely due to an irreducible component in the violation term.
Revised: • Based on our fairwashing theory, we establish requirements for a sufficient fairwashingdetection method and prove that fairwashing is impossible to avoid completely due to anirreducible component in the violation term.

Original: • We introduce the first method for fairwashing detection (Section 4.2).
Revised: • We introduce the first method for fairwashing detection (Section 4.2).

Original: Informed by our theo- retical results, we observe that fairwashing causes disparate effect on the subpopulation-wise fidelity distributions of the interpretable model with respect to the black-box predictions.We
Revised: Informed by our theoretical results, we observe that fairwashing causes disparate effect on the subpopulation-wisefidelity distributions of the interpretable model with respect to the black-box predictions.

Original: leverage this observation to propose a non-cooperative black-box access method, FRAUD- Detect , which utilizes KL divergence on per-subpopulation confusion matrices to distinguishbetween fairwashed and honest interpretablemodels.
Revised: Weleverage this observation to propose a non-cooperative black-box access method, FRAUDDetect , which utilizes KL divergence on per-subpopulation confusion matrices to distinguishbetween fairwashed and honest interpretable

Original: • Our empirical results demonstrate that FRAUD-Detect successfully detects fairwashingbased on the KL divergence between subpopulation-wise confusion matrices (Section 6).We show that the divergence over subpopulation confusion matrices can vary by over 0 .
Revised: models.• Our empirical results demonstrate that FRAUD-Detect successfully detects fairwashingbased on the KL divergence between subpopulation-wise confusion matrices (Section 6).We show that the divergence over subpopulation confusion matrices can vary by over 0 .

Original: 6between an honest and a fairwashed interpretable models.
Revised: 6between an honest and a fairwashed interpretable models.•

Original: • We illustrate the robustness of FRAUD-Detect against an informed adversary ( i
Revised: We illustrate the robustness of FRAUD-Detect against an informed adversary ( i

Original: e. , adishonest entity who attempts to fairwash while evading detection).
Revised: e. , adishonest entity who attempts to fairwash while evading detection).

Original: Our empirical results show that evading our detector comes at the cost of a significant increase in subpopulationgap, negating fairwashing.
Revised: Our empirical resultsshow that evading our detector comes at the cost of a significant increase in subpopulationgap, negating fairwashing.

Original: Specifically, a dishonest entity that jointly attempts to fairwashwhile evading detection only achieves parity gaps greater than 10 %.
Revised: Specifically, a dishonest entity that jointly attempts to fairwashwhile evading detection only achieves parity gaps greater than 10 %.

Original:  Preliminaries and Problem Formulation
Revised:  Preliminaries and Problem Formulation

Original: 2.1 Related Work
Revised: 2.1 Related Work

Original: Fairness.
Revised: Fairness.

Original: Many different formal definitions of algorithmic fairness have been proposed in themachine learning community [35].
Revised: Many different formal definitions of algorithmic fairness have been proposed in the machinelearning community [35].

Original: One of the main challenges of algorithmic fairness is the lackof consensus on a universally applicable fairness definition.
Revised: One of the main challenges of algorithmic fairness is the lack of consensuson a universally applicable fairness definition.

Original: More precisely, it has been formallydemonstrated that many such measures are incompatible with each other ( i.e. , they cannot be achievedjointly in some situations) [30].
Revised: More precisely, it has been formally demonstrated thatmany such measures are incompatible with each other ( i.e. , they cannot be achieved jointly in somesituations) [31].

Original: In addition, their differences are also rooted in their underlyingphilosophical and moral assumptions [27].
Revised: In addition, their differences are also rooted in their underlying philosophical andmoral assumptions [28].

Original: Thus, the choice of a particular fairness metric is usuallycontext dependent.
Revised: Thus, the choice of a particular fairness metric is usually context dependent.

Original: Nonetheless broadly speaking, there are two main families of fairness definitions:individual fairness and group fairness [23].
Revised: Nonetheless broadly speaking, there are two main families of fairness definitions: individual fairnessand group fairness [23].

Original: Individual fairness posits that “similar individualsshould be treated similarly”.
Revised: Individual fairness posits that “similar individuals should be treated similarly”.

Original: In contrast, group fairness strives to equalize statistical properties ofclassification outcomes across subpopulations created by partitioning the population based on asensitive attribute A such as gender.
Revised: In contrast, group fairness strives to equalize statistical properties of classification outcomes acrosssubpopulations created by partitioning the population based on a sensitive attribute A such as gender.

Original: In our work, we focus on a particular notion of group fairness,demographic parity, also called statistical group parity, which refers to observing equal probabilityof positive label prediction over subpopulations.
Revised: In our work, we focus on a particular notion of group fairness, demographic parity, also calledstatistical group parity, which refers to observing equal probability of positive label prediction over subpopulations.

Original: One of the reasons we rely on demographic parityas the fairness metric is that true labels will not be available in a realistic scenario (especially in fairwashing scenario described in Section 2.3).
Revised: One of the reasons we rely on demographic parity as the fairness metric is thattrue labels will not be available in a realistic scenario (especially in fairwashing scenario describedin Section 2.3).

Original: In general, the existence of true labels conflicts withthe assumption that when requesting decisions (or explanations), users (or auditors) do not haveaccess to these true labels [42].
Revised: In general, the existence of true labels conflicts with the assumption that whenrequesting decisions (or explanations), users (or auditors) do not have access to these true labels [42].

Original: Furthermore, using the predictions of a black-box model as the truelabels is paradoxical in the sense that the black-box model is assumed to be unfair.
Revised: Furthermore, using the predictions of a black-box model as the true labels is paradoxical in thesense that the black-box model is assumed to be unfair.

Original: Demographicparity enforces independence between the class predicted by the model and inclusion in a particularsubpopulation.
Revised: Demographic parity enforces independencebetween the class predicted by the model and inclusion in a particular subpopulation.

Original: Definition 1 (Demographic parity [15]) .
Revised: Definition 1 (Demographic parity [15]) .

Original: A classifier ˆ Y satisfies demographic parity with respect tosensitive attribute A
Revised: A classifier ˆ Y satisfies demographic parity with respect tosensitive attribute A

Original: if
Revised: if

Original: [Equation]
Revised: [Equation]

Original: Global black-box explanation methods focus on explaining the whole logic of black-box models bytraining an inherently interpretable surrogate model.
Revised: Global black-box explanation methods focus on explaining the whole logic of black-box modelsby training an inherently interpretable surrogate model.

Original: In this paper, we focus on fairwashing performedwith global explanation models.

Original: We refer the interested reader to Appendix A for the description ofother explanation methods.
Revised: We refer the interested reader to Appendix Afor the description of other explanation methods.

Revised: In terms of abusing explanations via fairwashing,

Revised: Fukuchi et al., similarly to us, try to detect fairwashing but with a different setting [25].

Revised: Slack et al.shows an alternate method of fairwashing that evades local model explanation techniques in a settingthat is comparable to ours but they do not attempt to detect the fairwashing [44].

Revised: We explore theseadditional fairwashing settings in Appendix B.

Original: 2.2 Desiderata for Global Black-box Explanations
Revised: 2.2 Desiderata for Global Black-box Explanations

Original: We consider the setting in which an entity learns a complex black-box model B ( · ) on a training set( X Tr , Y Tr , A ) , in which A denotes a binary-valued variable representing a sensitive attribute. 
Revised: We consider the setting in which an entity learns a complex black-box model B ( · ) on a trainingset ( X Tr , Y Tr , A ) , in which A represents a sensitive attribute. We maintain the setup from recentfairwashing literature [3, 2] and consider binary classifiers mapping M features into a binary label B : R M (cid:55)→ { 0 , 1 } . We also consider binary-valued sensitive attributes, A ∈ { 0 , 1 } .

Original: Notethat we do  not assume that A is necessarily used during the training of B ( · ) .
Revised: Note that we do 143 not assume that A is necessarily used during the training of B ( · ) .

Original: Given user queries,the entity uses the  predictions of B ( · ) as part of a high-stakes decision system.
Revised: Given user queries, the entity usesthe predictions of B ( · ) as part of a high-stakes decision system.

Original: Later, a group ofthese users may request an explicit explanation from the entity due to concerns over improper useof the sensitive attribute.
Revised: Later, a group of these users mayrequest an explicit explanation from the entity due to concerns over improper use of the sensitiveattribute.

Original: Such a scenario frequently arises in high-impact domains, such as bankloans and credit scoring.
Revised: Such a scenario frequently arises in high-impact domains, such as bank loans and creditscoring.

Original: Let X sg be a suing set comprising the unlabeled data of users demanding anexplanation for their particular outcomes (see Figure 1).
Revised: Let X sg be a suing set comprising the unlabeled data of users demanding an explanationfor their particular outcomes (see Figure 1).

Original: Consequently, an external auditing entityrequires the company to provide explanations in terms that are understandable to humans aboutthepredictions of the  black-box.
Revised: Consequently, an external auditing entity requires thecompany to provide explanations in terms that are understandable to humans about the predictions ofthe black-box.

Original: Here, we will assume that the explanation will be provided in the formof a global explanation.
Revised: Here, we will assume that the explanation will be provided in the form of a globalexplanation.

Original: This means that a simple interpretable model I ( · ) will be trained on a dataset X labeled by querying the black-box model B ( · ) trained on X Tr , such that I ( · ) accurately reflectsand explains the logic of B ( · ) in terms that are understandable to humans.
Revised: This means that a simple interpretable model I ( · ) will be trained on a dataset X labeledby querying the black-box model B ( · ) trained on X Tr , such that I ( · ) accurately reflects and explainsthe logic of B ( · ) in terms that are understandable to humans.

Original: Formalizing further the auditing desiderata for black-box model explanations, the interpretable model
Revised: Formalizing further the auditing desiderata for black-box model explanations, the interpretable model

Original: I ( · ) must accurately mirror: 1) the output predictions of the black-box model ( i.e. , fidelity) and 2) thefairness (according to a pre-defined metric) of the black-box model.
Revised: I ( · ) must accurately mirror: 1) the output predictions of the black-box model ( i.e. , fidelity) and 2) thefairness (according to a pre-defined metric) of the black-box model.

Original: Fidelity.
Revised: Fidelity.

Original: The interpretable model should satisfy the fidelity criterion for any X ∼ D .
Revised: The interpretable model should satisfy the fidelity criterion for any X ∼ D .

Original: Fidelity(defined in Appendix B) is difficult to perfectly achieve and challenging to measure for an auditorover all X ∼ D .
Revised: Fidelity(defined in Appendix C) is difficult to perfectly achieve and challenging to measure for an auditorover all X ∼ D .

Original: Thus, we additionally introduce the notion of empirical fidelity.
Revised: Thus, we additionally introduce the notion of empirical fidelity.

Original: Definition 2 (Empirical Fidelity) .
Revised: Definition 2 (Empirical Fidelity) .

Original: The empirical fidelity on a dataset X = { x i } Ni =1 including N data 154 points x i is defined as the relative accuracy of I ( · ) with respect to B ( · ) on X :
Revised: The empirical fidelity on a dataset X = { x i } Ni =1 including N data 161 points x i is defined as the relative accuracy of I ( · ) with respect to B ( · ) on X :

Original: EmpiricalFidelity
Revised: EmpiricalFidelity

Original: [Equation]
Revised: [Equation]

Original:  where the indicator 1 ( · , · ) outputs 1 if the output label of I ( · ) and B ( · ) are the same and 0 otherwise.
Revised: (1) where the indicator 1 ( · , · ) outputs 1 if the output label of I ( · ) and B ( · ) are the same and 0 otherwise.

Original: Fairness.
Revised: Fairness.

Original: As the audit of B ( · ) is expressly requested due to fairness concerns with respect to A , theinterpretable model should reflect the fairness violations or adherences of the black-box model.
Revised: As the audit of B ( · ) is expressly requested due to fairness concerns with respect to A , theinterpretable model should reflect the fairness violations or adherences of the black-box model.

Original: 2.3 Fairwashing Definitions
Revised: 2.3 Fairwashing Definitions

Original: To evade legal consequences of decision-making based on improper use of a sensitive attribute, adishonest company may perform fairwashing .
Revised: To evade legal consequences of decision-making based on improper use of a sensitive attribute, adishonest company may perform fairwashing .

Original: Fairwashing with respect to a pre-defined fairnessmetric permits the dishonest company to learn an interpretable model that hides the unfair behaviourof their black-box model.
Revised: Fairwashing with respect to a pre-defined fairness metric permits the dishonest company to learn an interpretable model that hides the unfair behaviourof their black-box model.

Original: In practice, fairwashing could also occur due to the fact that currentregulations [25] do not define what constitutes a valid explanation , thus leaving the possibility for themodel provider to choose the interpretable model that meets their needs [44].
Revised: In practice, fairwashing could also occur due to the fact that currentregulations [26] do not define what constitutes a valid explanation , thus leaving the possibility for themodel provider to choose the interpretable model that meets their needs [45].

Original: Fairwashing can be
Revised: Fairwashing can bequantified with respect to a particular fairness measure.

Original: Definition 3 (Fairwashing) .
Revised: Definition 3 (Fairwashing) .

Original: Let Γ I and Γ B define the fairness gaps of the interpretable model andblack-box model, respectively, with respect to a pre-defined fairness metric.
Revised: Let Γ I and Γ B define the fairness gaps of the interpretable model andblack-box model, respectively, with respect to a pre-defined fairness metric.

Original: For example, in terms ofdemographic parity (Definition 1):
Revised: For example, in terms ofdemographic parity (Definition 1):

Original: [Equation]
Revised: [Equation]

Original: We define the fairwashing violation γ > 0 as the difference between Γ I and Γ B :
Revised: We define the fairwashing violation γ > 0 as the difference between Γ I and Γ B :

Original: [Equation]
Revised: [Equation]

Original: Assumption 1.
Revised: Assumption 1.

Original: The fairwashing violation γ is non-negative.
Revised: The fairwashing violation γ is non-negative.

Original: Remark that the case of γ < 0 is theopposite of fairwashing since the interpretable model is displaying a bigger fairness gap than that ofthe black-box model, thus a model owner has no incentive to employ such an interpretable model.
Revised: Remark that the case of γ < 0 is theopposite of fairwashing since the interpretable model is displaying a bigger fairness gap than that ofthe black-box model, thus a model owner has no incentive to employ such an interpretable model.

Original:  Fairness Gap of the Black-Box Breeds Fairwashing
Revised:  Fairness Gap of the Black-Box Breeds Fairwashing

Original: We analyze whether it is possible to eliminate the risk of fairwashing altogether.
Revised: We analyze whether it is possible to eliminate the risk of fairwashing altogether.

Original: We characterizethe fairwashing violation γ and show that completely eliminating fairwashing is impossible: aninterpretable model explaining an unfair black-box model always has a non-zero fairwashing violation.
Revised: We characterizethe fairwashing violation γ and show that completely eliminating fairwashing is impossible: aninterpretable model explaining an unfair black-box model always has a non-zero fairwashing violation.

Original: Theorem 1.
Revised: Theorem 1.

Original: Assume ˆ Y B are the black-box model B ( · ) predictions, ˆ Y
Revised: Assume ˆ Y B are the black-box model B ( · ) predictions, ˆ Y

Original: I are the predictions of aninterpretable model
Revised: I are the predictions of aninterpretable model

Original: I ( · ) explaining B ( · ) and A is a sensitive attribute: 1) If B ( · ) does not satisfy 179 demographic parity, completely eliminating fairwashing is impossible; 2) A detector for fairwashingmeasuring false-positive rates and true-positive rates of I ( · ) with respect to B ( · ) is sufficient.
Revised: I ( · ) explaining B ( · ) and A is a sensitive attribute: 1) If B ( · ) does not satisfy 186 demographic parity, completely eliminating fairwashing is impossible (i.e., γ > 0 ); 2) A detectorfor fairwashing measuring false-positive rates and true-positive rates of I ( · ) with respect to B ( · ) issufficient.

Original: We provide a proof sketch here (see Appendix C for full proof).
Revised: We provide a proof sketch here (see Appendix D for full proof).

Original: Proof Sketch.
Revised: Proof Sketch.

Original: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which 183 D a denotes the distribution over data with attribute a ∈ A .
Revised: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which 191 D a denotes the distribution over data with attribute a ∈ A .

Original: Define F + a , the false-positive rate of I ( · )with respect to B ( · ) on X ∼ D a .
Revised: Define F + a , the false-positive rate of I ( · )with respect to B ( · ) on X ∼ D a .

Original: Denote the differences between T +0 and T +1 , and F +0 and F +1 as˜ δ, δ ′ , respectively.
Revised: Denote the differences between T +0 and T +1 , and F +0 and F +1 as˜ δ, δ ′ , respectively.

Original: We can express Γ I in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B | A by expanding termsand using Bayes’ formula:
Revised: We can express Γ I in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B | A by expanding termsand using Bayes’ formula:

Original: [Equation]
Revised: [Equation]

Original: Now that we have derived Γ I in terms of our desired terms, we can eliminate Γ I from the equationfor γ .
Revised: Now that we have derived Γ I in terms of our desired terms, we can eliminate Γ I from the equationfor γ .

Original: The resulting formula for γ in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B |
Revised: The resulting formula for γ in terms of ˜ δ, δ ′ , T +1 , F +1 , Γ B and Y B |

Original: A allows us to demonstrate 189 correctness , irreducibility and sufficiency .
Revised: A allows us to demonstrate 197 correctness , irreducibility and sufficiency .

Original: [Equation]
Revised: [Equation]

Original: Correctness.
Revised: Correctness.

Original: Note that if Γ B = 0 ( i.e. , black-box model is fair) then no-fairwashing is present, andno detector should mistakenly report fairwashing.
Revised: Note that if Γ B = 0 ( i.e. , black-box model is fair) then no-fairwashing is present, andno detector should mistakenly report fairwashing.

Original: We can verify this with Equation (13).
Revised: We can verify this with Equation (13).

Original: Under
Revised: Under

Original: Assumption 1, we know that γ ≥ 0 and as below terms are, by definition non-negative, thus:
Revised: Assumption 1, we know that γ ≥ 0 and as below terms are, by definition non-negative, thus:

Original: [Equation]
Revised: [Equation]

Original: Irreducibility.
Revised: Irreducibility.

Original: If Γ B ̸ = 0 , even if ˜ δ = δ ′ = 0 , i.e. when true-positive and false-positive rates are equal over subpopulations ( T +0 ={++ 
+}
Revised: If Γ B ̸ = 0 , even if ˜ δ = δ ′ = 0 , i.e. when true-positive and false-positive rates are equalover subpopulations ( T {++ 
+}

Original:  T +1 and F +0 =F +1 )
Revised: = T + 1 and F + 0 =

Original: , there exists an irreducible fairwashing violation:
Revised: F + 1 ), there exists an irreducible fairwashing violation:

Original: [Equation]
Revised: [Equation]

Original: since 1 + F +1 − T +1 > 0 except in trivial cases (see Appendix C for the proof).
Revised: since 1 + F +1 − T +1 > 0 except in trivial cases (see Appendix D for the proof).

Original: We further pointout that 1 + F +1 − T +1 is not a function of disparity in the interpretable model as the value doesnot measure disparity between subpopulations.
Revised: We further pointout that 1 + F +1 − T +1 is not a function of disparity in the interpretable model as the value does not measure disparity between subpopulations.

Original: Rather, 1 + F +1 − T +1 is a function of the differencein true-positive and false-positive rates ( T +1 and F +1 , respectively) within a single subpopulation.
Revised: Rather, 1 + F +1 − T +1 is a function of the differencein true-positive and false-positive rates ( T +1 and F +1 , respectively) within a single subpopulation.

Original: The sole measure of disparity in the gap above is black-box Γ B , which is a given constant.
Revised: The sole measure of disparity in the gap above is black-box Γ B , which is a given constant.

Original: Sufficiency.
Revised: Sufficiency.

Original: The remaining factors of the gap are only function of ˜ δ and δ ′ as Pr (cid:104)
Revised: The remaining factors of the gap are only function of ˜ δ and δ ′ as Pr (cid:104)

Original: ˆ Y B =
Revised: ˆ Y B =

Original: 1 | A
Revised: 1 | A

Original: = 0 (cid:105)and Pr (cid:104)
Revised: = 0 (cid:105)and Pr (cid:104)

Original: ˆ Y B =
Revised: ˆ Y B =

Original: 1 | A = 0 (cid:105) are also given constants.
Revised: 1 | A = 0 (cid:105) are also given constants.

Original: Therefore, a detector that measures ˜ δ and δ ′ issufficient to detect fairwashing.
Revised: Therefore, a detector that measures ˜ δ and δ ′ issufficient to detect fairwashing.

Original:  Detecting Fairwashing
Revised:  Detecting Fairwashing

Original: 4.1 Fairwashing In Practice
Revised: 4.1 Fairwashing In Practice

Original: Eliminating the fairness gap of the black-box model would be the ideal outcome as it would alsoremove the possibility for a dishonest model provider to perform fairwashing.
Revised: Eliminating the fairness gap of the black-box model would be the ideal outcome as it would alsoremove the possibility for a dishonest model provider to perform fairwashing.

Original: However, in practiceabsolute fairness is often unrealistic and even regulatory bodies tolerate small violations of fairnesslaws [18].
Revised: However, in practiceabsolute fairness is often unrealistic and even regulatory bodies tolerate small violations of fairnesslaws [18].

Original: In Section 3, we showed that fairness violations result in fairwashing violations unless theinterpretable model has a higher fairness gap than the black-box model, which would not be desirablefrom the point of view of the model provider.
Revised: In Section 3, we showed that fairness violations result in fairwashing violations unless theinterpretable model has a higher fairness gap than the black-box model, which would not be desirablefrom the point of view of the model provider.

Original: Therefore, not all fairwashing violations may have beenintended by the model owners.
Revised: Therefore, not all fairwashing violations may have beenintended by the model owners.

Original: These involuntary violations are in fact side-effects of the imperfectfidelity, which is inherent to optimizing the empirical fidelity (Definition 2) of the interpretable modelon a training set
Revised: These involuntary violations are in fact side-effects of the imperfectfidelity, which is inherent to optimizing the empirical fidelity (Definition 2) of the interpretable modelon a training set

Original: X tr ∼ D , max
Revised: X tr ∼ D , max

Original: EmpiricalFidelity ( I, B ; X tr ) .
Revised: EmpiricalFidelity ( I, B ; X tr ) .

Original: In contrast, deliberate fairwashing is the result of a similar optimization problem purposefullyconstrained to provide better fairness on a target suing set X sg ∼ D :
Revised: In contrast, deliberate fairwashing is the result of a similar optimization problem purposefullyconstrained to provide better fairness on a target suing set X sg ∼ D :

Original: Definition 4 (Fairwashing Optimization [2]) .
Revised: Definition 4 (Fairwashing Optimization [2]) .

Original: Given a black-box model B ( · ) and a suing set X sg ,fairwashing optimization is defined as learning an interpretable model I ( · ) from B ( · ) on X sg such 218 that the interpretable model has 1) high fidelity with respect to the black-box model and 2) is lessunfair than this black-box model:
Revised: Given a black-box model B ( · ) and a suing set X sg ,fairwashing optimization is defined as learning an interpretable model I ( · ) from B ( · ) on X sg such 226 that the interpretable model has 1) high fidelity with respect to the black-box model and 2) is lessunfair than this black-box model:

Original: [Equation]
Revised: [Equation]

Original: subjectto
Revised: subject

Original: [Equation]
Revised: [Equation]

Original: (7) in which ϵ is an upper bound on the fairness gap of the interpretable model on the suing set
Revised: in which ϵ is an upper bound on the fairness gap of the interpretable model on the suing set

Original: FairnessGap ( I ; X sg ) , and FairnessGap ( · , · ) constitutes a measure of fairness gap according to 222 some fairness metric – generically measured by the subpopulation gap in a fairness metric.
Revised: FairnessGap ( I ; X sg ) , and FairnessGap ( · , · ) constitutes a measure of fairness gap according to 230 some fairness metric – generically measured by the subpopulation gap in a fairness metric.

Original: Note thatϵ <
Revised: Note thatϵ <

Original: FairnessGap ( B ; X sg ) must be less than the fairness gap of the black-box model on the suingset FairnessGap ( B ; X sg ) to cause fairwashing violation γ > 0 (see Definition 3).
Revised: FairnessGap ( B ; X sg ) must be less than the fairness gap of the black-box model on the suingset FairnessGap ( B ; X sg ) to cause fairwashing violation γ > 0 (see Definition 3).

Original: 4.2 Proposed method: FRAUD-Detect
Revised: 4.2 Proposed method: FRAUD-Detect

Original: As demonstrated in Section 3, a fairwashing detection method that relies on the difference oversubpopulations in true-positive and false-positive rates of the interpretable model w.r.t the blackbox model ( ˜ δ and δ ′ respectively)
Revised: As demonstrated in Section 3, a fairwashing detection method that relies on the difference oversubpopulations in true-positive and false-positive rates of the interpretable model w.r.t the blackbox model ( ˜ δ and δ ′ respectively)

Original: is sufficient.
Revised: is sufficient.

Original: Inspired by this theoretical analysis, we proposeour fairwashing detector, FRAUD-Detect , which locates fairwashing violations.
Revised: Inspired by this theoretical analysis, we proposeour fairwashing detector, FRAUD-Detect , which locates fairwashing violations.

Original: FRAUD-Detectaims to distinguish between honest and fairwashed interpretable models approximating the unfairblack-box model B ( · ) as described in Section 2.2 under the additional following constraints: 1)without white-box access to the black-box and interpretable models as well as; 2) independently ofthe cooperation of the model provider being audited.
Revised: FRAUD-Detectaims to distinguish between honest and fairwashed interpretable models approximating the unfairblack-box model B ( · ) as described in Section 2.2 under the additional following constraints: 1)without white-box access to the black-box and interpretable models as well as; 2) independently ofthe cooperation of the model provider being audited.

Original: These constraints are introduced to conform torealistic scenarios in which entities desire to retain the confidentiality of their black-box model due tointellectual property rights and because they considered it as a company’s asset.
Revised: These constraints are introduced to conform torealistic scenarios in which entities desire to retain the confidentiality of their black-box model due tointellectual property rights and because they considered it as a company’s asset.

Original: Similarly to definitions of true-positive T + a and false-positive F + a rates of I ( · ) w.r.t B ( · ) on datawith attribute a ∈ A = { 0 , 1 } , we define the true-negative rate, T − a , and false-negative rate, F − a as: 
Revised: Similarly to definitions of true-positive T + a and false-positive F + a rates of I ( · ) w.r.t B ( · ) on datawith attribute a ∈ A = { 0 , 1 } , we define the true-negative rate, T − a , and false-negative rate, F − a as: T − [Equation]

Original: T − a = 
Revised: = Pr[ [Equation] |

Original: Pr[ ˆ Y[-I = 
-]
Revised: [Equation] Y[-I = 
-]

Original: | ˆ Y B = 0 , A = a ] , F − a =
Revised: [Equation] = 0 , A = a ] , F [Equation] =

Original: Pr[ ˆ Y[-I = 
-]
Revised: Pr [Equation] ˆ Y[-I = 
-]

Revised: [Equation]

Original:  | ˆ Y B = 1 , A = a ] for a ∈ A = { 0 , 1 } .
Revised: = [Equation] | ˆ Y B = 1 , A = a ] for a ∈ A = { 0 , 1 } .

Original: Note 239 that
Revised: Note 247 that

Original: T − a = 1 − F + a and F − a = 1 − T + a .
Revised: T − a = 1 − F + a and F − a = 1 − T + a .

Original: FRAUD-Detect computes the T + a , F + a , T − a , F − a for each subpopulation
Revised: FRAUD-Detect computes the T + a , F + a , T − a , F − a for each subpopulation

Original: a ∈ A = { 0 , 1 } using the predictions of the interpretable model and black-box model on the suing set
Revised: a ∈ A = { 0 , 1 } using the predictions of the interpretable model and black-box model on the suing set

Original: X sg .
Revised: X sg .

Original: Let
Revised: Let

Original: C 

Revised: C

Original: =
Revised: =

Original: [ T +0 , F +0 , T − 0 , F − 0 ] and C 1 =
Revised: [ T +0 , F +0 , T − 0 , F − 0 ] and C 1 =

Original: [ T +1 , F +1 , T − 1 , F − 1 ] be the (flattened) confusion matrices oftheinterpretable model w.r.t theblack-box model on X sg with attribute 0 and 1 , respectively.
Revised: [ T +1 , F +1 , T − 1 , F − 1 ] be the (flattened) confusion matrices of the

Original: Then,
Revised: Input: Query access to the interpretable model

Original: FRAUD-Detect computes the divergence between C 0 and C 1 using Kullback–Leibler (KL) as: [Equation] Recall that we demonstrated in Section 3 that the divergence in T +0 and T +1 ( ˜ δ )  andF +0 and F +1 ( δ ′ )were sufficient for a detection method. We can complement 1 our detection method with additional
Revised: I ( · ) , suing dataset X sg , Black-box model predictions on the suing set B ( X sg ) , sensitive attribute a ∈ A and a threshold ∆ > 0 .

Original: Output: (cid:26) T fairwashing is detected
Revised: Output: (cid:26) T fairwashing is detected

Original: F fairwashing is not detected 1: ˆ Y B ← B ( X sg )
Revised: F fairwashing is not detected 1: ˆ Y I ← I

Original: ▷ Query black-box model 2: ˆ Y I ←

Original: I ( X sg )
Revised: ( X sg )

Original: ▷ Query interpretable model 3: for i ← 0 , 1 do 4:
Revised: ▷ Query interpretable model 2: for i ← 0 , 1 do 3:

Original: T + i ← |{ ˆ y B =1 , ˆ y I
Revised: T + i ← |{ ˆ y B =1 , ˆ y I

Original: =1 ,a = i }| |{ ˆ y B
Revised: =1 ,a = i }| |{ ˆ y B

Original: =1 ,a = i }| ▷ TPR 5:
Revised: =1 ,a = i }| ▷ TPR 4:

Original: F + i ← |{ ˆ y B =0 ,
Revised: F + i ← |{ ˆ y B =0 ,

Original: ˆ y I =1 ,a = i }| |{ ˆ y B
Revised: ˆ y I =1 ,a = i }| |{ ˆ y B

Original: =0 ,a = i }| ▷ FPR 6:
Revised: =0 ,a = i }| ▷ FPR 5:

Original: F − i ← |{ ˆ y B =1 , ˆ y I =0 ,a = i }| |{ ˆ y B
Revised: F − [Table] ← |{ [Table]

Revised: y

Revised: [Table]

Revised: =

Revised: [Table]

Revised: [Table]

Revised: =

Revised: [Table]

Revised: [Table]

Revised: = i }| |{

Revised: [Table]

Revised: y

Revised: [Table]

Revised: =

Revised: [Table]

Original: =1 ,a = i }| ▷ FNR 7:
Revised: [Table] = [Table] | ▷ FNR 6:

Original: T − i ← |{ ˆ y B =0 ,
Revised: T [Table] ← |{ [Table]

Original: ˆ y I =0 ,a = i }| |{ ˆ y B
Revised: = [Table] [Table] =

Revised: [Table]

Revised: [Table]

Revised: =

Revised: [Table]

Revised: }| |{

Revised: [Table]

Revised: y

Revised: [Table]

Revised: =

Revised: [Table]

Revised: ,a = i }

Revised: [Table]

Original: =0 ,a = i }| ▷ TNR 8: C 0 ← [ T +0 , F +0 , T − 0 , F − 0 ] 9: C 1 ← [ T +1 , F +1 , T − 1 , F − 1 ] 10: C KL ← KL ( C 0 , C 1 ) ▷ Kullback–Leibler 11: if C KL > ∆ then 12: return T 13: else 14: return F divergences in T − a and F − a and measure the dissimilarity via KL divergence.
Revised: ▷ TNR 7: C 0 ← [ T +0 , F +0 , T − 0 , F − 0 ] 8: C 1 ← [ T +1 , F +1 , T − 1 , F − 1 ] 9: C KL ← KL ( C 0 , C 1 ) ▷ Kullback–Leibler 10: if C KL > ∆ then 11: return T 12: else 13: return F interpretable model w t on FRAUD-Detect computes the divergence between C and C 1 using Kullback Leibler (

Revised: ) as:

Revised: [Equation]

Original:  This choice is naturalas the KL divergence is commonly used to quantify the divergence over probability distributions—
Revised: [Equation] Recall that we demonstrated in Section 3 that the divergence in T +0 and T +1 ( ˜ δ ) and F +0 and F +1 ( δ ′ )were sufficient for a detection method. We can complement 1 our detection method with additionaldivergences in T − a and F − a and measure the dissimilarity via KL divergence. This choice is naturalas the KL divergence is commonly used to quantify the divergence over probability distributions—

Original: C a functions as a simplified representation of the probability distribution Y I | Y B , A = a .
Revised: C a functions as a simplified representation of the probability distribution Y I | Y B , A = a .

Original: For anhonest interpretable model, C KL is generally relatively low ( ˜ δ, δ ′ ≈ 0 ) as the only divergence arisesfrom general error in fidelity optimization and the fairness gap of the black-box model (as shownin Section 3).
Revised: For anhonest interpretable model, C KL is generally relatively low ( ˜ δ, δ ′ ≈ 0 ) as the only divergence arisesfrom general error in fidelity optimization and the fairness gap of the black-box model (as shownin Section 3).

Original: In other words, the honest interpretable model approximates the black-box modelequivalently across subpopulations.
Revised: In other words, the honest interpretable model approximates the black-box modelequivalently across subpopulations.

Original: However, for a fairwashed interpretable model, C KL growssignificantly, as the interpretable model is explicitly manipulated to optimize the fairness acrosssubpopulations and improve over the black-box model.
Revised: However, for a fairwashed interpretable model, C KL growssignificantly, as the interpretable model is explicitly manipulated to optimize the fairness acrosssubpopulations and improve over the black-box model.

Original: We quantify the distinction between honestand fairwashed interpretable models via a threshold ∆ > 0 on C KL such that if C KL > ∆ , the 257 interpretable model is considered fairwashed.
Revised: We quantify the distinction between honestand fairwashed interpretable models via a threshold ∆ > 0 on C KL such that if C KL > ∆ , the 265 interpretable model is considered fairwashed.

Original: Note here that due to the irreducibility result statedin Section 3, ∆ must be chosen such that ∆ is tightly greater than the irreducible term in order toproperly distinguish between accidental fairwashing that occurs as a result of the irreducibility andmalicious fairwashing.
Revised: Note here that due to the irreducibility result statedin Section 3, ∆ must be chosen such that ∆ is tightly greater than the irreducible term in order toproperly distinguish between accidental fairwashing that occurs as a result of the irreducibility andmalicious fairwashing.

Original: This procedure is detailed in Algorithm 1.
Revised: This procedure is detailed in Algorithm 1.

Original:  Evading FRAUD-Detect
Revised:  Evading FRAUD-Detect

Original: In this section, we investigate on whether a dishonest entity could evade FRAUD-Detect whileperforming fairwashing.
Revised: In this section, we investigate on whether a dishonest entity could evade FRAUD-Detect whileperforming fairwashing.

Original: We assume that this dishonest entity, which we call the adversary , isinformed about FRAUD-Detect .
Revised: We assume that this dishonest entity, which we call the adversary , isinformed about FRAUD-Detect .

Original: Namely, the adversary is aware of the fairwashing detection methodand desires to fairwash while evading detection by the auditor.
Revised: Namely, the adversary is aware of the fairwashing detection methodand desires to fairwash while evading detection by the auditor.

Original: To achieve this goal, the informedadversary has the objective of finding a fairwashed interpretable model that satisfies an additionalconstraint on the Kullback–Leibler (KL) divergence of the confusion matrices among subpopulations,
Revised: To achieve this goal, the informedadversary has the objective of finding a fairwashed interpretable model that satisfies an additionalconstraint on the Kullback–Leibler (KL) divergence of the confusion matrices among subpopulations,

Original: C KL in the previous section.
Revised: C KL in the previous section.

Original: To probe the robustness of our detector to an informed adversaryempirically, we explore the range of fairness gap given a fixed value of fidelity and a fixed value of
Revised: To probe the robustness of our detector to an informed adversaryempirically, we explore the range of fairness gap given a fixed value of fidelity and a fixed value of

Original: C KL via solving the informed adversary optimization problem.
Revised: C KL via solving the informed adversary optimization problem.

Original: Definition 5 (Informed Adversary Optimization) .
Revised: Definition 5 (Informed Adversary Optimization) .

Original: Given a black-box model B ( · ) , a suing set X sg ,a sensitive attribute A , a loss threshold v and a fairwashing detection threshold ∆ , an informedadversary aims to learn an interpretable I ( · ) such thatminimize
Revised: Given a black-box model B ( · ) , a suing set X sg ,a sensitive attribute A , a loss threshold v and a fairwashing detection threshold ∆ , an informedadversary aims to learn an interpretable I ( · ) such that

Original: [Equation]
Revised: [Equation]

Original: subject to
Revised: to

Original: [Equation]
Revised: [Equation]

Original: where C KL is the KL divergence between the subpopulation confusion matrices of X sg .
Revised: where C KL is the KL divergence between the subpopulation confusion matrices of X sg .

Original: The differences between the fairwashing optimization in Definition 4, and the informed adversaryoptimization of Definition 5 are two fold.

Original: F irst, the order of the maximand and the constraint isreversed.

Original: Both formulations are valid since each indicates priorities of the adversary, namely, whetherto put a hard limit on the fidelity loss or on the fairness gap of the interpretable model.

Original: Second, in

Original: Definition 5 we replaced the non-differentiable EmpiricalFidelity ( I, B ; X tr ) with a differentiablemeasure of loss L ( I ; X sg ) .

Original: In our empirical evaluations, we use the logistic regression loss.

Original: 1 1 − (cid:15) .
Revised: 1 1 − (cid:15) .

Original: C K L
Revised: C K L

Original: AB & DT .
Revised: AB & DT .

Original: 1 1 − (cid:15)
Revised: 1 1 − (cid:15)

Original: XGB & DT .
Revised: XGB & DT .

Original: 1 1 − (cid:15)
Revised: 1 1 − (cid:15)

Original: DNN & DT .
Revised: DNN & DT .

Original: 1 1 − (cid:15)
Revised: 1 1 − (cid:15)

Original: RF & DT .
Revised: RF & DT .

Original: 1 1 − (cid:15)
Revised: 1 1 − (cid:15)

Original: AB & LR .
Revised: AB & LR .

Original: 1 1 − (cid:15)
Revised: 1 1 − (cid:15)

Original: XGB & LR .
Revised: XGB & LR .

Original: 1 1 − (cid:15)
Revised: 1 1 − (cid:15)

Original: DNN & LR .
Revised: DNN & LR .

Original: 1 1 − (cid:15)
Revised: 1 1 − (cid:15)

Original: RF & LR .
Revised: RF & LR .

Original: P a r i t
Revised: P a r i t

Original: y ga p
Revised: y ga p

Original: o f I ( · )
Revised: o f I ( · )

Revised: The differences between the fairwashing optimization in Definition 4, and the informed adversaryoptimization of Definition 5 are two fold.

Revised: First, the order of the maximand and the constraint isreversed.

Revised: Both formulations are valid since each indicates priorities of the adversary, namely, whetherto put a hard limit on the fidelity loss or on the fairness gap of the interpretable model.

Revised: Second, in

Revised: Definition 5 we replaced the non-differentiable EmpiricalFidelity ( I, B ; X tr ) with a differentiablemeasure of loss L ( I ; X sg ) .

Original:  To solve the above optimisation problem, we consider the Rashomon set of high-fidelity interpretablemodels and compute the range of the fairness gap of interpretable models.
Revised: In our empirical evaluations, we use the logistic regression loss. To solve the above optimisation problem, we consider the Rashomon set of high-fidelity interpretablemodels and compute the range of the fairness gap of interpretable models.

Original: In short, given a classification task, the Rashomon Set [41] is defined as the set of almost-equally-accurate models (interpretablemodels in our case).
Revised: In short, given a classification task, the Rashomon Set [41] is defined as the set of almost-equally-accurate models (interpretablemodels in our case).

Original: More precisely, given a model class F , a loss function L D ( · ) over a dataset D 285 of interest, a reference model f ∗ ( e.g. , optimal model) and a performance threshold τ ∈ [0 , 1] , the 286 Rashomon set R s (
Revised: More precisely, given a model class F , a loss function L D ( · ) over a dataset D 293 of interest, a reference model f ∗ ( e.g. , optimal model) and a performance threshold τ ∈ [0 , 1] , the 294 Rashomon set R s (

Original: F , f ∗ , τ ) =
Revised: F , f ∗ , τ ) =

Original: { f ∈ F | L D ( f )
Revised: { f ∈ F | L D ( f )

Original: ≤ L D ( f ∗ ) + τ } .
Revised: ≤ L D ( f ∗ ) + τ } .

Original: We extend the Fairness in the Rashomon Set (FaiRS) algorithm [19] to compute the range of thefairness gap of interpretable models that can be generated over the set of high-fidelity modelssatisfying a constraint on the KL divergence.
Revised: We extend the Fairness in the Rashomon Set (FaiRS) algorithm [19] to compute the range of thefairness gap of interpretable models that can be generated over the set of high-fidelity modelssatisfying a constraint on the KL divergence.

Original: FaiRS exploits the so-called Rashomon Effect [14],which is an empirical phenomenon resulting in multiple models displaying the same performanceoverall ( e.g. , w.r.t their global accuracy) but have significant differences in terms of their individualpredictions [19].
Revised: FaiRS exploits the so-called Rashomon Effect [14],which is an empirical phenomenon resulting in multiple models displaying the same performanceoverall ( e.g. , w.r.t their global accuracy) but have significant differences in terms of their individualpredictions [19].

Original: In our setting, the Rashomon effect implies that several interpretable models canachieve the same high fidelity w.r.t the black-box model while displaying different values of fairnessgap.
Revised: In our setting, the Rashomon effect implies that several interpretable models canachieve the same high fidelity w.r.t the black-box model while displaying different values of fairnessgap.

Original: Thus, computing the range of the fairness gap of high-fidelity interpretable models that satisfy aconstraint on C KL quantifies the robustness of FRAUD-Detect to the informed adversary.
Revised: Thus, computing the range of the fairness gap of high-fidelity interpretable models that satisfy aconstraint on C KL quantifies the robustness of FRAUD-Detect to the informed adversary.

Original: To the best of our knowledge, the FaiRS algorithm of [19] is the only work proposing a practicalsolution to efficiently characterize the range of the fairness gap over the Rashomon set.
Revised: To the best of our knowledge, the FaiRS algorithm of [19] is the only work proposing a practicalsolution to efficiently characterize the range of the fairness gap over the Rashomon set.

Original: Alternativeapproaches would require measuring the range of the fairness gap over approximations of the
Revised: Alternativeapproaches would require measuring the range of the fairness gap over approximations of the

Original: Rashomon set obtained by either generating models using several hyperparameter values ( e.g. ,seeds [32, 21]) or brute-forcing over a particular class of model ( e.g. , depth seven decision trees [41]).
Revised: Rashomon set obtained by either generating models using several hyperparameter values ( e.g. ,seeds [33, 21]) or brute-forcing over a particular class of model ( e.g. , depth seven decision trees [41]).

Original: This is not practical due to the high computational cost and may lead to sub-optimal results.
Revised: This is not practical due to the high computational cost and may lead to sub-optimal results.

Original:  Empirical validation
Revised:  Empirical validation

Original: FRAUD-Detect aims to detect fairwashing by distinguishing between a fairwashed interpretablemodel and honest interpretable one during an audit.
Revised: FRAUD-Detect aims to detect fairwashing by distinguishing between a fairwashed interpretablemodel and honest interpretable one during an audit.

Original: Additionally, FRAUD-Detect seeks to be robusteven against an informed adversary, i.e. , demonstrate the ability to detect fairwashing even when adishonest entity is aware of FRAUD-Detect and attempts evasion.
Revised: Additionally, FRAUD-Detect seeks to be robusteven against an informed adversary, i.e. , demonstrate the ability to detect fairwashing even when adishonest entity is aware of FRAUD-Detect and attempts evasion.

Original: Therefore, we empirically validatethe performance of FRAUD-Detect in: 1) capturing a relationship between C KL and demographicparity gap of the interpretable model I ( · ) : C KL increases as the demographic parity gap of theinterpretable model decreases ( i.e. as fairwashing becomes more severe); 2) restricting the fairwashingcapabilities of the informed adversary: incorporation of C KL constraint into fairwashing problemprevents favorable demographic parity gaps for the interpretable model, precluding fairwashing.
Revised: Therefore, we empirically validatethe performance of FRAUD-Detect in: 1) capturing a relationship between C KL and demographicparity gap of the interpretable model I ( · ) : C KL increases as the demographic parity gap of theinterpretable model decreases ( i.e. as fairwashing becomes more severe); 2) restricting the fairwashingcapabilities of the informed adversary: incorporation of C KL constraint into fairwashing problemprevents favorable demographic parity gaps for the interpretable model, precluding fairwashing.

Original: We assess the performance of FRAUD-Detect using a diverse set of black-box architectures, interpretable models and datasets.
Revised: We assess the performance of FRAUD-Detect using a diverse set of black-box architectures, interpretable models and datasets.

Original: More precisely, we consider four architectures of black-box models:
Revised: More precisely, we consider four architectures of black-box models:

Original: Deep Neural Networks (DNN), AdaBoost (AB) [24], Gradient Boosted Decision Trees (XGBs)
Revised: Deep Neural Networks (DNN), AdaBoost (AB) [24], Gradient Boosted Decision Trees (XGBs)

Original: [17]and Random Forests (RFs) [13].
Revised: [17]and Random Forests (RFs) [13].

Original: We evaluate the approach on three real-world datasets correspondingto critical decision systems: Adult Income [22], Bank Marketing [33] and COMPAS [5].
Revised: We evaluate the approach on three real-world datasets correspondingto critical decision systems: Adult Income [22], Bank Marketing [34] and COMPAS [5].

Original: We relyon Logistic Regression (LR) and Decision Trees (DTs) as our interpretable models due to theirhigh performance and ease of explainability.
Revised: We relyon Logistic Regression (LR) and Decision Trees (DTs) as our interpretable models due to theirhigh performance and ease of explainability.

Original: We refer to Appendix D for details on these datasets,black-box models, interpretable models and implementation of FRAUD-Detect .
Revised: We refer to Appendix E for details on these datasets,black-box models, interpretable models and implementation of FRAUD-Detect .

Original: FRAUD-Detect successfully detects fairwashing.

Original: Figure 2 illustrates both the demographic paritygap and C KL of interpretable models as a function of ϵ .

Original: The bigger the value of ϵ , the stronger thefairwashing (see Definition 4).

Original: Any graph within the figures may be read from left to right as thedegree of fairwashing increases.

Original: As fairwashing becomes more severe, the demographic parity gap

Original: AB .65 .69 .

Revised: AB .65 .69 .

Original: XGB .7 .

Revised: XGB .7 .

Original: DNN .75 .78 .84 .86 .
Revised: DNN .75 .78 .84 .86 .

Original: RF 0 .0 .0 .0 .0 .1 .
Revised: RF 0 .0 .0 .0 .0 .1 .

Original: Fidelity 0 .0 .0 .0 .1 .
Revised: Fidelity 0 .0 .0 .0 .1 .

Original: P a r i t
Revised: P a r i t

Original: y ga p
Revised: y ga p

Original: o f I ( · ) ∆: .
Revised: o f I ( · ) ∆: .

Revised: FRAUD-Detect successfully detects fairwashing.

Original: decreases while C KL increases such that we observe a relationship between the demographic paritygapof interpretable models andthe associated C KL values 2 . 
Revised: Figure 2 illustrates both the demographic paritygap and C KL of interpretable models as a function of ϵ . The bigger the value of ϵ , the stronger thefairwashing (see Definition 4).

Revised: Any graph within the figures may be read from left to right as thedegree of fairwashing increases.

Revised: As fairwashing becomes more severe, the demographic parity gapdecreases while C KL increases such that we observe a relationship between the demographic paritygap of interpretable models and the associated C KL values.

Original: For example, decreasing the demographicparity gap of Decision Trees trained on a COMPAS dataset labelled by a RF black-box model byhalf, increases C KL by a factor of 4.
Revised: For example, decreasing the demographicparity gap of Decision Trees trained on a COMPAS dataset labelled by a RF black-box model byhalf, increases C KL by a factor of 4.

Original: The amount of change in C KL and in the demographic paritygap of fairwashed interpretable models differ across the dataset and black-box models due to thedifferent original fairness of the black-box models.
Revised: The amount of change in C KL and in the demographic paritygap of fairwashed interpretable models differ across the dataset and black-box models due to thedifferent original fairness of the black-box models.

Original: As C KL is sensitive to changes in fairwashing,
Revised: As C KL is sensitive to changes in fairwashing,

Original: FRAUD-Detect can be used to detect fairwashing by not permitting C KL to pass a certain thresholdvalue.
Revised: FRAUD-Detect can be used to detect fairwashing by not permitting C KL to pass a certain thresholdvalue.

Original: The C KL thresholds corresponding to 5% and 50% fairwashing are reported in Appendix F.
Revised: The C KL thresholds corresponding to 5% and 50% fairwashing are reported in Appendix G.

Original: Per-seed results are provided in Appendix G.
Revised: Per-seed results are provided in Appendix H.

Original: FRAUD-Detect is robust to the informed adversary.
Revised: FRAUD-Detect is robust to the informed adversary.

Original: Figure 3 shows the range of the demographicparity gap of high-fidelity fairwashed interpretable models subjected to a constraint on the C KL bysolving the optimization problem of Equation (9).
Revised: Figure 3 shows the range of the demographicparity gap of high-fidelity fairwashed interpretable models subjected to a constraint on the C KL bysolving the optimization problem of Equation (9).

Original: We use four types of black-box models (AB, XGB,
Revised: We use four types of black-box models (AB, XGB,

Original: DNN and RF).
Revised: DNN and RF).

Original: However, for simplicity, we performed the experiment using logistic regression asthe interpretable model.
Revised: However, for simplicity, we performed the experiment using logistic regression asthe interpretable model.

Original: The fairwashing detection thresholds were chosen from a wide range, i.e.∆ ∈ { 0 . 01 , 0 .
Revised: The fairwashing detection thresholds were chosen from a wide range, i.e.∆ ∈ { 0 . 01 , 0 .

Original: 10 , 0 . 20 } (the constraint on C KL in Equation (9)).
Revised: 10 , 0 . 20 } (the constraint on C KL in Equation (9)).

Original: A direct implication 339 for FRAUD-Detect is that using low fairwashing detection thresholds makes fairwashing difficult.
Revised: A direct implication 347 for FRAUD-Detect is that using low fairwashing detection thresholds makes fairwashing difficult.

Original: Consistently over all these results, imposing a constraint on the C KL significantly narrows the rangeof demographic parity gap of fairwashed interpretable models.
Revised: Consistently over all these results, imposing a constraint on the C KL significantly narrows the rangeof demographic parity gap of fairwashed interpretable models.

Original: For instance, on COMPAS with∆ = 0 . 2 , for an AB model that has demographic parity gap of 0 . 214 , fairwashing can producea completely fair ( i.e. , demographic parity gap of 0 . 0 ) high-fidelity interpretable model whileremaining undetected ( i.e. , C KL < 0 . 2 ).
Revised: For instance, on COMPAS with∆ = 0 . 2 , for an AB model that has demographic parity gap of 0 . 214 , fairwashing can producea completely fair ( i.e. , demographic parity gap of 0 . 0 ) high-fidelity interpretable model whileremaining undetected ( i.e. , C KL < 0 . 2 ).

Original: However, when C KL decreases to 0 .
Revised: However, when C KL decreases to 0 .

Original: 1 , the fairest high-fidelityinterpretable model obtained by the adversary exhibits a demographic parity gap of 0 . 107 .
Revised: 1 , the fairest high-fidelityinterpretable model obtained by the adversary exhibits a demographic parity gap of 0 . 107 .

Original:  Conclusion and Future Work
Revised:  Conclusion and Future Work

Original: Incorporating fairness and explainability in decision systems can help foster trust [28] in them bycharacterizing failure modes ( e.g. , unfairness) and providing assurances ( e.g. , fairness constraints)
Revised: Incorporating fairness and explainability in decision systems can help foster trust [29] in them bycharacterizing failure modes ( e.g. , unfairness) and providing assurances ( e.g. , fairness constraints)

Original: Auditing provides an oversight component and helps avoid first-order failures such as unfairness.
Revised: Auditing provides an oversight component and helps avoid first-order failures such as unfairness.

Original: Inthis sense, fairwashing can be seen as failure of auditing—a second-order failure that reduces trustin both fairness and explainability.
Revised: Inthis sense, fairwashing can be seen as failure of auditing—a second-order failure that reduces trustin both fairness and explainability.

Original: Second-order failures help characterize the limits of techniquesdesigned to reduce the risks of first-order failures.
Revised: Second-order failures help characterize the limits of techniquesdesigned to reduce the risks of first-order failures.

Original: Our work is the first to delineate the theoreticallimits of what is possible in auditing fairness and in containing the risk of fairwashing.
Revised: Our work is the first to delineate the theoreticallimits of what is possible in auditing fairness and in containing the risk of fairwashing.

Original: In practice,we demonstrated the possibility of performing fairwashing through joint optimization of fidelity andfairness constraints when providing explanations.
Revised: In practice,we demonstrated the possibility of performing fairwashing through joint optimization of fidelity andfairness constraints when providing explanations.

Original: This could thwart auditing processes that are solelybased on an analysis of the delivered interpretable model.
Revised: This could thwart auditing processes that are solelybased on an analysis of the delivered interpretable model.

Original: We address this issue by proposing anadditional auditing protocol querying both theblack-box and interpretable model.
Revised: We address this issue by proposing anadditional auditing protocol that queries the interpretable model.

Original: Finally, throughboth a theoretical framework and an experimental evaluation, we demonstrate that our fairwashingdetector cannot be evaded by an attacker, informed of our detector, without significantly degradingthe fidelity of model explanations.
Revised: Finally, through both a theoreticalframework and an experimental evaluation, we demonstrate that our fairwashing detector cannot beevaded by an attacker, informed of our detector, without significantly degrading the fidelity of modelexplanations.

Original: Future directions include the extension of the detector to otherfairness measures ( e.g. , measures incorporating ground-truth information) as well as extending theanalysis to multi-class setups.
Revised: Future directions include the extension of the detector to other fairness measures ( e.g. ,measures incorporating ground-truth information) as well as extending the analysis to multi-classsetups.

Original: However, these would both require additional assumptions on the datamanifold (to account for class correlations) to establish similar results as in Section 3.
Revised: However, these would both require additional assumptions on the data manifold (to accountfor class correlations) to establish similar results as in Section 3.

Original: Broader Impact Statement
Revised: Broader Impact Statement

Original: FRAUD-Detect may be used to provide evaluations of the fairness of model explanations, resulting inincreased trust in explanations, public expectation of model fairness, and model owner accountability.
Revised: FRAUD-Detect may be used to provide evaluations of the fairness of model explanations, resulting inincreased trust in explanations, public expectation of model fairness, and model owner accountability.

Original: Since FRAUD-Detect is non-cooperative, requires only query access from the black-box model, andresists evasion even against an informed adversary, it empowers auditors, protects the intellectualproperty and ownership of the model owners, and may resist gamification into a new arms-race systembetween fairwashing and auditor detection methods.
Revised: Since FRAUD-Detect is non-cooperative and resists evasion even against an informed adversary, itempowers auditors, protects the intellectual property and ownership of the model owners, and mayresist gamification into a new arms-race system between fairwashing and auditor detection methods.

Original: By empowering auditors, FRAUD-Detect alsoenforces democratic ideals of equality by providing an avenue for users to receive fairness-vettedmodel explanations for high-stakes decisions, further increasing public trust in AI and model auditing.
Revised: By empowering auditors, FRAUD-Detect also enforces democratic ideals of equality by providingan avenue for users to receive fairness-vetted model explanations for high-stakes decisions, furtherincreasing public trust in AI and model auditing.

Original: The potential risks of this work revolve around the fairness definition and datasets used in thispaper.
Revised: The potential risks of this work revolve around the fairness definition and datasets used in thispaper.

Original: Demographic parity, while befitting our setting (as discussed in Section 2.1) is not alwaysan appropriate fairness definition, depending on the model’s domain and application.
Revised: Demographic parity, while befitting our setting (as discussed in Section 2.1) is not alwaysan appropriate fairness definition, depending on the model’s domain and application.

Original: We recognisethat continual focus on a concept reinforces it
Revised: We recognisethat continual focus on a concept reinforces it

Original: and it’s ideology’s normalcy, and that our focuson demographic parity also reinforces the political and philosophical ideologies underlying it anddeprives alternate definitions of due attention.
Revised: and it’s ideology’s normalcy, and that our focuson demographic parity also reinforces the political and philosophical ideologies underlying it anddeprives alternate definitions of due attention.

Original: Likewise, as noted in Appendix D, our evaluation of
Revised: Likewise, as noted in Appendix E, our evaluation of

Original: FRAUD-Detect on the current popular fairness benchmark datasets normalizes these applications of
Revised: FRAUD-Detect on the current popular fairness benchmark datasets normalizes these applications of

Original: AI.
Revised: AI.

Original: We draw specific attention to the use of the COMPAS recidivism prediction dataset [5], and notethat we do not necessarily condone the use of AI models for this application.
Revised: We draw specific attention to the use of the COMPAS recidivism prediction dataset [5], and notethat we do not necessarily condone the use of AI models for this application.

Original: To mitigate these societal risks, we advocate for similar auditing measures for second-order failureswith respect to other fairness definitions (see Section 7).
Revised: To mitigate these societal risks, we advocate for similar auditing measures for second-order failureswith respect to other fairness definitions (see Section 7).

Original: We also acknowledge the importance ofinvestigating the impact of FRAUD-Detect in real-world scenarios, especially where the group ofusers suing for model explanations represents minority or sensitive demographics.
Revised: We also acknowledge the importance ofinvestigating the impact of FRAUD-Detect in real-world scenarios, especially where the group ofusers suing for model explanations represents minority or sensitive demographics.

Original: Checklist
Revised: Checklist

Original: For all authors.
Revised: For all authors...

Revised: (a) Do the main claims made in the abstract and introduction accurately reflect the paper’scontributions and scope?

Revised: [Yes]

Original: (b) Did you describe the limitations of your work?
Revised: (b) Did you describe the limitations of your work?

Original: [Yes] See Section 7.
Revised: [Yes] See Section 7.

Original: [Table]

Original: (c) Did you discuss any potential negative societal impacts of your work?
Revised: (c) Did you discuss any potential negative societal impacts of your work?

Original: [Yes]
Revised: [Yes]

Original: In the
Revised: In the

Original: [Table]

Original: Societal Impact Statement
Revised: Societal Impact Statement

Original: [Table] (d) Have you read the ethics review guidelines and ensured that your paper conforms to[Table]
Revised: (d) Have you read the ethics review guidelines and ensured that your paper conforms to

Original: them?
Revised: them?

Original: [Yes]
Revised: [Yes]

Original: [Table]

Original: If you are including theoretical results...
Revised: If you are including theoretical results...

Original: [Table]

Original: (a) Did you state the full set of assumptions of all theoretical results?
Revised: (a) Did you state the full set of assumptions of all theoretical results?

Original: [Yes] See Assump-
Revised: [Yes] See Assump-

Original: [Table]

Original: tion 1.
Revised: tion 1.

Original: [Table]

Original: (b) Did you include complete proofs of all theoretical results?
Revised: (b) Did you include complete proofs of all theoretical results?

Original: [Yes] See Appendix C3.
Revised: [Yes] See Appendix D

Original: If you ran experiments...
Revised: If you ran experiments...

Original: [Table] (a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
Revised: (a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URL)?

Original: [Yes] See supplemen- [Table] tal material.[Table]
Revised: [Yes] See supplemental material.

Original: (b) Did you specify all the training details (e.g., data splits, hyperparameters, how they
Revised: (b) Did you specify all the training details (e g., data splits, hyperparameters, how theywere chosen)?

Original: [Table]

Original: were chosen)?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: [Table] (c) Did you report error bars (e.g., with respect to the random seed after running experi-[Table]
Revised: (c) Did you report error bars (e g., with respect to the random seed after running experi-

Original: ments multiple times)?
Revised: ments multiple times)?

Original: [Yes]
Revised: [Yes]

Original: [Table] (d) Did you include the total amount of compute and the type of resources used (e.g., type [Table] of GPUs, internal cluster, or cloud provider)?
Revised: (d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?

Original: [Yes]
Revised: [Yes]

Original: [Table]

Original: If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
Revised: If you are using existing assets (e g., code, data, models) or curating/releasing new assets...

Original: [Table]

Original: (a) If your work uses existing assets, did you cite the creators?
Revised: (a) If your work uses existing assets, did you cite the creators?

Original: [Yes]
Revised: [Yes]

Original: [Table]

Original: (b) Did you mention the license of the assets?
Revised: (b) Did you mention the license of the assets?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: [Table]

Original: (c) Did you include any new assets either in the supplemental material or as a URL?
Revised: (c) Did you include any new assets either in the supplemental material or as a URL?

Original: [Yes]
Revised: [Yes]

Original: [Table]

Original: See supplemental material.
Revised: See supplemental material.

Original: [Table] (d) Did you discuss whether and how consent was obtained from people whose data you’re[Table]
Revised: (d) Did you discuss whether and how consent was obtained from people whose data you’re

Original: using/curating?
Revised: using/curating?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: [Table]

Original: (e) Did you discuss whether the data you are using/curating contains personally identifiable
Revised: (e) Did you discuss whether the data you are using/curating contains personally identifiable

Original: information or offensive content?
Revised: information or offensive content?

Original: [Yes] See supplemental material.
Revised: [Yes] See supplemental material.

Original: If you used crowdsourcing or conducted research with human subjects...
Revised: If you used crowdsourcing or conducted research with human subjects...

Original: (a) Did you include the full text of instructions given to participants and screenshots, if
Revised: (a) Did you include the full text of instructions given to participants and screenshots, if

Original: applicable?
Revised: applicable?

Original: [N/A]
Revised: [N/A]

Original: (b) Did you describe any potential participant risks, with links to Institutional Review
Revised: (b) Did you describe any potential participant risks, with links to Institutional Review

Original: Board (IRB) approvals, if applicable?
Revised: Board (IRB) approvals, if applicable?

Original: [N/A]
Revised: [N/A]

Original: (c) Did you include the estimated hourly wage paid to participants and the total amount
Revised: (c) Did you include the estimated hourly wage paid to participants and the total amount

Original: spent on participant compensation?
Revised: spent on participant compensation?

Original: [N/A]
Revised: [N/A]

Original: A Explanation methods
Revised: A Explanation methods

Original: Methods to facilitate the understanding of machine learning models are usually classified into twomain categories: transparent-box design and black-box model explanation [26].
Revised: Methods to facilitate the understanding of machine learning models are usually classified into twomain categories: transparent-box design and black-box model explanation [27].

Original: Transparent-boxdesign concerns methods that produce inherently interpretable models such as logistic regressorsand decision trees [12, 31].
Revised: Transparent-boxdesign concerns methods that produce inherently interpretable models such as logistic regressors anddecision trees [12, 32].

Original: Black-box model explanation corresponds to approaches designed toexplain how black-box models produce their outcomes.
Revised: Black-box model explanation corresponds to approaches designed to explainhow black-box models produce their outcomes.

Original: For instance, global explanations techniques( e.g. , TREPAN [20] and DECTEXT [11]) focus on explaining the whole logic of black-box modelsby training an inherently interpretable surrogate model.
Revised: For instance, global explanations techniques ( e.g. , TREPAN [20] and DECTEXT [11]) focus on explaining the whole logic of black-box models bytraining an inherently interpretable surrogate model.

Original: On the other hand, local explanations aimat explaining black-box models decisions on single data points ( e.g. , LIME [38],DICE [34] and GRADCAM [40]).
Revised: On the other hand, local explanations aim atexplaining black-box models decisions on single data points ( e.g. , LIME [38], and GRADCAM [40]).

Original: For simplicity, in this paper, we focus on fairwashing performed with globalexplanation models.
Revised: For simplicity, in this paper, we focus on fairwashing performed with global explanation models.

Original: B Fidelity
Revised: B Related Work

Revised: We highlight some of the differences in settings, assumptions and solutions between our work and[44, 25]:

Revised: Setting We rely on explainer models while they use a published data subset for fairness auditing.

Revised: Inparticular, they reveal some subset of the training data and their predictions.

Revised: We follow the settingconsidered in the literature introducing fairwashing [2, 3].

Revised: Slack et al.

Revised: explanations require inputperturbations while ours require the model owner to provide an interpretable model.

Revised: Assumptions Slack et al.

Revised: assume that fairness auditing is performed via model-agnostic localexplanations (e.g., LIME and SHAP) [44].

Revised: Both [44] and [25] assume an ideal detector withknowledge of the underlying training distribution of the model; and query access to the black-boxmodel.

Revised: However, we only assume access to black-box model predictions on the suing set.

Revised: We stressthat this dataset is available before any formal audit takes place (in the form of asking for modelexplanations).

Revised: This does not constitute query access to the black-box model; in fact it is only datasetaccess, which here is the predictions on the suing set.

Revised: Solutions [44, 25] attempt to detect fairwashing using a Kolmogorov-Smirnov (KS) test over theunderlying and company-provided subset distribution to determine if the data subset was honestlyprovided.

Revised: More precisely, they show that detection is difficult when fairwashing is conductedby minimizing the Wasserstein distance between the distributions while subject to fairness; thisoptimization also minimizes the upper-bound of the advantage (i.e., distinguishability) of the KS test.

Revised: This differs from our setting in which an informed malicious company must optimize the fairnesssubject to both fidelity (loss) and the detection threshold.

Revised: Explanation methods like LIME and SHAPperturb inputs to gauge feature relevance, inadvertently querying with synthetic data that may bedetected with out-of-distribution detection.

Revised: Queries determined to originate from explainers arefed to a fair model whereas in-distribution points are given to the biased model.

Revised: In contrast, ourdetection method is non-cooperative (we require no additional information from the black-box modelas the auditor already has the predictions on the suing set) and therefore does not rely on inputperturbations.

Revised: C Fidelity

Original: Definition 6 (Fidelity [20]) .
Revised: Definition 6 (Fidelity [20]) .

Original: The fidelity of an interpretable model I ( · ) with respect to a black-boxmodel B ( · ) is defined as the relative accuracy of I ( · ) with respect to B ( · ) :
Revised: The fidelity of an interpretable model I ( · ) with respect to a black-boxmodel B ( · ) is defined as the relative accuracy of I ( · ) with respect to B ( · ) :

Original: [Equation]
Revised: [Equation]

Original: where D is a data distribution, ˆ Y I is the prediction of the interpretable model and ˆ Y B is the predictionof black-box model.
Revised: where D is a data distribution, ˆ Y I is the prediction of the interpretable model and ˆ Y B is the predictionof black-box model.

Original: ˆ Y I and ˆ Y B are assumed to be both binary-valued random variables.
Revised: ˆ Y I and ˆ Y B are assumed to be both binary-valued random variables.

Original: C Additional Proofs
Revised: D Additional Proofs

Original: Below we iterate the full proof for Theorem 1, the main theorem in the paper.
Revised: Below we iterate the full proof for Theorem 1, the main theorem in the paper.

Original: Proof.
Revised: Proof.

Original: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which D adenotes the distribution over data with attribute a ∈ A .
Revised: Define T + a , the true-positive rate of I ( · ) with respect to B ( · ) on X ∼ D a , in which D adenotes the distribution over data with attribute a ∈ A .

Original: Define F + a , the false-positive rate of I ( · ) withrespect to B ( · ) on X ∼ D a :
Revised: Define F + a , the false-positive rate of I ( · ) withrespect to B ( · ) on X ∼ D a :

Original: [Equation]
Revised: [Equation]

Original: Note that by definition, T + a , F + a ∈
Revised: Note that by definition, T + a , F + a ∈

Original: We denote ˜ δ, δ ′ > 0 as the difference in value between
Revised: We denote ˜ δ, δ ′ > 0 as the difference in value between

Original: T +0 and T +1 , and F +0 and F +1 , respectively.
Revised: T +0 and T +1 , and F +0 and F +1 , respectively.

Original: Without loss of generality, let T +

Revised: Without loss of generality, let T +

Original: =
Revised: =

Original: T +0 + ˜
Revised: T +0 + ˜

Original: δ
Revised: δ

Original: and
Revised: and

Original: F +1 = F +0 + δ ′ where ˜ δ, δ ′ ∈ [0 , 1] .
Revised: F +1 = F +0 + δ ′ where ˜ δ, δ ′ ∈ [0 , 1] .

Original: To characterize γ
Revised: To characterize γ

Original: = Γ B − Γ I , we first express Γ I and Γ B in terms of ˜ δ , δ ′ , T +0 , T +1 , F +0 ,and F +1 , and Y B | A . Recall from Definition 3 that we define Γ I :=
Revised: = Γ B − Γ I , we first express Γ I and Γ B in terms of ˜ δ , δ ′ , T +0 , T +1 , F +0 ,

Original: Pr (cid:104)

Original: [-ˆ Y I = 
-]
Revised: [-ˆ Y I = 
-]

Original: | A = 0 (cid:105) − Pr (cid:104)

Original: [-ˆ Y I = 
-]
Revised: [-ˆ Y I = 
-]

Original: | A = 1 (cid:105) .

Original: We use Bayes’ formula to re-write Pr (cid:104)
Revised: We use Bayes’ formula to re-write Pr (cid:104)

Original: ˆ Y I = 

Revised: ˆ Y I =

Original: | A = a (cid:105) , a ∈ A = { 0 , 1 } in terms of the aforementioned variables.
Revised: | A = a (cid:105) , a ∈ A = { 0 , 1 } in terms of the aforementioned variables.

Original: For a ∈ { 0 , 1 } :
Revised: For a ∈ { 0 , 1 } :

Original: Pr

Original: [Equation]
Revised: [Equation]

Original: Given our derivation above
Revised: Given our derivation above

Original: [Equation]
Revised: [Equation]

Original: we continue by substituting terms into Γ I :
Revised: we continue by substituting terms into Γ I :

Original: [Equation]
Revised: [Equation]

Original: Now that we have derived Γ I in terms of our desired terms and Γ B
Revised: Now that we have derived Γ I in terms of our desired terms and Γ B

Original: [Equation]
Revised: [Equation]

Original: we eliminate Γ I from the equation 594 for γ :
Revised: we eliminate Γ I from the equation 632 for γ :

Original: [Equation]
Revised: [Equation]

Original: [Table]

Original: Corollary 1.
Revised: Corollary 1.

Original: Let T +1 , F +1 as defined in Section 3.
Revised: Let T +1 , F +1 as defined in Section 3.

Original: Then:
Revised: Then:

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Note that F +1 , T +1 ∈ [0 , 1] , therefore 1 + F +

Revised: Note that F +1 , T +1 ∈ [0 , 1] , therefore 1 + F +

Original: − T +1 ≥ 0 .
Revised: − T +1 ≥ 0 .

Original: We will show 1 + F +1 − T +

Revised: We will show 1 + F +1 − T +

Original: = 0if and only if T + = 1 and F + = 0 , i.e. if I ( · ) has perfect fidelity w.r.t.
Revised: = 0if and only if T + = 1 and F + = 0 , i.e. if I ( · ) has perfect fidelity w.r.t.

Original: B ( · ) .
Revised: B ( · ) .

Original: We re-arrange1 + F + 1 − T + 1{++
+} = 0 as:
Revised: We re-arrange1 + F +1 − T {++
+} = 0 as:

Original: [Equation]
Revised: [Equation]

Original: Since F +1 , T +1 ∈ [0 , 1] and the size of interval [0 , 1] is 1 − 0 =
Revised: Since F +1 , T +1 ∈ [0 , 1] and the size of interval [0 , 1] is 1 − 0 =

Original: 1 = T +1 − F +1 , T +1 and F +1 must beendpoints 1 and 0 , respectively.
Revised: 1 = T +1 − F +1 , T +1 and F +1 must beendpoints 1 and 0 , respectively.

Original: D Details on Experimental Setup of FRAUD-Detect
Revised: E Details on Experimental Setup of FRAUD-Detect

Original: In this section, we provide additional details on our experimental setup. 
Revised: In this section, we provide additional details on our experimental setup. [Table]

Original: Datasets.
Revised: Datasets.

Original: COMPAS [5] is a recidivism dataset released by ProPublica that includes data on 6 , 167offenders in prison from Broward County (Florida), with the classification task consisting in predicting whether an inmate will re-offend in the two years after their release.
Revised: COMPAS [5] is a recidivism dataset released by ProPublica that includes data on 6 , 167offenders in prison from Broward County (Florida), with the classification task consisting in predicting whether an inmate will re-offend in the two years after their release.

Original: 3 We consider race( African-American , Caucasian ) as the sensitive attribute in COMPAS.
Revised: 2 We consider race( African-American , Caucasian ) as the sensitive attribute in COMPAS.

Original: Adult Income [22] includes 48,842 profiles of individuals, each characterized by 13 attributes, drawn from the U.S census.
Revised: Adult Income [22] includes 48,842 profiles of individuals, each characterized by 13 attributes, drawn from the U.S census.

Original: The Adult Income binary classification task is to predict whether an individual’s income will exceed $50,000 a year.
Revised: The Adult Income binary classification task is to predict whether an individual’s income will exceed $50,000 a year.

Original: We use gender ( Female , Male ) as the sensitive attribute in Adult Income.
Revised: We use gender ( Female , Male ) as the sensitive attribute in Adult Income.

Original: Finally, Bank Marketing [33] constitutes information collected from a direct marketing campaignof a Portuguese banking institution between 2008 to 2013.
Revised: Finally, Bank Marketing [34] constitutes information collected from a direct marketing campaignof a Portuguese banking institution between 2008 to 2013.

Original: This dataset contains 41,188 profilesof individuals, each described by 20 attributes with the classification task being the prediction ofsubscription to a term deposit.
Revised: This dataset contains 41,188 profilesof individuals, each described by 20 attributes with the classification task being the prediction ofsubscription to a term deposit.

Original: We consider age ( between 30-60 or not between 30-60 ) as thesensitive attribute in Bank Marketing.
Revised: We consider age ( between 30-60 or not between 30-60 ) as thesensitive attribute in Bank Marketing.

Original: Training of black-box and interpretable models.
Revised: Training of black-box and interpretable models.

Original: For each experiment, the dataset is split randomlyinto three partitions: the training set (67%), the suing set (16.5%) and the testing set (16.5%),following the same procedure as in [3].
Revised: For each experiment, the dataset is split randomlyinto three partitions: the training set (67%), the suing set (16.5%) and the testing set (16.5%),following the same procedure as in [3].

Original: Each experiment is repeated with 10 separate random seeds.
Revised: Each experiment is repeated with 10 separate random seeds.

Original: The black-box models are learned from the training data points with their associated true labels.
Revised: The black-box models are learned from the training data points with their associated true labels.

Original: However, the interpretable models are trained from the suing set, which has been labelled by theblack-box model.
Revised: However, the interpretable models are trained from the suing set, which has been labelled by theblack-box model.

Original: More precisely, we use the exponentiated gradient algorithm [1] subject to thefairwashing constraint from the FairLearn Library [9] to learn the fairwashed interpretable model.
Revised: More precisely, we use the exponentiated gradient algorithm [1] subject to thefairwashing constraint from the FairLearn Library [9] to learn the fairwashed interpretable model.

Original: The exponentiated gradient algorithm when applied for fairness identifies the saddle point solutionjointly maximizing the fairness and minimizing the loss.
Revised: The exponentiated gradient algorithm when applied for fairness identifies the saddle point solutionjointly maximizing the fairness and minimizing the loss.

Original: To learn the fairwashed interpretablemodel of the informed adversary, we extend the Fairness In The Rashomon Set (FaiRS)
Revised: To learn the fairwashed interpretablemodel of the informed adversary, we extend the Fairness In The Rashomon Set (FaiRS)

Original: [19]framework by adding an additional constraint on C KL .
Revised: [19]framework by adding an additional constraint on C KL .

Original: FaiRS reduces the problem of minimizing theunfairness under two constraints on the fidelity and C KL to a saddle point problem, which is solvedwith the exponentiated gradient technique [1].
Revised: FaiRS reduces the problem of minimizing theunfairness under two constraints on the fidelity and C KL to a saddle point problem, which is solvedwith the exponentiated gradient technique [1].

Original: Table 1 summarizes the performance of the consideredblack-box models on each suing set.C K L AdaBoost and DT XgBoost and DT
Revised: Table 1 summarizes the performance of the consideredblack-box models on each suing set.

Original: DNN and DT

Original: RF and DT 0 . 0 0 .

Original: 8 1 . 0 1 − (cid:15) 0 .0 .0 .0 .0 .0 .

Original: C K L

Original: AdaBoost and LR 0 . 0 0 .

Original: 8 1 . 0 1 − (cid:15)

Original: XgBoost and LR 0 . 0 0 .

Original: 8 1 . 0 1 − (cid:15)

Original: DNN and LR 0 . 0 0 .

Original: 8 1 . 0 1 − (cid:15)

Original: RF and LR 0 .0 .0 .0 .

Original: P a r i t

Original: y ga p

Original: o f I ( · ) 0 .0 .0 .0 .

Original: P a r i t

Original: y ga p

Original: o f I ( · )

Original: Implementation.
Revised: Implementation.

Original: FRAUD-Detect solely requires the confusion matrices from the interpretablemodel for both the majority and minority group.
Revised: FRAUD-Detect solely requires the confusion matrices from the interpretablemodel for both the majority and minority group.

Original: These matrices are flattened, then compared via KLas outlined in Equation 8.
Revised: These matrices are flattened, then compared via KLas outlined in Equation 8.

Original: Experiments were conducted on Intel Xeon Silver 4110 CPUs (2.10 GHz,8GB RAM) and the KL divergence detector implemented in Python 3.8 using scikit-learn [36].
Revised: Experiments were conducted on Intel Xeon Silver 4110 CPUs (2.10 GHz,8GB RAM) and the KL divergence detector implemented in Python 3.8 using scikit-learn [36].

Original: Giventhat the KL divergence constraint is not a linear constraint on the confusion matrices, the extendedversion of FaiRS may fail to obtain the global minimum of the demographic parity gap on the setof high-fidelity interpretable models.
Revised: Giventhat the KL divergence constraint is not a linear constraint on the confusion matrices, the extendedversion of FaiRS may fail to obtain the global minimum of the demographic parity gap on the setof high-fidelity interpretable models.

Original: In our experiments, our approach converges for two datasets,namely COMPAS and Marketing, but does not achieve the global minimum of the demographicparity gap for Adult Income.
Revised: In our experiments, our approach converges for two datasets,namely COMPAS and Marketing, but does not achieve the global minimum of the demographicparity gap for Adult Income.

Original: We are planning to address this in future work by focusing on detectingfairwashing using linear constraints. 
Revised: We are planning to address this in future work by focusing on detectingfairwashing using linear constraints. [Figure] [Figure] [Figure]

Original: E Additional results
Revised: [Figure] C K L AdaBoost and DT XgBoost and DT

Revised: DNN and DT

Revised: RF and DT

Revised: [Figure]

Revised: C

Revised: [Figure]

Revised: RF

Revised: [Figure]

Revised: LR

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: P a r i t y ga

Revised: [Figure]

Revised: I (

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: P a

Revised: [Figure]

Revised: i t

Revised: [Figure]

Revised: [Figure]

Revised: [Figure]

Revised: F Additional results

Original: FRAUD-Detect successfully detects fairwashing.
Revised: FRAUD-Detect successfully detects fairwashing.

Original: Figures 4 and 5 illustrate both the demographicparity gap and C KL of interpretable models as a function of ϵ for two datasets (Adult income and
Revised: Figures 4 and 5 illustrate both the demographicparity gap and C KL of interpretable models as a function of ϵ for two datasets (Adult income and

Original: Bank marketing), respectively.
Revised: Bank marketing), respectively.

Original: FRAUD-Detect is robust to the informed adversary.
Revised: FRAUD-Detect is robust to the informed adversary.

Original: Figures 6 and 7 show the range of thedemographic parity gap of high-fidelity fairwashed interpretable models subjected to a constraint onthe C KL by solving the optimization problem of Equation (9).
Revised: Figures 6 and 7 show the range of thedemographic parity gap of high-fidelity fairwashed interpretable models subjected to a constraint onthe C KL by solving the optimization problem of Equation (9).

Original: We use four types of black-box models(AdaBoost, XgBoost, DNN and Random Forest).
Revised: We use four types of black-box models(AdaBoost, XgBoost, DNN and Random Forest).

Original: F Fairwashing Detection Thresholds
Revised: G Fairwashing Detection Thresholds

Original: Table 2 reports several fairwashing detection thresholds observed in Figures 2, 4 and 5. 
Revised: Table 2 reports several fairwashing detection thresholds observed in Figures 2, 4 and 5. [Table]

Original: G Per seed fine grain C KL versus fairness
Revised: H Per seed fine grain C KL versus fairness

Original: Figures 8, 9, and 10 show the per-seed results aggregated to form figures 2, 4, and 5, respectively.
Revised: Figures 8, 9, and 10 show the per-seed results aggregated to form figures 2, 4, and 5, respectively.

Original: In these figures (especially in 8 and 10), the order of increasing black-box fairnesses across the10 seeds is often replicated in the order of decreasing interpretable model C KL values across the C K L AdaBoost and DT
Revised: In these figures (especially in 8 and 10), the order of increasing black-box fairnesses across the10 seeds is often replicated in the order of decreasing interpretable model C KL values across thesame seeds; thus a seed with a low black-box fairness has higher C KL values than a seed with highblack-box fairness.

Original: XgBoost and DT DNN and DT RF and DT 0 . 0 0 . 8 1 . 0 1 − (cid:15) 0 .0 .0 .0 .
Revised: Some seeds from the logistic regression interpretable model (e.g. Figure 9 AdaBoost and logisticregression green and grey lines) show discontinuities at high epsilons where fairwashing is significant.

Original:  C K L AdaBoost and LR 0 . 0 0 . 8 1 . 0 1 − (cid:15) XgBoost and LR 0 . 0 0 .
Revised: There, the C KL values jump to infinity due to a division by zero in the KL log term when there are 0sin the minority subpopulation confusion matrix.

Original: 8 1 . 0 1 − (cid:15) DNN and LR 0 . 0 0 . 8 1 . 0 1 − (cid:15)
Revised: We also observe that for several seeds across all datasets and models, the fairness of the honestinterpretable model is imperfectly aligned and usually higher than the black-box fairness.

Original: RF and LR 0 .0 .0 .0 .0 .0 .0 . P a r i t y ga p o f I ( · ) 0 .0 .0 .0 .0 .0 .0 .
Revised: This isunintentional fairwashing, which may arise from the relatively small capacity of the interpretablemodel compared to the black-box model.

Original: P a r i t
Revised: I Bounds for KL Divergence of the Confusion Matrices

Original: y ga p o f I ( · )[Table]
Revised: First, denote γ 1 = Γ B (1 + F +1 − T +1 )

Original: same seeds; thus a seed with a low black-box fairness has higher C KL values than a seed with highblack-box fairness. Some seeds from the logistic regression interpretable model (e.g. Figure 9 AdaBoost and logisticregression green and grey lines) show discontinuities at high epsilons where fairwashing is significant. There, the C KL values jump to infinity due to a division by zero in the KL log term when there are 0sin the minority subpopulation confusion matrix.
Revised: =

Original: We also observe that for several seeds across all datasets and models, the fairness of the honestinterpretable model is imperfectly aligned and usually higher than the black-box fairness.
Revised: Γ B ( F − 1 + F +1 ) and similarly γ 0 =

Original: This isunintentional fairwashing, which may arise from the relatively small capacity of the interpretablemodel compared to the black-box model.
Revised: Γ B ( F − 0 + F +0 ) .

Original: H Experimental results with two independent variables
Revised: Depending on the context of the problem either subgroup could be the more sensitive subgroup, soin Equation (6) we implicitly assumed that subgroup is z = 1 : γ = γ 1 . Equation (8) defines thedetector based on the flattened confusion matrix, but since for a binary classification problem, therank of the confusion matrix is 2, it is enough to consider KL [Equation] 3 We will deriveupper and lower bounds on this divergence measure in terms of γ 0 , γ 1 , F + i , F − i ; i ∈ { 0 , 1 } .

Revised: First, note that γ 1 is the sum of type I F − 1 and type II F +1 detection errors; scaled by the unfairness of theblack-box model Γ B .

Revised: In a detection problem, typically, we bound the type I error and minimize thetype II error.

Revised: The particular choice of type I and type II error is context-dependent (are false positivesa bigger concern than false negatives or not?).

Revised: In the context of fairwashing, a reasonable choice isthat

Revised: [Equation]

Revised: That is, the cost of falsely detecting a fairwashed model is larger than missing fairwashing—due tothe legal and liability considerations, for instance.

Revised: We can bound the irreducible fairwashing gap γ i :

Revised: [Equation]

Revised: [Equation]

Revised: Using this we bound the aforementioned KL divergence:

Revised: [Equation]

Revised: Similarly,

Revised: [Equation]

Revised: [Equation]

Revised: Therefore, κ := KL (cid:18)(cid:20)

Revised: [Equation]

Revised: +

Revised: [Equation]

Revised: F

Revised: [Equation]

Revised: (cid:21)(cid:19) is bounded from below and above by:

Revised: [Equation]

Revised: Equivalently, we can re-write these bounds in terms of F + and T + to match Theorem 1 (using thefact that F − = 1 − T + ):

Revised: [Equation]

Revised: [Equation]

Revised: Therefore any choice between κ min := F +0 log (cid:16) F +0 F − 1 · γ 0 γ 1 (cid:17) and κ max :=

Original: In this section
Revised: F − 0 log (cid:16) [Table] is valid.κ = κ min is the most conservative choice which increases the risk of false discovery (of fairwashing), while κ = κ max is a permissive choice that minimizes the risk of false discovery at the expense ofincrease in the false negatives (not detecting fairwashing). A balanced choice for threshold could bethe intermediate point:

Original: [Figure]
Revised: [Equation]

Revised: where ∆( R ) =

Revised: R 0 /R 1 and ∆ l ( R ) = log ∆( R ) .

Revised: In the last equation, we have used the taylorexpansion of log(1 − x ) at x = 0 .

Revised: J Experimental results with two independent variables

Original:  we repeat experiments shown in Section 6 with the following adjustment:
Revised: In this section, we repeat experiments shown in Section 6 with the following adjustment:

Original: [Equation]
Revised: [Equation]

Original: in which we use only two independent variables of true-positive rate T + and false-positive rate F + .
Revised: in which we use only two independent variables of true-positive rate T + and false-positive rate F + .

Original: Figures 11, 12 and 13 show similar results as Figures 2, 4 and 5.
Revised: Figures 11, 12 and 13 show similar results as Figures 2, 4 and 5.

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: I Perfect Fairwashing is impossible
Revised: K Perfect Fairwashing is impossible

Original: Theorem 2.
Revised: Theorem 2.

Original: Consider binary-valued random variables ˆ Y B black-box model predictions, ˆ Y I proxy(interpretable) model predictions, and A a sensitive attribute; then these conditions cannot be satisfiedsimultaneously:
Revised: Consider binary-valued random variables ˆ Y B black-box model predictions, ˆ Y I proxy(interpretable) model predictions, and A a sensitive attribute; then these conditions cannot be satisfiedsimultaneously:

Original: Interpretable Model Demographic Parity
Revised: Interpretable Model Demographic Parity

Original: [Equation]
Revised: [Equation]

Original: Non-detectable
Revised: Non-detectable

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Let Equation (14) be equal to T + , and Equation (15) to F + .
Revised: Let Equation (39) be equal to T + , and Equation (40) to F + .

Original: Define the base rates of theblack-box model on subpopulations 0 and 1 as:
Revised: Define the base rates of theblack-box model on subpopulations 0 and 1 as:

Original: [Equation]
Revised: [Equation]

Original: Using the rule of total probabilities, for a ∈ [0 , 1] ,
Revised: Using the rule of total probabilities, for a ∈ [0 , 1] ,

Original: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Fairness according to parity requires that
Revised: Fairness according to parity requires that

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: Substituting Equation (16) and Equation (18) and re-organizing we have:
Revised: Substituting Equation (41) and Equation (43) and re-organizing we have:

Original: [Equation]
Revised: [Equation]

Original: Case 1: If T + = F + , by eliminating both and recognizing that (1 − µ 0 ) =
Revised: Case 1: If T + = F + , by eliminating both and recognizing that (1 − µ 0 ) =

Original: 1 − µ 0 , we see that theequality Equation (18) reduces to 0 = 0 which is trivially correct.
Revised: 1 − µ 0 , we see that theequality Equation (43) reduces to 0 = 0 which is trivially correct.

Original: Case 2: T + ̸ = F + Simplifying Equation (19) further and eliminating T+ − F +
Revised: Case 2: T + ̸ = F + Simplifying Equation (44) further and eliminating T

Original: [Equation]
Revised: [Equation]

Original: [-= 
-]
Revised: [-= 
-]

Original: [Equation]
Revised: [Equation]

Original: But these are probabilities, for Equation (18) to hold we must have that
Revised: But these are probabilities, for Equation (43) to hold we must have that

Original: [Equation]
Revised: [Equation]

Original: which means that the black-box predictions exclusively depend on the sensitive attribute.
Revised: which means that the black-box predictions exclusively depend on the sensitive attribute.

Original: In otherwords, for demographic parity to hold for the interpretable model, the black-box models predictionsall samples of one subpopulation A = 0 and reject the others A = 0 .
Revised: In otherwords, for demographic parity to hold for the interpretable model, the black-box models predictionsall samples of one subpopulation A = 0 and reject the others A = 0 .

Original: Clearly, this is unfair to onesubpopulation.
Revised: Clearly, this is unfair to onesubpopulation.

Original: To establish the impossibility result, we further show that such an assumption is equivalent to requiringcompletely inaccurate black-box models with maximum error rates.
Revised: To establish the impossibility result, we further show that such an assumption is equivalent to requiringcompletely inaccurate black-box models with maximum error rates.

Original: Consider the “true” base rates for each subpopulation:
Revised: Consider the “true” base rates for each subpopulation:

Original: [Equation]
Revised: [Equation]

Original: If interpretable demographic parity Equation (20) holds then, these base rates reduce to:
Revised: If interpretable demographic parity Equation (45) holds then, these base rates reduce to:

Original: [Equation]
Revised: [Equation]

Original: In other words, the black-box model predictions is completely independent of the true label andexclusively dependent on the sensitive attribute A .
Revised: In other words, the black-box model predictions is completely independent of the true label andexclusively dependent on the sensitive attribute A .

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: i t
Revised: i t

Original: [Figure]
Revised: [Figure]

Original: ga
Revised: ga

Original: [Figure]
Revised: [Figure]

Original: o f
Revised: o f

Original: I ( · )
Revised: I ( · )

Original: [Figure]
Revised: [Figure]

Original: 1 Fairness .
Revised: 1 Fairness .

Original: C K L
Revised: C K L

Original: AdaBoost and LR .
Revised: AdaBoost and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: DNN and LR .
Revised: DNN and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: RF and LR .
Revised: RF and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: XgBoost and LR .
Revised: XgBoost and LR .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: C K L
Revised: C K L

Original: AdaBoost and DT .
Revised: AdaBoost and DT .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: DNN and DT .
Revised: DNN and DT .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: RF and DT .
Revised: RF and DT .

Original: 1 Fairness .
Revised: 1 Fairness .

Original: XgBoost and DT
Revised: XgBoost and DT

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: DNN and LR .
Revised: DNN and LR .

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: consider of the

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]