Rethinking Counterfactual Explanations as Local and Regional Counterfactual Policies
Authors: Salim I. Amoukou
Nicolas J-B. Brunel
Venue: NeurIPS 2022 Submitted (2022)
Keywords: counterfactuals
algorithmic recourse
learning theory
random forest
interpretability
explainable models
tree-based models

Abstract: Among the challenges not yet resolved for Counterfactual Explanations (CE), there are  stability,  synthesis of the various CE and the lack of plausibility/sparsity guarantees. From a more practical point of view, recent studies show that the prescribed counterfactual recourses are often not implemented exactly by the individuals and demonstrate that most state-of-the-art CE algorithms are very likely to fail in this noisy environment. To address these issues, we propose a probabilistic framework that gives a sparse local counterfactual rule for each observation: we provide rules that give a range of values that can change the decision with a given high probability instead of giving diverse CE. In addition, the recourses derived from these rules are robust by construction. These local rules are aggregated into a regional counterfactual rule to ensure the stability of the counterfactual explanations across observations. Our local and regional rules guarantee that the recourses are faithful to the data distribution because our rules use a consistent estimator of the probabilities of changing the decision based on a Random Forest. In addition, these probabilities give interpretable and sparse rules as we select the smallest set of variables having a given probability of changing the decision. Codes for computing our counterfactual rules are available, and we compare their relevancy with standard CE and recent similar attempts.

Rethinking Counterfactual Explanations as Local and Regional Counterfactual Policies
Anonymous Author(s)
Affiliation Address
email
Abstract
Among the challenges not yet resolved for Counterfactual Explanations (CH), there are stability, synthesis of the various CE and the lack plausibility/sparsity guarantees. From a more practical point of view, recent studies show that the prescribed counterfactual recourses are often not implemented exactly by the individuals and demonstrate that most state-of of-thh-art CE algorithms are very likely fail in this noisy environment. To address these issues, we propose probabilistic framework that gives a sparse local counterfactual rule for each observation: we provide rules that give a range of values that can change the decision with a given high probability instead of giving diverse CF. In addition, the recourses derived from these rules are robust by construction. These local rules are aggregated into a regional counterfactual rule to ensure the stability of the counterfactual explanations across observations. Our local and regional rules guarantee that the recourses are faithful to the data distribution because rules use a consistent estimator of the probabilities changing the decision based on a Random Forest, In addition, these probabilities give interpretable and sparse rules as select the smallest set of variables having given probability of changing the decision. Codes for computing our counterfactual rules available, and compare their relevancy with standard CE and recent similar attempts.
2
10
11
12
14
16
18
Introduction
19
20 In recent years, many explanations methods have been developed for explaining machine learning models, with strong focus local analysis, i.e., generating explanations for individual prediction 22 (see [Molnar, 2022] for a survey). Among this plethora of methods, one of the most prominent and
21
28 active techniques are Counterfactual Explanations [Wachter et al., 2017b]. Unlike popular local
24 attribution methods, e.g., SHAP [Lundberg et al., 2020] and LIME [Ribeiro et al., 2016], which highlight the importance score of each feature, Counterfactuals Explanations (CE) describe the smallest modification to the feature values that changes the prediction to desired target. Although CE are intuitive and user-friendly by giving recourse in some scenarios (e.g., loan application), they have many shortcomings in practice. Indeed, most counterfactual methods rely on gradient- based
25
26
27
28
29 algorithm or heuristics approaches [Karimi al., 2020b], thus can fail identify the most natural explanations and lack guarantees. Most algorithms either do not guarantee sparse counterfactuals
30
31 (changes in the smallest number of features) or do not generate in-distribution samples (see [Verma et al., 2020, Chou et al., 2022] for a survey on counterfactuals methods). Although some works
33 [Parmentier and Vidal, 2021, Poyiadzi et al., 2019, Looveren and Klaise, 2019] try to solve the plausibility/sparsity problem, the suggested solutions are not entirely satisfactory.
35 In another direction, many papers [Mothilal et al., 2020, Karimi et al., 2020a, Russell, 2019] encour- ages the generation of diverse counterfactuals in order to find actionable recourse [Ustun et al., 2019]. Actionability is a vital desideratum, as some features may be non-actionable, and generating many
37
Submitted to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). Do not distribute.

---

38
makes the explanations less intelligible, and the synthesis of various CE or local explanations, in general, is yet to be comprehensively solved [Lakkaraju et al., 2022]. In addition, recently Pawelczyk et al. [2022] highlights a new problem of local CE called: noisy responses to prescribed recourses. Indeed, in real-world scenarios, some individuals may not be able to implement exactly the prescribed recourses, and they show that most CE methods fail this noisy environment. Therefore, we propose to reverse the usual way of explaining with counterfactual by computing Counterfactual rules. We introduce a new line of counterfactuals: we build interpretable policies for changing a decision with given probability that ensure the stability of the deduced recourse. These policies are optimal (in sparsity) and faithful to the data distribution. Their computation comes with statistical guarantees as they use consistent estimator of the conditional distribution. Our proposal is to find a general counterfactuals increases the chance of getting actionable recourse. However, the diversity of CE
39
40
41
42
43
44
45
47
49 policy or rule that permits changing the decision while fixing some features instead of generating
50 many counterfactual samples. One of the main challenges is identify the (minimal) set of features
51 that provide the best promising directions for changing the decision to the desired output. We also show this approach can be extended for finding a collection of regional counterfactuals, such that 58 have a global counterfactual policy for analyzing a model. An example of the counterfactual rules that we introduce is given in figure 1.
Local Counterfactual Rule
IF Age=50, Salary=sok, Sex=M, HoursPeeWeekkaddl THEN Loan=False
Age Salary "skk, sok]. Sex E [M. THEN Loan=True with Probability=0.9
Regional Counterfactual Rule
IF Age E [50, 80]. Salary E [s0, 50], Sex E [M, F] THEN Loan=False
IF Age [500 807 Salary é [sok, sok].
WoursPorWeek E [sob, 45b]
THEN Loan= True with Probability=o.0
Figure 1: Illustration of the local and regional Counterfactuals Rules that we introduced on a dataset with variables: Age, Salary, Sex, and HoursPerWeck. The Counterfactual Rules define intervals on the minimal subset of features to change the decision of a model prediction in the local counterfactual rule or the decision rule that applies on a sub-population in the regional counterfactual rule. In Blue, we have the proposed rules to change the decision.
54
55 2 Motivation and Related works
56
Most of the methods that propose Counterfactuals Explanations are based the approach of the seminal work of Wachter et al. [2017a]: the counterfactuals are generated by optimizing a cost, but this procedure does not account directly the plausibility of the counterfactual examples (see [Verma et al., 2020] for classification of CE methods). Indeed, a major shortcoming is that the adverse decision needed for obtaining the counterfactual is not designed to be feasible or representative of the underlying data distribution. However, some recent studies proposed ad-hoc plausibility constraint in the optimization, using for instance outlier score [Kanamori et al., 2020], Isolation Forest [Parmentier and Vidal, 2021] or a density-weighted metrics [Poyiadzi et al., 2019] to generate in- distribution samples. In another direction, Looveren and Klaise [2019] proposes use an autoencoder that penalizes out-of-distributin candidates. Instead of relying on ad-hoc constraints, we propose CE that gives plausible explanations by design. Indeed, for each observation, we identify the variables and associated ranges of values that have the highest probability of changing the prediction. We can compute this probability with a consistent estimator of the conditional distribution PYY]Xss. As consequence, the sparsity of the counterfactuals is not encouraged indirectly by adding a penalty term (lo or é1) as existing works [Mothilal et al., 2020]. Our approach inspired by the concept of Same Decision Probability (SDP) (introduced in [Chen et al., 2012]) that can be used for identifying the smallest subset of features to guarantee (with given probability) the stability of a prediction. This minimal subset is called Sufficient Explanations. In [Amoukou and Brunel, 2021], it has been shown
57
59
60
61
82
63
84
85
86
67
68
69
70
72

---

74
that the SDP and the Sufficient Explanations can be estimated and computed efficiently for identifying important local variables in any classification and regression models. For counterfactuals, we are interested in the dual set: we want the minimal subset of features that have a high probability of
75
76
77 changing the decision (when the other features are fixed). Another limitation of the current CE is their local nature and the multiplicity of the explanations produced. While some papers [Mothilal et al., 2020, Karimi et al., 2020a, Russell, 2019] promote the generation of diverse counterfactual samples to ensure actionable recourse, such diverse explanations should be summarized to be intelligible [Lakkaraju et al., 2022], but the compilation of local explanations is often a very difficult problem. To address this problem, we do not generate counterfactual samples, but we build rule Counterfactual Rules (CR) from which we can derive counterfactuals. Contrary to classic CE which gives the nearest instances with desired output, we find the most effective rule for each observation (or group of similar observations) that changes the prediction to the desired target. This local rule easily aggregates similar (Age=20, Salary=35k, HoursWeek=25h, Sex=M,....
78
79
80
81
83
85
86
counterfactuals. For example, if with Loan=False, fixing the variables Age and Sex and changing the Salary and HoursWeek change the decision. Therefore, instead of given multiples combination of Salary and HoursWeek (e.g. 35k and 40h or 40k and 55h, ...) that result in many instances, the counterfactual rule gives the range of values: IF HoursWeek E [35h, 50h], Salary E [40k, 50k], and the remaining features are fixed THEN Loan=True. It can be extended at a regional scale,
87
88
89
90
81
92 e.g.. given rule R = {IF Salary E [35k, 20k], Age E [20, 80] THEN FalneFalse). the regional Counterfactual Rule (CR) could be {IF Salary E [40k, 50k], HoursWeek E [35h, 50h] and the remaining rules are fixed THEN Loan=True). The main difference be- tween local and a global CR is that the Local-CR explain single instance by fixing the remaining 95
93
94
96 feature values (not used in the CR): while a regional-CI is defined by keeping the remaining variables
97 in given interval (not used in the regional-CR). Moreover, by giving ranges of values that guarantee
96
a high probability of changing the decision, we partly answer the problem of noisy responses to prescribed recourses [Pawelczyk et al., 2022] so long the perturbations are within our ranges.
99
100
Although the Local Counterfactual Rule is new, the Regional Counterfuctual Rule can be related to some recent works. Indeed, Rawal and Lakkaraju [2020] proposed Actionable Recourse Summaries (AReS). framework that constructs global counterfactual recourses in order to have global insight of the model and detect unfair behavior. While AReS is similar to the Regional Counterfactual
101
102
103
104
Rule, we emphasize some significant differences. Our methods can address regression problems and deal with continuous features. Indeed, AReS needs to discretize the continuous features, inducing a trf between speed and performance as noticed by [Ley et al., 2022]. Thus, too few bins result in unrealistic recourse, while too many bins result in excessive computation time. In addition, AReS uses greedy heuristic search approach to find global recourse, which might produce sub-optimal recourse. As we have already mentioned, the changes we provide overcome these two limitations because the consistency of our counterfactual is controlled by an estimation of the probability of changing the decision, and because we favor changes of minimum number of features. Another global CE framework has been introduced in [Kanamori et al., 2022] to ensure transparency: the Counterfactual Explanation Tree (CET) partitions the input space with decision tree and assigns an appropriate action for changing the decision of each subspace. Therefore, it gives a unique action for changing the decision of multiple instances. In our case, we offer more flexibility in the counterfactual explanations because we provide a range of possible values that guarantee a change do not make any assumption about the cost of changing the feature nor the causal structure. If we have such information, then we can add it as additional
105
106
107
108
110
112
114
115
116
with given probability. In our approach,
118
119
post-progssiing such that it can be made more explicit and more transparent for the final user as
120
required for trustworthy AI.
3 Minimal Counterfactual Rules
121
D, {(x,, Y))=...... such that (X,Y) P(x,r) where We assume that we have an i.i.d sample D,, =
122
123 X E X' (typically X' = RP) and Y E y. The output y can be discrete or continuous. We want to
124
explain the predictor / R2 + V, that has been learned with the dataset D,,. We use uppercase letters for random variables and lowercase letters for their value assignments. For given subset
125
S C [p). Xs = (X))ies denotes subgroup of features, and we write x = (g)) (with some
126
abuse of notation).
3

---

128
For an observation (z. y= f(z)), we have a target set 3 C 3/ such that $ Vr For the simple case of classification problem, V- = {y²} is the standard singleton such that y E 3 is different of y. Contrary to standard approaches, our definition of the counterfactual deals also with the regression case by considering 3 = (a. b] C IR; our definitions and computations of counterfactuals are the same for both classification and regression. We remind that the classic CE problem (defined only for classification) is to find a function a : X + X. such that for all observations æ E X. f(x) A y², and we have f(((x)) yy. With standard CE, the function is defined point-wise by solving an optimisation program. Most often a(-) is not a real function, as a(x) may be in fact a collection of (random) values (æi..... #p). A more recent point of view was proposed by Kanamori et al. [2022], and it defines a as a decision tree, where in each leaf L. the best perturbation ar is predicted and add it to all the instances E L.
129
130
131
132
135
135
136
137
139
Our approach is hybrid, because we do not propose a single action for each subspace of X' or sub-group of population, but give sets of possible perturbations. Indeed, Local Counterfactual Rule (Local-
140
CR) for y and observation a: (with f(x: & *)) is a rectangle Cs(æ;; ") = Ihes (as, bi), as, by E
141
142
R such that for all perturbations of æ (æs, ggg) obtained as xx = (zs, gg with 2s E Cs(a;; " and x* an in-distribution sample, then f (x*) is in 3* with a high probability.
143
Similarly, Regional Counterfactual Rule (Regional-CR) Cs +/*) is defined for y and
rectangle R. = I [T:=]la;bbb by]. a.,b, E E. if for all observations x (#5. E R, the perturbations obtained as x² = (z) with 2 E Cs ") and x" an in in-distribution sample are such that I (a*) is in @ with high probability.
by],
145
146
147
We build such rectangles sequentially, first, we propose to find the best directions S C [p] that offers
148
the best probability of change. Then, we find the best intervals (a:, b E S that change the decision
149
150
to the desired target. A central tool in this approach is the Counterfactual Decision Probability.
151
Definition& 3.1. Counterfactual Decision Probability (CDP). The Counterfactual Decision Prob- ability of the subset S C [1,p]. w.r.t x = (æ5, F5) and the desired target y (s.t. f(xx) & +)
152
153
is
CDPs (V+: = P (f(X) E [ = Dgg
154
The CDP of the subset S is the probability that the decision changes to the desired target y by sampling the features X s given Xs =gg. It is related to the Same Decision Probability SDPs x) = P (J(() E 3/ |Xs 5 = æs) used in [Amoukou and Brunel, 2021] for solving the dual problem of selecting the most local important variables for obtaining and maintaining the decision f(x) E y (where fxx E y C )). The set S is called the Minimal Sufficient Explanation. Indeed, SDPs( x). The computation of these probabilities is challenging and discussed in Scction 4. We now focus on the minimal subset of features S such that the model makes the desired decision with a given probability л.
156
157
158
we have (3/
160
161
162
Definition 3.2. ( (Minimal Divergent Explanations). Given an instance and desired target Ø++ is a Divergent Explanation for probability x > 0, if CDPs ( " x) 2 n, and no subset 2 of S satisfies CDPz (3/ x) 2 æ. Hence, a Minimal Divergent Explanation is a Divergent Explanation
163
165
with minimal size.
166
The set minimizing this probability is not unique, and we can have several Minimal Divergent Explanations. Note that the probability π represents the minimum level required for a set to be chosen for generating counterfactuals, and its value should be as high as possible and depends on the use case We have now enough material to define our main criterion for building a Local Counterfactual Rule (Local-CR):
187
168
189
170
Definition 3.3. (Local Counterfactual Rule). Given an instance æ., a desired target @ } f(x:). Minimal Divergent Explanation S, the rectangle Cs(a; = IIkEsla; bi], at, by E R is Local Counterfactual Rule with probability тe if
171
172
173
CRPs x, Cs(x: & P(f()) 2 [Xs E Cs(x; V.), Xg = x8) 2 nC. (3.1)
174
The CRPs is the Counterfactual Rule Probability.
175
The higher the probability Te is, the better the relevance of the rule Cs( 20r) is. for this instance. Given set S, we seek for the maximal rectangle in the direction S satisfying Definition 3.1.
176
177 In practice, we can observe that the Local-CR Cs(:; gy) for neighbors æ, x'' are often quite close, be- cause the Minimal Divergent Explanations are similar and the corresponding rectangles often overlaps.

---

Hence, this motivates generalisation of these Local-CR to hyperrectangle R II (a, a,l a,, by E R regrouping similar observations. We denote supp Rt {i [at, b.] * }J the support of the rectangle, and we extend the Local-CR to Regional Counterfactual Rules (Regional-CR), In order to do it, we denote Rg = ee (a,, 6,] the rectangle with intervals of R in supp RR) n and we also defines the corresponding Counterfactual Decision Probability CDP (Definition 3.1) for rule R and subset S as Cs ( R) P (f) E 3 X 3 E R. ). Therefore, we can also compute the Minimal Divergent Explanation for rule R using Definition 3.2 with the CDP for rules.
179
181
183
185
186
Definition 3.4. (Regional Counterfactual Rule). Given any rectangle R. desired target Wr, a Minimal Divergent Explanation S of R. the rectangle Cs(R.y") Ilies (a,, by] is a Regional Counterfactual Rule with probability TC if
187
CRPs(3' R. (R, ") & P(f(X) E 3 E Xs E Rs) 2 Te. (3.2)
189
CRPs R, (R)) the corresponding Counterfactual Rule Probability for rule R.
190
Local-CR and regional-CR differ slightly: for local, we condition by Xs = 2g in Eq. 3.1, while for regional, we condition by X 5 E Rg. For computing regional-CRi we can start for rectangle generated by any method, such as [Wang et al., 2017, Lin et al., 2020]. The only condition is that it contains homogeneous group. i.c. with almost the same output. However, by default we use as initial rules the Sufficient Rules derived in [Amoukou and Brunel, 2021] as it handles regression problem. The Sufficient Rules are minimal support rectangles define for a given output y
Remarks:
192
194
185
196
as Cs(( = Tiss (as, by] such that Vo: E X, 2g E () P()()) E X = xs)) 2 π.
Estimation of the CDP and CRP
197
198
In order to compute the probabilities CDPs and CRPs for any S, we use a dedicated Random Forest (RF) Mk.n that learns the model f to explain. Indeed, the conditional probabilities CDPs and CR can be easily computed from a RF by combining the Projected Forest algorithm [Bénard al., 2021a] and the Quantile Regression Forest [Meinshausen and Ridgeway, 2006]: hence we can æ)) We adapt the approach used in [Amoukou and Brunel, 2021] and remind for the sake of completeness, the computation of the estimate of SDPs
199
200
201
202
estimate consistently the probabilities CDPs (PPs("rr
203
4.1
Projected Forest and CDPs
205
The estimator of the SDPs is built upon a learned Random Forest [Breiman et al., 1984]. A Random Forest (RF) is predictor consisting of collection of & randomized trees (see [Loh, 2011] for detailed description of decision tree). For each instance æ. the predicted value of the j-th tree is denoted m, (z. θ2) where O2 represents the resampling data mechanism in the j-th tree and the successive random splitting directions. The trees are then averaged to give the prediction of the forest
207
209
as:
1 k
mi. (x,, Orok D.) =
mx; D.)
(4.1)
However, the RF can also be view as an adaptive nearest neighbor predictor. For every instance the observations in D, are weighted by Wn, 11,, Dn,, = I,...,n. n. Therefore, the prediction of RF can be rewritten
212
me. (x. 1 Bi),,Dn)=
Σ "n.i( (x; Or. D.).s
This emphasizes the central role played by the weights in the RF's algorithm, see [Meinshausen and Ridgeway, 2006, Amoukou and Brunel, 2021] for detailed description of the weights. Therefore, it naturally gives estimators of other quantities e.g., Cumulative hazard function [Ishwaran et al., 2008], Treatment effect [Wager and Athey, 2017], conditional density [Du et al., 2021]. For instance, used the same weights to estimate the
215
217
218
Meinshausen and Ridgeway [2006] showed that Conditional Distribution Function with the following estimator:
219
F = x. Oz.k. D,)
Σ Wn,i (x; Oz.. D,)
(4.2)
220
In another direction, Bénard et al. [2021a] introduced the Projected Forest algorithm [Bénard et al., 2021c,a] that aims to estimate E X s] by modifying the RF's prediction algorithm.
5

---

222
To estimate X æs] instead of E| = x using a RF, Bénard et al. [2021b] suggests to simply ignore the splits based on the variables not contained in S from the tree predictions. More formally, it consists of projecting the partition of each tree of the forest on the subspace spanned by the variables in S. The authors also introduced an algorithmic trick that computes the projected partition efficiently without modifying the initial tree structures. We drop observations down in the initial trees, ignoring the splits which use a variable not in S: when a
Projected Forest:
223
224
225
226
227
split involving variable outside of S is met, the observations are sent both to the left and right children nodes. Therefore, each instance falls in multiple terminal leaves of the tree. We drop the
229
new query point z down the tree, following the same procedure, and gather the set of terminal leaves where 2g falls. Next, we collect the training observations which belong to every terminal leaf of this collection, in other words, we keep only the observations that fall in the intersection of the leaves where æs falls. Finally, we average the outputs Y; of the selected training points to generate æs]. Notice that this algorithm converges asymptotically to the true projected conditional expectation x = æs)-
231
233
234
the estimation of F|Y|
235
As the RF, the PRF gives also weight to each observation. The associated PRF is denoted (m2)
237
ma k. (zs) = Ei=1 (n,) Therefore, as the weights of the original forest was used to estimate the CDF in equation 4.2, Amoukou and Brunel [2021] used the weights of the Projected Forest Algorithm to estimate the SDP as SDPs (z) = Σn= wn,((as)) e The idea is essentially to replace Y, by I &ye in the Projected Forest equation defined above. The authors also
238
239
240
241
show that this estimator converges asymptotically to the true SDPs. Therefore, we can estimate the
242
CDP with the following estimator
CDPs (3 x))
Σ
(4.3)
Remarks: Note that we only give the estimator of the CDPs of an instance The estimator of the CDPs of a rule R will be discussed in the next section as it is related to the estimator of the C'RPs.
244
4.2
Regional RF and CRPs
246
x. Cs(:; V/)) = P()(X) E "/") = P(f(X) E grr Xs Cs ", E Rg). For simplicity, we remove the dependency of the rectangles in V Based
In this section, we focus on the estimation of the
3 Xs Cs(a; ")" Xg æg) and CRPs(3r* R. Cs
248
249
on the previous Section, we already know that the estimators using the RF will be in the form of
CRPs s (3V. x. Cs(x)) = Σn= (n.i(x)) I y thus we only need to find the right weighting.
250
The main challenge is that we have condition based a region, e.g., X (s(z) X E Rg regionall-bseede instead of condition of type X = æg (fixed basue- as usually. However, we introduced a natural generalization of the RF algorithm to make predictions when the conditions are both regional-baa -based and fixed value-based. Thus, the case where there are only regional-based conditions are naturally derived.
252
253
254
255
Regional RF to estimate CRPs( , x. Cs(x)) = P(f(X) E 3r* |Xs Cs(x), æg): The algorithm is based on a slight modification of RF. Its works as follow: we drop the observations in the initial trees, if split used variable i E S, i.e. fixed value-based condition, we use the classic rules of RF, if x; & I, the observations go to the left children, otherwise the right children. However, if split used variable E S, i.e, regional-
257
259
260
condition, we use the rectangles Cs(æ)
I 19 [a,, b1]. The observations are sent to the left children if b, < f. right children if a; > t and if t E [a,, by] the observations are sent both to the left and right children. Therefore, we use the weights of the Regional RF algorithm to estimate the CRPs as in equation 4.3, the estimator is
261
262
263
CRPs s ( x. Cs(x)) Ei=1 Wn, (x) 1x. A more detailed version of the algorithm is provided and discussed in Appendix.
265
266
To estimate the CDP of a rule CDPs (3r+; R) P(f(X) E 3 x E R3), we just have to apply the projected Forest algorithm to the Regional RF, i.e., when a split involving a variable outside of S is met, the observations are sent both to the left and right children nodes, otherwise we use the Regional RF split rule, i.e., if an interval of Rg is below I, the observations go to the left children, otherwise the right children and if is in the interval, the observations go to the left and right children.
257
269

---

271
The estimator of the CRPs( R. (s))) for rule is also derived from the Regional RF. Indeed, it a special case of the Regional RF algorithm where there are only regional- based conditions.
272
5
Learning the Counterfactual Rules
273
274
We compute the Local and Regional CR using the estimators the previous section. First, we find the Minimal Divergent Explanation in the same way as Minimal Sufficient Explanation can be found all possible subsets exponential, we search the Minimal Divergent Subset among the K 10 most frequently selected variables in the RF ma, used to estimate the probabilities CDPs, C'RPs (K is an hyper--paamett to select according to the case and computational power). We also use any importance measure.
276
[mmukoo and Brunel, 2021]. As the exploration
278
280
Given an instance æ or rectangle R (and set ( and their corresponding Minimal Divergent Explanation S, we want to find rule Cs(æ)) ) = Iies (a;, b,] s.t. given X 2g or Xg E Rg and
282
Xs E (z))), the probability that Y E 3 is high. More formally, we want: ( E y X Cs(2), X3 = zz) ((f(X) E y E Cs(x), Xg E Rg) above TC:
283
284
The computation of the rectangles Cs(x)) = Ilis s|s,, be] relies heavily on our use of RF and on the
285
algorithmic trick of the projected RF. Indeed, the rectangles defining the rules arise naturally from RF,
286
while AReS [Rawal and Lakkaraju, 2020] relies on binned variables to generate candidate rules and
287
tests all these possible rules for choosing an optimal one. We overcome the computational burden
288
and the challenge of choosing the number of bins.
x,
X
17
LS]
X2
Xa
Figure 2: The partition of the RF learned to classify the toy data (Green/Blue stars). Its has 10 leaves. The explainee is the Blue triangle in leaf 5.
Figure 3: The partition of the projected Forest when we con- dition on Xo, i.e., ignoring the splits based on X; (the dashed lines).
Figure 4: The optimal CR for x when we condition given Xo zo is the Green region, its corresponds to the union of leaf and 4 of the forest
289 To illustrate the idea, we use a two-dimensional data (Xo, X1) with label represented as Green/Blus 280 stars in figure 2. We fit Random Forest classify this dataset and show its partition in figure 2. The explaince æ is the Blue triangle observation.
By looking at the different cells/lcaves of the RF, of x is 5 = X1. Indeed, in figure 3, we observe the leaves the Projected Forest when we do not S X1, thus projected the RF's partition only the subspace Xo- Its consists of ignoring all the splits in the other directions (here the X3 axxiss) thus x falls in the projected leaf 0.58.
guess that the Minimal Divergent Explanation
293
condition
295
(see figure 3) and its CDP is CDPx, (Green; x)) 10 Greea++7 Blue
10 Green
Finally, the problem of finding the optimal rectangle Cs(x) [a., by] in the direction of X1 s.t. the decision changes can be easily solved by using the leaves of the RF fact, by looking at the leaves of the RF (figure 2) of the observations that belong in the projected RF leaf 2 (figure 3) where falls, we see in figure 4 that the optimal rectangle to change the decision given X, = xo or being in the projected RF leaf 2 is the union of the intervals X₁ of the leaf 3 and 4 of the RF (see the Green
296
300
302
region of figure 4).
Given an instance æ and its Minimal Divergent Explanation S, the first step is the collect of the observations which belong to the leaf of the Projected Forest given S where in falls. It corre- sponds to the observations that has positive weights in the computation of the (DP
304
7

---

Σx= (n,s(zz)l (y, i.e., fay (xg) > 0}. Then, we used the partition of the original forest
306
307
to find the possible leaves Cs(x)) in the direction S. The possible leaves is among the RF's leaves
308
of the collected observations {a, (mg) > 0). Let denote [(x,)) the leaves of the observations
309
2; with ( > 0. A possible leaf is a leaf (.(;;)) s.t. CRPs(W** x. [s):)ss) = ( e
310
V Xxs E (s Xg = gg) 2 *c. Finally, we merge all the neighboring possible leaves to get
311
the largest rectangle, and this maximal rectangle is the counterfactual rule. Note that the union of the
312
possible leaves is not necessary connected space, thus we can have multiple counterfactual rules.
313
We apply the same idea to find the regional CR. Given a rule R and its Minimal Divergent Explanation S, we used the Projection given X E Rg to find the compatible observations and their leaves and combine the possible ones to obtain the regional CR that has CRPs( (3 R, (R)) 2 TC.
314
315
318
For example, if we consider the leaf 5 of the original forest as rule: If X E Leaf 5, then predict Blue. Its Minimal Divergent Explanation also = X1. The R-CR would also be the
317
318
Green region in figure 4. Indeed, if we satisfy the Xo condition of the leaf 5 and X₁ condition of the leaf 3 and 4, then the decision change to Green.
319
Experiments
320
321
To demonstrate the performance of our framework, we conduct two experiments on real-world datasets. The first consists of showing how can use the Local Counterfactual Rules for explaining
322
323
a regression model. In the second experiment, we compare our approaches with the 2 baselines methods in classification problem: (1) CET [Kanamori et al., 2022], which partition the input space using a decision tree and associate a vector perturbation for each leaf, (2) A AReS [Rawal and Lak Lakkarau, 2020] performs an exhaustive search for finding global counterfactual rules, but we used the implementation of Kanamori et al. [2022] that adapts the algorithm for returning counterfactuals samples instead of rules. We compare the methods only in classification problem as most prior works
324
325
326
327
328
329
do not deal regression problem. In all experiments, we split our dataset into train (75%)- - test (25%), and we learn a model f. a LightGBM (estimators=50. nb leaves=8), on the train set that is the explainee. We learn f's predictions on the train set with an approximating RF mab,n (estimators=20, max 10p that will be used to generate the CR with x = 0.9. The used parameters for AReS, CET are max rules=8, bins= 10 and max iterations=0000, max leaf=8, bins= respectively. Due to page limitation, the detailed parameters of each method are provided in Appendix.
330
331
332
334
335
Sampling CE using the Counterfactual Rules:
Notice that our approaches cannot be directly compare with the baseline methods since they all return counterfactual samples while we give rules (range of vector values) that permit change the decision with high probability. However, we adapt the CR to generate also counterfactual samples using a generative model. For example, given an (Ig. x æg), target e and its counterfactual rule Cs(:;; *), we want to find a sample x= = (zs, xg) with zs E Cs(a, ( s.t xx is an in-distribution sample and f(x*) E 2 Instead of using a complex conditional generative model as [Xu al., 2019, Patki et al., 2016] that can be difficult to calibrate, we use an energy-based generative approach [Grathwohl et al., 2020, Lecun et al.,
336
337
338
339
instance x
340
341
342
343
2006). The core idea is to find 2 E Cs(2, vY" s.l. x' maximize a given energy score to ensure that it is an in-distriuution sample. As an example of an energy function, we use the negative outlier score of an Isolation Forest [Liu et al., 2008]. We use Simulated Annealing (see [Guilmeau et al., 2021] for review) to maximize the negative outlier score using the information of the counterfactual rules Cs [ +) In fact, the range values given by the CR Cs(a;, 2 reduce the search space for øs drastically. We used the training set D,, to find the possible values i.e., we defined P., Ps as the list of values of the variable E S found in D,, and Ps- = {zs = (21...., Es) zg E Cs(x, y²), 2, E P.} the possible values of 2 respectively. Then, we sample =5 in the set Ps and use Simulated Annealing to find a x that maximizes the negative outlier score. Note that the algorithm is the same for sampling
344
345
347
319
351
352
CE with the Regional-CR. A more detailed version of the algorithm is provided in Appendix.
353
Finally, we compare the methods on unseen observations using three criteria. Correctness is the aver- age number of instances for which acting as prescribed change to the desired prediction. Plausibility the average number of inlier (predict by an Isolation Forest) in the counterfactual samples. Sparsity is the average number of features that have been changed, and especially for the global counterfactual methods (AReS, Regional-CR that do not ensure to cover all the instances, we compute Coverage that corresponds to the average number of unseen observations we cover.
354
355
357
8

---

359
Local counterfactual rules for regression: We give recourse for the California House Price dataset [Kelley Pace and Barry, 1997] derived from the 1990 U.S. census. We have information about each district (demography.... ...), and the goal is to predict the median house value of each district.
360
362
To illustrate the efficiency of the Local-CR, we select all the observations in the test set having price lower than 100k (1566 houses), and we aim to find the recourse that permit to increase their price we want the price y to be in the interval g 2200k, 250k]. For each instance æ. we compute the Minimal Divergent Explanation S, the Local-CR Cs [200k: 250k]) and a CE using the Simulated Annealing as described above. We succeed in changing the decision of all the observations
383
364
385
366
367
(Correctness = 1) and most of them passed the outlier test with Plausibilityy = 0.92. On top of that, our LRcal-CR have sparse support (Sparsity- = 4.45). For example, the Local-CR of the instance æ (Longitude=-11) 118.22 2, latitude=33. 8, housing median age-26, total rooms=703,
368
389
370
is total bedroom==202, population=757, households-212, median income=2.
Cm [200k; 2504)) = (total room E (546), total bedrooms E [214, 491] It means if total room and total bedrooms satisfy the conditions in Cs [200k: 250k]) and the remaining features of æ is fixed, then the probability that the price is in [200k, 250k] is 0.97.
372
We use 3 real- world datasets: Diabetes [Kaggle, 2016] contains diagnostic measurements and aims to predict whether or not patient has diabetes, Breast Cancer Wisconsin (BCW) [Dua and Graff, 2017] consists of predicting if a tumor is benign or not using the characteristic of the cell nuclei, and Compas [Larson al., 2016] was used to predict recidivism, and it contains information about the criminal history. demographic attributes, During the evaluation, we observe that ARes, CET are very sensitive to the number of bins and the maximal number of rules or actions as noticed by [Ley et al., 2022]. A bad parameterization gives completely useless explanations. Moreover, different model needs to be trained for each class to be accurate, while we only need to have a RF that has good precision.
Comparisons of Local-CR and Regional-CR with baselines (AReS, CET):
375
376
377
378
379
380
381
382
383
In table 1, we notice that the Local and Regional-CR succeed in changing decisions with a high accuracy in all datasets, outperforming A ReS and CET with large margin on BCW, and Diabetes. Moreover, we notice that the baselines struggle to change at the same time the positive and negative class, (e.g. CET has Acc=1 in the positive class, and 0.21 for the negative class BCW) or when they have a good Acc, the CE are not plausible. For instance. CET has Acc=0.98 and Psb=0 on Compas, meaning that all the CE are outlier. Regarding the coverage of the global CE, CET covers
384
385
386
387
388
389
all the instances as it partitions the space, but we observe that AReS has a smaller Coverage= (0.43, 0.44, 0.81} than the Regional-CR which has {1, }} for BCW, Diabetes, and Compas
390
381
respectively. To sum up. the CR is easier to train and provides more accurate and plausible rules than the baselines methods.
392
Table 1: Results of the Correctness (Acc). Plausibility, and Sparsity (Sprs) of the different methods. We compute each metric according to the positive (Pos) and negative (Neg) class.
COMPAS
BCW
Diabetes Psh
Ace
Pib
Ave
Psb
Sps
Avc
Sps
Pns
Neg
Pos
Neg
Pos
Neg
Pos
Neg
Pos
Neg
Pos
Neg
Pns
Neg
Prs
Neg
Pos
Neg
L-CR
0.9
0.87
0.73
2
1
1
0.96
9
7
0.97
1
0.99
0.8
3
4
R-CR
0.9
0.98
0.74
0.93
2
0.89
0.9
0.94
0.93
9
9
0.99
0.99
0.9
0.87
3
AReS
0.98
1
0.8
0.61
1
0.63
0.34
0.83
0.80
4
0.73
0.60
0.77
0.86
1
CET
0.85
0.98
0.7
0
2
0.21
0.6
0.80
8
0.84
1
0.60
0.20
6
6
Conclusion
393
394
Most current works that generate CE are implicit through an optimization process or a brunch of random samples, thus lacking guarantees. For this reason, we rethink CE as Counterfactual Rules. For any individual or sub-population, it gives the simplest policies that change the decision with high probability. Our approach learns robust, plausible, and sparse adversarial regions where the observations should be moved. We make central use of Random Forests, which give consistent estimates of the interest probabilities and naturally give the counterfactual rules we want to extract.
395
396
397
398
399
In addition, it permits us to deal with regression problems and continuous features. Consequently, our methods are suitable for all datasets where tree-based model performs well (e.g.. tabular data). A prospective work is to evaluate the robustness of our methods to noisy human responses, i.e., when the prescribed recourse not implemented exactly, and to refine the methodology for selecting the
400
401
403
threshold probabilities π and nC-
9

---

References
405
406
Salim Amoukou and Nicolas JB Brunel. Consistent sufficient explanations and minimal local rules for explaining regression and classification models. arXiv preprint arXiv:2// 1.04618. 2021.
408
Clément Bénard, Gérard Biau, Sébastion Da Veiga, and Erwan Scornet. Shaff: Fast and consistent shapley effect estimates via random forests. arXiv preprint arXiv:2 2021a.
410
Clément Bénard, Gérard Biau, Sébastien Veiga, and Erwan Scornet. Interpretable random forests via rule extraction. In International Conference Artificial Intelligence and Statistics, pages 937-945, PMLR, 2021b.
412
Clément Bénard, Sébastien Da Veiga, and Erwan Scomet. Mda for random forests: inconsistency, and a practical solution via the sobol-mda. arXiv preprint arXiv: (tt::2002.13347, 2021c
414
415
Leo Breiman, Jerome Friedman, Richard Olshen, and Charles Stone. Classification and regression trees. wadsworth int. Group, 37(15):237-255 1984.
416
417
S. Chen, Arthur Choi, and Adnan Darwiche. The same-decision probability: A new tool for decision making. 2012.
418
419
Yu-L Lian Chou, Catarina Moreira, Peter Bruza, Chun Ouyang, and Joaquim Jorge. Counterfactuals and causability in explainable artificial intelligence: Theory, algorithms, and applications. Informa- tion Fusion, 81:59-83, 2022. ISSN 1566-2535. doi: https://doi.or// org/ wwtf//.infffss........i.. 2021 11.00.111..........1..11.1111000 11.00. URL https: : www. sci weweedeeeeettt..mm com/ wwwence/articee//eppeeeeaaa /ttaac///atttccee//////////s2221
421
422
429
Qiming Du Gérard Biau, François Petit, and Raphaël Porcher. Wasserstein random forests and appli- cations in heterogeneous treatment effects. In International Conference Artificial Intelligence and Statistics, pages 1729-1737. PMLR, 2021.
425
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http:::///arcchre...cs uci... edu/ml.
427
Will Grathwohl, Kuan- Chan-Chich Wang, Joern- Henrikk Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. In International Conference Learning Representations, 2020.
429
431
Thomas Guilmeau, Emilie Chouzenoux, and Víctor Elvira. Simulated annealing: a review and a new scheme. pages 101-105, 07 2021. doi: 10.1 10011999999502222.9951117777
435
Hemant Ishwaran, Udaya B Kogalur, Fugene H Blackstone, and Michael Lauer. Random survival forests. The annals of applied statistics, 2(3):841 2008.
435
Kaggle. Pima indians diabetes database, 2016. URL https://wwrt kaggle. com/datasets/ pwtppiiiaaiinnns...iiiiimiiiiiii
436
uciml/pima- indiansn-iii-iii wwwbetes-datalleee
Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, and Hiroki Arimura. Dace: Distribution-aware counterfactual explanation by mixed-integer linear optimization. In IJCAI, 2020.
438
Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, and Yuichi Ike. Counterfactual explanation trees: Transparent and consistent actionable recourse with decision trees. In Proceedings of The 25th International Conference Artificial Intelligence and Statisties, PMLR 151:1846-1888 1870, 2022.
440
442
Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. Model-agnose counterfactual explanations for consequential decisions. ArXiv, http//905.111900 2020a.
444
Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. survey of algorithmic recourse: definitions, formulations, solutions, and prospects. CoRR. abs/2010.04050, 2020b. URL
446
https://arxiv. http://aaxi...orra////////////// 040500
R. Kelley Pace and Ronald Barry. Sparse spatial autoregressions. Statistics, Probability Letters, 33 URL
448
(3):291-297, 1997. ISSN 0167-7152. doi: http:://oooonylll.ooonsss6111112
com/
10

---

450
Himabindu Lakkaraju, Dylan Slack, Yuxin Chen. Chenhao Tan, and Sameer Singh. Rethinking 451
explainability as a dialogue: A practitioner's perspective. CoRR, abs/2202.01875, 2022. URL
org/ abs/ 2202. 01875.
Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. How we analyzed the compas recidivism algorithm, 2016. URL https://wwwt htt////wwppppbllieceeo org/articls how-we- analyzed- the compas-mmeeeeeeeeeee recidivism algorithm.
454
455
456
Yann Lecun, Sumit Chopra, and Raia Hadsell. A tutorial on energy-based learning. 01 2006.
457
Dan Ley, Saumitra Mishra, and Daniele Magazzeni. Global counterfactual explanations: Investiga- tions, implementations and improvements, 2022. URL htttps:///aaiiiii httpp//////aaw.o./a///aa///// /2044. 06917.
458
459
Jimmy J. Lin, Chudi Zhong, Diane Hu, Cynthia Rudin, and Margo I. Seltzer. Generalized optimal sparse decision trees. ArXiv, abss2000.086690 2020.
460
461
Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In 2008 eighth ieee international conference on data mining, pages 412-422. IEEE, 2008
463
Wei- Yin Loh. Classification and regression trees. Wiley Intendisciplinary Reviews: Data Mining and Knowledge Discovery, 1, 2011.
465
Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by proto- types. CoRR, 010/190770288444 2019, URL
486
025844.
Scott M. Lundherg, Gabriel Erion, Hugh Chen, Alex DeGrave, Jordan M. Prutkin, Bala Nair, Ronit Katz, Jonathan Himmelfarb, Nisha Bansal, and Su-In Lee. From local explanations to global understanding with explainable ai for trees. Nature Machine Intelligence, (()::2522-583) 2020.
468
470
Nicolai Meinshausen and Greg Ridgeway. Quantile regression forests. Journal of Machine Learning Research, 7(6), 2006.
472
Christoph Molnar. Interpretable Machine Learning. edition, 2022. URL httpss//cchristtthmm gi github. hww.h/teteeeeetaeennl.... book.
Ramaravind K. Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency. FAT '20, page 607-617, New York, NY, USA, 2020. Associa- tion for Computing Machinery. ISBN 978 4503693667. doi: 10. 10.1145/33109553377888
475
478
477
URL
478
https: org/10 1145/3351095 007.511010.3338500 3372850.
479
Axel Parmentier and Thibaut Vidal. Optimal counterfactual explanations in tree ensembles. CoRR, abs/2106.06631, 2021. URL httpt:ttt///aatt httpa/aawwiivaaaaaaaaaaaaaaa hbtp///bb/211..... 06631.
481
N. Patki, R. Wedge, and K. Veeramachaneni. The synthetic data vault. In 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 390-410, Oct 2016. doi: 10.1109/DSAA.201...99
483
484
Martin Pawelczyk, Teresa Datta, Johannes van-den Heuvel, Gjergji Kasneci, and Himabindu Lakkaraju. Algorithmic recourse in the face of noisy human responses, 2022. URL https: // //arxiv... org/ /bs///3.000666 06768.
485
Rafael Poyiadzi, Kacper Sokol, Raúl Santos-Rodrigus Tijl De Bie, and Peter A. Flach. FACE: feasible and actionable counterfactual explanations. CoRR, abs/11909.0933 2019. URLhttp: // //arxiv..o.. org/ abs/ 1909. 09369.
488
489
490
Kaivalya Rawal and Himabindu Lakkaraju. Beyond individualized recourse: Interpretable and interactive summaries of actionable recourses. Advances in Neural Information Processing Systems, 33:12187-12198 2020.
482
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144, 2016.
494
11

---

496
Chris Russell. Efficient search for diverse coherent explanations. In Proceedings of the Conference on Fairness, Accountability, and Transpareney. FAT 19, page 20-28. New York, NY, USA. 2019. Association for Computing Machinery. ISBN 9781 1450361 255. doi: 10. 10.1145/32285550332855566 URL batttss://daee orgoorr//// 1111.....13311 0114/32877500 3287569.
497
499
500
Berk Ustun, Alexander Spangher, and Yang Liu. Actionable recourse in linear classification. Pro- ceedings of the Conference on Fairness, Accountability, and Transparency, 2019.
501
502
Sahil Verma, John P. Dickerson, and Keegan Hines. Counterfactual explanttions for machine learning: A review. CoRR, 0bss22110.1099p 2020. URL https:///xxii www. httpg/abb/200000 10596.
504
Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Harv. JL & Tech., 31:841, 2017a.
505
506
Sandra Wachter, Brent Daniel Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. Cybersecurity, 2017b.
507
508
Stefan Wager and Susan Athey. Estimation and inference of heterogeneous treatment effects using random forests, 2017.
Tong Wang, Cynthia Rudin, Finale Velez, Yimin Liu, Erica Klampfl, and Perry MacNeille. A bayesian framework for learning rule sets for interpretable classification. J. Mach. Learn. Res., 18: 70:1-70:37, 2017.
511
512
513
Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante and Kalyan Veeramachaneni. Modeling tabular data using conditional gan. In NeurIPS, 2019.
514
Checklist
515
516
For all authors...
517
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] In Section 3 the two new explanation methods: The Local and Regional Counterfactual rules. Section 4 shows that our methods are consistent, contrary to prior works. Then, in Section 6 we demonstrate the performance of our new explanations w.r.f SOTA on real-world datasets. Finally, we provide Python Package that computes our methods. Additional experiments can be found in Appendix.
518
519
520
521
522
523
(b) Did you describe the limitations of your work? [Yes]] In conclusion, we emphasize that as our estimators are based on a Random Forest, our methods are suitable for all datasets which tree-hased models perform well. Therefore, it works well on tabular data, but is not adapted for big computer vision models for example.
525
526
527
(c) Did you discuss any potential negative societal impacts of your work? [No] Our con- tributions are fully dedicated to the positive societal impacts. Indeed, we propose new and better explanation methods. For instance, rules permit us to detect unfair behavior of model as A ReS but more accurately. On the other hand, the local counterfactual rules permit to give more robust recourse in real-world scenarios.
529
regional counterfactual
531
539
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes] We conform to the ethics review as we are not concerned by potential negative impacts (methodologies, data...), as the general ethical conduct.
535
537
2. If you are including theoretical results...
538
(a) Did you state the full set of assumptions of all theoretical results? [Yes] See Section 4.
539
(b) Did you include complete proofs of all theoretical results? [Yes] Our methods are based on the theoretical results of previous work. Thus, the complete proof's can be found in the given references.
540
541
3. If you experiments...
12

---

543
(a) Did you include the code, data, and instructions needed to reproduce the main ex- perimental results (either in the supplemental material as URL)? [Yes] All the codes to reproduce the results are given in https: htpp::///ttai thub. com/ anoxai/ counterfactual cacecluualrrelesseeeeee rulles
544
546
547
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See Appendix.
519
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes] We run the experiments multiple times.
550
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix.
552
553
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We use the implementation of AReS, and CET provided by Kanamori et al. at https: hat.t//.cc//eeehhccceeeeeeee We have cited the data used in the experiment Section.
855
557
558
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any assets either in the supplemental material URL? [Yes]
560
At https://gii //ggttttubb... com/ hwwp//nxai/////aa//aaa www.a/raacttataaaacc.c..aaa
581
(d) Did you discuss whether and how consent was obtained from people whose data you're
582
using/curating? [No] Not relevant
563
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [No] Not relevant
564
585
5. If you used crowdsourcing or conducted research with human subjects...
566
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [No] Not relevant
568
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [No] Not relevant
589
570
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [No] Not relevant
571
13


Reviews:
[{"review_id": "wOgG82jiOU", "reviewer": "Reviewer_CFsf", "summary": "To explain machine learning models, the paper proposes learning counterfactual example rules (regions) rather than generating individual counterfactual examples. This has benefits in stability and avoids the variance of producing individual examples. ", "questions": "* Can the approach from Anchors be used in a similar way to solve the CF regions problem?\n* If I understand correctly, the output policies/regions may not be 100% correct (i.e., may not flip the label). Wouldn't this violate the expectation of a counterfactual explanation?\n* Can you provide some examples of the policy learnt by the model, e.g., on Compas? A qualitative analysis of the regions and their interpretability will help.", "limitations": "The provided policies/regions may not be 100% correct. That is a limitation.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n* Useful formulation to think of counterfactual regions versus examples.\n* Technically sound method using random forests that avoids discretization parameters in past work\n* Evaluation is reasonable and compares to past work.\n\n### Weaknesses\n* The writing can be improved. The idea of using Random Forests and ignoring Sbar features is simple enough. It will be nice if the paper can provide intuitive examples as in Figures 2,3,4 before presenting the technical details.\n* possibly similarities to the Anchors paper that has not been discussed https://homes.cs.washington.edu/~marcotcr/aaai18.pdf\n\nThe paper solves a well-motivated problem. Counterfactual example methods, by their nature, can be unstable and lack guarantees. Providing rules or sets of feature values instead of individual examples is an important advance. I wish the authors had engaged more with explanations literature outside CFs that does propose such rules (e.g., the Anchors paper linked above). \nOther than that, I think the paper is technically strong and I like the use of Random Forest as an estimation tool given its non-parametric properties. The writing can be improved to make it more intuitive. It will help to describe section 5 in an algorithm or a table to understand the different steps. It looks like there are multiple steps in the optimization and it is unclear how they interact. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "aA5RCc-Djap", "reviewer": "Reviewer_VVsi", "summary": "The paper aims to address multiple problems with counterfactual\nexplanations that consist of single points, by replacing them by\nrectangular regions that switch the class $f(x)$ of an input x with high\nprobability. These rectangular regions can then be communicated in the\nform of rules, which are called Local Counterfactual Rules. There is\nalso an extension called Regional Counterfactual Rules, which explains\nregions instead of single inputs x.\n\nThe definitions of the local and regional counterfactual rules rely on a\ncertain conditional distribution of the inputs, which therefore has to\nbe estimated very accurately. An important observation of the paper is\nthat this estimation can be made computationally efficient if $f$ is first\napproximated by a random forest $\\hat{f}$.\n\nFinally there are experiments that compare the new methods to existing\nmethods on several data sets.\n\n", "questions": "The limitations of the paper seem very fundamental, and would require a\nsignificant revision to address.\n", "limitations": "Limitations of the approach have not been sufficiently addressed, as\ndiscussed above. In particular, it is not clear when it can be expected\nto work and when not, or how to detect whether it can be applied for a\nparticular type of data.\n\n", "rating": 3, "confidence": 5, "soundness": 1, "presentation": 2, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The structure of the paper is to discuss a series of limitations of\nexisting methods in the literature, and then to posit new definitions\nfor Local and Regional Counterfactual Rules. Unfortunately, however,\nthere is very little motivation for why the new definitions should be\nexactly the way they are, and what their strengths and weaknesses are.\nAnd they do strike me as having at least the following significant\nweaknesses (I will focus on Local Counterfactual Rules here):\n\n* Definition 3.3 of Local Counterfactual Rules only provides a necessary\n  requirement that the rectangles should satisfy, so it is in fact not\n  a full definition.\n* The rectangles produced by Local Counterfactual Rules may\n  actually be *implausible*, i.e. definition 3.3. leaves open the\n  possibility that P(C_S(x;y*)) is arbitrarily small, because it is only\n  a requirement about a probability conditional on C_S(x;y*).\n* The approach critically relies on estimating a conditional probability\n  for x. There should be a discussion on when this is reliable. For\n  instance, this cannot work well when explaining inputs x from\n  low-probability regions for which the exact distribution cannot be\n  estimated very well, or if the true distribution is very irregular, or\n  if the input dimension is large. The conditional probability is also\n  estimated for each possible rectangle C_S(x), which will inflate the\n  estimation error.\n* It is not clear how a user is supposed to obtain recourse from hearing\n  a rectangular region. Definition 3.3 implies that, if the user picks a\n  point from the region according to the true distribution of x, then\n  they will obtain recourse with high probability. But a) the user does\n  not know the true distribution of x, and b) even if they did, it would\n  seem unreasonable to expect them to modify their features in\n  accordance with this distribution. The introduction (line 98) claims\n  that their approach \"partly answer[s] the problem of noisy responses\n  to 99 prescribed recourses\" identified by Pawelczyk et al., 2022, but\n  this claim is not backed up.\n\nPresentation:\n\nThe presentation is sufficient, but clarity could still be improved\nand claims could be substantiated better. For example: \n- There are multiple cases where terms are unclear or become clear only\n  later, especially in the introduction. For example, line 51 refers to\n  \"the best promising directions\", but it is not clear by which measure\n  'best' is determined.\n- In line 34 it is claimed that solutions from prior work are \"not\n  entirely satisfactory\", but it is not specified in which way they are\n  not satisfactory.\n- Definition 3.2 confuses the distinction between a \"Divergent\n  Explanation\" and a \"Minimal Divergent Explanation\".\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "0dOErpyxBr8", "reviewer": "Reviewer_b9vt", "summary": "The authors propose to counterfactual rules instead of counterfactual samples as explanations -- i.e. the explanation is a (set of) rule(s) instead of single sample (counterfactual).", "questions": "None", "limitations": "None", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Good paper!\n\nOnly minor issues:\nI think an illustration inf the form of a 2d data set + decision boundary might help even more to understand Local and Regional Counterfactual Rules.\nI am bit concerned about generating rectangles for high-dimensional spaces (i.e. curse of dimensionality) -- maybe the authors could briefly comment on this.\nIf I understood it correctly, the users always used a LightGBM as a black-box model that is going to be explained. Since the authors propose a model agnostic method, I think evaluating on a single model only is enough -- the authors should consider several highly different models for their empirical evaluation.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Reject