Learning Models for Actionable Recourse
Authors: Alexis Ross
Himabindu Lakkaraju
Osbert Bastani
Venue: NeurIPS 2021 Poster (2021)
Keywords: recourse
actionable recourse
machine learning
adversarial training

Abstract: As machine learning models are increasingly deployed in high-stakes domains such as legal and financial decision-making, there has been growing interest in post-hoc methods for generating counterfactual explanations. Such explanations provide individuals adversely impacted by predicted outcomes (e.g., an applicant denied a loan) with recourse---i.e., a description of how they can change their features to obtain a positive outcome. We propose a novel algorithm that leverages adversarial training and PAC confidence sets to learn models that theoretically guarantee recourse to affected individuals with high probability without sacrificing accuracy. We demonstrate the efficacy of our approach via extensive experiments on real data.

Learning Models for Actionable Recourse
Alexis Ross"
Himabindu Lakkaraju
Harvard University
Harvard University
Allen Institute for Artificial Intelligence alexisr@allenai.
hlakkarajuuseas hwrvard. edu
Osbert Bastani
University of Pennsylvania obastan obastani@sen tani@seas........ upenn. edu
Abstract
As machine learning models are increasingly deployed in high-stakes domains such as legal and financial decision-makinn making, there has been growing interest post-hoc methods for generating counterfactual explanations. Such explanations provide individuals adversely impacted by predicted outcomes (e.g., applicant denied loan) with recourse-i.e -i.e,, description of how they can change their features to obtain positive outcome. We propose a novel algorithm that leverages adversarial training and PAC confidence sets to learn models that theoretically guarantee recourse to affected individuals with high probability, We demonstrate the efficacy of our approach with extensive experiments on real data.
1 Introduction
In recent years, there has been a growing interest in using machine learning to inform consequential decisions in legal and financial serisionn-aaking-ea deciding whether give an applicant loan [Hardt et al. [016]. bail to a defendant (Lakkaraju and Rudin [017]. or parole to prisoner ZZng al. 2017]. Because these decisions have an impact the lives of the concerned individuals, is critical to explain why the model made its prediction. Explanations are important not only to ensure that there are no issues with the way the prediction made (e.g., making sure the decision is free of racial/gender bias (Hardt et al. 2016) and does not suffer from causal issues [Bastani et al. 2017 )
but also to give the affected individual a justification for the decision. Thus, there has been great
deal of recent interest in explainable machine learning [ou et al. 2012
Wang and Rudin 2015
Ribeiro et 2016b
Lakkaraju al. 2019..
We focus on counterfactual explanations Wachter et al. 2017]. which specify how features can be changed to obtain different model prediction. These explanations can be used to provide individuals who are negatively impacted by model outcomes with actionable recourses-i.e. actions they can take receive a positive outcome [Ustun et al., 2019). For instance, for applicant denied a loan, an actionable recourse might be: "to get a loan, increase your income by $1 31,000." Such a recourse must satisfy two properties be actionable: (i) it only changes features that the individual can realistically modify-e., e.g., it cannot change race gender, but can change income, and (ii) the magnitude of the change must be reasonable.
actionable recourse like this may be considered necessary some settings because it provides an individual with agency over consequential decision that affects them.
Prior research has addressed the need for recourses through post-hoc algorithms for computing individual recourses corresponding to certain kinds of models-e.g. using integer linear programming
Work started at Harvard University.
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

in the context of linear models [Ustun et al. differentiable models (Wachter et al., 2017 J. actionable recourses exist; at best, Ustun et al.
2019], or using gradient descent on the input for
However, these approaches do not guarantee that
2019 guarantee that they find one if it exists, and only for linear models. In other words, many affected individuals may not even be prescribed any actions that they can take to change their outcome.
We aim to provide tools for guaranteeing the existence of actionable recourses for domains in which recourses may be necessary. We propose a novel algorithm for learning models in a way that is designed to ensure the existence of actionable recourses so that affected individuals receive recourse with a high probability. To the best of our knowledge, our approach is the first to train models for which recourse likely exist with high probability. It builds on adversarial training [Goodtellow et al. 2015a]. which is designed to ensure that models are robust to adversarial examples. At a high level, given binary classification model so + y/ where y = {0, }}, an adversarial example is a perturbation 8 A to an input E X' such that fo(x + 6) & fo(r). In typical adversarial training setting, we choose to be "small" in some sense (e.g.. in terms of L norm of E A) and aim to guarantee that adversarial examples do not exist-i.e. f(x 6) f(x) for all 8 E A.
In contrast, in our setting, we intuitively want to ensure that adversarial examples do exist. In particular, given an input x for which f(x) = O, we want there to exist recourse E A such that f(r + +) = 1, where in our case, A is the given set of all permissible recourses for the application domain. Thus, we adapt existing adversarial training algorithms to ensure the existence of recourse. As an added benefit, these algorithms can compute recourse significantly faster than existing techniques for general differentiable models Waacttrr et al. 2017
While adversarial training heuristically encourages recourse to exist, it does not provide any theoretical guarantees. We build on PAC confidence sets (Park et al. 2020 to guarantee that recourse exists with high probability (assuming the test distribution the same as the training distribution).
We evaluate our approach on four real world datasets that cover lending, recidivism, bail, and credit outcomes. Our results demonstrate that our approach is very effective at improving recourse rates (i.e., the probability that individuals are given recourse) without noticeably reducing accuracy.
In addition, we show that we achieve this improvement in recourse rates without noticeably harming the quality of recourses or the brittleness of the underlying model. Firstly, we empirically demonstrate that our approach improves the rate at which models provide recourses that are grounded in reality: We find that our approach encourages the existence of recourses that both obey causal constraints driven by real-world causal relationships and are in-distribution to the original data. Secondly, we show that our approach encourages the existence of robust recourses (i.e., recourses that result in positive outcomes even when changed in small ways). Lastly, we show that these improvements in recourse rates do not render the underlying classifier brittle
Related work. Beyond Wachter et al. [201) 7]. Ustun et al. [019]] other approaches have been proposed for generating recourses [Zhang et al.
2018 Hendricks et al. 2018 Mothilal et al.) 2020
However, all these works focus on how to compute recourse for given predictive models; in contrast, our goal is to learn predictive models that provide recourses at high rates. Any of these methods can be used in conjunction with ours. Our work also builds on adversarial training [Szegedy et al. 2014 Goodfellow al. 2015b Bastani et al., 2016 Shaham et al. 2018 While recent work in model explainability has leveraged adversarial training to improve robustness of explanations [Lakkaraju et al. 2020b), their goal is to reduce the rate of adversarial examples, whereas ours is to increase the rate of recourses.
Looveren and Klaise 2019 Poyiadzi et al.) 2020 Karimi et al. 2020a ble
2 Problem Formulation
Preliminaries. Consider a binary classifier fa X' + V, where x E X C Ra are the features, y [ = {0, 1 are the labels, and 0 E θ C Re are the parameters. We assume that ʃo has the form fa(x) = (x) 2 ob). where ge: : X + [0, 1] is a scoring function and & E R is a decision threshold. We assume that go is differentiable in x- i... Vaga exists almost everywhere
"Our code is available at http::///htttooooooo.o.cssssssss
'Note that we have focused on real- features. We discuss how our algorithm can be extended to handling categorical features in Section B.4

---

Recourse. We seek to ensure that individuals given negative outcomes by fo are also given recourse.
Definition 2.1. Given a classifier fo : + y and a set A f R an input x & has recourse there exists d E A such that f(x + 5) = 1.
We use Rg to denote the set of inputs for which recourse exists. The specifie design of the set of permissible recourses A S I is domain specific. We assume that 0 E A; then, recourse 1 by taking ā = 0. In addition, we assume that A is a polytope -ie A {6 R Ab + b 2 0} for some A R and b R That is, A can be expressed as a set of affine constraints. This assumption is required for computational tractability,
automatically exists for positive outcomes (x)
A standard choice is A {ō |0]1>> S dmax), which says that the recourse can suggest changes to any feature by a bounded amount. We can apply this constraint after an affine transformation of that appropriately scales different covariates. In addition, we often want to restrict features- e.g., to avoid suggesting that an individual change their age, we can constrain d; = O, or to avoid suggesting that an individual decrease their income to qualify for loan, we can constrain d; 2 0. In principle, can also tailored to individuals- e.... disallow suggesting increased education for individuals who cannot afford to do so. Finally, A should be designed large enough so it includes a plausible recourse for every individual, yet small enough to ensure that the recourses do not overburden the individuals. All of these considerations are domain-spectin our experiments in Section
describe our choices for datasets in
Probably Approximately has REcourse (PARE). Our goal is to ensure that recourse exists for most individuals. Given a confidence level E R2D, we say that @ approximately has recourse if
Pp(x) x E X 2 1 - E,
ie., recourse exists for fe with probability at least 1 - w.r.t. the distribution p(z) over individuals.
Then, our goal to design an algorithm for estimating the model parameters 0 so that so approxi- mately has recourse. To do so, our algorithm takes as input held-out calibration dataset Z S X x ) of examples (ar, y) ~ p, where p(2, y) the distribution over labeled examples, and outputs model parameters (2). Then, as in the probably approximately correct (PAC) learning framework [Valiant 1984), our algorithm might additionally fail due to the randomness in Z. Thus, given second confidence level a E Rzo, we say that i Probably Approximately has REcourse (PARE) if
PP(Z) (0(Z) approximately has recourse
2 1 a,
where p(Z)= = II (Z.)) p(z, v). In other words, our algorithm produces model that approximately has recourse with probability at least - 8 over p(2).
Constructing a PARE classifier. Note that we can trivially obtain a PARE classifier fo by choosing the decision threshold 9, O, in which case fx) = for all x E X. However, this approach is undesirable since assigns a positive outcome all individuals. Instead, we want to maximize the performance of fo (e.g., in terms of accuracy, Fi score, etc.) subject to a constraint that fo is PARE. Thus, we divide the problem of constructing a PARE classifier into two parts (i) increasing recourse rate: we train go in way that heuristically increases the rate at which inputs x E X' have recourse (for any fg), and (ii) guaranteeing recourse: we choose Og to guarantee that the resulting fe is PARE.
3 Our Algorithm
We describe our algorithm for learning models that satisfy PARE while achieving good performance. We describe Step (increasing recourse rate) in Section 3.1 and Step (guaranteeing recourse) in Section
3.3
We describe ways to compute recourse in Section 3.2
3.1 Step 1: Increasing Recourse Rate
Background: adversarial training, Consider classifier fo X - y and perturbations A C R=x Given x E X, an adversarial example (Szegedy et al., 2014] for x is a perturbation E A such that fo(x + 5) fo(x) i.e., the perturbation ò (restricted be small) changes the predicted label of Adversarial examples are undesirable because they indicate that So is not robust to small changes to the input that should not affect the class label (e.g., according to human predictions). Thus, there has
3

---

been a great deal of interest in designing algorithms for improving robustness to adversarial examples. The basic approach is adversarial training [Goodiellow al. 2015b), which dynamically computes adversarial examples for inputs in the training set and adds these to the objective as additional training examples, as in data augmentation. (x))( y) is the loss for training example V), they seek to compute
particular, given a loss function l
+ IR where
0" = arg min (0) where (A( = Ep(r.w) {(go(z). v)
max ((g + 6), 6).8)
EEe
SEA
In (0), the first term is the supervised learning loss, the second is the adversarially robust loss (i.e., encourage Go to be robust to adversarial examples x + 8), and À E Rzo is a hyperparameter.
The challenge in optimizing (0) is computing the maximum over 8 E A. To address this challenge, existing adversarial training algorithms leverage approximations enabling efficient computation d.
Our approach. We use adversarial training to learn model Sa for which inputs E X have recourse at higher rates compared to models trained using conventional approaches. There are two key differences compared to ad training. First, want recourse to exist, which corresponds to encouraging the existence of adversarial examples. Second, we only care about changing negative labels fo(x) 0 to positive ones fo(x + 5) 1, not vice versa. Thus, want solve
(n( = Epta.) ((go(()::) v) + X min (go + +6),,) 1)
0" = arg min (n))) where 8ge
(I)
SEA
Compared C,, E has a different second term in two ways: (i) the maximum over 5 with minimum, and (ii) the label y is replaced with the label 1. We note that when À = 0, Eq. iis supervised learning.
We optimize Eq. T using adversarial training [Goodfellow al. 2015a Shaham et al. 2018
Lakkaraju et al. 2020a which performs stochastic gradient descent on ( The key challenge to computing Vo Voen(( (0) is computing the gradient of the second term, which can be rewritten follows:
V min (gg (x + 5), 1) = Vel(ge (x + 8*), 1),
BEA
where 8* is the perturbation that maximizes the likelihood of positive outcome, or minimizes the loss between predicted and positive outcomes-i.e.
8* arg min (ggo + 6).1)
Computing 8 is computationally challenging; thus, (ggo( + 0), x ((ga(z), 1) V,l(go(r), Using this approximation, Eq. becomes
Taylor approximation of the loss
8 =ar Barggin min (gg 8), 1) arg Vzl(ge 1) SEA
BEA
where we have dropped the term E(go (z), .. 1) since it is constant with respect to 8. Finally, note that the optimization problem on the last step is linear program (LP), since we have assumed that ó e A be expressed set of affine constraints and since the objective is linear in 6. In summary, optimize Eq. Jsiin stochastic gradient descent, where at each step we solve an LP to approximate the second term-i.e.. given parameters and example (1)) W:) gradient step i, and step size E R> on step i, we use the stochastic gradient update
0,41 = 0, + Th (Vel(go(z)) u) + Vol(go (x: + 1)) where 8 = arg min (zl(go<)),1)
dEA
3.2 Computing Recourse
So far, have focused on how to train model fe that provides recourse. Once we have trained fe- we still need a way to compute recourse for given individual æ-iie., an algorithm : X + for computing 8, A(x) such that f(x 8.)) 1. We describe three such algorithms; in general, any algorithm designed to output recourses can be used [Karimi et al.] 2020a Poyiadzi et al. 2020).
Gradient descent. The approach proposed in Wachter et al 2017 can directly be applied to compute recourse. They solve the problem
8g = arg min({(go(x + 5, + X 1
SEA

---

where X' E R20 is a hyperparameter, using gradient descent on ö. The term ||āÍ|2 is designed to encourage the recourse be small, which often desirable in practice (we have excluded from our formulation for simplicity). While this approach is generally effective, it can be very slow since we need to solve an optimization problem for each individual.
Adversarial training, We can also use adversarial training to compute recourse-i.e..
= arg min V (x))) T)
SEA
This approach approximates the gradient descent approach, but can be computed much more effi- ciently. Furthermore, since our objective in Eq. i is designed to encourage this specific perturbation to provide recourse, it performs nearly as well as gradient descent when fa is trained with À > O.
Linear approximation. Finally, Ustun et al. (2019) propose an approach to compute recourse when fo(x) 1(8Tx 2 (b) is linear model. In this case, they compute 6₂ using an integer linear program (II.P. For nonlinear models, we can instead use the linear approximation of near I. Letting 8- = A(x; Bo. 3) be the recourse generated by their algorithm, and using the Taylor expansion (a = go (x) + V gx (x)T(=' (x' z),, we can use their approach to compute the recourse
6_ ((x; Oo go)),, (x))
However, this approach only works well when go is approximately linear as a function of x; otherwise, the Taylor approximation may be poor and 6 may not satisfy the desired condition fo + 6x) 1.
3.3 Step 2: Guaranteeing Recourse
Finally, we describe how we choose Og to ensure fo provides recourse with high probability. Note that 8 also controls the fraction of individuals given a positive outcome without the need for recourse; thus, we choose 0o to optimize the performance of fe subject to a constraint that fo is PARE.
Background: PAC confidence sets. We build on work constructing PAC confidence sets (Park et al. 2020j. Given x E X' they construct model he.- X' - P()) (where P is the power set) that returns the set of all labels y with score above threshold R-ic.,
{ (0(())
if yf
Th (x) = ty E E ho(a, y) 2 r where he(x, v)
{1 (o(2)
if - O.
Given e E R200 we say T is approximately correct if
P Pp(na) (y E hx)
i.e., (x) contains the true label for with high probability over p( v). Note that 0 satisfies this condition, since ha.o(x) = {0, 1} for all æ. The goal is to choose T as large as possible while ensuring approximate correctness.
Park et 22200 proposes an estimator è that takes as input (i) the pretrained model ge X [0, 1].
(iI) a calibration dataset Z C X' x y' of i.i.d. samples (I, y) ~ p. and (iii) confidence levels e, a E Rx,, and constructs threshold +(2) [0, 1] that probably approximately correct (PAC):
Pp(z)[((Z) is approximately correct] 2 11a
In other words, +(Z) is approximately correct with probability at least 1 - a according to p(7). Their approach leverages the fact that + +(Z) is an estimator for single parameter; thus, they can use learning theory to obtain PAC guarantees [Kearns et al. 1994].
PARE models via PAC confidence sets. Given model go X - R, an algorithm A: X A for computing recourse, and calibration set Z S X' x ), our algorithm leverages PAC confidence sets to choose threshold 0g = (2( that ensures the resulting model fa satisfies the PARE constraint. First, our algorithm uses the PAC confidence set algorithm to construct the new calibration dataset
z' = {(x + A(x), (z, v) E Z}.
Intuitively, Z' says that the "correct" label for every input x + should be 1-i.e, the recourse constructed by A should satisfy fo(æ + A(x)) 1.
5

---

Metries
Adult
Compas
Bail
German
Baseline
Ours
Baseline
Ours
Baseline
Ours
Baseline
Ours
Performance
Fl score
0.697
0.636
0.739
0.717
0.775
0.760
0.447
0,419
Accuracy
0.830
0.787
0.667
0.565
0.643
0.629
0.600
0.527
Precision
0.621
0.555
0,655
0.561
0.646
0,644
0.364
0.317
Recall
0.799
0.752
0.850
0.991
0.968
0.930
0.583
0.638
Recourse neg
Linear approx.
0.220
0.053
0.156
0.068
0.102
0.029
0.204
0.086
Gradient desc.
0.210
0.496
0.579
1.000
0.317
1.000
0.804
0.864
Adversarial train.
0.007
0.498
0.000
0.967
0.000
0.993
0.127
0.968
Recourse all
Linear approx.
0.453
0.328
0.773
0.982
0.957
0.981
0.607
0.593
Gradient desc.
0.461
0,661
0.883
1.000
0.970
1.000
0.890
0.920
Adversarial train.
0.321
0.659
0.722
0.999
0.952
0.999
0.517
0.980
Table I: Performance and recourse for the baseline model (A 0) and the model trained with our
algorithm (A 0.8), and for each of the three algorithms for computing recourse in Section 3.2 We
show mean results across 3 random data splits and bold the higher value between the baseline and our algorithm.
Then, our algorithm constructs h2))(2) using go. Z', and the given e, a. The PAC guarantee in Eq. says that with probability at least over p( p)Z), have
PP( (Prewi he. 2 - 2
where (z',8') Z' is the example constructed from (z, y) Z as described above. Note that the outer probability over p(Z) since Z' is deterministic function of the random variable Z. Plugging in the definitions x' = x A(x) and y' 1, Eq. becomes
P PA( P Premm) [1 E ho. + A(x)) 2 21-a.
and plugging in the definition of he.r, it becomes
Pp(z) [P "p(=, (90 + A(x)) +(2))
21
(5)
Then, our algorithm returns d Z) = +(Z') (with given parameters 0 as the remaining parameters). Since Eq. is equivalent the PARE condition, we have:
Theorem 3.1. 0 satisfies the PARE condition.
Experiments
We evaluate our approach and show how it can effectively improve recourse rates while preserving accuracy. We also demonstrate how it can improve correctness and robustness recourses. In Appendix B we provide additional results on how our approach affects fairness of models, well as results on an NLP task with discrete covariates to demonstrate the flexibility of our framework.
4.1 Experimental Setup
Datasets. We use four real-worle datasets. The first contains adult income information from the 1994 United States Census Bureau (Dua and Grafi 2017]. It includes information about adults' demographics, education, and occupations. Each adult is labeled as making below or > $50K a year, which can thought of as a proxy for whether an individual will be able to repay loan or not. The second contains information collected by Propublica about criminal defendants' compas recidivism scores [Angwin al., 2016). This dataset includes information about defendants' demographics and crimes, and each defendant is labeled as having either high low likelihood of reoffending,

---

adult
bail
Recoursa IAll Recourse (Neg) Perfommanca
R
0.2
0.
0022440600101224111122 00020405008
5.0224 00G020000091001100000 211.0200000000101111111111111111
Figure How performance and recourse vary with À 0 is the baseline and X is our approach. We plot means and standard errors across 3 random data splits. The first row shows performance metrics; the second and third show recourse metrics. Performance: F Accuracy. "gradient descent" "adversarial training".
Recourse Algorithm:
66
Thrsred IN
Figure 2: How performance and recourse vary with 6g for our approach (A 0.8, left) and the baseline (A = 0, right). Performance:
Fi Accuracy. Recourse:
Recourse neg Recourse all.
as measured by the compas assessment tool. The third dataset represents bail outcomes from two different U.S. state courts from 1990-2009 (Schmidt and Witte 1988). It includes information about individuals' criminal histories and demographics. Each defendant labeled as having a high or low risk of recidivism. The fourth dataset is the german credit dataset (Dua and Graff 2017], which contains individuals' demographic, personal, and financial information. Each applicant is labeled as either having high low credit risk.
We standardize all continuous features have mean and variance 1. We randomly split each dataset into 80% train and 20% validation sets. We use random data splits and report the splits for the rest of this section, unless otherwise specified. For compas and adult, we hold out 500 examples from the validation set form test set; for german, we hold out 100 examples. For bail, we evaluate on a 500-instance subset of the test set. All results are reported on the test set
across
Models. All models are neural networks with 3 100-node hidden layers, dropout probability 0.3, and tanh activations. For evaluation, we choose the epoch achieving the highest validation F; score.
Parameters. We experimented with values between 0.0 to 2.0 increments of 0.2 and found that À = 0.8 provided the best tradeoff between F1 score and recourse rate across all datasets; evaluate how our results vary with À below. For A, we choose the set of actionable features (i.e., features that can changed part of the recourse) based on the dataset (two features for adult, bail, german; one for compas); we set 8max = 0.75 after standardizing features. (See Appendix [B.2 experiments investigating the effect of varying values of dmax). We also include domain-specific linear constraints in A e.g., for the adult dataset, recourse can only require that hours worked increases. See Appendix for more details about our choices of A.
*Our processed datasets have sizes: adult z= 32.5K, compas 6K, bail z 8K, german 1K.
7

---

We choose the decision threshold Bg to maximize Fl score rather than to obtain PAC guarantees, since our goal is to understand the tradeoff's between model performance and recourse rates with fixed underlying predictive model fo- We evaluate the effects of rigorously choosing On in Section
Baselines. To the best of our knowledge, our approach is the first train models with the objective of providing recourse at a higher rate. Thus, we compare to baseline that omits our recourse loss-iee.,
= For our approach and this baseline, we evaluate the performance of each of the three different algorithms for computing recourse described in Section 3.2 We use the alibi implementation
Klaise et al. 2019 of the gradient descent algorithm for computing recourse |aachter et al. 2017
and set the imttial value of the hyperparameter X' = 0.001. We use LIME [Ribeiro et al. 2016a as our linear approximation method with the approach proposed by |Ustum et al. 2219).
Metrics. We evaluate our approach and the baselines with the following metries: (i) standard
performance metrics of accuracy and Fj score, (ii) "recourse neg," the proportion of instances
with negative original outcomes that receive recourse such that f(x) = 0 but f(x + 5*) = 1, and (ii)) "recourse all," the proportion of all instances such that either f(x) = or f(x) f(z + 8*) 1. Metric (iii) is most useful for measuring rate of positive outcomes, since we want to include individuals who are originally assigned a positive outcome.
but
4.2 Efficacy of Our Framework
In Table M we show the performance and recourse rates of models trained with our approach and baseline models. Overall, our approach significantly improves recourse without sacrificing performance. Across datasets, models trained using our approach offer recourse at significantly higher rates than the baseline model, for both the "adversarial training" and "gradient descent" approaches to computing recourse. We do not observe this trend when using "linear approximation" to compute recourse; in this case, both the baseline and our approach perform poorly. We believe this effect can be explained by poor LIME approximations of fe. which are exacerbated by adversarial training since it increases the nonlinearity of fo. Figure [ shows how these results vary with A: Fi scores and accuracies are relatively stable as a function of A.
In Figure 2 we plot how these results vary with 6g (for classifiers trained with À = 0 and À = 0.8 on a single data split), using the "adversarial training" method of computing recourse High values of On lead to lower recourse rates in all cases. The curves for performance are similar for the baseline and for our approach, but the decline in recourse values begins at lower thresholds in the case of the baseline. Thus, our approach improves recourse for most choices of 6o without sacrificing performance. The trade-off between recourse and performance depends on the dataset. For adult, performance increases while recourse decreases since the majority label in this dataset is negative, whereas for bail and compas, performance and recourse both decrease as 0g increases since the majority label is positive.
Performance under PARE guarantees. Next, we show that we can often obtain PARE guarantees without significantly reducing performance. We compare the performance of choosing the decision threshold 0 to maximize Fi score to that of On chosen to obtain PARE classifiers. Specifically, for the latter, we compute On using the approach described in Section 3.3 with parameters& < = a 0.05. Then, we evaluate models at thresholds in 10 equally spaced increments from to the upper bound and fix do to maximize Fl score. For these experiments, we use the "adversarial training" algorithm to compute recourse; we observed similar trends for the "gradient descent" algorithm.
Results are shown in Table 2 In all cases, choosing Gy to satisfy the PARE condition yields a = 0.95, which validates our theoretical guarantees.
classifier that returns recourse a rate 2 Furthermore, on all three datasets, the F1 score does not significantly decrease when imposing the PARE condition. We do see decrease in F1 score for the baseline model on the adult dataset, but the decrease for our model is smaller, suggesting that our end-to-end framework of training models and fixing O is successful at guaranteeing recourse without big drop in accuracy
"Note that the "recourse neg" values are low for À = 0 because we use the "adversarial training" method to compute recourse, which builds on fast linear approximation and thus does not exhaustively find recourses. We this method instead of the more effective "gradient descent" method for efficiency, since the latter is less efficient and would require recomputing recourses for each threshold.
We can obtain a weaker theoretical guarantee at a smaller cost in performance. For instance, applying PARE model in Table 2 with
0.25 results in an Fl score of 0.613 on the adult dataset.
8

---

Adult
Compas Recourse
Bail
German
Fi
Recourse
Fi
Fi
Recourse
Fi
Recourse
BL Fi max
0.697
0.321
0.739
0.722
0.775
0.952
0.447
0.517
+ PARE
0.400
0,981
0.722
0.972
0.777
0.996
0.442
0.997
Ours Fi max 0.636
0.659
0.717
0.999
0.760
0.999
0.419
0.980
Ours + PARE
0.526
0.974
0.721
0.999
0.776
0.999
0.457
1.000
Table 2: Impact of choosing decision threshold 0g to satisfy the PARE constraint. We show F1 scores for the baseline model and the model trained with our algorithm using two methods of determining fg: (i) maximize the Fi score, and (ii) guarantee that the model satisfies the PARE constraint (+PARE, bolded). For the baseline model (BL), À = O; for our model (Ours), À = 0.8.
Recourse (Neg)
05
Figure 3: "Recourse neg" for the german dataset using the causal recourse computing algorithm proposed by Karimi et al. [2020b) We show means and standard errors across 20 random data splits for varying values of X.
Groundedness of recourses. One key question is whether the recourses of models trained with our approach are grounded in reality-i.e., whether they plausible modifications of the ground truth label. For instance, if an individual is denied a loan, they should be given a recourse such that if they take the specified action, they actually increase their likelihood of paying back the loan; individual may increase their income and be given a loan, but still fail to repay it. We want to ensure that our training approach increases the rate of recourses that obey causal relationships in the world, rather than recourses that exploit spurious correlations between features. Directly evaluating groundedness of recourses is challenging, since we do not know the ground truth labels for suggested recourses. Thus, we evaluate whether they are causally grounded and in-distribution.
First, we measure the rate at which causally grounded recourses offered by models trained with our approach. We apply the algorithm for computing causal recourses (CR) introduced by Karimi al. [2020b] to evaluate whether proposed training algorithm improves the rate at which causally grounded recourses are offered. Because CR requires access to an underlying structural causal model (SCM) of the world, we only experiment with the german dataset, for which Karimi et al. (022b) provide an associated SCM. We measure the proportion of test instances x for which the CR algorithm computes valid recourse that satisfies the constraint that perturbations be bounded by 6mx -i.. "recourse neg") i As shown in Figure 3 with increasing X, the rate at which recourses are by the CR algorithm increases. This finding suggests that our training algorithm encourages the existence of causally grounded recourses.
Second, we evaluate whether the recourses offered by models trained with our approach are in- distribution with respect to the original training data. In line with prior work (Slack et al. 2020]. train classifiers to distinguish between original data instances and recourses computed by the "gradient descent" algorithm for models trained with our approach (A = 0.8). In Table 3 we report held out test The low classifier accuracies across all
the accuracies of these classifiers datasets indicate that these recourses are indistinguishable from original data instances. Thus, our framework encourages the existence of recourses that are in-distribution the original data.
Robustness. Another key question is whether the recourses generated using our approach are robust- ie., whether small changes to the recourse result in valid recourses. For instance, if an individual
We select hyperparameters for the CR algorithm that maximize "recourse neg" on the train set.

---

Adult
Compas
Bail
German
Neural Network
0.54
0.56
0.52
0.51
Random Forest
0.53
0,55
0.51
0,51
Logistic Regression
0.51
0.48
0.48
0.47
Table 3: Accuracies of classifiers trained to distinguish between original data instances and recourses computed using the "gradient descent" algorithm for computing recourse for models trained on a single random data split with our approach (A 0.8).
Adult
Compas
Bail
German
Robustness Exp.
Recourse Alg.
BL
Ours
BL
Ours
BL
Ours
BL
Ours
Grad, desc.
1.000
0.865
1.000
1.000
1.000
1.000
0.949
0.898
Recourse
Advers. train.
1.000
0.841
1.000
1.000
1.000
1.000
0.857
1.000
Model
0.976
0.926
0.980
1.000
0.994
0.996
0.970
0.920
Table The first row shows the percentage of recourses found that are robust to noise. The second row shows the percentage of test inputs that are robust to noise in the recourse dimensions. We show results for models trained with varying (A indicates the baseline, and À 0.8 indicates our approach) on single data split. For each model, we show results for the "gradient descent" and "adversarial training" algorithms for computing recourse in Section 3.2
increases their income by more (or even slightly less) than the amount suggested in the recourse, they would expect still be provided with positive decision.
For each computed recourse 8, we compute noisy recourse 8' by adding i.i.d. Gaussian noise to each actionable feature of 8-i.e., .. 8 6, + (0, 0.1). We consider recourse ö robust if its noisy recourse is valid-i.e. if f(x + 8') = 1. In the top row of Table we report the percentage of recourses found that are robust for baseline model and model trained with our adversarial approach. As shown, our training approach does not significantly reduce the robustness of recourses: There is a slight drop in robustness for the adult dataset and for the "gradient descent" recourse algorithm for the german dataset; however, for compas and bail, recourses remain robust.
Effect on classifier brittleness. We also investigate the effect of our adversarial training approach on model brittleness. particular, we measure how sensitive models trained with our adversarial algorithm are to noises in the recourse dimensions, as compared to baseline models. We add i.i.d. Gaussian noise, as described above, the actionable dimensions of original inputs, and compute the proportion of test instances for which the trained model is robust to this noise. As shown in the bottom row of Table f our training approach does not significantly increase model brittleness- observe small drop in model robustness in the recourse dimensions for the adult and german datasets and a small increase in robustness for the compas and bail datasets. These results suggest that our training approach effectively ensures recourse without rendering classifiers brittle.
5 Conclusion
We have proposed novel algorithm for training models that are guaranteed to provide recourse for reversing adverse outcomes with high probability. Our experiments show that our algorithm trains models that provide recourse at high rates without sacrificing accuracy compared to traditional learning algorithms. Future work includes extending our techniques beyond binary classification.
Acknowledgements
We would like to thank the anonymous reviewers for their insightful feedback. This work is supported in part by the NSF awards #1IS-2008461, #IIS-2040989, and #CCF-1910766 and research awards from the Harvard Data Science Institute, Amazon, Bayer, and Google, The views expressed are those of the authors and do not reflect the official policy or position of the funding agencies.
10

---

References
Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. How we analyzed the compas recidivism algorithm. Propublica, 2016.
Osbert Bastani, Yani Ioannou, Leonidas Lampropoulos, Dimitrios Vytiniotis, Aditya Nori, and Antonio Criminisi. Measuring neural net robustness with constraints. In Advances in neural information processing systems, pages 2613-2621, 2016.
Osbert Bastani, Carolyn Kim, and Hamsa Bastani. Interpreting blackbox models via model extraction. arXiv preprint 1rrXivv1755.000 705.08504, 2017.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4188 Minneapolis, Minnesota, URL https
June 2019. Association for Computational Linguistics. doi: 10. 10.11866501
// //www.c www.wb.cwwbb.o org/ http//antooloolll1/9 1423
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http:///archive.icc.. uci.edu/m
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015a.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In ICLR, 2015b.
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. In Advances in neural information processing systems, pages 3315-3322 2016.
Lisa Anne Hendricks, Ronghang Hu, Trevor Darrell, and Zeynep Akata. Generating counterfactual explanations with natural language. arXiv preprint arXiv: 1xXiv:8806008800 2018.
Amir-Hossein Karimi, Gilles Barthe, Borja Balle, and Isabel Valera. Model-agnosta counterfactual
explanations for consequential decisions. In International Conference on Artificial Intelligence and Statistics, pages 895-905, 2020a.
Amir-Hossein Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic recourse: from counter- factual explanations to interventions, 2020b.
Amir-Hossein Karimi, Julius von Kügelgen, Bernhard Schölkopf, and Isabel Valera. Algorithmic recourse under imperfect causal knowledge: a probabilistic approach, 2020c.
Michael J Kearns, Umesh Virkumar Vazirani, and Umesh Vazirani. An introduction to computational learning theory. MIT press, 1994.
Janis Klaise, Arnaud Van Looveren, Giovanni Vacanti, and Alexandru Coca. Alibi: Algorithms for monitoring and explaining machine learning models, 2019. URL https://gi thtb//gttthuu..g com/ SeldonI0/alibi
Himabindu I. Lakkaraju and Cynthia Rudin. Learning cost-effective and interpretable treatment regimes. In Artificial Intelligence and Statistics, pages 166-175, 2017.
Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. Faithful and customizable explanations of black box models. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pages 131-138. ACM, 2019.
Himabindu Lakkaraju, Nino Arsov, and Osbert Bastani. Robust and Stable Black Box Explanations. Ieml, 2020a.
Himabindu Lakkaraju, Nino Arsov, and Osbert Bastani. Robust and stable black box explanations. In ICML, 2020b.

---

Arnaud Van Looveren and Janis Klaise. Interpretable counterfactual explanations guided by proto- types. 2019.
Yin Lou, Rich Caruana, and Johannes Gehrke. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 150-158, 2012.
Seyed-Mohsen Moosavi-Dezfoolli, Alhussein Fawzi, and Pascal Frossard. Deepfool: A simple and accurate method to fool deep neural networks. 20/6 IEEE Conference on Computer Vision and Pattem Recognition (CVPR), pages 2574-2582, 2016.
Ramaravind Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, pages 607-617, 2020.
Sangdon Park, Osbert Bastani, Nikolai Matni, and Insup Lee. Pac confidence sets for deep neural networks via calibrated prediction. In ICLR, 2020.
Rafael Poyiadzi, Kacper Sokol, Raul Ranios-Rodrrguez, Tijl De Bie, and Peter Flach. Face: Feasible and actionable counterfactual explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, AIES 200, page 344-350, New York. NY. USA, 2020. Association for Computing Machinery. ISBN 9781450371100.
doi: 00114553375662.33755550 URL
https://doi. org/ 10. 1145/ 3375627 3375850
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should trust you?": Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference Knowledge Discovery and Data Mining, San Francisco. CA. USA. August 13-17, 2016, pages 1135-11444 144, 2016a.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin, why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135-1144, 2016b.
Peter Schmidt and Ann D. Witte. Predicting recidivism in north carolina, 1978 and 1980. Inter- university Consortium for Political and Social Research, 1988. URL httph:h/tt/attt wwtp:://./wwoop.ooo.oooo hrffil/ss/Diggttttaaa//////11113
pdf
Uri Shaham, Yutaro Yamada, and Sahand Negahhan. Understanding adversarial training: Increasing local stability of supervised models through robust optimization. Neurocomputing, 307:195-204, 2018.
Dylan Slack, Sophie Hilgard, Emily Jia, Sameer Singh, and Himabindu Lakkaraju. Fooling lime and shap: Adversarial attacks on post hoc explanation methods. In Proceedings of the AAAI/ACM
Conference on AI. Ethics, and Society, AIES '20, page 180-186, New York, NY, USA, 2020. Association for Computing Machinery. ISBN 9781450371100 doi: 041144/3375627.3375555
URL httpss://dol. org/ 10. 0145/3375660 33755830
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL hRLphttss: wwwp://wwwwwwww..///p/p// awwwww............ org. orgg/anthooogg/D........ (Di-Loogyy//-1111
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. In ICIR, 2014.
Berk Ustun, Alexander Spangher, and Yang Liu. Actionable recourse in linear classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, pages 10-19, 2019.
Leslie G Valiant. A theory of the learnable. Communications of the ACM, 27(1)::1334 1142, 1984.
S. Wachter, Brent D. Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr. European Economics: Microeconomics & Industrial Organization eJournal, 2017.
12

---

Fulton Wang and Cynthia Rudin. Falling rule lists. In Artificial Intelligence and Statisties, pages 1013-1022, 2015.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Huggingface's 's transformers: State-of-the-at natural language processing, 2019.
Jiaming Zeng, Berk Ustun, and Cynthia Rudin. Interpretable classification models for recidivism prediction. Journal the Royal Statistical Society: Series A (Statistics in Society), 3(180))68--72. 2017.
Xin Zhang, Armando Solar-Lezama, and Rishabh Singh. Interpreting neural network judgments via minimal, stable, and symbolic corrections. Advances in Neural Information Processing Systems, pages 4874 4885, 2018.
Zhengli Zhao, Dheeru Dua, and Sameer Singh. Generating natural adversarial examples. In International Conference on Learning Representations (ICLR), 2018.
13


Reviews:
[{"review_id": "n_p_l7vytD", "reviewer": "Reviewer_tuPn", "summary": "The paper develops an adversarial training framework for training models such that they are likely to have recourse options (small changes to the input which lead to switching decisions from 0 to 1); and a method for providing a PAC-inspired guarantee that an algorithm has recourse options with high probability.", "questions": "", "limitations": "", "rating": 6, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I'm going to separate my review into general concerns about the setup, and then comments on the technical merits of the paper that assume that the general concerns are mitigated. The technical details of this paper are strong but they require that the setup makes sense (and that's where most of my concerns are) - so it's helpful to separate the two. \n\n**[Setup  concerns]** The key premise of this paper is that negative automated decisions require \"small\" actionable recourse with high probability. They give the example of an \"applicant denied a loan, a counterfactual explanation might be: “to get a loan, increase your income by $1,000”\", and want such recourse to occur with high probability. Given the massive impact that some of these systems have on applicants (bail decisions, recidivism, etc.), this is a desirable property (although it is not obvious that most recourses should be \"small\"): if we're going to use machine learning to assist in such consequential decisions, at the very least we should provide a recourse avenue to those who get an undesirable prediction; and recourse based on counterfactual inputs has become a popular approach in the explainable AI community. \n\nThat said, I find the idea of using adversarial-style training (proposed in this paper) to ensure a model has a way of providing recourse worrying, because it asks to ensure that counterfactual inputs for recourse exist under the model without addressing whether or not such inputs *should* lead to a positive decision in reality? Overall very little attention is paid to whether, $f_\\theta(x+\\delta) = f(x+\\delta)$  - i.e. whether the proposed recourse would be labeled as true under some true labelling process, $f$? This seems very problematic given that adversarial examples are usually used to highlight the arbitrariness of machine learning decisions when input data differs by small amounts from the training distribution. If the recourse suggestions have no grounding in reality---and the proposed adversarial training procedure seems likely to make this problem worse---then I'd worry that you're asking people to make arbitrary (and likely costly) changes to their lives in order to satisfy a model (e.g. it's good that it's possible to enforce constraints to ensure the procedure doesn't suggest decreases to income to qualify for a loan, but such constraints shouldn't be necessary if the proposed recourse set had any grounding in reality). \n\nThe results shown in the recourse correctness are give some defence against this critique, but I don't know that I share their conclusions - in Figure 3, $\\lambda = 0$ and  $\\lambda = 2.$ are not significantly different from each other, and generally there is a lot of overlap in the confidence intervals, so I'm not sure you can conclude that the training algorithm \"training algorithm encourages the existence of causally grounded recourses,\" especially from only a single dataset. \n\nThis concern is the primary reason for my vote to reject. That said, this isn't my area of research, so I'll happily increase my score if the rebuttal and / or other reviewers convince me this isn't such an issue, hereafter I assume that these concerns are not an issue when evaluating the technical contribution...\n\n**[Technical merit]** There are two parts to the approach - the adversarial training, and an approach to choosing the decision threshold such that we ensure a model \"Probably Approximately has REcourse\" (PARE; analogous to PAC guarantees). Both leverage existing techniques but were very well executed, and I particularly enjoyed the way that the paper uses PAC confidence sets [Park et al 2020] to ensure recourse with high probability. I could imagine such guarantees being required by regulators.\n\nThe experiments show that the method is works as intended and provides recourse options that are relatively robust.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See [setup concerns] above.\n", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}, {"review_id": "kPZgjkJd17H", "reviewer": "Reviewer_7Srb", "summary": "The paper proposes the idea of training machine models that are effectively recourse ready. The authors propose leveraging adversarial training and PAC confidence sets to train models that \"theoretically guarantee recourse\". The experiments with four datasets demonstrate increased recourse rates while sacrificing accuracy mildly. \n", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Originality: \nThe idea of a training recourse ready machine learning model is interesting. Looking for small perturbations that change the outcome has been the driving idea behind algorithmic recourse. However, thinking about a model that promotes recourse is unique.\n\nQuality: \nThe paper appears to be technically sound. I have a few reservations in regards to the evaluations. \n\nHow were the features (the two features) selected for evaluation? One of the problems with providing recourse is that mutability is subjective. How will the model behave when we consider more or different features? How do the training time and recourse rates change if we increase the number of features?\n\nWe need a sensitivity analysis around $\\delta$ as well. And could the authors discuss whether making a model recourse ready has any impact on diversity requirements?\n\nI am unable to understand why the model provides CR? At best Fig 3 shows that $\\lambda$ has effectively no net effect. And results show that CR works on one domain only and with limited features, which is not convincing.\n\nClarity: \nThe paper is well written and well organised. \n\nSignificance: \nThe idea is both interesting and relevant to the algorithmic recourse community. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "None discussed sufficiently.\n", "review_text": "", "extra_scores": {"ethical_concerns": "None", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5-6"}, "extra_text": {}}, {"review_id": "Wzad2-3mynK", "reviewer": "Reviewer_teEu", "summary": "The authors contribute to the field of algorithmic recourse by considering how to train models such that recourse options will exist with high probability with theoretical guarantees. To do so, the authors use a method which incorporates an approach from the study of adversarial training and PAC confidence sets. ", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1) Interesting research direction with potential real impact on the field of algorithmic recourse. \n2) Novel, as far as I can tell. \n3) Clearly written and motivated.\n\nWeaknesses:\n1) Since the paper uses a reversed version of a method that was originally introduced to reduce model brittleness, I'm not sure there was enough discussion of the effect of using such an approach on model robustness for which adversarial training was first suggested to begin with. While the authors do provide some empirical evidence in Table 3 that adding Gaussian noise did not make the models catastrophically fail. However, I think further discussion and perhaps additional results with more realistic noise additions/some examples of datasets from adversarial training literature could have made this point more convincing. \n2) I believe Wachter  et al. 2017, which is cited in this work, also included a discussion on the connection between adversarial training and counterfactuals/recourse. I'm not sure the authors acknowledged this enough (unless I missed it, happy to stand corrected).\n3) Presentation and experimental section can be a bit improved, see suggestions below.\n\n\nMinor comments:\n\n1) Line 55-6 vs 57-8 -- don't they say the same thing in different words? Could you not drop one of them?\n2) Line 63 -- it becomes clear later in the experimental section, but it might be helpful to readers to understand what you mean by \"robust recourses\". Robust how?\n3) Section 3 might benefit from a summary of the 3 steps of the method in an algorithm(s) block(s).\n4) Line ~153 -- might be good to quickly state $\\Delta$ is constructed based on use case and domain knowledge/user specified?.\n5) Line 175 -- \"$g_\\theta$ is approximately linear as a function of x\" -- is this supposedly falsifiable from data? Is this a strong assumption in certain cases? Might help to discuss a bit more.\n6) Line 197 -- \"uses the PAC confidence... constructs the new...\", should \"constructs\" be changed to \"to\"? Typo?\n7) Eq. 4 -- extra closing bracket?\n8) Table 1 -- Consider adding titles to the two bottom lines, it's a bit confusing as it stands.\n9) Line 215, 222 -- cite the adult income dataset? and bail outcomes?\n10) Fig 2. -- why are results from the German dataset not included for this analysis? + I might have missed this, but why would recourse neg be 0 *for any threshold* under $\\lambda=0$? And for all datasets? Is it just a matter of scale that makes it hard to see? I don't think based on other papers it make sense for recourse rates to be so bad?\n11) Lines 227-231 -- Is this just describing 3 fold cross-validation? If no, perhaps clarify the difference + what is the difference between the procedure for bail vs. adults/compas in this aspect?\n12) Line 254-5 -- why would this be the most useful measure of recourse? Why would we want to provide recourse to individuals who already received a positive outcome?\n13) Table 2 -- why does it seem to be organized as a transpose of table 1 instead of consistent with it?\n14) Fig. 3 -- can you describe why recourse neg shouldn't be more monotonic in $\\lambda$? Do you have an intuition?\n15) Use of the term \"significantly\" could be more cautious.\n16) Footnote 4 -- \"maximize 'recourse neg\" on the train set\" -- can you explain why train and not validation?\n17) Bibliography -- Line 370, 372 -- repeated citation?\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I believe the authors are aware of societal impact of their work, and proposed it to begin with to have a positive societal impact. I did not see much discussion of limitations, however.", "review_text": "", "extra_scores": {"ethical_concerns": "No specific ethical concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10"}, "extra_text": {}}, {"review_id": "92LqZlaveZv", "reviewer": "Reviewer_jpk7", "summary": "The author(s) proposed an adversarial training-based classification algorithm that guarantees affected individuals receive recourse with a high probability. The paper considers an interesting topic. The numerical experiments demonstrate the effectiveness of the proposed algorithm. ", "questions": "", "limitations": "", "rating": 6, "confidence": 1, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Pros**\n\n* The paper is written clearly and well-organised.\n\n* The topic is interesting. The author(s) considered classification algorithms that ensure adversarial examples do exist. \n\n**Cons**\n\n* It remains unclear to me why it is useful to consider algorithms with actionable recourses in practice. It would be helpful if the author(s) could discuss this in more detail.\n\n* The author(s) considered counterfactual explanations but did not adopt a causal framework for problem formulation. It would be better if a counterfactual outcome framework is used to present the problem, the methods and the results. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "I suggest the author(s) to discuss more about the motivations of learning classifiers with actionable recourses. In addition, the author(s) might want to adopt a counterfactual outcome framework to formulate their problems. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2h"}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper studies how to train models that offer good opportunity for recourse, quantifying what that might mean in terms of probabilities and providing algorithms with guarantees on these. The paper is an interesting and valuable contribution to the area of algorithmic recourse and was well reviewed by the referees. Many useful and detailed suggestions were made and the authors should incorporate these into a camera ready version.
Decision: Accept (Poster)