Towards Robust and Reliable Algorithmic Recourse

1 Introduction . Over the past decade , machine learning ( ML ) models are increasingly being deployed to make a variety of highly consequential decisions ranging from bail and hiring decisions to loan approvals . Consequently , there is growing emphasis on designing tools and techniques which can provide recourse to individuals who have been adversely impacted by predicted outcomes [ 30 ] . For example , when an individual is denied a loan by a predictive model deployed by a bank , they should be provided with reasons for this decision , and also informed about what can be done to reverse it . When providing a recourse to an affected individual , it is absolutely critical to ensure that the corresponding decision making entity ( e.g. , bank ) is able to honor that recourse and approve any re-application that fully implements the recommendations outlined in the prescribed recourse Wachter et al . [ 31 ] . Several approaches in recent literature tackled the problem of providing recourses by generating local ( instance level ) counterfactual explanations 2 [ 31 , 26 , 12 , 21 , 18 ] . For instance , Wachter et al . [ 31 ] proposed a gradient based approach which finds the closest modification ( counterfactual ) that can result in the desired prediction . Ustun et al . [ 26 ] proposed an efficient integer programming based approach to obtain actionable recourses in the context of linear classifiers . There has also been some recent research that sheds light on the spuriousness of the recourses generated by counterfactual/contrastive explanation techniques [ 31 , 26 ] and advocates for causal approaches [ 3 , 14 , 15 ] . All the aforementioned approaches generate recourses under the assumption that the underlying predictive models do not change . This assumption , however , may not hold in practice . Real world ∗Equal contribution 2Note that counterfactual explanations [ 31 ] , contrastive explanations [ 13 ] , and recourse [ 26 ] are used interchangeably in prior literature . Counterfactual/contrastive explanations serve as a means to provide recourse to individuals with unfavorable algorithmic decisions . We use these terms interchangeably to refer to the notion introduced and defined by Wachter et al . [ 31 ] 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . settings are typically rife with different kinds of distribution shifts ( e.g , temporal shifts ) [ 22 ] . In order to ensure that the deployed models are accurate despite such shifts , these models are periodically retrained and updated . Such model updates , however , pose severe challenges to the validity of recourses because previously prescribed recourses ( generated by existing algorithms ) may no longer be valid once the model is updated . Recent work by Rawal et al . [ 24 ] has , in fact , demonstrated empirically that recourses generated by state-of-the-algorithms are readily invalidated in the face of model shifts resulting from different kinds of dataset shifts ( e.g. , temporal , geospatial , and data correction shifts ) . Their work underscores the importance of generating recourses that are robust to changes in models i.e. , model shifts , particularly those resulting from dataset shifts . However , none of the existing approaches address this problem . In this work , we propose a novel algorithmic framework , RObust Algorithmic Recourse ( ROAR ) for generating instance level recourses ( counterfactual explanations ) that are robust to changes in the underlying predictive model . To the best of our knowledge , this work makes the first attempt at generating recourses that are robust to model shifts . To this end , we propose a novel minimax objective that can be used to construct robust actionable recourses while minimizing the recourse costs . Second , we propose a set of model shifts that captures our intuition about the kinds of changes in the models to which recourses should be robust . Next , we outline an algorithm inspired by adversarial training to optimize the proposed objective . We also carry out theoretical analysis to establish the following results : i ) we quantify the probability of invalidation for recourses generated without accounting for model shifts , and ii ) we derive an upper bound on the relative increase in the costs incurred due to robust recourses ( proposed by our framework ) to the costs incurred by recourses generated from existing algorithms . Our theoretical results further establish the need for approaches like ours that generate actionable recourses that are robust to model shifts . We evaluated our approach ROAR on real world data from financial lending and education domains , focusing on model shifts induced by the following kinds of distribution shifts – data correction shift , temporal shift , and geospatial shift . We also experimented with synthetic data to analyze how the degree of data distribution shifts and consequent model shifts affect the robustness and validity of the recourses output by our framework as well as the baselines . Our results demonstrate that the recourses constructed using our framework , ROAR , are substantially more robust ( 67 – 100 % ) to changes in the underlying predictive models compared to those generated using state-of-the-art recourse finding technqiues . We also find that our framework achieves such a high degree of robustness without sacrificing the validity of the recourses w.r.t . the original predictive model or substantially increasing the costs associated with realizing the recourses . 2 Related Work . Our work lies at the intersection of algorithmic recourse and adversarial robustness . Below , we discuss related work pertaining to each of these topics . Algorithmic recourse As discussed in Section 1 , several approaches have been proposed to construct algorithmic recourse for predictive models [ 31 , 26 , 12 , 21 , 18 , 3 , 14 , 15 , 7 ] . These approaches can be broadly characterized along the following dimensions [ 29 ] : the level of access they require to the underlying predictive model ( black box vs. gradients ) , if and how they enforce sparsity ( only a small number of features should be changed ) in counterfactuals , if counterfactuals are required to lie on the data manifold or not , if underlying causal relationships should be accounted for when generating counterfactuals or not , whether the output should be multiple diverse counterfactuals or just a single counterfactual . While the aforementioned approaches have focused on generating instance level counterfactuals , there has also been some recent work on generating global summaries of model recourses which can be leveraged to audit ML methods [ 23 ] . More recently , Rawal et al . [ 24 ] demonstrated that recourses generated by state-of-the-art algorithms are readily invalidated due to model shifts resulting from different kinds of dataset shifts . They argued that model updation is very common place in the real world , and it is important to ensure that recourses provided to affected individuals are robust to such updates . Similar arguments have been echoed in several other recent works [ 28 , 13 , 20 ] . While there has been some recent work that explores the construction of other kinds of explanations ( feature attribution and rule based explanations ) that are robust to dataset shifts [ 16 ] , our work makes the first attempt at tackling the problem of constructing recourses that are robust to model shifts . Adversarial Robustness The techniques that we leverage in this work are inspired by the adversarial robustness literature . Wachter et al . were the first to remark on similarities between counterfactual generation and adversarial attacks , but did not leverage this connection to develop robust recourse [ 31 ] . It is now well established that ML models are vulnerable to adversarial attacks [ 10 , 4 , 2 ] . The adversarial training procedure was recently proposed as a defense against such attacks [ 19 , 1 , 32 ] . This procedure optimizes a minimax objective that captures the worst-case loss over a given set of perturbations to the input data . At a high level , it is based on gradient descent ; at each gradient step , it solves an optimization problem to find the worst-case perturbation , and then computes the gradient at this perturbation . In contrast , our training procedure optimizes a minimax objective that captures the worst-case over a given set of model perturbations ( thereby simulating model shift ) and generates recourses that are valid under the corresponding model shifts . This training procedure is novel and possibly of independent interest . 3 Our Framework : RObust Algorithmic Recourse . In this section , we detail our framework , RObust Algorithmic Recourse ( ROAR ) . First , we introduce some notation and discuss preliminary details about the algorithmic recourse problem setting . We then introduce our objective function , and discuss how to operationalize and optimize it efficiently . 3.1 Preliminaries . Let us assume we are given a predictive modelM : X →Y , where X ⊆ Rd is the feature space , and Y is the space of outcomes . Let Y = { 0 , 1 } where 0 and 1 denote an unfavorable outcome ( e.g. , loan denied ) and a favorable outcome ( e.g. , loan approved ) respectively . Let x ∈ X be an instance which received a negative outcome i.e. , M ( x ) = 0 . The goal here is to find a recourse for this instance x i.e. , to determine a set of changes that can be made to x in order to reverse the negative outcome . The problem of finding a recourse for x involves finding a counterfactual x′ = x+ for which the black box outputs a positive outcome i.e. , M ( x′ ) =M ( x+ ) = 1 . There are , however , a few important considerations when finding the counterfactual x′ = x+ . First , it is desirable to minimize the cost ( or effort ) required to change x to x′ . To formalize this , let us consider a cost function c : X × X →R+ . c ( x , x′ ) denotes the cost ( or effort ) incurred in changing an instance x to x′ . In practice , some of the commonly used cost functions are ` 1 or ` 2 distance [ 31 ] , log-percentile shift [ 26 ] , and costs learned from pairwise feature comparisons input by end users [ 23 ] . Furthermore , since recommendations to change features such as gender or race would be unactionable , it is important to restrict the search for counterfactuals in such a way that only actionable changes are allowed . Let A denote the set of plausible or actionable counterfactuals . Putting it all together , the problem of finding a recourse for instance x for whichM ( x ) = 0 can be formalized as : x′ = arg min x′∈A c ( x , x′ ) s.t M ( x′ ) = 1 ( 1 ) Eqn . 1 captures the generic formulation leveraged by several of the state-of-the-art recourse finding algorithms . Typically , most approaches optimize the unconstrained and differentiable relaxation of Eqn . 1 which is given below : x′ = arg min x′∈A ` ( M ( x′ ) , 1 ) + λ c ( x , x′ ) ( 2 ) where ` : Y × Y→R+ denotes a differentiable loss function ( e.g. , binary cross entropy ) which ensures that gap betweenM ( x′ ) and favorable outcome 1 is minimized , and λ > 0 is a trade-off parameter .

Summary: This paper deals with the robustness of algorithmic recourse options given to users by common methods in the field, as models get updated over time. In particular, unlike previous works, this one proposes a solution that aspires to withstand distributional shifts in training data over time, space, correction to measurements, etc. The proposed solution, ROAR, is inspired by adversarial training. The work also introduces a lower bound on probability of invalidation of recourse options of models trained normally, as well as upper bound the added cost the output of ROAR might incur. Finally, the authors compare the performance of ROAR, as well as to a causal version of ROAR (combined with Karimi et al.'s MINT) to those of existing popular methods, and offer a sensitivity analysis, demonstrating on a synthetic dataset the effect of distributional shifts on validity of recourse options.