Learning Recourse on Instance Environment to Enhance Prediction Accuracy
Abstract
Machine Learning models are often susceptible to poor performance on instancessampled from bad environments.
For example, an image classifier could providelow accuracy on images captured under low lighting conditions.
In high stake
ML applications, such as AI-driven medical diagnostics, a better option could beto provide recourse in the form of alternative environment settings in which torecapture the instance for more reliable diagnostics.
In this paper, we proposea model called R ECOURSE N ET that learns to apply recourse on the space ofenvironments so that the recoursed instances are amenable to better predictions bythe classifier.
Learning to output optimal recourse is challenging because we donot assume access to the underlying physical process that generates the recoursedinstances.
Also, the optimal setting could be instance-dependent — for examplethe best camera angle for object recognition could be a function of the object’sshape.
We propose a novel three-level training method that (a) Learns a classifierthat is optimized for high performance under recourse, (b) Learns a recoursepredictor when the training data may contain only limited instances under goodenvironment settings, and (c) Triggers recourse selectively only when recourseis likely to improve classifier confidence.
We experiment with synthetic and realworld datasets to show the efficacy of our proposed approach.
Introduction
The performance of any supervised learning model depends strongly on the quality of input instances.
However, in practice, instances may be of suboptimal quality when generated in adverse environmentsettings.
For example, even an expressive image classification model may misclassify an image shotat an extreme close-up or at a wrong angle or under poor lighting [13, 28].
Despite large training sizes,such unfavorable instances can deteriorate model performance which can have serious consequencesin high stake scenarios like AI guided crop monitoring [21], automatic disease diagnosis fromimages [22] and AI driven accessibility enhancement for the hearing impaired.
Mitigating the effect of such unfavorable instances entails the design of recourse mechanism torecommend alternative environment settings that yield instances revealing the target class.
Forexample, in low cost smartphone based medical diagnosis [22] where imaging is performed bynon-experts, such recourse mechanisms can interactively recommend camera settings that yieldimages optimal for the downstream diagnosis model.
The optimal camera settings could be labeldependent.
For example, the best camera angle for recognizing an aeroplane could be different fromthe angle for recognizing poles.
More formally, the problem we seek to address in this paper is as follows.
We have an object z in thephysical space (e.g. a crop) with an unknown true label y (e.g. type of disease).
Let β ∈ B be theenvironment setting under which we capture a digital representation x of z to diagnose the label from an downstream classifier f θ ( x ) .
Our goal during recourse is to recommend an alternative setting β ′(if any) to the user for getting a different representation x ′ of z where f θ ( x ′ ) is more likely to becorrect than f θ ( x ) .
The above problem can be thought of as an instance of algorithmic recourse, on which there has beenmuch work in recent years [30, 29, 23, 8, 32, 11].
These methods recommend recourse actions onthe instance space x , which is difficult to realize on raw data for objects such as images and speech.
Instead we propose to intervene at the level of the environment which generates the instance viaan unknown physical process.
We view the contributions of our present work under three facets asexplained below:(i)
Novel framework for recourse mechanism.
We propose R ECOURSE N ET , a trainable recoursemechanism which recommends modified actions to the end user so that, if acted upon the environment,it can generate instances with improved accuracy.
R ECOURSE N ET consists of three components: (1)a classifier f θ , (2) a recourse trigger π (3) a recourse recommender network g ϕ .
Given an instance( x , β ) , the recourse trigger π first decides whether to recommend recourse for x .
If so, the recourserecommender g ϕ suggests an alternative environment β ′ .
Using these, the user generates a newinstance x ′ , on which f θ would give the correct label with potentially higher confidence.(ii)
Three level training proposal.
The main challenge of R ECOURSE N ET is that we do not assumeaccess to the latent physical process Z that generates an x ′ given a β ′ during training.
Instead wetrain with a fixed labeled dataset containing (latent) objects z i rendered as instances { x ij } undera small but variable set B i of observed settings { β ij } .
We show that direct end-to-end training ofa combined likelihood training settles on easy local minima, and fails to provide good recourse.
Training them stage wise also is challenging; we list some of these.
For f θ , training on the entiredataset may be suboptimal since instances in poor settings, where recourse will be asked, may misleaddecisions on good instances.
For g ϕ , we have no direct supervision of good β for a given ( x ij , β ij ) .
For π , simple heuristics like choosing to recourse examples where f θ has low confidence does notguarantee improved accuracy.
Our training strategy employs careful scheduling and decoupling ofthe training of the three modules via proxy functions.
This achieves substantial gains over simpleend-to-end training and existing methods of training classifiers with data selection based purely onnoise [20, 2, 5, 12, 14, 17, 24, 31].(iii) Characterization of recourse conditions.
We provide theoretical characterizations to identifythe circumstances under which recourse will enhance prediction accuracy.
Specifically, we showthat given an instance x , if the recourse recommender suggests a modified environment that is closeto at least one of the training environments resulting in an improved accuracy, then the recourse isbeneficial.
Moreover, if there exists some environment which improves the accuracy by a substantialmargin, then even a modestly calibrated recourse recommender can lead to improved accuracy.
Related work
Our work is closely related to (i) Algorithmic recourse, (ii) Learning with triage and (iii) Machinelearning with environment perturbation.
Algorithmic recourse :
In recent years, there is an increasing interest in designing recourse on theinstance space [30, 29, 23, 8, 32, 11, 25, 9] for a wide variety of applications.
For example [30, 29]aim to improve fairness; [15, 7, 10] aim to train the models so that the predicted output is preservedunder strategic perturbation of the instance space.
Another line of work called strategic classification[7, 15, 10] deals with applying causal interventions to instances.
However, these work learn therecourse action on the instance space, whereas, our goal is to design recourse action on the observedenvironment.
An additional challenge in our setting is that the impact of the environment on theinstance is latent and we do not assume presence of enough labeled data to learn a generative modelfor complex real-world instances under different environments.
Learning with triage: A recent line of work [20, 2, 5, 12, 14, 17, 24, 31] aims to learn when tooutsource a subset of instances to human and assign the rest of the examples to machine so thatmachine and human together achieve superior performance than what they would have achievedindependently.
However, in our problem, humans do not participate in prediction task but they onlygenerate new instances under the recommended environments.
Machine learning under environment perturbations: Machine learning models are sensitive toenvironments under which data is generated [13, 28].
For example [13], show that simple parametricperturbations on the Shapenet dataset can flip class labels.
In another related work [18] suggestsinterventions on the environment using policy gradients to train a recourse model.
However, theyassume the availability of a human through out the training loop to generate data in an on-demandbasis.
We make no such assumptions and train with a fixed labeled dataset.
Proposed approach
In this section, we first formally present our problem, present our training methodology, and thentheoretically characterize the settings under which recourse is possible.
Problem formulation Let Z denote a space of objects, B denote a space of environment settingswhich could be real-valued or discrete or mixed, and X denote a space of instances obtained via alatent physical process Z : Z × B → X .
Given a latent object z ∈ Z and an environment settingβ ∈ B , we get an instance x ∈ X =
R d x i.e. , x = Z ( z, β ) .
Each object z has a label y ∈ Y
with|Y|
= K .
We are interested in inferring the object’s label using a trained classifier f θ .
During training,for each of the latent set of objects { z i } i ∈ D , we are given a true label y i and for a small set of settings
B i ⊂ B , we are given instance { x ij } j ∈ B i .
Thus, we view the training data as a set of examples{ y i , { x ij , β ij } j ∈ B i } i ∈
D
We use V to index all the examples, i.e. , V = ∪ i ∈ D {{ i } × B i } .
As statedearlier, our goal is to design a recourse mechanism that given a representation x obtained of a latentobject z under given settings β will recommend an alternative β ′ if the resultant x ′ =
Z ( z, β ′ ) isexpected to yield more accurate prediction under f θ .
Note that Z is not accessible to us duringtraining
and we assume in this work that it is difficult to learn Z or infer z from the available labeleddata D .
Our goal instead is to use D to learn both f θ and the recourse mechanism.
3.1 Training R ECOURSE N ET
R ECOURSE N ET consists of three components:
A classifier f θ : X × Y → [0 , 1] which aims to capture the likelihood of the label y given aninstance x ,
i
e. f θ ( y | x ) approximates Pr( y | x ) .
A recourse recommender network g ϕ : X × B × B → [0 , 1] , that suggests a modified environment
β ′ ∼ g ϕ
( • | x , β ) such that if the user (via Z ) were to regenerate a new instance x
′ using β ′ the
classifier is likely to provide higher accuracy.
A recourse trigger network π : X × B → { 0 , 1 } which is a binary decision function.
Here,π ( x , β ) = 1 indicates that we decide to perform recourse on the environment and the β ′ suggested
by g ϕ should be used to regenerate the instance.
Training objective.
Given a set of examples with { y i , { x ij , β ij } j ∈ B i } i ∈ D , we aim to find θ, ϕ andπ by solving the following optimization problem:
[Equation]
subject to,
[Equation]
≤ b
[Equation]
and π
[Equation]
ij
[Equation]
ij
[Equation]
Here, b indicates the maximum number of examples which can undergo recourse.
The first termin the objective (1) (1 − π ( • , • )) log f θ ( • | • ) accounts for examples that do not need recourse andthe second term π ( • , • )log f θ ( • | • ) accounts for those that need recourse.
End to end training ofthe optimization problem (1)— (2) is challenging since we do not have an analytical form of Z andtraining such a process will be difficult.
We propose to train the three components f θ , g ϕ , π in acarefully designed three-stage process that we describe next.
Training the classifier f θ .
Training f θ on the entire training data may be sub-optimal becauseinstances in poor settings would be subject to recourse, and the classifier should instead focuson instances after recourse as the above training objective suggests.
For training f
θ first we eschew the involvement of Z and g ϕ from the training objective (1) by noting that π ( x ij , β ij ) =
1only if f θ ( y i | Z ( z i , argmax β g ϕ ( β | x ij , β ij )))
≥ f θ ( y i | x ij ) .
Therefore, we replace the term
Z ( z i , argmax β g ϕ ( β | x ij , β ij )) with some instance ( x ir , β ir ) for some r ∈ B
i of the same objectz i such that the predicted classification accuracy on x ir is better than the original instance x ij bya certain margin ∆ .
Specifically, given ( x ij , β ij ) , we first define Rec ∆ ( θ, x ij , y i ) as the set ofenvironments which would improve the log-likelihood of the gold label by at least a margin ∆ i.e. ,
[Equation]
and then we pose the following training problem to learn θ .
[Equation]
subject to,
[Equation]
Since our budget is limited, one needs to spend it on only those instances which not only suffer frompoor accuracy, but can also lead to new instances that promote f θ to predict the correct label.
Thepresence of a non-zero margin ∆ ensures such a condition.
In Section 3.2, we provide the conditionsunder which such a recourse set will exist.
Given π ( x ij , β ij ) ∈ { 0 , 1 } , we first define the set R = { ( i, j ) | π ( x ij , β ij ) = 1 } .
Then, we canwrite the objective (4) as
[Equation]
which gives us the problem of subset selection in conjunction with parameter estimation.
Note thatthe involvement of R as an optimization variable renders the above problem challenging even iflog f θ ( y | x ) is concave in θ .
Thus, we resort to a greedy algorithm [19, 5, 16, 33] to solve thisoptimization problem (summarized in Algorithm 1).
It is an iterative routine, which picks up aninstance ( x ij , β ij , y i ) at every iteration which will maximize the training objective.
Given an update Rat step k ≤ b , it chooses a candidate instance ( i, j ) which maximizes F ( θ k
( R ∪{ ( i, j ) } ) , R ∪{ ( i, j ) } ) ,where θ ( S ) = max θ F ( θ, S ) .
We would like to highlight that, by definition of the set Rec ∆ , inclusionof ( i, j ) in R either improves the log-likelihood or keeps it at the same value obtained in the previousiteration.
Formally, we can say that F ( θ k +
( R ∪ { ( i, j ) } ) , R ∪ { ( i, j ) } ) ≥ F ( θ k ( R ) , R ) .
Learning g ϕ .
Our overall objective (1) is not differentiable with respect to ϕ because of theargmax β g ϕ ( • ) input to f θ and the unknown Z .
We first get rid of the argmax term via the following
Require: The set of all instances
V = ∪
i ∈ D
( i, B i ) , the budget b 1: R ← ∅ 2: θ 0 ( ∅ ) ← T RAIN ( F ( • , ∅ )) 3: for k ∈ [ b ] do 4: for ( i, j ) ∈
V \ R do 5:
L [( i, j )] =
F ( θ k
( R ∪ { ( i, j ) } ) , R ∪ { ( i, j ) } ) 6: end for 7: ( i ∗ , j ∗ ) ← argmax ( i,
j ) ∈
V \ R L [( i, j )] 8:
R ← R ∪ { ( i ∗ , j ∗ ) } 9: θ k +
( R ) ← T RAIN ( F ( • , R ))
10: end for 11: Return R, θ ∗ ( R )
Algorithm 2: R ECOURSE N ET
Require: The set of all instances
V = ∪
i ∈ D
( i, B i ) , the budget b , δ 1: I NITIALIZE ( f θ , g ϕ ) 2: ˆ θ,
R ← G REEDY
A LGORITHM ( V, b, f θ ) 3: ˆ ϕ ←
T RAIN R ECOURSE R ECOMMENDER ( V, D δ , g ϕ ) //
Solve (8) 4: π ←
C OMPUTE R ECOURSE T RIGGER (ˆ ϕ, ˆ θ, V ) //
Use Eq.
Return ˆ θ, ˆ ϕ, π approximation:
[Equation]
[Equation]
Next we account for the unknown Z by partitioning all examples in D into two groups — the set D δwhich contains groups with at least one instance where good β s are available ( i.e. max r f θ ( y i | x ir ) >1 − δ ), and the remaining objects D − D δ where no good instances are available.
For the instancesin D δ , we can jointly train θ and ϕ .
For the ones in D − D
δ we replace the loss achievable by agiven β by an estimated accuracy f CF ( y i | x ij , β ) of getting the correct prediction when the settingβ ij is replaced by β for the i -th object.
We estimate this quantity as the average classifier accuracyon objects with similar labels and under settings β .
In general, for continuous y, β this can be fitas a regression problem.
For discrete y, β , simple fractional estimates were found adequate in ourexperiments.
We compute these estimates by defining the following counterfactual:
[Equation]
where I [ • ] is an indicator function and ˆ θ is the output of Algorithm 1.
With these two terms, wemaximize the following objective:
[Equation]
Computation of π .
Our training objective (1) suggests that π ( x ij , β ij )
= 1 only if f θ ( y i | x ij ) <
f θ ( y i | x ′ ij =
Z ( z i , β ′ ij ))
where β ′ ij = argmax β g ˆ ϕ
( β | x ij , β ij )) .
Since the recourse budget islimited, we cannot obtain x ′ ij for all instances to compute π .
Therefore, in practice, we usef CF ( •| x ij , β ′ ij ) as a proxy for f θ ( • | x ′ ij =
Z ( z i , β ′ ij )) .
Specifically, we setπ ( x ij , β ij ) =
I [ f CF ( y max | x ij , β ′ ij ) > f (cid:98) θ ( y max | x ij )] where y max = argmax y
f ˆ θ ( y | x ij ) (9)
We call our overall training method as R ECOURSE N ET , which is summarized in Algorithm 2.
Dataset
[Figure]
[Table]
[Figure]
[Table]
[Figure]
[Table]
Synthetic
[Table]
Shapenet-Large [Table] light color)
[Table]
Shapenet-
[Table]
[Table]
light color)
[Table]
Speech Commands
[Table]
[Table]
[Table]
, illumination, contrast)
3.2 Theoretical Analysis
In this section we present the conditions on θ, ϕ, π under which R ECOURSE N ET will be successful inproviding recourse.
The proofs of the propositions are given in Appendix B.
Proposition 1 Assume that Z is L β -Lipschitz with respect to β , the model log f θ ( y | x ) is L xLipschitz with respect to x .
Given i ∈ D and j ∈ B i , if the set Rec ∆ ( θ, x ij , y i ) is non-empty and therecourse network g ϕ gives a modified β ′ ij such that || β ′ ij − β || ≤ ϵ for some β ∈ Rec ∆ ( θ, x ij , y i ) ,then, for ∆ > tL x
L β ϵ with t > 1 we have:
[Equation]
[Equation]
The above proposition suggests that as long as g ϕ ( • | x ij , β ij ) is close to some β ∈ Rec ∆ ( θ, x ij , y i ) ,then the accuracy provided by the classifier f θ improves.
One of the key assumption of this propositionis the non-emptiness of Rec ∆ ( θ, x ij , y i ) .
In the following proposition, we find the requirements forsuch conditions in terms of the true classifier f θ ∗ .
Proposition 2 Let us assume that the true conditional distribution of y given x is f θ ∗ , log f θ ( y | x ) is L θ -Lipschitz w.r.t.
θ and ||
θ − θ ∗ || ≤ δ .
Given i ∈ D and j ∈ B i , if Rec ∆ 0 ( θ ∗ , x ij , y i ) is non-emptyfor some ∆ 0 > 2 L θ δ , then Rec ∆ ( θ, x ij , y i ) is non-empty for ∆ <
∆ 0 − 2 L θ δ .
Moreover, if therecourse network g ϕ gives us a modified β ′ ij such that || β ′ ij − β || ≤ ϵ for some β ∈ Rec ∆ ( θ, x ij , y i ) ,then, for ∆ 0 > 2 L θ δ + tL β L x ϵ with t > 1 we have:
[Equation]
Experiments
In this section, we experiment with several datasets to show that R ECOURSE N ET ’s training strategyoutperforms existing methods or simpler alternatives.
Our experiments are designed to answer thefollowing research questions through empirical evaluations:
In the training of f θ , what is the impact of subsetting the training set when compared with defaultalternatives like training on all available labeled data.2.
In deciding when to trigger recourse, how important is to model expected accuracy under recourse,in contrast to just asking recourse on low confidence examples?3.
In training the recourse recommender, how important was it distinguish between objects withand without good β s?
During inference, how important is it to make instance specific recourse recommendations instead of a single ideal beta?
We could not find any existing benchmark that records different environment settings under whichobjects are rendered.
Thus we generate datasets that admit causal relationship across x , β , z and y asfollows: we first sample a class label from the class prior y ∼ Pr( • ) and then we choose B i settingsby sampling β s drawn from a Pr( β | y ) .
Finally we generate x under the B i chosen environments.
We generate 4 datasets of varying complexities as shown in the Table 1. [Table]
Shapenet-Large.
Shapenet consists of three dimensional models of many kinds of objects that canbe mapped into two dimensional pixel maps under various environments [3].
Each environmentβ represents the camera settings provided by (view, zoom level, light color) .
We select|Y|
= 10 classes and draw 250 objects from each class to obtain a total of | D | = 2500 objects.
Foreach object, we draw B i =
4 different β s from a set of |B| = 9 possible camera settings and renderthem under these settings.
Among the four environments, we ensure that each z i contains a β thatrenders it properly with a probability 0 . 8 .
To make the task challenging, we corrupt the rendered x ijusing various kinds of noise from the image corruptions library 1 .
In particular, we corrupt x ij if β ijis not a good choice for z i so as to make learning of such settings difficult for f θ .
Shapenet-Small : This dataset differs from Shapenet-Large in the number of environments underwhich each object is rendered.
Among the two environments, each z i contains a good β withprobability 0 .
This dataset is more challenging than Shapenet-Large because of scarcity in thenumber of objects that contain atleast one x ij that produces good accuracy.
This makes the objective(6) difficult to learn.
Here also we add noise to x ij in a manner similar to Shapenet-Large.
The testset for both Shapenet-Large and Shapenet-Small is same, and contains 80 objects per class; each ofthem rendered under all 9 camera settings β thus contributing to 7200 images.
Speech Commands Dataset.
This dataset consists of textual commands that can be converted tospeech under different environments β defined by (pitch, speed, noise) sampled from B with|B| = 60 .
We select |Y| = 20 commonly used Alexa commands and render them to speech signalswith a frame width of 0 .
5 seconds using Google text to speech library 2 .
These speech signals arethen processed into 2 D mel spectrograms [27].
In particular, the training dataset consists of 2000z i rendered under | B i | = 5 environments each thereby contributing to 10000 samples.
The test setcontains 200 z i s rendered under all 60 β s thereby containing 12000 speech samples.
Further details about dataset preparation and results on synthetic datasets are provided in Appendix C.
4.1 RQ1: Impact of subsetting the training data in learning f θ
We compare our iterative greedy proposal to train f θ with two other baselines as follows:
Full data: Here we train f θ over the entire training dataset.2.
One-shot subsetting:
Here we subset all b examples at once unlike our iterative algorithm 1.
i.e.we compute L ( i, j ) for all samples given θ 0 ( ∅ ) and choose the ones that incur top- b values into Rand then maximize F ( • , R ) to obtain ˆ θ .
Table 2 shows the recourse accuracy of f θ at 100% recourse when learned under these three differenttraining strategies.
For all three methods we use g ϕ trained using our objective (8) to obtain recourserecommendations.
We observe that our iterative greedy algorithm to train f θ consistently outperformsthe model trained with the entire data.
This establishes the importance of training classifiers differentlywhen recourse is an option.
A classifier that is trained only on instances with ’good’ environmentsettings is more suitable for classification under recourse, even in data hungry deep learning models.
Simply subsetting by removing the worst b instances is significantly worse than our iterative algorithm.
4.2 RQ2: Impact of modelling expected recourse accuracy in triggering recourse
We compare our proposal for f θ and π with two other baselines adapted from the work of [24].1 https://github.com/
[Figure]
Score based recourse trigger: Here, we train f θ on entire training data.
Then during inference,given a budget b , we seek recourse on the least b confident predictions of f θ .2.
Full automation based recourse trigger: Here also, we train f θ on entire training data.
Thenfor recourse trigger, we learn an error predictor trained on the loss incurred by the classifier ontraining examples.
During inference, for a budget b , we seek recourse on those examples that
incur the b highest predicted losses
Details about the neural architecture of the error predictor is
provided in Appendix E
Figure 2 summarizes the comparison of recourse trigger π against the two baselines proposed inliterature [24].
Unlike our greedy algorithm, the other two methods propose full training for f θ andthus they are inferior at 0% recourse.
The steepness in the recourse accuracy for our proposed π ismore in comparison to other baselines because it prioritizes not just the instances that suffer frompoor accuracy for recourse but also the ones that respond better to recourse by means of modellingthe expected recourse accuracy.
Our method suggests recourse only when the expected gains thatwe calculate using f CF is positive, and performs much better than methods based purely on currentclassifier confidence or an estimate of the confidence.
4.3 RQ3: Impact of treating objects with and without good renderings differently inlearning g ϕ
We compare our g ϕ against three other methods as follows.
Only ϕ : This model takes a form similar to g ϕ and learns to recourse the instances ( i, j ) thatincur top 50% losses in the training data to β ir where r is obtained from argmax r f θ ( y i | x ir ) .
R ECOURSE N ET without f CF :
This model trains g ϕ using the objective (6).3.
Constant: This method entails a constant β recommendation independent of the features ( x , β ) .We
select constant β as the one that achieves the best training accuracy.
To elucidate the importance of finding the objects that have no good β and thereby including them inthe set D − D δ in the g ϕ objective (8), we perform an ablation study of our proposal with three otherbaselines as listed above.
The results presented in Figure 3 shows the following observations.
Only ϕ model performs poorly on Shapenet-Large and performs on par with Constant method onother datasets.
Because many groups do not have β that produce good accuracy, Only ϕ receives noisysupervision during training.
(2) R ECOURSE N ET without f CF achieves a decent fit on Shapenet-Largeand Specch datasets but fails miserably on the Shapenet-Small dataset.
Because Shapenet-Small has| B i | = 2 , we can see that 50% examples force the recourse recommender to predict the input β asis under the joint objective (6).
This renders identity function as a strong local maxima which themodel struggles to avoid during training.
This brittleness of R ECOURSE N ET without f CF to objectswith no good β motivates the need for our current objective 8.
The supervision provided by thef CF term in our g ϕ objective (8) guides instances in the set D − D δ and thus achieves better recourseaccuracy.
(4) One good competitor to our g ϕ across the datasets is constant prediction which brings
[Figure] [Figure]
us to the other half of RQ3 – Is an instance independent constant β recourse recommendation alwaysadvisable?
We try to answer this question by probing the average error incurred by these methods for each class.
Figure 4 summarizes the results for 5 classes which shows that unlike baselines, our g ϕ garnersmodest to best accuracy across classes consistently.
The performance of constant method in the
Shapanet datasets can be attributed to the fact that many objects in it admit a unique good β .
However,this is not the case in the speech dataset because we found no one β to dominate in performanceacross classes.
As a result, the recourse accuracy suffers in the speech dataset with constant βprediction.
Thus we conclude by saying that it is always good to make instance specific recourserecommendations.
Conclusions
In this paper, we proposed R ECOURSE N ET that aims to make recourse recommendations to instancesthat are sampled from poor environments.
R ECOURSE N ET has three components: (1) classifier f θ ,(2) Recourse recommender g ϕ
and (3) Recourse trigger π .
We learn these components using a novelthree level training objective without having to model the latent physical generator Z .
Moreover, ourtheoretical results assure that under mild conditions, recourse is beneficial.
These results in effect,press the need for recourse in order to obtain quality predictions from a model.
The experiments onsynthetic and real-world datasets show that our method outperforms several baselines.
Our work opens up many areas of future work.
It would be interesting to extend R ECOURSE N ETto regimes where the space of environment variable β s can be continuous.
In medical applications,some environment variables may be restrictive and can cause harm to the patients.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope
(c) Did you discuss any potential negative societal impacts of your work?
[N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[Yes] See Section 3.2(b)
Did you include complete proofs of all theoretical results?
[Yes] Included in theAppendix.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] Code isincluded in the Supplementary Material and datasets are provided in an anonymizedURL.
(b) Did you specify all the training details (e g., data splits, hyperparameters, how they were chosen)?
[Yes] See Section 4 and Appendix
(c) Did you report error bars (e g., with respect to the random seed after running experiments multiple times)?
[Yes] See Section
(d) Did you include the total amount of compute and the type of resources used (e g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Appendix4.
If you are using existing assets (e g., code, data, models) or curating/releasing new assets...
(a)
If your work uses existing assets, did you cite the creators?
[Yes](b) Did you mention the license of the assets?
[N/A]
(c) Did you include any new assets either in the supplemental material or as a URL?
[Yes]Datasets are included in an anonymized URL
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[Yes]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount [Table] spent on participant compensation?
[N/A]