Title: INSPIRE: A Framework for Integrating Individual User Preferences in Recourse
Authors: Prateek Yadav
Peter Hase
Mohit Bansal
Venue: ICLR 2023

Abstract: Most recourse generation approaches optimize for indirect distance-based metrics like diversity, proximity, and sparsity, or a shared cost function across all users to generate recourse. The latter is an unrealistic assumption because users can have diverse feature preferences which they might be willing to act upon and any changes to any undesirable feature might lead to an impractical recourse. In this work, we propose a novel framework to incorporate the individuality of users in both recourse generation and evaluation procedure by focusing on the cost incurred by a user when opting for a recourse. To achieve this, we first propose an objective function, Expected Minimum Cost (EMC) that is based on two key ideas: (1) the user should be comfortable adopting at least one solution when presented with multiple options, and (2) we can approximately optimize for users' satisfaction even when their true cost functions (i.e., costs associated with feature changes) are unknown. EMC samples multiple plausible cost functions based on diverse feature preferences in the population and then finds a recourse set with one good solution for each category of user preferences. We optimize EMC with a novel discrete optimization algorithm, Cost-Optimized Local Search (COLS), that is guaranteed to improve the quality of the recourse set over iterations. Our evaluation framework computes the fraction of satisfied users by simulating each user's cost function and then computing the incurred cost for the provided recourse set. Experimental evaluation on popular real-world datasets demonstrates that our method satisfies up to 25.9% more users compared to strong baselines. Moreover, the human evaluation shows that our recourses are preferred more than twice as often as the strongest baseline.


Review discussion:
Strengths
Problem statement: The authors address a pertinent problem faced by the algorithmic recourse research community.
While prior approaches assume a cost function that is shared by all users, the authors formulate the problem to overcome this assumption, thus extending the research boundary.
Solution: The solution - EMC objective and COLS are quite intuitive, non-trivial, and simple to follow. The search procedure is also theoretically grounded.
Experiments: The authors conduct exhaustive experiments to validate the approach, including fairness analysis and ‘simulated’ human subject experiments resulting in strong evidence suggesting the approach’s efficacy.
Weakness
I have two major concerns with the proposed cost functions.
Firstly, I believe it is impractical if not unrealistic to expect users to express the cost function in the required form. Users might provide some sort of ordinal rankings of the attributes, but to expect cardinal utilities for feature changes is unrealistic. More evidence from the authors to support their definition of cost function will strengthen the paper.
Secondly, even though the authors claim on modeling unknown user cost function, it is strictly not unknown. There is a strong assumption-  the user cost function belongs to a larger family of functions. The structure of the cost function remains the same. Is there any evidence to suggest that the behavior of the cost functions are fundamentally different?
That is, the cost functions (lin and per) are uncorrelated on a given dataset?
While the experiments using distribution shifts(Q5) are perhaps in the right direction, evidence supporting the behavior of cost functions will further strengthen the observations from Figure 2 in the appendix.
Furthermore, the results presented in Figure 3 are incomplete. It would help to see the distribution shift spanning from [0, 1], instead of stopping at 0.3.
There are also concerns with the human subject experiments.
Firstly, asking annotators to pretend to be the end users of the system takes away the real-life changing situations under which customers might use the system. This has severe trust issues- A company is deploying a system for their clients based on hypothetical experiences of annotators. I recognize that designing and conducting an experiment involving real-users is a daunting task and also appreciate the authors’ effort in attempting to address this issue.
However, I worry that these experiments involving pretend annotators may inaccurately portray the acceptance and usability of such systems . It is important to perform human subject experiments to validate the method, but the experimental design needs more care and technical soundness.
Secondly, the experiment is basically asking the pretend annotators to check if the proposed recourse is following the feature scores. Considering that INSPIRE suggests recourses satisfying the feature scores, it is not surprising that the users favored INSPIRE over actionable recourses. I would rather argue that the acceptance rate of INSPIRE (57%) is quite low! More importantly, the experiment does not measure whether the recourse generated by INSPIRE is in accordance with the user’s cost function (not the vector $p_u$), which would require the user to provide a cost function!
I would also like to see some discussion on selecting $k$ in the feature satisfaction metric. How sound is it to assume a constant $k$ across the users? How does one decide on this value in the real world?
A minor concern is the comparison of INSPIRE against other actionable recourse approaches. One might view the comparison on cost metrics as unfair as the other methods are not designed to consider the user cost functions.
The paper is well organized and written.
However certain sections relegated to the appendix should be part of the main paper.
For example, the discussion on fairness that is not central to the contribution of the algorithm can be swapped with the experiments on distribution shifts between training and testing.
There are some typos that can be easily corrected.
For example - the subscript $t$ is missing in some of $S$ in algorithm 1.
NO.
**Response to weakness point 1:** Firstly, we agree with you that it is impractical for users to exactly specify their cost functions or feature preferences. We do not assume this and is exactly stated in our problem definition in section 2, in section 3.2, and in our introduction paragraph 4.
That is why in section 3.1, we specify a procedure to sample cost functions from our proposed distributions that are used to optimize for the EMC objective.
Second, incorporating individual user preferences in recourse is hard if no estimates of preferences are provided as inputs. As mentioned in section 3.1, paragraph 5, point 4, during the recourse generation if the user's feature preferences are provided then our method can incorporate them in the cost function sampling algorithm (Appendix Algorithm 4) to directly account for and optimize for them. In cases when no such preferences are provided, we work with the assumption of having a cost function distribution that users in a population follow. This is a reasonable and much weaker assumption than other prior works in this setting that assume a known and shared cost function across all the users, see section 5 last paragraph.
Moreover, during the simulated evaluation, each user has a different cost function according to their preferences which is not observed during the generation, as opposed to past works where a single shared cost function is used both during recourse generation and evaluation.
Our family of cost functions (aka our distribution) builds upon the well-established ideas of percentile and linear costs from the literature [1] which are fundamentally different because the linear costs increase linearly with change in a feature whereas percentile cost uses population feature statistics.
Moreover, different cost types do not need to be completely uncorrelated but need to align with how users might characterize cost.
As noted by you, our distribution shift experiments test potential scenarios when our assumptions might be violated and tests robustness with respect to them. The results from these experiments provide validity to our choices, if these are not enough we request you to be more specific about your concerns.
**Human Eval:** We would like to note that our experiment is an artificial experiment to simulate the process as we are not the company deploying the ML model that possesses the relevant user data and hence we cannot run real user evaluation studies. Our human study represents a novel human study in the right direction relative to past work where there is no human verification at all. In this study, we emulate the cases where there are trust compliance officers (human annotators in our study) in the organization that can ask users for their high-level preferences regarding features and assesses the generated counterfactuals before presenting them to the user to maintain user trust. In the study, we use three annotators to average out the variance across trust compliance officers to provide more robust results.
As mentioned in section 4.1 - evaluation details, during recourse generation, we use random cost function samples from the distribution to induce diversity in the solution set, these samples are different from the user’s cost function C_u^#, which is used for evaluation.
Hence, we do not optimize directly for the user cost function C_u^# or its corresponding feature preferences when generating the recourse, and our method is not supposed to have 100% preference over the baseline. A 57% preference compared to 25% for AR is a big improvement.
Lastly, our automatic simulated user evaluation which reports FS@k and other cost metrics does exactly what you mentioned about evaluating with respect to the user’s simulated cost function C_u^#. For the human study, we evaluate based on feature preference scores because as you stated users can maybe provide preference but it's hard for them to specify their cost functions in the current form.
We appreciate that you understand we cannot conduct a real user study for this application and we hope that our explanation of the design of the study makes sense to you. We will clarify this further in the main paper to avoid misleading interpretations and hope that this addresses your concern.
[1]  Actionable Recourse in Linear Classification, Ustun et al, FAT 2019
**Discussion on selecting k:** To compare different recourse methods' performance, the value of k just acts as a threshold to quantify user satisfaction. Usually, small changes in k do not affect the rank ordering of the methods. To illustrate this we also report the main table results for different values of k in Appendix Q13 and Table 11.
In practice, the value of k has to be estimated by doing some user survey where users are provided with multiple recourse options and asked if they are comfortable in adopting the recourse, given that the cost for the provided recourses is known to the deployer they can use this information to estimate an appropriate value of k.
**Potential Unfair Comparision:** During recourse generation, our method does not have access to the user’s cost function making it a fair comparison. Please refer to the previous answer about human eval performance.
Moreover, if cost-based evaluation is something we care about then all methods should do well on it, regardless of whether they try to optimize for it or not.
Finally, we would like to note that our work not only presents a recourse method but also the INSPIRE framework that could be used by the community to account for user preferences by further developing, better cost function distributions, objective functions, and optimization methods. INSPIRE provides a common ground to evaluate recourse methods while accounting for user preferences which further increases the research value and usability of our work to progress this important research direction. We believe that this would be useful for the community. We hope that our response clarified on your main concerns and if so please consider amending your score. Thank you so much for your effort and time!
Dear reviewer, this is a gentle reminder for our response above to answer your questions. We hope that we have addressed all of your questions, If there are any other questions or clarifications please let us know and we will answer those too asap. If you are satisfied with our responses then please consider updating your scores. Thanks again!
The paper is interesting, and it tries to tackle one of the challenges of algorithmic recourse. They outline a method that accounts for different cost functions and provides recourse sets from which the user can select.
However, from the problem statement and assumptions, the problem the authors try to solve does not match the claim made by the paper.
From Section 2 and Section 3, it seems that the assumption just shifted from having a shared cost function to a shared set of distributions describing all the possible user cost functions. D_{perc}, D_{lin} and D_{mix} need to be well-design to provide meaningful results. In a paper titled “integrating individual user preferences in recourse,” I would expect to see a way to integrate a single user’s preferences to provide recourse while the user is still not considered in the estimation process.
The authors do define preferred features (FP) as a means to incorporate real individualised preferences (see Section 3.1).
However, they assume that the user can provide them directly (which is generally not true), and they do not elaborate on them much further. I think providing a way to estimate them would be an important contribution instead.
In Section 3.4, the authors argue that obtaining user feedback to assess satisfaction is challenging. This statement is not true since there is extensive literature [1] about estimating user preferences via several types of interaction, which has been applied in many fields, such as recommender systems.
Minor:
Section 2 does not specify exactly how the cf is converted into actionable feedback for the user. Does the user select the less costly cf from the recourse set only? or is this cf presented in a “sequential” way to the user (e.g., first, get a degree, second, get a better job, etc.)?
[1] Viappiani, Paolo, Boi Faltings, and Pearl Pu. "Preference-based search using example-critiquing with suggestions." Journal of artificial intelligence Research 27 (2006): 465-503.
The paper tries to address a relevant problem in the area of algorithmic recourse. It is also well-written and very detailed. The reviewer did not check the source code nor tried to run the experiments, but it seems the work is detailed enough to be reproducible by other researchers.
NO.
**Integrating user preference:** We partially agree with this point and would like to note that incorporating individual user preferences in recourse is hard if no estimates of it are provided as inputs from the user. As mentioned in section 3.1, paragraph 5, point 4, during the recourse generation if the user's feature preferences are provided then our method can incorporate them in the cost function sampling algorithm (Appendix Algorithm 4) to directly account for and optimize for them. In cases when no such preferences are provided, we work with the assumption of having a cost function distribution that users in a population follow. As mentioned in Section 5, last paragraph, this is a reasonable and much weaker assumption than other prior works in this setting that assume a known and shared cost function across all the users.
Moreover, during the simulated evaluation, each user has a different cost function according to their preferences, as opposed to past works where a single shared cost function is used both during recourse generation and evaluation.
Furthermore, we would like to note that our work is not just providing a recourse method but we are providing a new framework to generate and evaluate recourse while accounting for user preference.
Hence, our formulation is much more realistic and challenging and incorporates individual users' preferences either directly if provided as input or relaxes the assumptions of past works to a great extent.
**Collecting user preference:** In our works, we can work in the case when no preferences are provided by sampling cost functions with random feature preferences. In cases where users can provide some sort of preferences, it is possible to collect binary preference judgments over features from users and then fit a preference model like BTL used in [1] to obtain preference scores. There are different existing techniques for this that could be adopted, and we consider them outside the scope of our as our aim is to present a novel recourse method and not reimplement methods for preference inference from data. As mentioned in Section 3.1, paragraph 4, lines 5-9, for the purposes of sampling distributions, we do not assume these feature preferences to be known and choose to randomly sample them, which allows us to model different feature preferences users’ in a population might possess. See Appendix Algorithm 4 for more details on the sampling.
**User feedback to assess satisfaction:** There is a bit of a misunderstanding here. We mean that we cannot collect real recourse preferences of users who are exposed to ML decisions in the wild [2] as we are not the company deploying the ML model. There is no risk in eliciting user preferences in an artificial study, so we have to simulate this environment to the best of our ability in our computational experiments and human study.
**Converting CF to actionable feedback:** The user is provided with all the cfs in the recourse set and we assume that a rational user will select the least costly option according to them.
[1] Beyond individualized recourse: Interpretable and interactive summaries of actionable recourses, NeurIPS 2020
[2] Formalizing Trust in Artificial Intelligence: Prerequisites, Causes, and Goals of Human Trust in AI, Faact 2021
Dear reviewer, this is a gentle reminder for our response above to answer your questions. We hope that we have addressed all of your questions, If there are any other questions or clarifications please let us know and we will answer those too asap. If you are satisfied with our responses then please consider updating your scores. Thanks again!
Strengths :
The problem statement was well specified formally, which was extremely helpful in understanding the work (but mostly because of the absence of a running example).
The method evaluates both on the devised metrics as well as existing metrics to compare with baselines.
COLS seems to be a simple genetic algorithm like method which I can see could be used in further research in this direction.
Weaknesses :
I would have really appreciated it if the work had a running example to explain “recourse”, the role of the EMC objective, diverse cost function, COLS and so on. Readability of the paper was severely hampered without it.
Several works in other literature like database systems and web mining have used measures like coverage & at least some seminal work should be cited.
The method relies on a good cost function generation method. Were there any studies conducted to support that the method would be able to cover human preferences? [With real humans in the loop?]
Clarification :
Why is monotonicity a useful property for COLS? (so long the desired candidate is part of the recourse set?) Infact, I would argue hill-climbing becomes monotonic if I’m maintaining the best found so far, however its still not sound, so I’m not sure why Theorem 3.1 is presented.
How is the feasibility of the generated data point ensured?
Would you agree that Algorithm 1 seems like some variant of a class of genetic algorithms [If so this should be pointed out]?
Doesn’t this line of works that estimate user cost matrix raise questions about user-identifiability? [Where maybe through such intricate cost matrices these AI systems can uniquely identify users without their knowledge?]
Originality / Novelty / Reproducibility :
I find that through text and supplementary material, the work should be reproducible.
NO.
**Running Example:** A standard use case of recourse methods is when a user applies for credit. In this case, the decision is made by a Black Box Model (BBM) based on the features of the user. In cases, when the credit application is denied, the user can request actions (recourse) they can take to get their credit approved. The recourse method generates multiple counterfactuals for the user. Consider these two counterfactual examples provided to the user – (1) “increasing your income by 10000 dollars and reducing your debt by 5000 dollars would result in your credit approval”, and (2) “increasing your income by 5000 dollars and reducing your debt by 10000 dollars would result in your credit approval”. These provided counterfactuals can be of varying levels of difficulty for different users depending on their preferences (which are encoded by their true cost function).
Hence, we can use a cost function to obtain the cost incurred by the user (under their true cost function) when adopting the counterfactual. If we do not know the user’s true cost function which captures their feature preferences, then we sample multiple cost functions from a distribution. Each of these sampled cost functions captures different feature preferences this user might possess but we are not certain if any of these preferences are close to the user's true preference. The EMC objective measures the average cost of a recourse set with respect to all of these sampled cost functions that capture different feature preferences.
It tells how would the recourse set perform on average if it was provided to multiple users with different feature preferences but similar initial features. COLS is used iteratively to refine the recourse set to minimize the EMC objective to generate a recourse set. If this was helpful then we can add this to the main paper.
**Citing Database literature for coverage:** Thanks for pointing this out. Are you referring to metrics like Average Effective Throughput [1]? If not can you please point us to some references and we will add them?
**Goddess of Cost Functions:** Our percentile and linear transition costs are based on the ideas from popular past works like Actionable Recourse [2] and hence are well grounded in the literature to relate to human preferences.
Moreover, as specified in section 3.1 paragraph 4 and Appendix Algorithm 4, the cost function samples are generated by randomly sampling multiple feature preferences, and hence by design, it accounts for all the possible feature preferences different users can have. Given this, we believe it is not necessary to have humans in the loop when designing the cost functions as we follow well-grounded past work but we agree that such a human-in-the-loop thing would add more user trust to the method.
Moreover, given that we are not the company deploying these models, it is impossible for us to keep real users in the loop.
**Monotonicity of COLS:** First, we note that we maintain the best set as opposed to the best counterfactual.
Second, monotonicity emerges as a consequence of maintaining the best set in COLS and is presented to demonstrate the performance cannot decrease over iteration which helps in convergence.
**Feasibility of generated datapoint:** During the recourse generation, we can query the black-box (BB) model to get the BB model’s outcome for the perturbed data point and store only the data points with the positive class in the best set. Note that given infinite access to the BB model, we can potentially query all possible feature states and hence we restrict the forward pass budget for all the methods as specified in section 4.1 baselines.
**Relation to Genetic Algorithms:** Yes, we agree that Algorithm 1 can be categorized as a genetic algorithm [3], we will mention this and cite related works in the paper.
Moreover, user-identifiability is out of the scope of this work because when users apply they provide us with their data in order to classify them.
Hence, the user’s identity is already known and we do not add to any privacy concerns beyond what user classification already poses.
Furthermore, as mentioned in Section 1 paragraph 4, we are sampling multiple cost functions that might be relevant for the user but are not directly associated with them.
Hence, we are not directly trying to assign or estimate the user’s cost function which preserves their identity.
[1] Metrics for Performance Evaluation of Distributed Application Execution in Ubiquitous Computing Environments
[2] Actionable Recourse in Linear Classification, FAT 2019
[3] Genetic Algorithms in Search, Optimization and Machine Learning, Goldberg, David E., 1989.
Dear reviewer, this is a gentle reminder for our response above to answer your questions. We hope that we have addressed all of your questions, If there are any other questions or clarifications please let us know and we will answer those too asap. If you are satisfied with our responses then please consider updating your scores. Thanks again!
I have to say in advance that I did not understand all parts of the paper fully, therefore my feedback should be taken with a grain of salt. Which parts I did not comprehend I will describe shortly.
Strengths
The idea of the paper and its constitutent parts are discussed in a clear manner. Technical explanations are used sensefully, the algorithm is helpfully added in pseudocode, most unfamiliar terms are introduced. It also appears to be concise regarding the brevity and thus aims to convey the core information on as little space as possible. Visualizations and tables are used to help the reader, as well as comprehensive supplementary material. Reproduction of results is encouraged by providing the source code, an ethics statement further makes some important statements about the ethical nature of the work.
Proposals for modification
A stronger explanation of the motivation and conrete use case would be very welcome, complemented by a handful of examples to help the reader picture it. The purpose, concept and feasibility of deploying this method in practical terms could be described in a discussion section, which presently is really very short.
Further, terms like "user satisfaction" and "fairness" are very charged concepts that tend to be multi-dimensional, in this work however they appear to have been mathematically formalized without discussion of the validity of such an approach. A brief exploration of the conceptual meanings attached to these terms and reflection on how they are made operational in this context would be helpful.
Lastly, the user experiments, if I understood correctly, were conducted with only three users, who were additionally close to the research team -- a rather low number to draw general conclusions from, while also inviting biases of different sorts to eschew study results. This limitation should be mentioned in the text and user studies should perhaps be conducted with a broader, more diverse sample.
Points that are unclear to me
I did not fully understand the problem that the presented work is solving. It appears that in some situations users are provided with an algorithmic classification result that they would like to change (=recourse), and the way in which this change is executed is detailed in this work. To do this, counterfactuals to the regular result are provided to perhaps better meet the users' expectations, until the user agrees with a classification result. Perhaps my background is limiting my capacity to picture the use case, but I can't imagine in which kind of situation this interaction between user and system takes place. Taking the recidivism example, would the user be the judge, the defendant, the attorney, or even the data scientist? In which realistic case would the user be presented with the data features, and how should they know what they would mean? I am missing a rather large explanation of this part of the problem.
Similarly, I did not understand how the user studies in particular took place. It appears that some individuals were presented with the system, performed tasks and gave feedback, but in which setting and with how many users is not clear to me. In the research fields I know, this is vital information. I realize that some of the informationis provided in the supplement, but I would argue this is relevant in the main text as well.
Further, I do not understand why the reserach questions show up so late in the outline together with the related work. Intuitively, I would have expected the research questions to come earlier and be answered after the experiment description.
Also, the related work section provides information that would be relevant for comprehension of the text before.
Finally, the relevance of proximity, sparsity and feature diversity for example in relation to the optimization eluded me, as they were called "objectives", but before there was discussion of the "objectives" of actionability and feasibility -- where the definitions also appear to overlap.
In short, I could not make out the connection between the presented objectives and the problem formulation.
I cannot readily judge quality, novelty and reproducibility, for this I know too little of the work connected to this paper.
NO.
**Definition User Satisfaction:** As mentioned in Section 3.4 - Proposed Metrics, we say that a user is satisfied if the cost of acting on recourse for them is below a threshold k. This is true because below a cost threshold k all counterfactuals are relatively easy to adopt, i.e., actionable and feasible.
Moreover, we use the same threshold across all methods which allows for a fair relative comparison of methods. In spirit, this is similar to defining metrics like Precision@k, and top-k Accuracy. The metric FS@k is a concrete way to quantify user satisfaction. We also report results for other values of k in Appendix A2 Q13 and Table 11.
**Contextual Definition Fairness:** Fairness in this context means that the recourse method should be able to provide valid recourse and satisfy a similar fraction of users from both subpopulations. For a more detailed discussion on fairness, its importance, and sources of bias, please read our ethics statement and Q3 first paragraph.
**Concrete Use Case and User Study:** A standard use case is when users apply for credit. In this case, there are a large number of applications, and the decision is made by a Black Box Model (BBM) based on the features of the user. In this case, the user is the person applying for credit. In cases, when the credit application is denied, the user can request actions (recourse) they can take to get their credit approved. The recourse method generates multiple counterfactuals for the user. Consider these two counterfactual examples provided to the user – (1) “increasing your income by 10000 dollars and reducing your debt by 5000 dollars would result in your credit approval”, and (2) “increasing your income by 5000 dollars and reducing your debt by 10000 dollars would result in your credit approval”.
Hence, the user is not required to understand the features or the complete counterfactuals as they are conveyed in a simple manner to the user. The provided counterfactuals to the user can be of varying levels of difficulty for them depending on their preference.
Hence, ideally, we would like to provide them with counterfactuals based on their ease of changing different features. Perhaps the user can quickly reduce debt as opposed to increasing income, making reducing debt a more preferred feature.
Finally, the user can either adopt a solution provided to them or they can reject the fairness/validity/acceptability of the classifier [1]. If you find this helpful, we can add this to the main paper.
We would like to note that our experiment is a study to simulate the process as we are not the company deploying the ML model that possesses the relevant user data and hence we cannot run real-world scale user evaluation studies. Our human study represents a novel human study in the right direction relative to past work where there is no human verification at all. In this study, we emulate the cases where there is a trust compliance officer (human annotators in our study) in the organization that can ask users for their high-level preferences regarding features like income, debt, education, changing jobs, etc and assess the generated counterfactuals before presenting them to the user to maintain user trust. In the study, we use three annotators to average out the variance across trust compliance officers to provide more robust results. We will add more details about this in the paper.
**Objectives of Recourse generation and the relevance of proximity, sparsity and feature diversity:** The main “objective” of the recourse method is to generate actionable and feasible recourse for the user. Diversity, proximity, and sparsity are the “objective functions” used by past works to generate actionable and feasible recourse. We will clarify this further in the paper and move the related work section after the introduction.
[1] The philosophical basis of algorithmic recourse, Venkatasubramanian, et, al, FAT20
Dear reviewer, this is a gentle reminder for our response above to answer your questions. We hope that we have addressed all of your questions, If there are any other questions or clarifications please let us know and we will answer those too asap. If you are satisfied with our responses then please consider updating your scores. Thanks again!
My background (deep learning, NLP, AI alignment) is quite different from the background of this paper, and thus my review should be taken with a huge grain of salt (I have more questions than judgments).
Strengths:
+ The paper is written very clearly.
+ There are many details included in the paper that allow for ease of reproducibility (including hyperparameter details, information in the appendix, a reproducibility statement, etc.)
+ The problem tackled by the paper seems important. Incorporating individual user preferences into recourse generation seems like a good idea. The results are quite strong, including on their human evaluations.
+ The paper performs an ablation of COLS and EMC, finding that each piece is important (though particularly EMC). In general the set of experiments performed in the paper is very thorough (see also Appendix A.2).
+ I really like the formatting of Section 4.2 -- it lays out a sequence of research questions along with the findings as supported by the evidence in the paper. In my opinion this is exactly how papers should be written.
Weaknesses:
- As an outsider to the field, the paper could have done a slightly better job explaining how the datasets + evaluation methodologies used in the paper (Adult-Income and COMPAS) relate to recourse generation 'in the wild'.
In particular, it's not obvious to me that recourses involving changing the space of user features is the most natural way for useful recourse to happen.
For example, in the Adult-Income dataset it feels like some recourses that could be generated involve changing the user's age or marital status.
In practice would this mean that the user would have to go get married to change the decision made by the algorithm? That seems quite unrealistic (let alone changing age from 34 to 84, as seen in Figure 8). What seems more natural are interventions that lay outside of the technical domain (eg having a human review your application rather than a model). Perhaps this should be more naturally viewed as a kind of interpretability, for explaining the models' decisions (as alluded to in Section 1).
However, it seems like this would be most naturally done using interpretable ML methods like decision trees to make the judgments in the first place.
Overall, I find myself confused about how these datasets apply to real-world recourse generation and thus, am unclear about the impact of the proposed method.
- Similarly, diving into the human evaluation instructions in Figure 7 in the Appendix, I feel like I would have a hard time making a decision about whether or not a recourse is reasonable, or whether one recourse is better than another.
For instance, if a user has a preference_score > 0 for age, and the recourse suggest changing their age, is that reasonable? It seems like the main thing to do is a manual investigation to ensure that the features being changed correspond to positive preference_scores, but it feels like this could be done automatically -- it's not clear to me how much human evaluation is adding (as someone who is usually a huge fan of human evaluation).
Clarity, quality, and reproducibility are all very high. It's difficult for me to judge novelty but it also seems sufficient.
Questions / comments:
- Not a contribution of this paper, but I'm a bit confused about about some of the properties of the  Adult-Income dataset. What is the difference between marital status and relationship (which can only be husband or non-husband)?
- Figure 2 in the Appendix is basically unreadable.
NO.
**Response to Weakness - 1:** First, we would like to point out that these are standard datasets used in the recourse generation literature[1,2].
Second, we agree with your assessment that ideally deploying interpretable ML models (like decision trees) would simplify the problem of recourse generation.
In practice, black-box models (BBMs) are deployed because they perform much better than their interpretable counterparts.
Moreover, in many real-world scenarios, the number of applications is high and it is not feasible to have human reviewers for all applications before making a decision.
Hence, these models are used mostly as a filtering process or in some cases to guide a decision-maker.
For example, filtering is done for credit card applications based on either provided or predicted income of the applicant/user.
Hence, there is close real-world applicability for adult-income datasets.
Similarly, the COMPASS dataset is used to predict the probability of recidivism by a defendant which is then used to grant bail.
Hence, for defendants who are denied bail, providing recourse to getting bail is helpful.
Lastly, as mentioned in Section 2 each feature has a type, the features can be mutable, conditionally mutable, or immutable. There might be specific users who consider marital status to be editable as they are already about to get married.
Hence, in this work, we do not assign types like immutable, or conditionally immutable to features based on our intuition as we lack authority on this and do not have access to the user population. We delegate these feature-type choices to the deployer of the method based on a population study or input from the users. Note that, this does not undermine any of the findings of our methods because the algorithm could be run again after imposing different feature types if needed.
**Response to Weakness - 2:** We believe human evaluation is valuable because it helps us to account for the subjectivity of incorporating different factors to decide if recourse is reasonable. A preference score > 0 implies changing this feature is allowed but this does not automatically make the recourse reasonable. Reasonable means that if a user is provided with the recourse they would be happy to adopt it, which is influenced by several factors like – (1) how strongly edited features are preferred, 2) how much change is required in features, 3) are more preferred features changed more than less preferred features, 4) how many features are changed, etc.
Hence, we used three annotators to average out the effect of how they weigh these different factors to assess reasonability.
**Adult Income Dataset Features:** As mentioned in Section 4.1 datasets, the features were binarized for the main experiments as the most relevant baseline works only with binary categorical features. This results in the features being similar. For results on the non-binarized version please refer to appendix Appendix A2 Q11 and Table 8.
**Appendix Figure 2:** The almost uniform heatmap demonstrates that the method is robust to distribution shifts along the axis of percentile and linear costs. We will clarify this further in the appendix.
[1] CARLA: A Python Library to Benchmark Algorithmic Recourse and Counterfactual Explanation Algorithms, NeurIPS 2021
[2] Beyond Individualized Recourse: Interpretable and Interactive Summaries of Actionable Recourses, NeurIPS 2020
Dear reviewer, this is a gentle reminder for our response above to answer your questions. We hope that we have addressed all of your questions, If there are any other questions or clarifications please let us know and we will answer those too asap. Thanks again!

Meta-review:
role: PC
data: metareview:_summary,_strengths_and_weaknesses: This paper considers the problem of recourse generation when user preferences are unknown and the preferences can vary between users. To this end, the authors develop a method to propose a diverse set of counterfactuals based on a novel objective function and assumptions about the possible cost functions of the users. They develop an optimization algorithm (similar to genetic algorithms) to efficiently compute the set of counterfactuals. Furthermore, they evaluate their approach on synthetic and semi-real-world datasets and observe good performance in comparison to several baselines.

Strengths:
* The paper is mainly well written and easy to follow.
* The considered setting is clearly more realistic as compared to existing work.
* The proposed approach is sensible and intuitive.
* Experimental results indicate improvements over existing work.

Weaknesses:
* Strong assumption about individual user cost-functions, i.e., availability of a distribution over possible functions; and while I partly agree with the authors that this is more general than what has been done, it would have been interesting to understand more deeply which cost functions are prevalent in reality. In any case, this aspect should be clarified, also in the title of the paper.
* The user study is quite limited and might be biased; 3 annotators close to the research team were used.
* Recourse generation for the user study is unclear. I have read the reviewer's concern in that regard and also the authors response and checked the paper again - I think the question of whether feature preferences are used for recourse generation is still not clarified (the paper leaves this open as far as I can tell, the authors' response is not clear in that regard: "Hence, we do not optimize directly for the user cost function C_u^# or its corresponding feature preferences when generating the recourse." - the statement before does not tell about the usage of feature preferences). 
* Novelty wrt to other subpopulation based approaches is limited.
* Human evaluation is rather artificial.

The reviewers gave mixed scores for this paper and the authors did their best to clarify questions and concerns. I have also considered the authors' concern regarding one of the reviews but think this is based on a misunderstanding of what the reviewer wanted to say (cost-function versus feature preferences; unclarity about usage of feature preferences as also explained above). Unfortunately there was little active discussion between reviewers and authors. In our internal discussion we reached the consensus that although the paper is addressing an interesting problem, we are recommending rejection of the paper. The paper could be improved by developing the investigations regarding distribution shifts in more detail, improving the human evaluation, and discussing the relation to other subpopulation based approaches in detail.
justification_for_why_not_higher_score: The weaknesses (mainly assumptions about cost function and novelty) are too substantial for the reviewers to warrant acceptance.
justification_for_why_not_lower_score: n/a

Decision: Reject