Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper investigates the problem of recourse generation -- generating plans that allow a user to change the decision of a model to some other option. The paper proposes to incorporate the individuality of user preferences into both training and evaluation. This involves both a new objective function, Expected Minimum Cost (EMC), and a new discrete optimization algorithm (COLS). They report improved performance (in terms of % of satisfied simulated users and actual human evaluations) over baselines across 2 datasets: Adult-Income (predicting whether someone has an income >$50k based on a set of features) and COMPAS (predicting whether defendants will commit another crime within 2 years). 
strength_and_weaknesses: My background (deep learning, NLP, AI alignment) is quite different from the background of this paper, and thus my review should be taken with a huge grain of salt (I have more questions than judgments). 

Strengths: 
+ The paper is written very clearly. 
+ There are many details included in the paper that allow for ease of reproducibility (including hyperparameter details, information in the appendix, a reproducibility statement, etc.)
+ The problem tackled by the paper seems important. Incorporating individual user preferences into recourse generation seems like a good idea. The results are quite strong, including on their human evaluations.
+ The paper performs an ablation of COLS and EMC, finding that each piece is important (though particularly EMC). In general the set of experiments performed in the paper is very thorough (see also Appendix A.2).
+ I really like the formatting of Section 4.2 -- it lays out a sequence of research questions along with the findings as supported by the evidence in the paper. In my opinion this is exactly how papers should be written.

Weaknesses:
- As an outsider to the field, the paper could have done a slightly better job explaining how the datasets + evaluation methodologies used in the paper (Adult-Income and COMPAS) relate to recourse generation 'in the wild'. In particular, it's not obvious to me that recourses involving changing the space of user features is the most natural way for useful recourse to happen. For example, in the Adult-Income dataset it feels like some recourses that could be generated involve changing the user's age or marital status. In practice would this mean that the user would have to go get married to change the decision made by the algorithm? That seems quite unrealistic (let alone changing age from 34 to 84, as seen in Figure 8). What seems more natural are interventions that lay outside of the technical domain (eg having a human review your application rather than a model). Perhaps this should be more naturally viewed as a kind of interpretability, for explaining the models' decisions (as alluded to in Section 1). However, it seems like this would be most naturally done using interpretable ML methods like decision trees to make the judgments in the first place. Overall, I find myself confused about how these datasets apply to real-world recourse generation and thus, am unclear about the impact of the proposed method.
- Similarly, diving into the human evaluation instructions in Figure 7 in the Appendix, I feel like I would have a hard time making a decision about whether or not a recourse is reasonable, or whether one recourse is better than another. For instance, if a user has a preference_score > 0 for age, and the recourse suggest changing their age, is that reasonable? It seems like the main thing to do is a manual investigation to ensure that the features being changed correspond to positive preference_scores, but it feels like this could be done automatically -- it's not clear to me how much human evaluation is adding (as someone who is usually a huge fan of human evaluation).



clarity,_quality,_novelty_and_reproducibility: Clarity, quality, and reproducibility are all very high. It's difficult for me to judge novelty but it also seems sufficient.


Questions / comments:
- Not a contribution of this paper, but I'm a bit confused about about some of the properties of the  Adult-Income dataset. What is the difference between marital status and relationship (which can only be husband or non-husband)? 

- Figure 2 in the Appendix is basically unreadable.
summary_of_the_review: Overall, this paper seems to make a strong contribution to the field of recourse generation. It is clearly written, the proposed method seems valuable, and improves significantly upon baselines. I remain quite uncertain in my review given that I have little background in this field, and I remain confused about the applicability of the datasets used to real-world recourse generation, but nonetheless I tentatively recommend acceptance.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper introduces a new framework to create recourse to algorithms with higher user satisfaction than comparable approaches, using Expected Minimum Risk and a novel optimization algorithm. Recourse here is meant to enable users to make changes in algorithmic classification, altering the result according to their preferences. Other recourse methods reportedly did not take into account individual preferences but optimized for several general metrics, such as proximity and diversity. Here this is achieved by drawing from several distributions of counterfactuals which are spread out so that the likelihood of increasing user satisfaction is higher. The proposed method is reportedly fairer, more performant and leads to higher user satisfaction.
strength_and_weaknesses: I have to say in advance that I did not understand all parts of the paper fully, therefore my feedback should be taken with a grain of salt. Which parts I did not comprehend I will describe shortly.

Strengths

The idea of the paper and its constitutent parts are discussed in a clear manner. Technical explanations are used sensefully, the algorithm is helpfully added in pseudocode, most unfamiliar terms are introduced. It also appears to be concise regarding the brevity and thus aims to convey the core information on as little space as possible. Visualizations and tables are used to help the reader, as well as comprehensive supplementary material. Reproduction of results is encouraged by providing the source code, an ethics statement further makes some important statements about the ethical nature of the work.

Proposals for modification

A stronger explanation of the motivation and conrete use case would be very welcome, complemented by a handful of examples to help the reader picture it. The purpose, concept and feasibility of deploying this method in practical terms could be described in a discussion section, which presently is really very short.
Further, terms like "user satisfaction" and "fairness" are very charged concepts that tend to be multi-dimensional, in this work however they appear to have been mathematically formalized without discussion of the validity of such an approach. A brief exploration of the conceptual meanings attached to these terms and reflection on how they are made operational in this context would be helpful. Lastly, the user experiments, if I understood correctly, were conducted with only three users, who were additionally close to the research team -- a rather low number to draw general conclusions from, while also inviting biases of different sorts to eschew study results. This limitation should be mentioned in the text and user studies should perhaps be conducted with a broader, more diverse sample. 

clarity,_quality,_novelty_and_reproducibility: Points that are unclear to me

I did not fully understand the problem that the presented work is solving. It appears that in some situations users are provided with an algorithmic classification result that they would like to change (=recourse), and the way in which this change is executed is detailed in this work. To do this, counterfactuals to the regular result are provided to perhaps better meet the users' expectations, until the user agrees with a classification result. Perhaps my background is limiting my capacity to picture the use case, but I can't imagine in which kind of situation this interaction between user and system takes place. Taking the recidivism example, would the user be the judge, the defendant, the attorney, or even the data scientist? In which realistic case would the user be presented with the data features, and how should they know what they would mean? I am missing a rather large explanation of this part of the problem.
Similarly, I did not understand how the user studies in particular took place. It appears that some individuals were presented with the system, performed tasks and gave feedback, but in which setting and with how many users is not clear to me. In the research fields I know, this is vital information. I realize that some of the informationis provided in the supplement, but I would argue this is relevant in the main text as well.

Further, I do not understand why the reserach questions show up so late in the outline together with the related work. Intuitively, I would have expected the research questions to come earlier and be answered after the experiment description. Also, the related work section provides information that would be relevant for comprehension of the text before.
Finally, the relevance of proximity, sparsity and feature diversity for example in relation to the optimization eluded me, as they were called "objectives", but before there was discussion of the "objectives" of actionability and feasibility -- where the definitions also appear to overlap. In short, I could not make out the connection between the presented objectives and the problem formulation. 

I cannot readily judge quality, novelty and reproducibility, for this I know too little of the work connected to this paper.  
summary_of_the_review: Personally I had issues of understanding, which could also result from my lack of familiarity with the topic. I think that the paper can be rendered more comprehensible and stronger in terms of motivation and context with a few changes. Elaborating on the use case of the presented method and providing examples can go a long way, restructuring could also help. In terms of the technical execution and mathematical formulations the work appears sound to me. However, better defining the user, their satisfaction and their perceived fairness is important, as well as discussing limitations of the approach.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The work proposes an algorithm, Cost-Optimized Local Search (COLS) to optimize the objective Expected Minimum Cost (EMC) [proxy to the original cost matrix of a user] to come up with a recourse set (actionable and feasible requests) personalized to the preferences of a user.
strength_and_weaknesses: Strengths : 
The problem statement was well specified formally, which was extremely helpful in understanding the work (but mostly because of the absence of a running example).

The method evaluates both on the devised metrics as well as existing metrics to compare with baselines. 

COLS seems to be a simple genetic algorithm like method which I can see could be used in further research in this direction.

Weaknesses : 
I would have really appreciated it if the work had a running example to explain “recourse”, the role of the EMC objective, diverse cost function, COLS and so on. Readability of the paper was severely hampered without it. 

Several works in other literature like database systems and web mining have used measures like coverage & at least some seminal work should be cited.

The method relies on a good cost function generation method. Were there any studies conducted to support that the method would be able to cover human preferences? [With real humans in the loop?]

clarity,_quality,_novelty_and_reproducibility: Clarification : 

Why is monotonicity a useful property for COLS? (so long the desired candidate is part of the recourse set?) Infact, I would argue hill-climbing becomes monotonic if I’m maintaining the best found so far, however its still not sound, so I’m not sure why Theorem 3.1 is presented.

How is the feasibility of the generated data point ensured?

Would you agree that Algorithm 1 seems like some variant of a class of genetic algorithms [If so this should be pointed out]?
Doesn’t this line of works that estimate user cost matrix raise questions about user-identifiability? [Where maybe through such intricate cost matrices these AI systems can uniquely identify users without their knowledge?]

Originality / Novelty / Reproducibility : 
I find that through text and supplementary material, the work should be reproducible.

summary_of_the_review: The work attempts to solve an interesting problem of accounting for specific user preferences while coming up with recourse recommendations. The reading could be improved with addition of more examples. The work utilizes three baselines over two datasets for their evaluations.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper describes INSPIRE, a methodology to provide recourse sets to users from which they can select a counterfactual instance which can overturn a potentially bad decision taken by a machine learning model. The authors show a method to efficiently sample counterfactuals tailored to the personal user’s cost function, thus making them more appealing to the users. They present COLS, a novel optimization algorithm which optimizes for the EMC (Expected Minimum Cost) and can refine the recourse sets iteratively. Lastly, the authors demonstrate the advantages of their method with an extensive evaluation.
strength_and_weaknesses: The paper is interesting, and it tries to tackle one of the challenges of algorithmic recourse. They outline a method that accounts for different cost functions and provides recourse sets from which the user can select. However, from the problem statement and assumptions, the problem the authors try to solve does not match the claim made by the paper. 

From Section 2 and Section 3, it seems that the assumption just shifted from having a shared cost function to a shared set of distributions describing all the possible user cost functions. D_{perc}, D_{lin} and D_{mix} need to be well-design to provide meaningful results. In a paper titled “integrating individual user preferences in recourse,” I would expect to see a way to integrate a single user’s preferences to provide recourse while the user is still not considered in the estimation process. 

The authors do define preferred features (FP) as a means to incorporate real individualised preferences (see Section 3.1). However, they assume that the user can provide them directly (which is generally not true), and they do not elaborate on them much further. I think providing a way to estimate them would be an important contribution instead. 

In Section 3.4, the authors argue that obtaining user feedback to assess satisfaction is challenging. This statement is not true since there is extensive literature [1] about estimating user preferences via several types of interaction, which has been applied in many fields, such as recommender systems. 

Minor:
Section 2 does not specify exactly how the cf is converted into actionable feedback for the user. Does the user select the less costly cf from the recourse set only? or is this cf presented in a “sequential” way to the user (e.g., first, get a degree, second, get a better job, etc.)?

[1] Viappiani, Paolo, Boi Faltings, and Pearl Pu. "Preference-based search using example-critiquing with suggestions." Journal of artificial intelligence Research 27 (2006): 465-503.

clarity,_quality,_novelty_and_reproducibility: The paper tries to address a relevant problem in the area of algorithmic recourse. It is also well-written and very detailed. The reviewer did not check the source code nor tried to run the experiments, but it seems the work is detailed enough to be reproducible by other researchers. 

summary_of_the_review: The paper presents a new methodology to obtain counterfactuals to achieve recourse. The authors try to move away from the shared cost assumption by providing a sample-based method which can mimic many different cost functions. However, the method itself is very involved, and it does not really provide user-tailored counterfactuals.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: Existing recourse generation methods optimize for any single objective that is shared across different users not paying attention to a user’s individuality. This might result in impractical recourses due to undesirable feature changes. Different individuals’ preferences may associate different costs to the same feature-change. Thus modeling integrating user-preference into recourse generation enhances its acceptability in the real-world. The unavailability of user-specific feature change costs further compounds the problem. The authors solution - INSPIRE provides each user with a recourse set that contains multiple options such that if possible, there is at least one feasible solution adhering to the user’s unknown personal feature preference. A diverse set of recourses are generated assuming the user-specific cost is sampled from a specific distribution. The authors propose COLS, a discrete optimization procedure to generate the recourses. The results from simulated user-preferences on benchmarking datasets demonstrates the solution’s efficacy. 
strength_and_weaknesses: Strengths

Problem statement: The authors address a pertinent problem faced by the algorithmic recourse research community. While prior approaches assume a cost function that is shared by all users, the authors formulate the problem to overcome this assumption, thus extending the research boundary. 

Solution: The solution - EMC objective and COLS are quite intuitive, non-trivial, and simple to follow. The search procedure is also theoretically grounded. 

Experiments: The authors conduct exhaustive experiments to validate the approach, including fairness analysis and ‘simulated’ human subject experiments resulting in strong evidence suggesting the approach’s efficacy.

Weakness

I have two major concerns with the proposed cost functions. Firstly, I believe it is impractical if not unrealistic to expect users to express the cost function in the required form. Users might provide some sort of ordinal rankings of the attributes, but to expect cardinal utilities for feature changes is unrealistic. More evidence from the authors to support their definition of cost function will strengthen the paper. Secondly, even though the authors claim on modeling unknown user cost function, it is strictly not unknown. There is a strong assumption-  the user cost function belongs to a larger family of functions. The structure of the cost function remains the same. Is there any evidence to suggest that the behavior of the cost functions are fundamentally different? That is, the cost functions (lin and per) are uncorrelated on a given dataset? While the experiments using distribution shifts(Q5) are perhaps in the right direction, evidence supporting the behavior of cost functions will further strengthen the observations from Figure 2 in the appendix. Furthermore, the results presented in Figure 3 are incomplete. It would help to see the distribution shift spanning from [0, 1], instead of stopping at 0.3.

There are also concerns with the human subject experiments. Firstly, asking annotators to pretend to be the end users of the system takes away the real-life changing situations under which customers might use the system. This has severe trust issues- A company is deploying a system for their clients based on hypothetical experiences of annotators. I recognize that designing and conducting an experiment involving real-users is a daunting task and also appreciate the authors’ effort in attempting to address this issue. However, I worry that these experiments involving pretend annotators may inaccurately portray the acceptance and usability of such systems . It is important to perform human subject experiments to validate the method, but the experimental design needs more care and technical soundness. Secondly, the experiment is basically asking the pretend annotators to check if the proposed recourse is following the feature scores. Considering that INSPIRE suggests recourses satisfying the feature scores, it is not surprising that the users favored INSPIRE over actionable recourses. I would rather argue that the acceptance rate of INSPIRE (57%) is quite low! More importantly, the experiment does not measure whether the recourse generated by INSPIRE is in accordance with the user’s cost function (not the vector $p_u$), which would require the user to provide a cost function!

I would also like to see some discussion on selecting $k$ in the feature satisfaction metric. How sound is it to assume a constant $k$ across the users? How does one decide on this value in the real world?

A minor concern is the comparison of INSPIRE against other actionable recourse approaches. One might view the comparison on cost metrics as unfair as the other methods are not designed to consider the user cost functions. 
clarity,_quality,_novelty_and_reproducibility: The paper is well organized and written. However certain sections relegated to the appendix should be part of the main paper. For example, the discussion on fairness that is not central to the contribution of the algorithm can be swapped with the experiments on distribution shifts between training and testing.

There are some typos that can be easily corrected. For example - the subscript $t$ is missing in some of $S$ in algorithm 1.

summary_of_the_review: Overall, I find the paper to be a work-in-progress on an important problem on generating recourses. While the methodology is technically sound, the fundamental issues related to the cost function assumptions, and the human subject experiments make me lean towards a reject. I would like to see this work getting published eventually.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper considers the problem of recourse generation when user preferences are unknown and the preferences can vary between users. To this end, the authors develop a method to propose a diverse set of counterfactuals based on a novel objective function and assumptions about the possible cost functions of the users. They develop an optimization algorithm (similar to genetic algorithms) to efficiently compute the set of counterfactuals. Furthermore, they evaluate their approach on synthetic and semi-real-world datasets and observe good performance in comparison to several baselines.

Strengths:
* The paper is mainly well written and easy to follow.
* The considered setting is clearly more realistic as compared to existing work.
* The proposed approach is sensible and intuitive.
* Experimental results indicate improvements over existing work.

Weaknesses:
* Strong assumption about individual user cost-functions, i.e., availability of a distribution over possible functions; and while I partly agree with the authors that this is more general than what has been done, it would have been interesting to understand more deeply which cost functions are prevalent in reality. In any case, this aspect should be clarified, also in the title of the paper.
* The user study is quite limited and might be biased; 3 annotators close to the research team were used.
* Recourse generation for the user study is unclear. I have read the reviewer's concern in that regard and also the authors response and checked the paper again - I think the question of whether feature preferences are used for recourse generation is still not clarified (the paper leaves this open as far as I can tell, the authors' response is not clear in that regard: "Hence, we do not optimize directly for the user cost function C_u^# or its corresponding feature preferences when generating the recourse." - the statement before does not tell about the usage of feature preferences). 
* Novelty wrt to other subpopulation based approaches is limited.
* Human evaluation is rather artificial.

The reviewers gave mixed scores for this paper and the authors did their best to clarify questions and concerns. I have also considered the authors' concern regarding one of the reviews but think this is based on a misunderstanding of what the reviewer wanted to say (cost-function versus feature preferences; unclarity about usage of feature preferences as also explained above). Unfortunately there was little active discussion between reviewers and authors. In our internal discussion we reached the consensus that although the paper is addressing an interesting problem, we are recommending rejection of the paper. The paper could be improved by developing the investigations regarding distribution shifts in more detail, improving the human evaluation, and discussing the relation to other subpopulation based approaches in detail.
justification_for_why_not_higher_score: The weaknesses (mainly assumptions about cost function and novelty) are too substantial for the reviewers to warrant acceptance.
justification_for_why_not_lower_score: n/a
Decision: Reject