Grounding and Validation of Algorithmic Recourse in Real-World Contexts: A Systematized Literature Review

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 14:23:57

Abstract: The aim of algorithmic recourse (AR) is generally understood to be the provision of "actionable" recommendations to individuals affected by algorithmic decision-making systems, in an attempt to offer the capacity for taking actions that may lead to more desirable outcomes in the future. Over the past few years, AR literature has largely focused on theoretical frameworks to generate "actionable" counterfactual explanations that further satisfy various desiderata, such as diversity or robustness. We believe that algorithmic recourse, by its nature, should be seen as a practical problem: real-world socio-technical decision-making systems are complex dynamic entities involving various actors (end users, domain experts, civil servants, system owners, etc.) engaged in social and technical processes. Thus, research needs to account for the specificities of systems where it would be applied. To evaluate how authors envision AR "in the wild", we carry out a systematized review of 127 publications pertaining to the problem and identify the real-world considerations that motivate them. Among others, we look at the ways to make recourse (individually) actionable, the involved stakeholders, the perceived challenges, and the availability of practitioner-friendly open-source codebases. We find that there is a strong disconnect between the existing research and the practical requirements for AR. Most importantly, the grounding and validation of algorithmic recourse in real-world contexts remain underexplored. As an attempt to bridge this gap, we provide other authors with five recommendations to make future solutions easier to adapt to their potential real-world applications.

Main paper: 
 introduction: 
 Algorithmic decision-making (ADM) tools are frequently seen as a way to improve decision processes in a variety of high-stakes domains such as public administration [47, 146] or healthcare [45, 87] .Deep learning models have attracted much attention due to their perceived high performance, but the predictions of such models cannot be interpreted by humans, hence end users -both individuals subjected to algorithmic decisions and decision-makers operating on them -are placed in a position where they are unable to understand the grounds of a prediction, act on it, or trust it [159] .To help address this problem, a variety of explanation methods has been proposed. Of particular interest for this paper are counterfactual explanations (CEs) that attempt to explain the predictions for individual instances of data, taking the form of conditional statements such as "if the value of featurex was a instead of b, the model would have predicted class y instead of z". They are perceived to be an attractive approach to explanation that does not require "opening the black box" [151] and have been argued to align with the ways that humans naturally reason about events [84] .Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute.CEs are also seen as the go-to method for algorithmic recourse (AR), or the generation of actionable recommendations that provide people with the knowledge needed to achieve more desirable predictions in ADM systems. Recourse is distinct from the "explanation" or "justification" of algorithmic decisions, and more closely related to the notion of contestability of Artificial Intelligence [7] in that it aims not only to improve the trust in the algorithm, but also embrace human agency [142] .Algorithmic recourse is an inherently practical problem in that it resembles a bureaucratic complaint process: an individual unhappy with some decision engages with a representative of the issuing organization, in an attempt to overturn it. Yet, we observe that much of the existing work is highly theoretical, with little consideration of whether it could be applied in organizational settings [see also 18] . Deploying AR in realistic systems without analyzing its mechanics in a broader context and without knowing what types of dynamics are expected to arise is bound to lead to unanticipated outcomes. Many of them will be undesirable and even potentially unsafe, and impossible to validate with respect to a set of requirements because the requirements for AR are necessarily socio-technical.Societal and institutional components of algorithmic recourse are the focal point of our work, as we look beyond the typical technical considerations to assess the practical aspects of the problem.To that end, we contribute a systematized review of 127 publications that address the goals of algorithmic recourse and we evaluate to what extent they incorporate such practical considerations.We characterize our approach as systematized because we follow a fully systematic approach to the collection of publications, but their selection is not necessarily exhaustive [46] as many impactful ideas in computer science are published only in the form of pre-prints. Based on our analysis, we also provide other authors with five recommendations on how to improve the practicality of AR research.The rest of the manuscript is structured as follows. In Section 2 we elaborate on the background of our work. Then, in Section 3 we describe our approach to this review. Next, Section 4 introduces our findings. Section 5 provides a discussion of our results, introduces our recommendations, and addresses the limitations of the current work. Finally, Section 6 forms the conclusion to this paper.Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?Answer: [Yes] Justification: Our main claim is that existing research on recourse is disconnected from the practical requirements of systems where it would be applied (see Section 4 and Section 5.1).Our claim is supported by a systematized literature review which is the contribution of this work (Section 3). These are reflected in the abstract and the introduction.Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
on algorithmic recourse: 
 Algorithmic -or actionable, individual -recourse was introduced in [138] as "the ability of a person to change the decision of the model through actionable input variables", building on the earlier work of [151] who argued that CEs are a psychologically-grounded way to (1) help decision-subjects understand an algorithmic decision, (2) provide them with information needed to contest it, and (3) inform about actions that could be taken to overturn it. For instance, consider a person who has unsuccessfully applied for a loan; they may then receive AR such as "if you requested $5000 less, you would qualify for this loan". The key consideration for AR is "actionability", which entails that the recipient of the recommendation should be capable of implementing it. If they had been informed "if you were 10 years younger, you would qualify for the loan", they would have still received a valid CE, but not recourse. More recently [69] has recast the problem as reasoning about minimal interventions on the structural causal model. This formulation (at least theoretically) addresses an important shortcoming of "correlational" recourse. Without accounting for the downstream causal effects of actions, an individual may exert more effort than necessary and still fail to achieve the target outcome. Indeed, counterfactuals are an inherently causal concept [103] .We note that problems similar to AR have been studied under a variety of different names: actionable knowledge discovery [e.g., 2], action rules mining [e.g., 110], inverse classification [e.g., 5], why not questions [e.g., 58], or actionable feature tweaking [134] . These alternative formulations have generally focused on "business" knowledge, rather than individual recommendations, but ultimately the goal of all these approaches is to extract information from a (black-box) model that allows the user -an individual or a decision-maker -to act. We highlight them to emphasize that AR does not have to be achieved through the means of CEs. Rather CEs should be seen as one of the means to achieve AR, particularly promising in that they do not require expert-level understanding of the model to be useful. Nonetheless, we decide to distinguish between the literature on AR (commonly equated with actionable CEs), and these alternative formulations in our work.Existing research has generally considered AR in simplistic settings that are far removed from real-world socio-technical decision-making systems, where it would be implemented as a process.For example, such systems are dynamic [113, 137] , must support the implementation of AR at scale [9, 94] , and involve various stakeholders beyond the end users [17, 151] . Moreover, if the intended goal of AR is to help individuals subjected to algorithmic decisions in an effective manner, research must entail a rich understanding of "actionability" to account for the differences between them [142] .Several groups of authors have previously surveyed the landscape of counterfactual explanations in general, and algorithmic recourse specifically. Perhaps the most relevant to our work is [71] , which discusses five deficits of research on CEs, with a special focus on the (lack of) psychological grounding.Another pertinent publication is [70] , which attempts to unify the definitions and formulations of AR in existing literature, but the work primarily focuses on technical aspects. Next, [143] develops a rubric to compare counterfactual explainers (equated with AR) and identifies 21 research challenges.While these also remain mostly technical, several of them are relevant to our work, for instance, CEs "as an interactive service to the applicants" or reinforcing "the ties between machine learning and regulatory communities". More recently, [48] reviewed and benchmarked a number of CE generators, but AR is only a secondary consideration in the work. We also highlight [130] , which is the only systematic review of counterfactual and contrastive approaches to date. The authors understand CEs as a way to justify model predictions (i.e., they are different from AR). We agree with this distinction in that CEs can be useful for reasons other than recourse, such as model debugging [e.g., 1, 122].Finally, although not reviews, [13] and [142] are particularly relevant to our work, offering critical perspectives on AR and addressing multiple shortcomings of recourse literature.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We highlight and discuss the three main limitations of our work in Section 5.2.Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.• The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
methods: 
 In this section, we briefly discuss our approach to the literature review following the SALSA -Search, Appraisal, Synthesis, Analysis -framework introduced in [46] . We also provide a more detailed description to allow for the reproduction of our process in the supplementary materials. Figure 1 presents our process in the form of a PRISMA flow diagram [97] .We make use of three search engines to collect the initial set of studies: ACM Digital Library, IEEE Xplore, and SCOPUS. Given the previously mentioned blurry distinction between AR and CEs, we consider the papers discussing either problem. In a small scoping review, we identify several keywords common to publications on recourse, as well as several equivalent terms to build the query.We search in titles, abstracts, and keywords, arriving at 3092 records after de-duplication. To facilitate the screening process, we employ the open-source ASReview tool, which makes use of an active learning approach to re-order the set of publications, such that the most relevant ones are always "at the top of the stack" [139] . The researchers behind the tool suggest employing a stopping rule measured in the number of consecutive irrelevant records, which we set to 30, or 1% of the entire dataset. We accept all papers that focus on algorithmic recourse and counterfactual explanations, completing the screening after evaluating 1040 abstracts, leading to 499 relevant records.We observe that some important publications may be missing from our results. For instance, [151] was published in a legal journal that is not indexed by computer science search engines. Thus, we decide to augment the set of records by applying snowballing, which has been shown as a good alternative to databases in systematic reviews in software engineering [162] . We collect the references for the top 50 (10%) "most impactful" publications, measured by the number of citations. While this introduces several pre-prints into our result set [52, 61, 91, 113, 143, 150] , we decide not to exclude them. Our review remains primarily concerned with peer-reviewed work. After adding the snowballed references to our dataset, we are left with 2018 records for the second screening with ASReview. This time, we look for publications that specifically refer to the problem of AR, "actionable" CEs, or modifying outcomes of automated decision-making systems. We employ a stricter stopping rule to minimize the risk of false negatives, completing the screening after 60 consecutive irrelevant records with 203 records considered for full-text appraisal. To allow for complete reproducibility of the search process, we provide an extended discussion (including queries) in the technical Appendix A.We were able to retrieve all of the remaining 203 documents. For each document, we require that the authors explicitly cite recourse as the center of interest, or look at (1) explanations (2) provided for individual instances (3) with the goal of acting upon them (4) in an attempt to modify the predictions (5) of a classification model. We exclude 51 publications as they are not on topic, primarily because they focus on CEs for the sake of explanation. Four works in this category look at (what they call) recourse but extend the problem to settings beyond the scope of this review: recommender systems [31, 43, 145 ], text classification [37] , and anomaly detection [27] . Further 15 publications are duplicates, typically pre-prints of other documents that were included in the review. Next, 8 documents were published before [151] that sparked the research on AR, and thus we exclude them as well. These look at the alternative formulations discussed earlier in Section 2.1. Finally, 2 documents are not publications: one is an abstract of a talk, and the other is a student poster. For each document, we answer a number of questions relating to the practical considerations introduced by the authors.Records identified from:• ACM Digital Library (n = 1267)• IEEE Xplore (n = 513) • SCOPUS (n = 2139)Records removed before screening: • Duplicates (n = 783) • Only meta-data (n = 44)Studies included in the review (n = 127): 3, 4, 6, 8, 9, 12-16, 19-22, 24-26, 28-30, 34-36, 38, 39, 41, 42, 44, 48-57, 60-67, 69, 70, 72-83, 85, 86, 88-96, 98-102, 104-109, 111-118, 120, 121, 123-129, 131-133, 135-138, 141-144, 147-158, 160, 161, 163-165 Figure 1 : Identification of studies via databases and snowballingTo compile the results we carry out a standard thematic content analysis following the approach presented in [40] . First, we explore the data extracted from the set of publications relevant to each question to find the commonalities, which serves as the grounds for creating the initial set of codes.We evaluate the documents against these codes and keep track of any other considerations. If such considerations appear in multiple documents, we create new codes for them. Afterward, we reevaluate all documents against the new code. As the coding exercise is carried out by one author, they do a third pass over all documents to double-check for potential errors. Finally, where relevant, we cluster the codes into larger themes. In this analysis we only look at the explicit statements provided by the authors, we do not attempt to infer their understanding of the problem. Thus, the numbers provided in Section 4 should be understood as describing how algorithmic recourse is discussed in the literature. For brevity, we focus our discussion on the main themes, but we still highlight specific publications if we observe that the authors introduce novel, highly relevant considerations that do not fit into other themes. Finally, even though we also evaluated the technical aspects of the proposed solutions -requirements for methods and datasets used in evaluations -they are not covered in this review. Instead, we point the interested readers to [48, 70, 143] .A pervasive assumption in the literature is that all challenges of AR require purely technical solutions. For instance, many authors emphasize the importance of causal modeling to guarantee recourse, but the models that aim to be explained are themselves not causal. Similarly, to improve the performance of CE generators many authors turn to deep generative models [35, 42, 61, 67, 81, 90, 99] . Not only do they explain the data rather than the model [10] , but more importantly they shift the problem from improving the trust in non-interpretable models, to attempting to trust non-interpretable explainers. Although a socio-technical perspective on AR brings its own challenges, such as accounting for the roles of stakeholders involved in the provision of recourse, it creates important opportunities. For example, developing "recourse contracts" [34, 39] or designing feedback processes to account for imperfect robustness.4. Accounting for emergent effects. Decision-making systems involve multiple individuals who may be interested in receiving recourse and may have competing interests. Research on AR should, from the onset, explore group-level effects such as external costs or fairness. While this may require expanding the boundaries of analysis, it is necessary to anticipate the emergent outcomes of recourse.These may even occur due to the multi-system dynamics of AR: recommendations implemented by an individual to improve their outcomes in one system will affect them in other contexts [see also 13].5. Attending to other operational aspects. Finally, the artifacts of AR research should be practitioner-friendly. On the one hand, this requires being explicit about the position of the proposed methods in a broader system, for example, in the form of end-to-end case studies that allow practitioners to better understand the benefits of the proposed solutions. On the other hand, this suggests that authors should attempt to move away from merely providing scripts for experiments, and focus on developing well-documented frameworks that can be adapted to different ADM systems.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Justification: Our work does not rely on any experiments, so this question is not applicable.Nonetheless, we provide the complete list of publications covered in this review. We will also release the review data upon acceptance.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/pu blic/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
results: 
 The following nine sections introduce the results of the thematic analysis. For each question, we explain why it is relevant to the analysis and examine the main themes. We also highlight highly important but underexplored themes. We start with the general points such as contributions and definitions in Sections 4.1 to 4.3. Then, in Sections 4.4 to 4.7 we investigate the societal components of AR research. Finally, in Sections 4.8 and 4.9 we look at the aspects relevant to practitioners.4.1 What types of contributions do the authors choose to make to the AR research?We start by looking at the main goals of the collected publications to validate our assumption that AR literature is primarily concerned with technical solutions. We annotate each entry with at most two codes based on the form of contributions. By far the largest group is propose methods, which applies to 88 (69.3%) out of the 127 publications. These are primarily generators for individual CEs, but we also find 18 (14.2%) documents that propose other methods. Next, 20 (15.7%) publications develop theoretical frameworks, for instance by grounding AR in user studies or providing critical perspectives on the problem. Further, 15 (11.8%) focus on empirical or theoretical analyses of the properties of AR and another 15 publications apply it in a variety of domains. We did not identify any applications evaluated with humans in the loop. Then, 5 (3.9%) publications benchmark existing methods, while 3 (2.4%) review them. We make our annotations available in technical Appendix B.We also evaluate what is understood as the problem to be addressed by AR mechanisms. In particular, what are the criteria to satisfy authors' definitions of recourse. A similar question was posed by [70] who combined six definitions into "recourse can be achieved by an affected individual if they can understand and accordingly act to alleviate an unfavorable situation, thus exercising temporallyextended agency", but this approach was far from systematic. Instead, we are interested in the underlying concepts. 74 (58.3%) publications explicitly define AR, 16 (12.6%) mention it but do not include a definition, while 37 (29.1%) do not mention AR, even though they align with its (overall) goals. The most common theme is overturning undesirable decisions, present in 47 definitions (63.5% of all definitions), but specifically overturning algorithmic decisions is mentioned only 43 (58.1%) times. It is generally understood that AR is provided to affected individuals (44, or 59.5%) but 4 (5.4%) definitions consider stakeholders more broadly. Actionability as a requirement for recourse is noted in only 39 (52.7%) definitions. Then, 20 (27.0%) publications specifically mention counterfactual explanations as means to AR, while 26 (35.1%) include various other technical considerations in the definitions, such as "changes to actionable input variables" or "desired classes".We also point to several themes that are, interestingly, underrepresented. Only 18 (24.3%) documents mention explanation, justification, or understanding of a decision as the pre-requisite for AR. Next, 10 (13.5%) highlight future-orientation or other temporal aspects of the provided recommendations.Although "consequential settings", typically bank lending, are given as examples in nine (12.2%) definitions, they are never explicitly mentioned as the scenarios where recourse ought to be provided, which may be akin to the "enjoyment of recourse" as defined by [142] where people are aware that there exists a way to reverse undesirable decisions. 1 8 publications (10.8%) promote AR as an ability.Finally, only 2 (2.7%) publications require that recourse accounts for the preferences of its recipients.As we observe, "actionability" is a concept that underpins AR but we discover that, in general, its understanding is limited. 91 (71.6%) publications attempt to define what it means (for a CE) to be actionable. Most commonly, in 48 (52.7%) out of 91 definitions, it is understood as acting only on directly-mutable features, 6 (6.6%) distinguish that features may be indirectly-mutable but still not actionable, while 22 (24.2%) also highlight that feature values may need to be constrained. Next, 19 (20.9%) definitions rely on a tautology that actionability means people can take actions, 11 (12.1%) emphasize that these actions must be successful or lead to change, and 3 (3.3%) further require that they are aligned with people's real-world objectives. Only 14 (15.4%) definitions put users at the center stage, indicating that actionability depends on the user or their preferences, while 2 (2.2%) highlight the importance of the context [144, 156] , for instance, that the ability to act on a recommendation may change over time. Importantly, ethical considerations are never mentioned as the pre-requisite for actionability, but we find some broader discussions about this [e.g., 142].What is the role of end users? What other stakeholders are envisioned in the AR process?Given that AR is to be implemented in socio-technical systems that include a variety of actors, we are interested in the types of stakeholders acknowledged in the literature. A total of 105 publications provide explicit consideration of this type. In general, end users subject to algorithmic decisions are envisioned to be the recipients of AR, but this is not always the case: it may also be provided to experts [e.g., 21, 22, 76] or organizations [e.g., 65, 72, 147] , which highlights that in some cases AR may be carried out on behalf of the affected individuals. In any case, 47 (44.8%) publications in the subset agree that end users should inform actionability, but it is rarely clear how these preferences should be specified. User-friendly (interactive) interfaces are a consideration in only 14 (13.3%) documents. A total of 29 (27.6%) publications envision domain experts as someone who inform the recourse process. They are either expected to inform actionability in the AR system or provide other forms of knowledge, typically in the form of a causal structure. Besides the experts, authors of 35 (33.3%) papers have discussed a variety of stakeholders. Most commonly system owners [e.g., 20, 34, 38, 89] , but also auditors [e.g., 138, 158] , data scientists [e.g., 28, 82] , developers [e.g., 22, 131] , practitioners [e.g., 100, 156] , regulators [e.g., 28, 120] , or even potential attackers [102] .With the multitude of challenges that stand ahead of real-world AR, we are interested in the considerations that motivate existing work. The main theme we find is ensuring proper individual actionability, which is addressed in 46 (37.4%) of 123 publications relevant to this question. This is typically achieved with the encoding of user preferences as constraints, but other means include providing diverse CEs. In fact, tackling specific desiderata for AR (beyond actionability) is the second largest area of research with 28 (22.8%) publications. Various other technical challenges are considered in 24 (19.5%) documents, for example, integrating background knowledge [e.g., 16, 62, 64, 98] , or incorporating feature importance [e.g., 4, 6, 96, 116] . We also find 19 (15.4%) publications that discuss the problem of communicating recourse to the end users. 16 (13.0%) focus on the dynamics of real-world systems, typically addressing the robustness of AR [e.g., 75, 91, 93, 137] , while 14 (11.4%) look at recourse in multi-agent systems. This also relates to performance considerations emphasized in 15 (12.2%) of documents. Causality drives research in 14 (11.4%) cases. We also find several themes that are under-emphasized: only 9 (7.3%) publications are directly motivated by research in psychology, while ethics of AR are emphasized in only 7 (5.7%) documents.While the previous section looked at the considerations that drive existing research, in this section we distill the recommendations for future research going beyond the improvement of own work, which are provided in 74 documents. Causality is highlighted as a challenge in 22 (29.7%) of them, while other technical considerations are given in 20 (27.0%) cases. These range from robustness [e.g., 51, 117, 137] , support for categorical features [e.g., 36, 157] , or distinguishing between valid CEs and adversarial examples [101] . Next, 19 (25.6%) documents highlight the importance of ensuring proper individual actionability, which also relates to communicating recourse to the end users (9, or 12.2%) and supporting realistic cost functions (8, or 10.8%). Ethics of AR are highlighted in 11 (14.9%) publications, for example, that AR research may detract from other obligations of model owners [77, 133] . The same number of publications emphasize the need to (1) ground research in user studies, and (2) accommodate for the dynamics of real-world systems. Privacy or security is highlighted in 10 (13.5%) documents, while the abuse of recourse, such as strategic behaviors, surfaces in 7 (9.4%) papers. Other challenges include improving performance (8, or 10.8%), considering multi-agent systems (4, or 5.4%), and developing legal frameworks (4, or 5.4%) for recourse. We also highlight several challenges particularly relevant to our work: (the usefulness of) recourse is perceived as difficult to evaluate in practice [41, 60, 115] , it must account for individual, contextual, societal, and even cultural factors [123] , which further means that engagement with recourse mechanisms and the likelihood of its implementation are context-dependent [e.g., 6, 42, 128] .Real-world systems entail the implementation of recourse by more than one agent, which may introduce group-level dynamics. Nonetheless, out of 119 documents relevant to this question, 93 (78.2%) seem to understand recourse as a purely individual phenomenon. Among the remaining 26 documents we find considerations for several different group-level effects. Various perspectives on the problem of fair AR, covering both individual and group formulations are addressed by [12, 36, 52, 120, 121, 131, 149, 154] . Next, [9] shows that the implementation of AR on a large scale may lead to domain and model shifts, which introduce unexpected costs for the stakeholders. 2 In [42] the authors focus on another negative consequence of AR at scale, showing that it may reinforce social segregation. The impact of the "right to be forgotten", where data deletion requests trigger model retraining that may invalidate existing recourses is addressed in [75] . Then, [94] develop a game-theoretic framework for AR in multi-agent settings, attempting to optimize for "social welfare" rather than the profits of individual agents. We find two further similar perspectives on recourse: [38] proposes auditing and subsidies to minimize the risks of strategic behaviors in a multi-agent setting, while [136] attempts to incentivize actual improvement for a population of agents. Finally, [65] provides a framework that generates transparent and consistent recourses for a sub-population.We also note two other lines of research that account for the remaining documents with group-level considerations. First, in a causal setting [e.g., 68, 73] subpopulations are necessary to estimate the interventional effects on individuals. Second, several works highlight the importance of global insights into the data [22, 41, 44, 78, 108, 112, 152] , such as recourse summaries [78, 112] .We now explore the different forms of "real-world" evaluations, going beyond quantitative experiments, which are present in 51 publications. Most commonly, in 28 (54.9%) of those, the authors make use of case studies presenting the methods in an end-to-end manner. Among those, the application of recourse in the Hired.com marketplace goes furthest in simulating real-world conditions for AR [89] , but the recommendations are still not evaluated with humans in the loop. Further, 9 (17.6%) documents include other forms of short walk-through examples. We also identify 14 (27.5%) papers that evaluate the methods with user experiments, 10 of which involve non-expert users and 4 involve expert users. While we do not observe any interviews with non-expert users, we find 1 (2.0%) publication where experts are interviewed [22] . Other involvement of non-experts applies to [116] , where they inform the development of methods. Other involvement of experts is featured in two documents where they evaluated the outputs of methods [25, 132] . Altogether, end users were involved in 17 publications, which is only 13.3% of all publications covered in our study, even more striking than the 21% of CE methods evaluated with user studies as reported in [71] .Finally, we note that the lack of availability of well-documented open-source code may be an important obstacle to the application of AR in real-world systems. For all 116 publications that involve some form of computational experiments, we verify whether the source code is publicly available. If the authors do not explicitly link to their code in the paper, we attempt to find it independently. Ultimately, we collect open-source implementations for 64 (55.2%) publications. Then, for each of them, we evaluate the quality of documentation. The instructions on the general usage (such as installation and workflow) are provided with 27 (41.5%) repositories, while instructions on the reproduction of results in 23 (35.4%). In 19 (29.2%) cases we find walk-through tutorials, typically in the form of Jupyter Notebooks, although we note that they differ in quality. For instance, 5 repositories include code-only notebooks with no further textual explanation that could guide the practitioner. Implementations for 4 papers include more "professionalized" documentation [9, 86, 100, 156] . The latter sets a golden standard as it further includes a tutorial video and a live demo. We do not find any additional materials for practitioners for 13 (20.0%) of the available implementations. 
discussion: 
 Regardless of whether AR can be normatively expected or not [77] , many systems can genuinely benefit from recourse mechanisms, especially when the interests of the system owner and the end users are aligned [72] , such as in the healthcare system to improve the well-being of patients [76, 96, 155] ,or on the online platforms that attempt to improve the experience of their users [89, 134] . Nonetheless, the values and norms underlying recourse -trust, agency, fairness, safety, and so on -are emergent properties of systems where recourse mechanisms would be introduced. Such norms can only be understood and evaluated when accounting for the technical, social, and institutional components of the system [32] , but the latter two remain largely unexplored in the recourse literature.Recourse is not inherently safe or unsafe, but its (incorrect) implementation may lead to the emergence of unsafe dynamics, such as the unexpected costs to stakeholders as discussed by [9] or the reinforcement of social segregation addressed in [42] . While it may be too challenging to provide accurate system-level evaluations at this stage of research, authors can still expand the boundaries of their analyses to account for global effects or look at the position of recourse mechanisms in the broader context of a complete socio-technical AI system [33] . As AR is a "reality-centric AI" problem [140] by its nature, working towards its integration into existing systems will require a design-oriented approach, potentially with specific systems in mind. The "Abstraction Traps" discussed by [119] in the context of research on fair machine learning apply here: that technical solutions designed for one social context cannot be directly repurposed for another application, that values to which they are expected to adhere to cannot be captured with mathematical formulas, that their insertion into an existing process will impact its behavior, or that the best solutions may not necessarily be technical.It is perhaps most telling that only 12% of surveyed publications attempt to apply recourse in realistic settings. We will discuss two of these settings to highlight the stark differences in system properties.Most of the applications included in our review focus on the provision of actionable individual recommendations to students [3, 4, 24, 109, 126, 135, 160] . In this relatively low-stakes domain almost any recourse will be actionable in that following a personalized set of learning activities does not require any resources other than time. Even then, the system involves multiple actors -students, teachers, parents -whose interactions will impact the process, for example, because students may fail to benefit from certain learning activities without additional support. Conversely, we find several publications where authors attempt to provide recourse in the high-stakes medical domain [76, 96, 155] . Here, recommendations must be tailored to the preferences, resources, or lifestyles of patients in order to have a chance of being actionable. Moreover, certain aspects of their implementation fully rely on other actors, such as a clinician prescribing the medications. Finally, it may happen that recourse does not exist at all when the outcomes of a patient cannot be improved.We distill our findings into five key recommendations. First, in Sections 4.2, 4.3 we observed that operational definitions for recourse are still unavailable. Second, Sections 4.4 and 4.8 underlined little consideration for people involved in recourse processes. Third, Sections 4.5, 4.6 highlighted the overwhelmingly technical approaches to recourse. Fourth, Section 4.7 stressed the lack of group-level analyses. Fifth, from Sections 4.8, and 4.9 we learned about the missing consideration of practitioners.1. Broadening the scope of research. AR is generally seen as a service for affected individuals, but this formalization may be unnecessarily limiting. In fact, in many systems, these individuals may be unable to directly act on recommendations [see also 142] . Instead, we propose to operationalize the aim of AR as the provision of recommendations aligned with the preferences of non-expert users in an attempt to help them improve outcomes in an ADM setting, which emphasizes that providing easy to understand and individually actionable recommendations remains the key research problem.2. Engaging end users, affected individuals, and communities. AR solutions are rarely evaluated with humans. Instead, they attempt to satisfy a variety of desiderata formulated by authors and assessed in an automated manner. Sparsity, proximity, or mutability of features are far from perfect proxies for individual actionability. For AR to be truly useful, it must be able to satisfy the preferences of its end users. Research is also necessary to learn about the needs of the affected individuals concerning recourse, and to validate its potential contributions and inherent limitations. Authors may also benefit from the rich literature on human-computer interaction [e.g., 11, 23] or psychology.Our review is not without shortcomings. Most importantly, for each paper the extraction and coding of data was performed by a single author, which means that the quantitative results may be imperfect.We account for this by focusing the analysis on the overarching themes represented in existing publications, thus, even if another researcher would have carried out the coding in a somewhat different manner, they should arrive at similar results and our analysis remains valid. Additionally, as our review ultimately looks at the authors' perception of recourse, we do not want to misconstrue their views. Thus, we do not infer any considerations unless they are provided explicitly. Our reading may be more strict than intended by the authors and the numbers reported in our results may be underestimated. At the same time, we believe that if certain considerations are deemed important by the researchers, they would choose to be explicit about them. Finally, although we followed a systematic process, we cannot claim that we collected AR literature in an exhaustive manner due to the specificities of computer science publishing. Thus, we acknowledge that there may exist some insightful publications addressing recourse that have not been covered in this literature review. 
conclusions: 
 Algorithmic recourse concerns the provision of recommendations aligned with the preferences of non-expert users of algorithmic decision-making systems to help them achieve more desirable outcomes in the future. Existing research on the topic is predominantly theoretical, even though recourse, in expectation, is a real-world problem with strong practical implications. To that end, we conducted a systematized literature review of 127 publications that focus on algorithmic recourse, and more generally on actionable counterfactual explanations. We evaluated the practical considerations provided by the authors. Our findings indicate that, indeed, AR tends to be perceived as a (predominantly) technical problem. Although we think highly of fundamental research, we note that for algorithmic recourse to leave computer science labs, it must be more strongly grounded and validated in the real world, and consider the requirements for systems that include not only technical but also social and institutional components. To help bridge this gap, we synthesize a list of five recommendations for other authors that aim to reinforce recourse as a practical problem. We believe that AR should not be seen as only a simple ad-hoc solution to improve the acceptance of black-box models in consequential domains, but rather as a full-fledged socio-technical mechanism that can benefit many systems and improve the agency of affected individuals and decision-makers across a variety of settings.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [NA] Justification: Our work does not rely on any experiments, so this question is not applicable.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
experiment statistical significance: 
 Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [NA] Justification: Our work does not rely on any experiments, so this question is not applicable.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [NA] Justification: Our work does not rely on any experiments, so this question is not applicable.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: Although this is not covered in a separate section, positive and negative societal impacts of our work (and algorithmic recourse in general) are a key consideration throughout this paper. See for instance Section 1 or Section 6.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.• The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.• If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: Our work poses no such risks, so this question is not applicable. We do not introduce any data or models.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.• Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.• We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [NA] Justification: Our work does not use existing assets (other than the referenced papers), so this question is not applicable. All papers covered in the review are referenced in sufficient detail, so that the readers can access them.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL.• The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.• For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.• If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [NA] Justification: Our work does not release any new assets, so this question is not applicable.We release the paper with the most permissible license available for NeurIPS submissions.Finally, we will release the review data upon acceptance.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA] Justification: Our paper does not involve crowdsourcing or research with human subjects, so this question is not applicable. The work was in its entirety carried out by the authors.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.• According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. Justification: Our work does not involve crowdsourcing or research with human subjects, so this question is not applicable. We did not require an IRB approval or equivalent to carry out this work. 
 
 Appendix: While our discussion of the search process in Section 3.1 in the main body of the document is complete, we also provide an extended version of this discussion to allow for full reproducibility.We make use of 3 search engines to collect the initial set of studies: ACM Digital Library, IEEE Xplore, and SCOPUS. Given the blurry distinction between AR and CEs, we consider the papers discussing either problem. In a small scoping review, we identify several keywords common to publications on recourse, as well as several equivalent terms to build the query shown below.("Machine Learning" OR "Artificial Intelligence" OR "Algorithmic Decision*" OR "Consequential Decision*" OR Classif* OR Predict* OR "Explainable AI" OR AI OR XAI) AND (((Counterfactual OR Contrastive OR Actionable) AND Explanation*) OR ((Algorithmic OR Individual* OR Actionable) AND Recourse) OR Counterfactual?)We modify this query to account for the semantic differences between the search engines.For ACM Digital Library: Title:(( "Machine Learning" OR "Artificial Intelligence" OR "Algorithmic Decision*" OR "Consequential Decision*" OR classif* OR predict* OR "Explainable AI" OR ai OR xai ) AND ( ( ( counterfactual OR contrastive OR actionable ) AND explanation* ) OR ( ( algorithmic OR individual* OR actionable ) AND recourse ) OR counterfactual? )) OR Abstract:(( "Machine Learning" OR "Artificial Intelligence" OR "Algorithmic Decision*" OR "Consequential Decision*" OR classif* OR predict* OR "Explainable AI" OR ai OR xai ) AND ( ( ( counterfactual OR contrastive OR actionable ) AND explanation* ) OR ( ( algorithmic OR individual* OR actionable ) AND recourse ) OR counterfactual? )) OR Keyword:(( "Machine Learning" OR "Artificial Intelligence" OR "Algorithmic Decision*" OR "Consequential Decision*" OR classif* OR predict* OR "Explainable AI" OR ai OR xai ) AND ( ( ( counterfactual OR contrastive OR actionable ) AND explanation* ) OR ( ( algorithmic OR individual* OR actionable ) AND recourse ) OR counterfactual? ))For IEEE Xplore:((("All Metadata":"Machine Learning" OR "All Metadata":"Artificial Intelligence" OR "All Metadata":"Algorithmic Decision*" OR "All Metadata":"Consequential Decision*" OR "All Metadata":classif* OR "All Metadata":predict* OR "All Metadata":"Explainable AI" OR "All Metadata":ai OR "All Metadata":xai ) AND ((("All Metadata":counterfactual OR "All Metadata":contrastive OR "All Metadata":actionable ) AND "All Metadata":explanation* ) OR ( ("All Metadata":algorithmic OR "All Metadata":individual* OR "All Metadata":actionable ) AND "All Metadata":recourse ) OR "All Metadata":counterfactual? )))For SCOPUS: TITLE-ABS-KEY ( ( "Machine Learning" OR "Artificial Intelligence" OR "Algorithmic Decision*" OR "Consequential Decision*" OR classif* OR predict* OR "Explainable AI" OR ai OR xai ) AND ( ( ( counterfactual OR contrastive OR actionable ) AND explanation* ) OR ( ( algorithmic OR individual* OR actionable ) AND recourse ) OR counterfactual? ) )The search is carried out on January 12 th 2024 in titles, abstracts, and keywords, with 1267 results from ACM Digital Library (The ACM Guide to Computing Literature), 513 results from IEEE Xplore, and 2139 results from SCOPUS. This leads to a total of 3919 results, which are imported to the Zotero reference management software for de-duplication. After removing the duplicates, we are left with 3136 results, 44 of which are the meta-data of conference proceedings that we also remove.To facilitate the screening process, we employ the open-source ASReview tool, which makes use of an active learning approach to re-order the set of publications, such that the most relevant ones are always "at the top of the stack" [139] . We run ASReview on the default settings, i.e.:Feature extraction technique: TF-IDF Classifier: Naive Bayes Query strategy: Maximum Balance strategy: Dynamic resampling (Double)The researchers behind the tool suggest employing a stopping rule measured in the number of consecutive irrelevant records, which we set to 30, or 1% of the entire dataset. We accept all papers that focus on algorithmic recourse and counterfactual explanations, completing the screening after evaluating 1040 abstracts (33.67% of the dataset), leading to 504 (16.30%) records among which we identify further 4 duplicates to remove. This results in the reported number of 499 relevant records.We observe that some important publications may be missing from our results. For instance, [151] was published in the Harvard Journal of Law & Technology that is not indexed by computer science search engines. Thus, we decide to augment the set of records by applying snowballing, which has been shown as a good alternative to databases in systematic reviews in software engineering [162] .We decide to make use of citation counts as a proxy for impact. Due to the lack of a suitable tool that would provide unbiased citation counts for all papers in our dataset, we collect them from Google Scholar. Unfortunately, citation counts on Google Scholar tend to be inflated, but as we make use of snowballing purely to enrich the dataset, these does not impact the validity of our study. We manually collect Google Scholar citation counts for all 499 results from the first screening on January 27 th and 28 th , order them descendingly, and collect references for the top 50 (10%) "most impactful" publications. Snowballing results in a total of 1519 new records. Indeed, we observe that [151] (mentioned above) is referenced by 39 of the 50 publications used for snowballing.While this strategy introduces several pre-prints into our result set [52, 61, 91, 113, 143, 150] , we decide not to exclude them. Our review remains primarily concerned with peer-reviewed work. Here, we also note that [114] , which we collected as a pre-print has been published between the search and appraisal. As such we decided to evaluate its published version and refer to it in this paper.After adding the snowballed references into our dataset, we are left with 2018 records for the second screening with ASReview, again on the default settings. This time, we look for publications that specifically refer to the problem of AR, "actionable" CEs, or modifying outcomes of automated decision-making systems. We employ a stricter stopping rule to minimize the risk of false negatives, completing the screening after 60 consecutive irrelevant records. We evaluate 538 results (26.71% of the dataset), with 203 (10.06%) relevant results that are considered for full-text appraisal. This concludes the extended discussion of the search process. ✓ [61] ✓ [81] ✓ [85] ✓ [138] ✓ 2020 [35] ✓ [86] ✓ [136] ✓ [20] ✓ [26] ✓ [44] ✓ [67] ✓ [66] ✓ [99] ✓ [104] ✓ [107] ✓ [120] ✓ [112] ✓ [13] ✓ [142] ✓ 2021 [69] ✓ ✓ [137] ✓ ✓ [41] ✓ [49] ✓ [53] ✓ [73] ✓ [150] ✓ [105] ✓ [19] ✓ [22] ✓ [63] ✓ [64] ✓ [88] ✓ [98] ✓ [115] ✓ [117] ✓ [153] ✓ [161] ✓ [121] ✓ [55] ✓ [12] ✓ [113] ✓ [125] ✓ [4] ✓ [82] ✓ [89] ✓ [96] ✓ [135] ✓ [152] ✓ [160] ✓ [100] ✓✓ ✓ [34] ✓ ✓ [6] ✓ [25] ✓ [50] ✓ [62] ✓ [158] ✓ [83] ✓ [56] ✓ [79] ✓ [80] ✓ [90] ✓ [93] ✓ [106] ✓ [111] ✓ [132] ✓ [131] ✓ [144] ✓ [65] ✓ [101] ✓ ✓ [24] ✓ ✓ [70] ✓ ✓ [15] ✓ [16] ✓ [94] ✓ [118] ✓ [133] ✓ [157] ✓ [128] ✓ [149] ✓ [28] ✓ [109] ✓ [126] ✓ [48] ✓ ✓ [143] ✓ ✓✓ ✓ [29] ✓ ✓ [116] ✓ ✓ [9] ✓ ✓ [42] ✓ ✓ [75] ✓ ✓ [147] ✓ ✓ [156] ✓ ✓ [155] ✓ ✓ [54] ✓ [123] ✓ [14] ✓ [72] ✓ [30] ✓ [51] ✓ [91] ✓ [92] ✓ [95] ✓ [108] ✓ [127] ✓ [129] ✓ [141] ✓ [154] ✓ [163] ✓ [164] ✓ [165] ✓ [78] ✓ [148] ✓ [74] ✓ [77] ✓ [124] ✓ [38] ✓ [57] ✓ [102] ✓ [3] ✓ [76] ✓ [8] ✓ [60] ✓ 2024 [21] ✓ [114] ✓NeurIPS Paper ChecklistJustification: We have reviewed the NeurIPS Code of Ethics and we confirm that our work conforms to it in every respect.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.• The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).Financial domain dominates the evaluations as well, with 90 of 116 evaluations on non-synthetic data making use of at least one finance-related dataset, most commonly German Credit Data [59] with 51 uses.Such "endogenous dynamics" were postulated earlier in the first version of[113] dated December 22 nd 2020, but this discussion has been completely removed from the subsequent versions of the pre-print. 


Decision: The paper conducts a thorough review of the existing literature on algorithmic recourse, focusing on the disconnect between theoretical approaches and practical applications in real-world settings. The authors review 127 publications and identify key themes, challenges, and gaps in the current research, emphasizing the need for more grounded and practical considerations in the development of algorithmic recourse methods. The paper concludes with five recommendations aimed at bridging the gap between research and practical implementation.

The reviewers generally agree that the paper is well-written, comprehensive, and provides valuable insights into the current state of algorithmic recourse research. There is a strong consensus among the reviewers that this is a good survey paper.

The main decision, therefore, depends on whether NeurIPS is the right venue for it. After confirming with the SAC, the call for papers requires submissions to contain "new and original research," and unfortunately, this paper does not satisfy that requirement. I tentatively recommend rejection on this ground but will leave a note for the SAC/PC to make the final decision. If the paper is rejected, reviewer yANZ has provided a list of pointers on where this submission might be better suited, and we strongly encourage the authors to incorporate the reviewer comments and submit the work to another venue.

Reviews: review: paper_summary: paper_summary: The authors present a survey regarding algorithmic recourse scientific literature. In their work, the authors analyze what types of contributions do the authors choose to make to the AR research, what are the criteria covered in the authors’ definitions of AR, what are the criteria covered in the authors’ definitions of actionability, the roles of end users, what types of real-world considerations motivate existing research, what types of real-world considerations are seen as challenges for future work, what types of group-level dynamics are addressed in the existing research, what are the approaches to the realistic evaluation of proposed methods, and what are the open source and documentation practices in AR research. They conclude their paper by providing recommendations on how to make future algorithmic recourse solutions better suited for real-world needs.
strength_weakness: strength_weakness: We did not identify important weaknesses. While an extensive survey could be created following this one, providing in-depth details for each of the sections, we understand this cannot be done within the constraints established for this venue.
questions: questions: We consider the work to be interesting and relevant. We consider the review to be concise yet relevant, and that could be extended later, providing more fine-grained insights on the topic of algorithmic recourse. Nevertheless, we would like to point to the following improvement opportunities:

(1) - the authors in their abstract mention multiple times the actionable component of algorithmic recourse, which is present in some of the subsections, but in others, the link to this aspect is less clear and could be enhanced. 

(2) - did the authors find any works considering actionability in a wider sense, e.g., that some action could be taken even by machines? What are the implications and concerns in such cases?
limitations: limitations: The authors have adequately acknowledged the limitations of their work.
paper_summary: paper_summary: The paper is a review of algorithmic recourse (AR) literature. The authors deploy a systematic framework to investigate research trends in algorithmic recourse and evaluate their incorporation of practical concerns like societal and institutional considerations of AR, or lack thereof. The review finds that current research is  focused on methods and technical considerations. The authors encourage researchers in AR to consider real-world implications of their work and conduct user studies.
strength_weakness: strength_weakness: While I agree with the points being made and appreciate the findings in the paper, I question their novelty. As mentioned in Section 4.6, there are papers (albeit in smaller numbers than we would want) that already provide real-world examples and attempt to discuss ethics within recourse. Previous work by [Doshi-Velez and Kim](https://arxiv.org/pdf/1702.08608), [Vaughan and Wallach](https://www.jennwv.com/papers/intel-chapter.pdf) have called for more user studies in interpretable ML, which resulted in studies like [Sixt et al.](https://openreview.net/pdf?id=v6s3HVjPerv). Considering that many researchers working on recourse is also in the field of ML interpretability, I am not sure if the paper's results and call for more user studies are very substantive.

Spending more time differentiating this work from other related works (especially other literature reviews like [70]) rather than listing their contributions in Section 2.2 may be helpful in making your case.

A more thorough discussion and evaluation of results (attempted in Section 5) may resolve some of these questions. As it stands, there is a disconnect between Section 4 and 5. The message of the first part of Section 5 (lines 318-350) is not clear. The second paragraph of the section does not seem to be a discussion of survey results but rather an argument the authors are trying to make (without using the results). The paper would benefit from expanding on the contents in lines 352 to 356, pointing to results in Section 4 and bridging them to the suggestions in Section 5.1.

The paper reads more like a position paper, trying to convince researchers in algorithmic recourse to not only focus on technical methods (which, again, I agree with). But I am not sure if a literature review or a position paper suitable for NeurIPS, considering its call for papers, does not seem to suggest so.
questions: questions: - What was the motivation for this literature review?
- Why do you say that recourse should be aligned with the *preferences* of the end-users?
limitations: limitations: Yes
paper_summary: paper_summary: The paper provides a comprehensive review of the algorithmic recourse research literature, concentrating on understanding the recourse research "in the wild", by focusing on the practical application of these techniques in real-world scenarios. The authors then provide some suggestions to practitioners to push future research to better practical applications.
strength_weakness: strength_weakness: I feel NeurIPS is not the right venue for this kind of contribution, since this paper does not provide the level of technical novelty required by the conference. Being a review, I think it does not fit the requirement of "new and original research" given by the Call of Papers. I suggest the authors not be discouraged, since I think the contribution is still valuable for the community. Potential other venues I believe are more in line with the scope of this work could be the following (the order is random):
- IJCAI Survey Track (https://ijcai24.org/call-for-papers-survey-track/)
- ACM FAccT (https://facctconference.org/)
- AAAI/ACM AIES (https://www.aies-conference.com/2024/)
- ICML Position Papers Track (https://icml.cc/Conferences/2024/CallForPositionPapers)
- ACM Computing Surveys (https://dl.acm.org/journal/csur) 
- TMLR (https://jmlr.org/tmlr/)

Lastly, I would like to point out some potential additional papers on algorithmic recourse which could complement some remarks made by the authors:
- Line 182 "We did not identify any applications evaluated with humans in the loop": there has been some development in providing human-in-the-loop algorithms to identify better recourse options:
  - [1] De Toni, Giovanni, et al. "Personalized Algorithmic Recourse with Preference Elicitation." Transactions on Machine Learning Research, https://openreview.net/forum?id=8sg2I9zXgO
- Recommendation 4, "Accounting for emergent effects": there has been some research regarding providing recourse to multiple individuals, where they are competing for a limited pool of resources, looking also at the fairness of these systems:
  - [2] Fonseca, João, et al. "Setting the right expectations: Algorithmic recourse over time." Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization. https://dl.acm.org/doi/pdf/10.1145/3617694.3623251
  - [3] Bell, Andrew, et al. "Fairness in Algorithmic Recourse Through the Lens of Substantive Equality of Opportunity." arXiv preprint arXiv:2401.16088, https://arxiv.org/pdf/2401.16088

I also point the authors to some new papers considering human-in-the-loop interfaces for recourse (Recommendation 1, Section 5.1):
- [4] Esfahani, Seyedehdelaram, et al. "Preference Elicitation in Interactive and User-centered Algorithmic Recourse: an Initial Exploration." Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization. https://dl.acm.org/doi/pdf/10.1145/3627043.3659556
- [5] Koh, Seunghun, Byung Hyung Kim, and Sungho Jo. "Understanding the User Perception and Experience of Interactive Algorithmic Recourse Customization." ACM Transactions on Computer-Human Interaction. https://dl.acm.org/doi/pdf/10.1145/3674503
questions: questions: I do not have any questions for the authors.
limitations: limitations: The authors have highlighted the limitations of their work in Section 5.2.
paper_summary: paper_summary: This paper provides a review of previous works that study "algorithmic recourse", i.e. conceptual and practical approaches for giving people actionable recommendations to change how they are impacted by algorithmic systems. This literature is deeply connected with counterfactual explanations and understanding models through small changes to test data, answering questions such as "how would the model M produce a different output if changed attribute x about myself". The authors review 127 archival publications and answer 9 questions about how these works frame and study algorithmic recourse.
strength_weakness: strength_weakness: Overall, I do think the current draft may not achieve the full impact that a future revision could provide.

The current discussion section feels like it largely echoes other calls in the community to apply systems thinking to ethical/responsible/pro-social AI/ML initiatives, and while each of the suggestions has some connection to one of the analyses, the current draft is not totally clear about the extent to which these recommendations stem from the findings vs. are motivated by first principles. The paper is overall very critical of the AR field, i.e. "Why hasn't this field engagement with any real world deployments". However, it's also not entirely clear in the current draft how any of the general recommendations would be applied in specific AR domains.

One aspect of the paper that I think would have been most directly helpful to the NeurIPS audience in particular would be to take a stance on how recourse should be defined -- is the definition on line 62 "endorsed" by the paper? Is the "imagine a counterfactual input x*" an advisable approach to take for future work. Does this review support the definition, highlight core definitional or epistemic issues, etc.

Ultimately, given the intended goals of this draft, it seems success and impact (on top of the core empirical contribution provided by writing a systematized review) here are dependent on the ability for the provided recommendations to shape future research positively. While the five recommendations here could have a some positive impact, taking a stronger stance on the core definitions and framing of recourse "tasks" could have an even larger impact.
questions: questions: The primary questions that I would pose to the authors would be:
- Is it possible to use the current "data" (i.e. selected papers) to provide more actionable domain-specific recommendations and/or pragmatic guidance about which contexts are likely to see real-world engagement with AR?
- What is needed to get organizations that develop and/or operate algorithmic systems to engage with recourse? Are there circumstances that the current literature treats as "futile"?

I think using the data that's already been collected and analyzed, focusing on domain-specificity of recommendations could go very far in strengthening the draft.
limitations: limitations: - No major concerns regarding unmentioned social impacts.
- Regarding the limitations of systematized literature review, the current draft discusses these reasonably.

Comments: title: All Official Comments are answered
comment: Dear Reviewers,

We have now answered all four Official Comments submitted by you during the discussion period. 

We will keep monitoring OpenReview in case you have any last-minute clarification questions, and do our best to properly address them in time.

Once again we would like to thank you for the productive discussions,  
All the best,  
The Authors
We also acknowledge having seen the response of Reviewer bnWe and thank them for the confirmation.
> I am still unconvinced that NeurIPS is the right venue for this contribution, so I will not change my evaluation.

> the points raised by the paper (although all true) are far from giving a novel and original view on the topic

We can see where you are coming from, but we have a different reading of the Call For Papers.

While we cannot infer how the authors of the CFP understand the "new and original research" clause, we still believe that the draft fulfills its requirements. As mentioned in the "global rebuttal" and the response to Reviewer pd5W, the novel aspects of our work include:

1. evaluating how the *problem* ("tasks") of algorithmic recourse is understood by authors;
2. quantifying the insights about this problem, showing, e.g., little interest in looking at AR in the context of specific domains;
3. providing a toolkit for other researchers and practitioners to facilitate this systems-oriented outlook on recourse.

We acknowledge and cite in the draft other works that provide various recommendations for AR research that are pertinent (although typically they are challenges for *research* rather than challenges for *research practices*), so we understand if the third point cannot be considered a fully novel contribution of our work. 

Nonetheless, to the best of our knowledge points 1. and 2. have not been addressed in the literature before. Given that AR is a broad challenge, we find the lack of its shared understanding, as seen in our results, an important obstacle for the research field.

In our opinion, the "original research" requirement of the CFP is fulfilled due to our choice of the research method: the systematic  (systematized) character of this literature review makes it empirical in nature. It involved data collection, analysis, and interpretation, akin to more common forms of contributions at NeurIPS. Thus, we go beyond "only" summarizing the existing literature, as would generally be the case in non-systematic reviews.  

---

> I find it difficult to imagine raising awareness of algorithmic recourse and engaging with industry and governmental partners through a simple poster session

We agree with you that increasing industry and governmental engagement with algorithmic recourse cannot be achieved by any single publication, but we believe that our work is a good point of departure in that direction. On the one hand, the draft is introductory enough to be valuable for people who are not yet familiar with the field (as also recognized by Reviewer pd5W who commended Section 2 on providing solid background). On the other hand, even researchers who have much experience with research on algorithmic recourse can benefit from a more grounded, shared understanding of the problem. 

In any case, we focused our recommendations on key factors that will be relevant for researchers regardless of their exact interest in the field. For instance, the lack of open-source code or documentation will be an important obstacle to the uptake of all forms of algorithmic recourse solutions. We are aware that such recommendations can be perceived as "simple", but they relate to substantial shortcomings that we have identified based on our results. 

Following the discussion with Reviewers Wg7R and pd5W we will also make sure that practitioners receive further guidance on how to apply this form of systems-oriented analysis to implement recourse in specific domains.  

---

> Probably, a nicer way to attract attention to the shortcomings of algorithmic recourse could have been either propose a tutorial (...) or a workshop on the topic (...)

We appreciate the suggestion on tutorials or workshops, this is something we will definitely consider as a next step!
Thank you for getting back to us!

---

>Since I slightly missed the main contribution of the paper, it might be worth taking some time to make that clearer in the paper. 

That is very good feedback. We will outline the contribution of this work in Section 1. Unless you have other suggestions in that regard, we will make use of the (summarized) explanation provided above in our response.

---

>I find the term "socio-technical" vague and unnecessary --- readers reading papers on algorithmic recourse already know its close relation to our lives

>The same goes for "society and institutional components". It's an abstract concept that is not easy to grasp (at least for me).

We fully agree with you that readers, in general, will understand that recourse would have impacts on people's lives. We talk about these social and institutional dimensions of algorithmic recourse problem in a somewhat broader sense, similar to how they are understood in fields such as systems safety, e.g.:  
* Leveson, N. G. (2012). _Engineering a Safer World: Systems Thinking Applied to Safety_ (p. 69). The MIT Press.
* De Bruijn, H., & Herder, P. M. (2009). System and actor perspectives on sociotechnical systems. _IEEE Transactions on systems, man, and cybernetics-part A: Systems and Humans_, _39_(5), 981-992.

With this, we aim to emphasize that the understanding of recourse in a specific domain will be influenced by factors such as the involved stakeholders or the organizational processes (see, e.g., lines 377-379), besides the technical factors that are the focus of the existing body of research. Moreover, many problems associated with algorithmic recourse, such as the meaning of actionability or its propensity to lead to unexpected emergent dynamics, can be understood only when accounting for all these components. In any case, we are happy to provide explicit definitions for these terms in the Introduction for ease of exposition.   

---

> Provided that the camera ready version includes an improved discussion section that weaves in its findings, I am willing to adjust my score.

Of course! To recap, we will reorganize Section 5 so that it starts with a longer discussion of the results, including what is now lines 352-356 to better explain the provenance of our five recommendations. Next, we will associate with each recommendation the guidelines for practitioners (from our answer to Reviewer Wg7R), explaining how these can be implemented in specific domains. Finally, we will integrate the current lines 318-350 as they follow from the recommendations, reducing the "disconnect" that you have highlighted in your initial review.

We are confident that these changes can be duly implemented within ≈three-fourths of the additional page in the camera-ready version of the draft, leaving enough space to better differentiate our work and the earlier reviews, as we also discussed.

---

> I doubt 1) whether the findings are sufficiently novel and 2) contributions are technical enough for NeurIPS, despite their significance in AR research. I will defer this decision to ACs.

We respect your judgment and appreciate your deferral of this decision to the ACs.

---

*Edit:* We have not received any notification from OpenReview that the above comment was received, so we are updating it to ensure that a notification was sent to the Reviewer. There are no changes in our response compared to its earlier version.
Thank you for the comments. We acknowledge we have reviewed the responses and have no further comments.
Thank you for the clarifications provided in the rebuttal. After reading the rebuttal and the other reviewers, I am still unconvinced that NeurIPS is the right venue for this contribution, so I will not change my evaluation.

My main concern is still the lack of novel methodological contributions required by the Call for Papers. I praise the work done by the authors, but the points raised by the paper (although all true) are far from giving a novel and original view on the topic. 

Regarding point 1 made by the authors in the general rebuttal, I find it difficult to imagine raising awareness of algorithmic recourse and engaging with industry and governmental partners through a simple poster session. Probably, a nicer way to attract attention to the shortcomings of algorithmic recourse could have been either propose a tutorial (https://neurips.cc/Conferences/2024/CallForTutorials) or a workshop on the topic (https://neurips.cc/Conferences/2024/CallForWorkshops). 
Moreover, other surveys on algorithmic recourse and, more broadly, counterfactual explanations (e.g., [1,2]) gained a lot of traction even if not submitted to NeurIPS.  

[1] Karimi, Amir-Hossein, et al. "A survey of algorithmic recourse: contrastive explanations and consequential recommendations." ACM Computing Surveys 55.5 (2022): 1-29.

[2] Verma, Sahil, et al. "Counterfactual explanations and algorithmic recourses for machine learning: A review." ACM Computing Surveys (2020).
Appreciate the detailed answers!

Since I slightly missed the main contribution of the paper, it might be worth taking some time to make that clearer in the paper. Perhaps the reason why I understood the main takeaway as calling for more user studies is from the title "grounding and validation of algorithmic recourse in real-world contexts", which one would achieve by running user studies (especially in the context of academic research).

Additional things I missed in my first review:
- I find the term "socio-technical" vague and unnecessary --- readers reading papers on algorithmic recourse already know its close relation to our lives
- The same goes for "society and institutional components". It's an abstract concept that is not easy to grasp (at least for me).

Provided that the camera ready version includes an improved discussion section that weaves in its findings, I am willing to adjust my score.

Having said that, I am still not confident that the content of the paper is suitable for NeurIPS. As Reviewer yANZ pointed out in their review, the Call for Papers mentions "new and original research" (the paper doesn't fit in the "interdisciplinary" category). I doubt 1) whether the findings are sufficiently novel and 2) contributions are technical enough for NeurIPS, despite their significance in AR research. I will defer this decision to ACs.
Thank you for the reply and the increased score! We are very happy that our answers satisfied your concerns about the contents of the draft — your suggestions have been very helpful, we have no doubts that going further in the recommendations will strengthen the document. 

We also appreciate your repeated support for our choice of the venue.

Should you have any further questions, please let us know.
Thank you for these detailed answers to the questions. This rebuttal

- answers above questions about defining / redefining recourse. I think this can easily be improved in a camera ready
- clarifies relation to prior work
- provides quite a few a concrete additional empirical insights that can be added to flesh out some additional aspects of the paper

Overall, this addresses some of my concerns. I expect the main lingering concern will be the question of venue fit. As argued in the global rebuttal, I can see this kind of paper fitting into the "catch all" category. Furthermore, as noted in my original review I agree this is a strong argument for trying to publish this work in venues that are most likely to impact recourse implementations in practice. That said, I expect this may come down to subjective interpretation of the CFP here.