Title: Learning Gradient Boosted Decision Trees with Algorithmic Recourse
Venue: NeurIPS 2025
Accepted: yes
Keywords: algorithmic recourse

gradient boosted decision trees

counterfactual explanation
Abstract: This paper proposes a new algorithm for learning gradient boosted decision trees while ensuring the existence of recourse actions. Algorithmic recourse aims to provide a recourse action for altering the undesired prediction result given by a model. While existing studies often focus on extracting valid and executable actions from a given learned model, such reasonable actions do not always exist for models optimized solely for predictive accuracy. To address this issue, recent studies proposed a framework for learning a model while guaranteeing the existence of reasonable actions with high probability. However, these methods can not be applied to gradient boosted decision trees, which are renowned as one of the most popular models for tabular datasets. We propose an efficient gradient boosting algorithm that takes recourse guarantee into account, while maintaining the same time complexity as the standard ones. We also propose a post-processing method for refining a learned model under the constraint of a recourse guarantee and provide a PAC-style analysis of the refined model. Experimental results demonstrated that our method successfully provided reasonable actions to more instances than the baselines without significantly degrading accuracy and computational efficiency.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on Algorithmic Recourse. The paper proposes a new gradient boosted decision tree framework, called Resource-Aware Gradient Boosted Decision Tree (RABIT), which can provide both accurate predictions and executable resource actions. A post-processing method is also proposed to refine the leaf weights, and a PAC-style guarantee is provided. Experimental results on four datasets show that RABIT outperforms the baseline while maintaining comparable accuracy and efficiency.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths \n\nThe issues this paper focuses on are important, especially since some of the features cannot be modified as mentioned in the introduction.\n\nThe paper gives a PAC-style guarantee.\n\n## Weaknesses\n\nThe experiments are not sufficient, for example, only four datasets are used, and the analysis of some experimental results is not very clear.\n\nQUESTIONS:\n1. There are few data sets used in the experiment. Why did you choose these four data sets?\n\n2. Is it only binary classification? How about multiple classifications?\n\n3. In the introduction section, it was mentioned that some features, such as age and race, cannot be modified. How did you reflect this in your experiment?\n\n4. “Figure 3 shows the average accuracy and recourse ratio for each γ. We can see that the recourse ratio 294 (resp. accuracy) was improved by increasing (resp. decreasing) γ on almost all the datasets, which 295 suggests that we could balance their trade-off by tuning “ I disagree with this statement. Figure 3 can only show that there is a trade-off, but it cannot show that RABIT can balance this trade-off.\n\n5. Are the experimental results in Sections 5.1 and 5.2 using post-processing or not? What are the ablation experiments with and without post-processing?\n\n6. Why is the accuracy lower and the variance larger on adult and bail from t=100 to t=150? The same thing happens on FICO and COMPAS from t=150 to t=200.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose a recourse-aware gradient-boosting algorithm (RABIT) that considers recourse actions. They also propose a post-processing task of modifying a learned tree ensemble model to satisfy the constraint on recourse guarantee and theoretically and empirically demonstrate the efficacy of RABIT in comparison to vanilla gradient boosting algorithms.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The authors clearly state the problem and present theoretically and empirically sound results. \n- They theoretically prove that with and without recourse guarantees, they can get an approximate solution in O(ND) time. \n- To ensure recourse guarantees for unseen test instances, authors also propose a leaf refinement approach and show a PAC-style bound on the estimation error of the surrogate risk.\n- Authors empirically test RABIT on four commonly used real-world datasets, provide key details about the experimentation process in the main paper and appendix, and avail their code. \n\n**Weaknesses**\n- Actionability and mutability of features. \nThe authors consider two cases: RABIT, which places less emphasis on actionability, and OAF, which restricts itself to actionable features. \nWhile I understand the premise of the author's argument and appreciate the two versions, it is unclear to me whether the recourse generated based on the RABIT-trained trees respects actionability constraints.\nFor example, even if RABIT does not explicitly incorporate actionability, do the generated recourses avoid suggesting unrealistic or non-actionable changes, such as increasing age by 10 years? While technically viable, the recommendation is not actionable and could encourage gaming behavior rather than genuine improvement.\nAdditionally, regarding the example on lines 31–36, would it be possible for the authors to use another example?\n\n- Price of recourse.\nThe authors use the maximum percentile shift costs function. However, Ustun et al. mention that this might not correctly reflect the difficulty\nof recourse. How easy(hard) would it be to equation 5 to other forms of the price of recourse?\n\n- Experiments.\nWhile the authors do a great job of illustrating the experimental results, the explanations of the observations are sparse. \nFor example, can the authors provide explanations for why OAF (Figure) consistently has the lowest accuracy and why the recourse ratio is close to 1 and 0.5 in other datasets, or why RABIT consistently has the highest ratio? Why is the gamma parameter better at 0.002 to 0.004?\n\nQUESTIONS:\n- Actionability and mutability of features. \nThe authors consider two cases: RABIT, which places less emphasis on actionability, and OAF, which restricts itself to actionable features. \nWhile I understand the premise of the author's argument and appreciate the two versions, it is unclear to me whether the recourse generated based on the RABIT-trained trees respects actionability constraints.\nFor example, even if RABIT does not explicitly incorporate actionability, do the generated recourses avoid suggesting unrealistic or non-actionable changes, such as increasing age by 10 years? While technically viable, the recommendation is not actionable and could encourage gaming behavior rather than genuine improvement.\nAdditionally, regarding the example on lines 31–36, would it be possible for the authors to use another example?\n\n- Price of recourse.\nThe authors use the maximum percentile shift costs function. However, Ustun et al. mention that this might not correctly reflect the difficulty\nof recourse. How easy(hard) would it be to equation 5 to other forms of the price of recourse?\n\n- Experiments.\nWhile the authors do a great job of illustrating the experimental results, the explanations of the observations are sparse. \nFor example, can the authors provide explanations for why OAF (Figure) consistently has the lowest accuracy and why the recourse ratio is close to 1 and 0.5 in other datasets, or why RABIT consistently has the highest ratio? Why is the gamma parameter better at 0.002 to 0.004?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-26"}

comments: SUMMARY:
This paper proposes *recourse loss* which measures how easily a prediction can be flipped with a low-cost, feasible action. This paper is the first to extend recourse guarantees to GBDTs: the training objective combines standard predictive loss and recourse loss via a trade-off parameter. The recourse loss is non-differentiable, so the authors derive a smooth upper bound using techniques like log-sum-exp, enabling gradient boosting.

Post-hoc refinement treats the tree ensemble as a linear model in its leaves and solves a constrained optimization to reduce recourse loss on unseen data. The optimization is made feasible by being reformulated into a tractable unconstrained problem using a Lagrangian multiplier tuned on a calibration set.

STRENGTHS AND WEAKNESSES:
Pros:

The paper is generally well-written and the empirical result is strong. The contribution appear to be significant within a community: previous recourse-aware learning methods targeted NN which rely on differentiability, or classification trees. This paper is the first to extend recourse guarantees to GBDTs. 

Cons:

There isn't an ablation study on the post-hoc refinement. Does the proposed method not make sense at all without the refinement?

QUESTIONS:
See Weaknesses

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a new framework, Recourse-Aware gradient Boosted decIsion Trees (RABIT) that learns tree ensemble models with recourse guarantees. The authors further proposes a post-processing procedure for modifying an existing tree ensemble model for better recourse outcomes on unseen data. They run a empirical experiments to verify that the resulting models have recourse and do not significantly sacrifice predictive accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\nThe authors have framed the problem very well. The problem they aim to address is significant and seems to be up to date with the literature. Experiments are well-designed.\n\n## Weaknesses\n**Significance of Experimental Results**\n\nThe paper can benefit from a more extensive discussion of the experimental results rather than just describing the outcome. The authors should take the extra step and answer: what significance do these results have? (i.e., real-life impact)\n\n**Set of Feasible Actions**\n\nStudies on algorithmic recourse heavily depend on how the set of feasible actions is defined. The set that the authors have presented in Section 2.1 (lines 94-95) is a convex relaxation of the true set of feasible actions. I have my reservations about this setup in effectively recovering recourse actions that are feasible in practice.\n\nQUESTIONS:\n- Why do we assume that $l_{01} (y,\\hat{y}) \\leq l(y,\\hat{y})$? (Line 80)\n- Shouldn't the 0-1 loss be $l_{01}(y,\\hat{y} = \\mathbb{I}[y\\cdot\\hat{y} < 0])$?\n- Can the authors elaborate on how each term of $\\xi_{t}$ can be computed in $\\mathcal{O}(I_{s})$? Is this because $\\mathcal{A}$ is an $l_{\\infty}$ ball around $x$?\n- Is Proposition 5 really necessary?\n- The current framework aims to have one point with the desired prediction for each data point. Can we modify the framework to allow more recourse provision for each data point (i.e., rather than having just one with minimal cost)?\n- Any guesses on when post-processing is as effective as RABIT ensembles for standard tree ensembles?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}

comments: SUMMARY:
The paper studies how to train boosted tree ensembles to maximize both the predictive accuracy and the likelihood of obtaining counterfactual explanations. It shows how to adapt the training procedure and provides theoretical results on guarantees of finding feasible recourse actions.

STRENGTHS AND WEAKNESSES:
The paper is very well written. Its main argument is convincing and the methodology is well-supported.

My main concern is w.r.t the choice of Feature Tweaking as an oracle for finding an algorithmic recourse in the experiments. This algorithm is well-known to be outperformed by several more recent approaches, such as the work of Kanamori et al, Cui et al (cited in the paper) but also:
- Parmentier, A., & Vidal, T. (2021, July). Optimal counterfactual explanations in tree ensembles. In International conference on machine learning (pp. 8422-8431). PMLR.
- Lucic, A., Oosterhuis, H., Haned, H., & de Rijke, M. (2022, June). FOCUS: Flexible optimizable counterfactual explanations for tree ensembles. In Proceedings of the AAAI conference on artificial intelligence (Vol. 36, No. 5, pp. 5313-5322).
It is likely that the trend in results would maintain with alternative algorithms for algorithmic recourse, but it is a worthy experiment to show how large the gap in guaranteed recourse depends on the underlying algorithm.

QUESTIONS:
- What part specifically in the approach of Kanamori et al. [ 28 ] makes it impossible to adapt the method to gradient-boosted trees?

LIMITATIONS:
See weaknesses.
TITLE: Official Review
CLARITY: 4
IMPACT: 3
ORIGINALITY: 3
DATE: 2025-07-05