Learning Actionable Counterfactual Explanations in Large State Spaces
Authors: Keziah Naggita
Matthew Walter
Avrim Blum
Venue: Accepted by TMLR ()

Abstract: Recourse generators provide actionable insights, often through feature-based counterfactual explanations (CFEs), to help negatively classified individuals understand how to adjust their input features to achieve a positive classification. These feature-based CFEs, which we refer to as \emph{low-level} CFEs, are overly specific (e.g., coding experience: \(4 \to 5+\) years) and often recommended in a feature space that doesn't straightforwardly align with real-world actions. To bridge this gap, we introduce three novel recourse types grounded in real-world actions: high-level continuous (\emph{hl-continuous}), high-level discrete (\emph{hl-discrete}), and high-level ID (\emph{hl-id}) CFEs.

We formulate single-agent CFE generation methods for hl-discrete and hl-continuous CFEs. For the hl-discrete CFE, we cast the task as a weighted set cover problem that selects the least cost set of hl-discrete actions that satisfy the eligibility of features, and model the hl-continuous CFE as a solution to an integer linear program that identifies the least cost set of hl-continuous actions capable of favorably altering the prediction of a linear classifier. Since these methods require costly optimization per agent, we propose data-driven CFE generation approaches that, given instances of agents and their optimal CFEs, learn a CFE generator that quickly provides optimal CFEs for new agents.  This approach, also viewed as one of learning an optimal policy in a family of large but deterministic MDPs, considers several problem formulations, including formulations in which the actions and their effects are unknown, and therefore addresses informational and computational challenges.  

We conduct extensive empirical evaluations using publicly available healthcare datasets (BRFSS, Foods, and NHANES) and fully-synthetic data. For negatively classified agents identified by linear and threshold-based binary classifiers, we compare the proposed forms of recourse to low-level CFEs, which suggest how the agent can transition from state \(\mathbf{x}\) to a new state \(\mathbf{x}'\) where the model prediction is desirable. We also extensively evaluate the effectiveness of our neural network-based,  data-driven CFE generation approaches. Empirical results show that the proposed data-driven CFE generators are accurate and resource-efficient, and the proposed forms of recourse offer various advantages over the low-level CFEs.

Keziah Naggita Published in Transactions on Machine Learning Research (06/2025)
Learning Actionable Counterfactual Explanations in Large State Spaces
knoggita@ttic.cm edu
Toyota Technological Institute at Chicago
Matthew R. Walter
movaltr edu
Toyota Technological Institate at Chicago
Avrim Blum
avrim@ttic..o edu
Toyota Technological Institute at Chicago
Reviewed OpenReview: https: openreeeee. net/ forum? id= www.:t/wwwppuuueeetterraeaaaamma
Abstract
Recourse generators provide actionable insights, often through deature-based counterfactual explanations (CFEs), to help negatively classified individuals understand how to adjust their input features to achieve positive classification. These feature- based CFEs, which refer to low-level CFFs, are overly specific (e-g., coding experience: 4 - 5+ years) and often recommended in feature space that doesn't straightforwardly align with real- world actions. To bridge this gap, we introduce three novel recourse types grounded in real-world actions: high-level continuous (bl-continuous)) high-level discrete (hl-discrete), and high-level ID (hl-id) CFEs.
We formulate single-agent CFE generation methods for hl-discrete and hl-contimuous CFEs. For the hi-discrete CFE, we cast the task as a weighted set cover problem that selects the least cost set of hl-discrete actions that satisfy the eligibility of features, and model the hllcontinuous CFE solution to an integer linear program that identifies the least cost set of hilcontinuous actions capable of favorably altering the prediction of a linear classifier. Since these methods require costly optimization per agent, we propose data-driven CFE generation approaches that, given instances of agents and their optimal CFFs, learn a CFF generator that quickly provides optimal CFEs for new agents. This approach, also viewed as one of learning an optimal policy in family of large but deterministic MDPs, considers several problem formulations, including formulations in which the actions and their effects unknown, and therefore addresses informational and computational challenges.
We conduct extensive empirical evaluations using publicly available healthcare datasets (BRFSS, Foods, and NHANES) and fully-synthetic data. For negatively classified agents identified by linear and threshold-beee recourse to low-level CFEs, which suggest how the agent can transition from state x to a new state where the model prediction is desirable. We also extensively evaluate the effective- ness of our neural network- based, data-driven CFE generation approaches. Empirical results show that the proposed data-driven CFE generators accurate and resourco-efficient, and the proposed forms of recourse offer various advantages over the low-level CFEss
binary classifiers, we compare the proposed forms of
1 Introduction
Machine learning models are incrcasingly being used guide high-stakes docision-making processes. Given the potential impact on individuals' livelihoods, society demands transparency and the right to an expla- nation, outlined in Articles 13 15-1 of the Furopean Parliament and Council of the EU (2016) General

---

Published in Transactions on Machine Learning Research (06/2025)
The agent profile (app1)
The agent profile (app2)
Calcium (mg): 309
PhysActivity:
Carbohydrate (gm): 109.15
Pruits: 0
Copper (mg):: 0.425
Veggies:
Dictary fiher (gm): 4.1
AnyHealthcare:
Tron (mR):
LowBP: O
Magnesium (mg): 96
NoSmoke:
LowChol: 0
HealthBMI:
The agent profile (app3)
Education:
BSc in computer science
Calcium (mg): 309 113
Coding experience: 4 years
Carbohydrate (gm):
Leadership: none
109.45 45.3376000000000000
Published research: none
Copper (mg):
Fruite:
0.425 0.2129550000000000000
Veggies: 0 -
Dictary fiher (gm):
LowBP:
4.1 - 50. 500111995000000000211
LowChol: 0 0++
Iron (mg):
HealthBMI: 0 -
4.08 - 42. 42.7284400000000002
Coding experience: 4 - 5+ years Leadership: none - 1+ years Published research: none 1+
Magnesium (mg): 57
The low-level
The low-level CFE
The low-level CFE
Take "leavening agents: cream
Pickup the "Ksinapril and atorvastatin preseription from the pharmocy" (doi) Adopt "a keto dšet" (daz)
tartar" (ca1)
Take " "fish, tuna, Rihh, canned water, druined solids" (caz)
Complete the Google Al residency program
The hl-continuous CFE
The hi-discrete CFE
The hl-id CFE
Figure 1: A comparative analysis of low-level CFEs with three high-level types: hl-continuous, hi-discrete, and hl-id, offering actionable insights to help negatively classified agents (with initial profiles/statee in grey) achieve positive outcomes from binary classifiers; dietary changes to achieve a healthy waist-to-hip top ratio (app1), guide an agent (app2) to meot wellness check critoria, and help an agent (app3) qualify for an AI junior research engineer role. In all cases, the low-level CFE precisely specifies which features to modify and by how much. The hl-continuous CFE adjusts multiple features numerically, e.g., corresponding to features in the profile (app3), ca2 = [8.0, 61.5, 0.195, 0.2, 3.72, 2.0, ... ca2 = [17.0, 0.0, 0.05, 0.0, 1.63, 23.0.-... but the agent does not need to understand the exact changes to follow the CFE. The hl-discrete CFE ensures features meet specifie eligibility thresholds without the agent needing to know the exact adjustments to take the CFE, e.g., da₁ = [o, 0, 0, 0, 1, 1,0,1,0,, -,,,,...,,,,. dag = [0, 1, 1, 0, 0, L..... The hl-id CFE provides a single overarching high-level action that favorably modifies all the features should. This figure illustrates how different types of CFEs might impact the agent's 's ability interpret and act given recourse.
Data Protection Regulation and Article 13 of the European Union (2025) AI Act. A critical aspect this transparency is understanding how individuals (agents) can modify their input features to achieve a desired outcome, such a positive label in binary classification setting. Recourse or counterfactual explanation (CFE) generators that provide actionable insights offer one such solution (Wachter et al., 2018; Ustun et al., 2019; Joshi el al., 2019; Dandl et al., 2020; Mothilal et al., 2020; Karimi et al., 2021; 2022)."
A popular form of recourse generation, actionable recourse (Ustun et al., 2019), generates feature-based CFEs, specifying precise adjustments to features (state) to ensure that the new features collectively result in a positive classification. For comparison with our work, we refer to these as low-level CFEs. While helpful, low-level CFFs are overly specífic and might be challenging to translate into real-world-like actions. To address this limitation, we introduce three novel forms of recourse that align with real-world actions: high-level continuous (hl-continuous), high-level discrete (hl-discrete), and high-level ID (hl-id) CFEs.
Figure 1 presents an illustrative example of hypothetical recourse across three binary classification tasks, highlighting the distinctions between low- and high-level CFEs. Specifically, the hl-contimuous CFEs involve
Following prior work (Pawelezyk al., 2022; Rasouli Chich Yu, 2024; Jiang al., 2024; Verma et al., 2024), the terms recourse and CFE interchangeably. For detailed discussion and nuanced differentiation of these terms, we refer the reader Section 2 Karimi et al. (2022) and Section 3.3 of Verma al. (2024).
2

---

Published in Transactions on Machine Learning Research (06/2025)
general and predefined real-world-like actions that modify multiple features simultancously through numerical menus. Similarly, hl-discrete CFFs also stem from real-world actions, each of which might affect several features, with the effect on each feature assumed to be known. However, unlike hl-continuous CFEs, which adjust feature values, hl-discrete CFEs modify feature eligibility. The hl-discrete CFEs use binary vector actions to ensure all features meet a predefined threshold, reducing decisions to unit simple yes/no questions. It particularly efficient scenarios where feature satisfiability depends on a. set threshold, such as level- one pecision-making in wellness checks (see Figure 1 and Appendix A). Lastly, the hl-id CFE encapsulates either hl-contimuous or hl-discrete CFEs into single overarching action, encompassing all necessary changes without detailing specific feature modifications. This level of abstraction relies on domain knowledge and often conveys significant implicit information.
Our contributions. First, we introduce three forms of recourse: hl-contimuous, hl-discrete, and hl-id CFEs, that bridge the gap between feature-based and real-world actions (Sections 1 and 2). Second, we propose single-agent CFE gencration methods that leverage predefined, worl-kord actions to generate optimal CFEs. Specifically, we formulate single-agent hl-discrete CFE generation as weighted set cover problem and single-agent hl-continuous CFE generation as an integer linear programming (ILP) problem (Section 3).
Third, we propose data-driven approaches that, given instances of agents and their optimal CFEs (agent- CFE dataset), learn a CFE generator that will quickly provide optimal hl-continuous, hl-discrete, and hl-id CFFs for new agents. Unlike the expensive single-agent CFE generation approach, the data-driven methods are more computationally friendly. Additionally, these methods are especially favorable when historical instances of agents and their optimal CFE data are available or aggregatable, recourse generators can operate inde- pendently of decision-makers, query access to the classifier is restricted or unavailable, and the actions along with their costs and explicit effects on features are unknown (Section 4). To the best of our knowledge, these alternative forms of recourse and the data-driven approach for generating CFEs are novel contributions.
Finally, we conduct extensive experiments on 30 agent-CFE. datasets derived from real-world healtheare datasets: BRFSS, Foods, and NHANES. We chose these datasets due to the availability of a wealth of publicly accessible data, which allows us to effectively demonstrate the benefits of incorporating real-world- like actions in CFE generation and sufficiently explore the impact of a larger action space (or action grid according to Ustun et al. (2019)), enabled by the high number of actionable features with broad value ranges. Alongside these real-world datasets, we also include 34 fully-synthelic agent agent-CFE datasets for comparison. We extensively compare the low-level CFEs with the hl-contimuous and hl-discrete CFEs, and provide an in-depth analysis of the performance of the data-driven hl-continuous, hl-discrete, and hl-id CFE generators under various settings (Sections and 6). Our code can be accessed here.
2 Background
We consider a binary classification setting, where an agent in the state E X' receives either a positive (desirable) or negative (undesirable) outcome under model f(x).. The state space X' consists of all valid agent states, each represented by a feature vector capturing attributes such as age and calcium The model operates over this space, and the CFE generator searches within it to identify alternative states with favorable model outcomes. Although we focus on binary classification, our data-driven CFE framework generalizes to other settings. Given an undesirable outcome, the CFE generator suggests actionable changes to move the agent at state x to a new state x' E X where the model prediction is desirable. Actionable recourse (Ustun et al., 2019) provides low-level CFEs that spocify featuro-level changes needed to reach such state.
The low-level CFE generator. Ustun et al. (2019) proposed an ILP-based low-level CFE generator (Equation 1) that generates a low-level CFE to help an agent change an undesirable model outcome to a desirable one.
min cost(a;))
s.t. f(x - a) = 0
(1)
a E A(x),
3

---

Published in Transactions on Machine Learning Research (06/2025)
where 8" is the desired model outcome, A(x) denotes the set of feasible actions given the imput x, and the function cost(:;x) A(x) + R4 encodes the preferences between these actions. When Equation is feasible, the optimal actions that modify the features (i.e., x + a) and lead to a desirable model outcome are recommended to the agent (cf. Figure 1). We refer the reader to Ustun et al. (2019) for a more detailed description and to Appendix B.2.1 for -ataset-specific experimental setup and supplementary examples of low-level CFEs.
Shortcomings of low-level CFEs and their generators. We note two limitations of low-level CFEs in comparison to proposed forms of recourse (hl-continuous, hl-discrete, and hl-id CFE), and two, we compare the low-level CFE generator to the proposed data-driven CFE generation approach (Barocas et al., 2020; Karimi et al., 2022; Verma et al., 2024).
First, low-level CFEs are feature- based and highly specific (e.g., in Figure 1, appl, calcium (mg): 309 - 113), which may overwhelm agents and introduce additional costs to translate the CFE into implementable steps. In contrast, our proposed CFEs are better aligned with real-world scenarios, offering what you see is what you get actionable insights (see Figure 1). Furthermore, with low-level CFEs, details about the actions the agent implements (the number of them needed, which features they would simultaneously modify, and costs to incur) are often unknown beforehand, potentially leading to a mislcading price of recourse and related metrics such as sparsity (few modified features) and proximity (closeness of final state to initial state) (Barocas et al., 2020).
Second, although a CFE (e.g., complete the Google AI residency program in Figure 1) could have been optimal for several agents with different but close profiles (e-g., one has coding experience: 4 and another 3), the low-level CFE being too specific, would give the agents different CFEs (coding experience: 4 - 5+ years and coding experience: 3 + 5+ years). In contrast, our proposed recourse generation approaches are both agent-specific (tailored to an agent's 's initial state) and generalizable (providing similar recommendations to agents with comparable profiles).
Lastly, while most low-level CFE generators operate on a single-agent basis (Karimi et al., 2022; Verma et al., 2024), recent work by Pedapati al. (2020); Rawal & Lakkaraju (2020); Kanamori et al. (2022); Ley et al. (2023) and Carrizosa et al. (2024) propose global low-level CFE generators capable of producing CFEs for multiple agents. However, these methods generate feature-based CFEs and require, at a minimum, query access to the classifier. In contrast, we propose data-driven CFF generation approaches that, given historical mappings between agents and their optimal CFEs, such as healthcare intervention records or high school counsclors' past successful college recommendations, can learn to generate CFEs with real-world-like actions for multiple new agents without requiring re-optimization. Additionally, the proposed data-driven approaches work well in settings where access to critical information, such as sufficient classification training data, classifier, or a comprehensive list of actions and their costs, is restricted or inaccessible.
3 The Proposed Single-agent CFE Generators
This section outlines the single-agent CFE generators for the proposed hl-contimnous and hl-discrete CFFs. Each generator relies on predefined, real-world-lik actions to solve optimization problems for CFE genera- tion: the weighted set cover problem for hl-discrete CFEs and ILP for hl-contimuous CFEs.
3.1 The Single-agen hl-continuous CFE Generation
Below, we formally define hl-continuous actions and the single-agent hl-contimnous CFE generation process for outcomes predicted by linear classification models.
Definition 1. (hl-continuous action): An hl-continuous action is a signed (+) and predefined real-world action whose cost and varied effects on an agent's input features are predefined and known. For example, in Figure 1, the hl-continuous action: ca1: take "leawening agents: cream of larlar" modifies 6+ features by a known amount and the agent incurs a cost (e.g., estimated average price in USD) that is known apriori.
The singe-agent hl-continuous CFE generator. This generator produces an hl-continuous CFE by solving integer linear program (IIP). Given the profile of negatively classified agent x and a set of
4

---

Published in Transactions on Machine Learning Research (06/2025)
hl-continuous actions with known costs (defined above), the objective is to identify the lowest-cost subset of hllcontinuous actions that, when taken, modify the agent's features to achieve positive classification. The ILP is of the form:
minimize
Σ
cost
jEJ
s.t.
c²
Σ Dj (2r, 1, -Cc"x +
(2)
JEJ
E (0,1), aj E (0,1), VjEJ
where J denoles the indices of the hl-continuous actions, with each action represented by vector Vj and with predefined cost, cost, E R4. The boolean variable aj indicates the inclusion (az 1) exclusion (aj = 0) of the jth hl-continuous action, while Cj encodes the sign of this action, reprosenting addition (c = 1) or subtraction (e, 0). The coefficients and intercept are the parameters of the linear classifier, and & is a small positive value that ensures striet inequality.
3.2 The Mingle-agent hl-discrete CFE Generation
Below, we formally define hi-discrete actions and the single-agent hl-discrete CFE generation process based a threshold classifier that determines feature cligibility.
Definition 2. (hl-discrete action): An hl-discrete action represents a binary vector that adds capabilities to specific features to meet the eligibility threshold. For example, consider the agent state x = 00,0, 0, 0, 1] and the hl-discrete action V, = [1, 1, O, O, 0j. When taken, the hl-discrete action adds capabilities to features I and 2 of x, transforming it to now state 1,,00 1]. Although we focus on binary actions, the formulation is extensible to more general cases.
The singe-agent hl-discrete CFE generator. This generator produces an hl-discrete CFE by solving a weighted set cover problem. Specifically, it identifies the lowest-cost subset of hl-discrete actions, cach with predefined cost, that negalively classified agent E {0, 1)" (e.g., someone deemed health risk) can lake to achieve a desirable classification (e.g., no longer classified as a health risk). The problem can be formally defined as follows:
minimize
Σ costraj
jEJ
Σ dgay + H, 2 ti E
s.t.
JEJ
a [0, 1}. d,s E {0, 1].
where J are the indices of the hl-discrete actions, each represented by a vector vj and with predefined cost:
cost, E R4. The threshold classifier t (t1,t2, (t.......... tn) over features classifies agent state x positive
if z; 2 t. Vi E [n], and negative otherwise. The binary variable aj denotes inclusion (a, = 1) exclusion (a: = 0) of the jth hi-discrete action, while dys indicates whether the jth hi-discrete action transforms (adds capabilities to) the feature i of the agent state x, ie., when performed, the new agent state x + V, = x' is such that x > H, and x/ 2 t.
The trained data- driven CFF generator
hl-continuous CFE hi-discrete CFE
Input: X
Output: CFF
hl-id CFE
Figure 2: Given an agent state (profile) x, data-driven CFE generator trained on instances of agents and their optimal CFEs (agent- CFF dataset) generates high-level CFE (hl-continuous, hl-discrete or hl-id) for the agent without the need for generator re-optimization access to the decision classifier.
5

---

Published in Transactions on Machine Learning Research (06/2025)
4 The Proposed Data-driven CFE Generators
This section, supplemented by Appendix C, dctails the three proposed data-driven CFE generators: hl- continuous, hilddisrret,, and hl-id (see Figure 2). Each generator learns from instances of agents and respective optimal CFEs, defined as the least cost CFE that leads to a favorable model outcome. Once trained, these generators can produce optimal CFEs for new agents without re-optimization. Empirical results demonstrate that even shallow deep-learning architectures perform strongly at this task, that is, generate correct CFEs, the least cost CFEs that favorably flip the model outcome.
The data-driven approaches are computationally more efficient than single-agent CFE generation approaches, which require optimization for each new agent. Furthermore, they are particularly favorable when agent- agent-CFE data is available or aggregatable from various sources. They allow recourse generation to operate indepen- dently of decision-makert function without direct access to the classifier, and handle scenarios where action costs and their explicit effects on features are unknown.
4.1 The Data-driven hl-continuous CFE Generator
We develop a data-driven hl-continuous CFE generator trained on agent hllcontinuous CFE training dataset, consisting of agents and their corresponding optimal hl-continuous CFFs. Each CFE defines a set of hl- continuous actions along with their associated costs. For instance, a CFE might include actions {action-a, action-b, action-c} with corresponding costs fcost-a, cost-b, cost-c). The generator learns from this data to produce hl-continuous CFEs for new agents without requiring generator re-optimization. Below are further details of the generator architecture.
Given the ggent-hl-contimmon CFE training dataset, we train a neural network model to learn to generate hllcontinuous CFEs for testing set agents. Specifically, the generator neural network model with three hidden layers, cach containing 2, 000 neurons. The model incorporates &2 regularization, dropout, and batch normalization. The training process uses the Adam optimizer (Kingma & Ba, 2015) with early stopping, restoring the best weights after a patience level of 300. The model trains with batch size of 6, 000 for an average of 5, 000 epochs. To ensure accurate data-driven hl-contimuous CFE generation for both training and testing set agents, we optimize the model loss function LиC given by:
M LHc Σ m==
Jajm log(ajm) + (1 - ajm) log(1 4jm))
(4)
where ājm is the predicted probability and ajm is the true indication of a presence (1) or absence (0) of the 5th hl-continuous action in agent m's hl-continuous CFE. There are possible hl-continuous actions and M agents in the agent-hl-comm holnn-hh--ontiuuuss CFE training dataset.
4.2 The Data-driven hl-discrete CFE Generator
We propose a data-driven hl-discrete CFE generator, trained using the agent CFE training dataset and evaluated on the agent CFE testing dataset. Each agent CFE dataset comprises instances of agents and their optimal hl-discrete CFEs that specify set of hi-discrete actions and associated costs.
Given the agent-hl-discree CFF training dataset, we design sequential encoder-decoder model to generate hl-discrete CFEs for new agents without generator re-optimization. The model configuration was dependent on the experimental setting. On average, we used 500 training epochs with a batch size of 128, a dropout rate of 0.4, a learning rate of 0.0005, and either the mean squared error or binary cross-entropy loss as the objective function. The models, on average, consisted of three layers, each using ReLU activation functions.
4.3 The Data-driven hl-id CFE Generator
The data-driven hl-id CFE generator supervised learning model trained on agent-hl-id CFE dataset of agent- pairs consisting of agents' initial states (profiles) and their corresponding optimal hl-id CFEs and associated costs. Fach hl-id CFF, as introduced in Section 1 and illustrated in Figure 1, is a single high-level

---

Published in Transactions on Machine Learning Research (06/2025)
action encapsulating all required changes without specifying individual feature modifications. When trained, the generator learns to generate CFFs for new agents without requiring re-optimization. Below, we provide details of the generator architecture.
Given the agent-hl-ic hl-id CFEs training dataset, we design the data-driven hl-id CFE generator as a neural network model with an average of two hidden layers, each consisting of 2000 neurons, & regularization, dropout, and batch normalization. We used the Adam optimizer (Kingma & Ba, 2015) and implemented early stopping and restoration of the best weights after patience level of 360. On average, we set the batch size to 2000 and the number of epochs set to 3000. To ensure that the data-driven hl-id CFE generator performs well on the training dataset and accurately generates hl-id CFEs for agents in the testing set, we optimize the model loss function LHID given by:
M Σ
LID
1
Σ
[akm log(ā
M
where ājm is the predicted probability and agm is the true indication of the wth CFE being the hi-id CFE (1) or not (0) for the mht agent. There are K possible hi-id CFEs and M agents in the training dataset.
5 Experimental Setup
This section provides detailed description of the experimental setup, including the evaluation metrics employed and the methodology for generating the 34 fully-synthetic agent CFE datasets and the 30 semi- synthetic agent-CFE datasets from real-world healthcare data sources: BRFSS, Foods, and NHANES.
5.1 Real-world Datasets
Below, we outline the extraction and preprocessing of the real-world datasets used in our experiments, including their statistical descriptions. We split all datasets into an 80/20 ratio for training and testing.
The Foods, BMI, and WHR datasets. We extracted the Foods dataset from USDA, Agricultural Research Service, Nutrient Data Laboratory (2016); Awram (2024) and the BMI (body mass index) and WHR (waist-to-hip) hip ratio) datasets from NHANES body measurement surveys (CDC, 1999; ICPSR at the University of Michigan, 2024), covering the years 1999 to pre-pandemic 2020.
To ensure commonality in actionability features between the (Foods, BMI) and the (Foods, WHR) dataset pairs, we selected intersectional nutritional intake features: protein (gm), carbohydrate (gm), dietary fiber (gm), calcium (mg), iron (mg), magnesium (mg), phosphorus (mg), potassium (mg), sodium (mg), zinc (mg), copper (mg), selenium (meg), vitamin C (mg), niacin (mg), vitamin B6 (mg), total folate (mag), vitamin B12 (meg); total saluraled fally acids (gm), lotal monounsaturrtted fally acids (gm), and lotal pokgunsuturated fatty acids (gm).
After preprocessing, for example, removing missing data and ensuring that selected nutritional intake fea- tures were a subset of the intersectional ones, the Foods dataset contained 3901 food items. Each item includes nutritional composition. We added two cost attributes: Monetary cost (in USD, obtained via web scraping) and Caloric cost (reflecting total caloric content, sourced from (Caputo, 2023)). In our experi- ments, costs+costs serves as the hl-continuous action space, where food items represent actions, and the cost attributes define the cost constraints. Based on cost type, we define two forms of hl-continuous actions: (i) monestyy cost and (ii) Foods+caloric cost.
The BMI dataset after preprocessing contained 50918 agents, each with 3 demographic features and 19 nutrient intake features, classified as either healthy (1) or unhealthy (0) BMI. On the other hand, the WHR dataset contained 9120 agents, each with 3 demographic features and 20 nutrient intake features, classified as either healthy (1) or unhealthy (0) WHR. For additional preprocessing details, see Appendix B.1.1.
The BRFSS dataset. We extracted the Behavioral Risk Factor Surveillance System (BRFSS) dataset from Teboul (2024): Centers for Disease Control and Prevention (2024). After preprocessing, e.g., removing missing data, the dataset was reduced to 13, 799 agents, each represented by 16 binary health risk features.
7

---

Published in Transactions on Machine Learning Research (06/2025)
These include: LowBP, LowChol, HealthBMI, NoSmoke, NoStroke, NoCHD, PhyssAtivitty Fruits, Veggies, RighhAlcohooConssunn, AnyHealtheare, DocbeCost, GoodGenHHth,, GoodMentHlthh, GoodPhysFHlhh and NoDij- fWolk. For additional details, see Appendix B.1.2.
5.2 Single-agent CFE Generation
Here and in Appendices 1. B.2.2, and B.2.3, we describe the generation of low-level, hil-oontinooss and hl-discrete CFEs using single-agent CFE generators (i.e., Equations 1, 2, 3). Since the agents and their computed CFEs will also be used to train and evaluate data-driven CFE generators, we generate CFEs for negatively classified agents in the BMI, WHR, and BRESS training and testing datasets.
For BMI and WHR datasets, only intersectional nutritional features are considered actionable, whereas all features are actionable for the BRESS dataset. We trained binary classifiers to identify agents requiring CFEs and identify classifier parameters to use in single-agent CFE generators. Fine-tuned logistic regression models for BMI and WHR achieved test accuracies of 78% and 85.18%, respectively. For the BRFSS dataset, which focuses on wellness checks, threshold classifier t = 116 achieved 100% accuracy.
The single-agent low-level CFE generation. We generated a low-level CFE for each negatively classified agent in the BMI, WHR, and BRFSS training/ /tsstintsttgggg datasots. We accomplished this by using the agent's 's initial state and the parameters of the trained binary linear decision-making classifiers for each dataset, along with the ILP framework defined in Equation 1.
The single-agent hl-continuous CFE generation. For the BMI and WHR training/testingg datasets, we use the negatively classified agents alongside two types of hl-continuous actions: moons-monetary costs and Foloric costs to create hllcontimuous CFFs. Using the ITP framework defined in Equation 2, we generate two distinct forms of hl-continuous CFEs for each agent: the optimal set of food items with minimal monetary cost and the optimal set of food items with minimal caloric cost.
The single-agent hl-discrete CFE generation. Lastly, using the BRFSS training/testing set agents, the threshold classifier (t =t= 116), and 100 synthetically generated hl-discrete actions (each of length 16) with associaled costs, applied Equation 3 to generate a hl-discrete CFE for each agent. Each CFE represents an optimal set of hl-discrete actions with minimal costs for each agent.
5.3 Data-driven CFE Generation
This section, along with Appendices B.2, B.3, and C, describe the creation of agent CFE datasets (where the CFE is either hl-contimuous, hl-discrete, or hl-id), and their role in data-driven CFE generation.
The semi-synthetic agent-CHi datascts. Using agent states and their optimal CFEs from Section 5.2, we construct training and lesting agent-CFF CFE datasels. First, we generate: agent- -hl-continuous CFE train/test datasets for BMI, 2 agent hl-continuous CFE train/test datasets for WHR, and agent hl-discrete CFE train/test datasets for BRFSS.
Then, given the following agent- CFE datasets: the -i-ootii--on CFE train/test datasets for BMT, created using monetayy costs as hl-continuous actions; the agent hl-continuous CFE train/test datasets for WHR, created using Foods+calorie costs as hl-continuous actions; and the agent hi-discrete CFE train /test datasets for BRFSS, we generate 3 gpenthh-idd CFE datasets. Specifically, for each agent- CFE dataset, we create a unique identifier for the CFE that denotes the single overarching action, resulting in an agent hl-id dataset corresponding to instances of agents and their hl-id CFE.
Lastly, for each of the agent CFF datasets described above, generated three variations based the frequency of CFEs in the dataset: all (includes all data), >10 (CFEs with more than 10 agents), and >40 (CFEs with more than 40 agents) varied frequency of CFEs agent CFE datasets.
The fully-syathetic agent- CFE datasets. We use the ILP defined in Equation to generate five variants of the agent-hl-discrete CFE datascts: varied dimensionality, frequency of CFEs, information access, feature satisifiability, and actions access. Below, we briefly describe the varied dimensionality and frequency of CFEs datasets and include more details about these and other variants in Appendix B.3.
8

---

Published in Transactions on Machine Learning Research (06/2025)
For varied dimensions agent C datasets, generated datasets with 20, 50, and 100 dimensions (action- able features), where we set the agent's feature to with probability Pf: and each discrete action can add capabilities to a feature with probability Lastly, we created three variod frequency of CFEs datasets: all, >10, and >40, and agent-hl-id CFE datasets for each varied dimensions agent CFE dataset, using similar approach as the semi-synthetic agent CFE datasets described above.
The cost of each action depends on the features it transforms.
Data-driven CFE generators. Given the semi-synthetic and fully-syuthetie training agent CFe datasets, train the corresponding data-driven generators described in Section 4 and evaluate their effectiveness the testing agent-CFE datasets. See Appendix C for supplemental details.
5.4 Evaluation and Comparative Analysis Metrics
We compare single-agent generated low-level CFEs to both hl-continuous and hl-discrete CFEs. Additionally, we assess the performance of data-driven CFE generators. The metrics used for comparison and evaluation are detailed below and in Appendix D.
Accuracy of data-driven generators. To assess the accuracy of the proposed data-driven CFE generators, use zero-one loss (see Equation 6), which checks if the generated CFE matches the true CFE I, defined the least cost CFE that. favorably flips the model outcome.
0 if if
Leval (1,f) - i
(6)
Comparison metrics. We analyze various factors related to the use of CFEs, including the average number of actions taken, the number of modified features, the proportion agents sharing the same optimal CFE, and the overall improvement measured as the distance between an agent's initial state and its final state after following a CFE. Assuming CFEs encourage truthful responses, we refer to this as agent improvement. We compare these factors when agents follow low-level CFE versus high-level CFF, either hl-continuous hl-discrete. The comparative analysis focuses CFÉs generated by single-agent CFE generators for negatively classified agents in the training sets of three datasets: BMI, WHR, and BRFSS. To ensure fair comparison, we include only agent-CFFs pairs where both low- and high-level CFEs are available, as the low-level CFE generator (ef. Equation 1) occasionally fails to produce a CFE.
To assess how much each variable, e.g. number of modified features varies across groups, for example, between male and female agents, we compute the coefficient of variations (Equation 7), normalized measure of dispersion calculated the ratio of the standard deviation to the mean of the variable
coefficient of variation(v) standard deviation,
x 100
mean,
6 Experimental Results
In this section, we provide comprehensive empirical evidence showcasing the strong performance of our data- driven CFE generators and their advantages over single-agent CFE generators. Furthermore, highlight the advantages of hl-continuous and hl-discrete CFEs, which offer actionable insights that closely align with real-world action spaces, over featuro-based low-level CFEs.
6.1 Comparison of low-level CFEs to the hl-continuous and hl-discrete CFEs
Below and in Appendices E.1 and E.2, we provide empirical evidence to show that, compared to low-level CFEs, both hl-continuous and hi-discrete CFEs involve fewer actions but lead to more improvement involving more modified features, are easier to personalize, and simplify the design and interrogation of CFE generators for fairness issues.
Fewer actions but higher improvement and more modified features. Our results indicate that hl-continuous and hi-discrete CFFs require fewer actions while yielding higher improvements and modi-
9

---

improvement #individuals/CFE Published in Transactions on Machine Learning Research (06/2025)
43,29
low-level CFE
hl-continuous CFE
Coefficient of variation (%)
40
12764.6
Average the variable
4484.5
10
20
12.88
10
27.5
10
15.9
10
9.1
9.1
low-level CFE
hd-continuous CFE
10
1.0
(b) Coefficient of variation in num- ber of modified features across inter- sectional (race, age, gender) groups
#actions
#features
Average of the variables
Figure 3: On the WHR dataset, (a) compared to low-level CFEs, to take hl-continuous CFEs require fewer actions but modify significant number of features and result in higher improvement. On average, each hl- continuous CFE is optimal for multiple agents, with high frequency of 27.5 per CFE, unlike the low-level CFEs with 1.0. Additionally, as shown in (b), thore is greater variability in the number of features modified across sensitive groups when using low-level CFEs, suggesting lower fairness than with hl-continuous CFEs. For more details on (a), see Appendix Figures 9, 10, and 11; for (b), see Appendix Figure 13(c) and 13(d).
fying more features than low-level CFEs. In contrast, low-level CFEs involve more actions but result lower improvements despite modifying high number features. For instance, while on average, on the WHR dataset, the hl-continuous CFEs require only 2 actions yet achieve significantly higher improve- ment (12, 765), the low-level CFEs involve 9 actions but yield much lower improvement (4, 484.5) (see Figure 3(a)).
In low-level CFE generation, sparsity (small number of modified features) and proximity (new agent state after taking the CFE close to the initial state) are often a primary goal due to actionable insights being part of the feature space (Ustun et al., 2019; Verma al., 2024). We observe perfect positive correlation between the number of modified features and actions taken (Kendall's correlation between actions taken and improvement achieved in low-level CFEs (Kendall 5.41e-227). However, for hllcontinuous CFEs, the correlation between the nunber of modified features and actions taken is positive but weaker (Kendall's T = 0.722, p-value = 0.0) and there is almost no relationship between the number of actions taken and improvement achieved (Kendall t = 0.0625, p-value = 3.21e-06). Despite requiring fewer actions (2), hl-contimuous CFEs modify significantly more features (16) compared to low-level CFEs, which involve actions and modify features (Figures and 3(a)).
1.0, p-value = 0.0) and positive 0.368, p-ealue-
Consequently, unlike taking low-level CFEs, agents using hllcontinuous or hi-discrete CFEs tend to become more "positive" or "qualified" after taking fewer actions. That is, proposed form of CFEs require fewer actions but result in greater improvement, broader feature modifications, and lower costs in both interpretation and execution (see Figures 1 and 3(a), and Appendix E.1 and Figures 9, 10, 11 and 12).
Personalization and fairness. Since both hl-discrete actions and hl-continuous actions are predefined and real-world-like, it easier and more transparent to examine the hi-ddiscrete and hl-continuous CFE gencrators and generated CFEs for potential fairness issucs, and tailor the CFE generation agents' needs. For example, our data-driven hi-continuous CFE generators can produce CFEs for agents who place greater importance monetary costs over caloric costs.
Morcover, agents across the interscctional sensitive (race, age, and gender) groups (c.g., Hispanic, 21-40, Female) take a comparable number of actions, modilying a closely similar number of features, achieving comparable improvements, and incurring closely similar costs when using hi-contimuous CFEs, as indicated by the low coefficient of variation Figure 3(b) and Appendix Figures and 14. In contrast, taking low-level CFEs results in higher variability across the groups in all these variables (see Figure 3(b) and A Appendices F.2.2 and F.2.1). Thus, high-level CFFs yield fairer outcomes than low-level CFFs.
10

---

Published in Transactions on Machine Learning Research (06/2025)
Accuracy of CFE generators
Effect of frequency of CFEs
hl-continuous
hl-discrete
hl-id
all
>10
>40
BMI
0.92 1 0.0053 0.92 1 0.0176
0.94 0.0045
20-dim
0.84 0.0060
0.89 0.0052
0.94 1 0.0042
WHR
0.97 0.0107
20-dim
0.97 1 0.0028
0.98 0.0021
0.99 1 0.0014
BRFSS
0.98 = 0.0102
0.99 + 0.0050
BMI
0.90 = 0.0057
0.91 0.0055
0.92 : 0.0053
20-dim
0.94 + 0.0042
0.99 + 0.0014
BRFSS
0.70 + 0.0182
0.86 + 0.0158
0.98 : 0.0102
Table 1: (Ieft) The accuracy of the hl-continuous, hl-discrete, and hl-id data-driven CFE generators on the testing set agents for >40, BMI, BRFSS, WHR, and fully-syuthetic (20-dim): 20-dimensional, datasets. (right) The data-driven CFE generators' accuracy decreases with a decrease in the frequency of CFEs (number of agents for whom given CFE is optimal) in the agent CFE training set, regardless of the dataset type. Specifically, training and testing on the (20-dim): the 20-dimensional agent-hh-ddisceft CFE dataset, (20-dim)*: the 20-dimensional agent hl-id dataset, (BMI): the BMI agent bllcontimuous CFE dataset, and (BRFSS): the BRFSS agent-hl-discrets CFE datasct all show this trend. The data-driven CFE generators are accurate (Ieft) and their accuracy improves as the frequency of CFEs increases, with those trained highest CFE frequency dataset (>40) performing best (right).
Lastly, our data-driven hl-discrete CFÉ generators effectively generate CFEs for all agents, regardless of action restrictions or feature satisfiability variations. This strong performance holds even without explicit knowledge of these variations (see Appendices E.2.3 and E.2.4).
6.2 The Data-driven CFE generators are Accurate and sesource-efficienn
Our results show that the proposed data-driven CFE generators are resource-efficient in terms of both limited information access and low computational overhead. These generators, without requiring re-optimization, accurately and efficiently generate CFEs for new agents after being trained under restrictive constraints such as no query access to the classifier or, in the case of the data-driven hl-id CFE generator, without knowledge of the cost and impact of actions on agent states. Refer Table (lleft) and Appendices E.3, E.4, and E.5).
In contrast to the overly specific low-level CFEs, which are generally unique to each agent, hl-continuous and hl-discrete CFFs are often optimal for a broad range of agents (refer to Figures and 3(a) and Appendix Figure 18). The removal of the need for re-optimization for each new agent, combined with the general applicability of the actions to agents, enhances the scalability of our proposed CFE generators compared to low-level generators. Additionally, because the actions in the hil-continouss and hl-discrete CFFs are both general and predefined, they are more transparent and easier interpret (see Figure 1), making them cheaper and more desirable than the overly specific and unique low-level CFEs.
However, our results show that the accuracy of the proposed data-driven generators declines with the low frequency of CFEs (sce Table 1 (right)) and the generalizability of CFE generation decreases with an increase in the number of actionable features. We observed that this is due to the growing uniqueness of CFEs to agents (see Table 1 (right)) and Section E.6). Data augmentation mitigates the negative effects of low CFE frequency. For instance, on the all 20-dimensional dataset, data augmentation improves accuracy from 0.969 to 0.982.
Lastly, the performance of data-driven CFE generators improves as model complexity increases. For example, a discrete agent hllid dataset, the neural network model outperforms the Hamming distance method (see Appendix Figure 20). Future rescarch could explore more advanced data-driven models for CFE generation and techniques like federated learning to enable CFE generation under limited access to agent-CFF data and privacy constraints.
7 Limitations and Ethical Considerations
The decision-maker must have access to data on instances of agents and their corresponding optimal CFEs to train the proposed data-driven CFE generators. Although this level of access mitigates some information access challenges, such needing at least. query access to the classifier and representative prediction training
11

---

Published in Transactions on Machine Learning Research (06/2025)
data or having an exhaustive list of actions and the associated costs, obtaining a historical agent-CFE dataset may still pose significant challenges. Future research could investigate techniques like federated learning and secure multi-party computation to facilitate collaborative training of robust CFE generators under varied privacy and data access constraints.
While our proposed data-driven CFE generators are agnostic to the underlying classífication model, the proposed single-agent hl-continuous CFE generators rely on linear classifiers. Although the linear models may not always be optimal, they can outperform non-linear models in some contexts (Wainer, 2016). When non-linear classifiers are preferred, one practical alternative is to approximate them by linear models (e.g., (Bshouty & Long, 2012; Li, 2015; Shalizi, 2020; Liu et al., 2021)), enabling CFE generation with the proposed single-agent CFE generator. Extending the generator to produce exact CFEs for non-linear models in a scalable and computationally efficient manner remains a challenging but promising avenue for future research.
Additionally, our formulations of hl-continuous and hi-discrete CFEs restrict them to being defined as a set of actions. More generally, one could consider settings where the order of actions matters, such as where CFE corresponds to an optimal policy for an agent in deterministic Markov decision process (MDP). Further, one could consider actions whose effects are stochastic, and CFE then corresponds to optimal policy for the agent in a general MDP.
Since the proposed approaches to data-driven CFE generation are closely related to data-driven algorithm design, ethical concerns related to data-driven algorithms, e.g., potentially propagating and exacerbating biases in historical agent-CFE data and the potential for flawed resource allocation, might apply our proposed CFE generators. Future research should investigate these ethical implications in greater depth.
Although our experiments primarily use healthcare datasets, our data-driven CFE generation approach generalizes to a broad spectrum of real-world scenarios, such as college admissions, loan applications, judicial systems, and other settings. Future works could expand our setup to other data settings and informational access challenges. Lastly, we caution readers that the experimentally generated CFEs from our empirical analyses are intended solely for illustrative purposes, and readers should not use them for self-trealment.
8 Related Work
The proposed single-agent hl-continuous and hildiscrete CFE generation approaches are in principle, similar to search-based optimization CFE generation frameworks (Ramakrishnan et al., 2020), single-agent ILP recourse generation approaches (Cui et al., 2015; Gupta et al., 2019; Ustun et al., 2019), and CFE generation methods based on logic and answer-set programming (Bertossi, 2020; Liu & Lorini, 2023; Marques-Silva, 2024). However, unlike these approaches, ours uses predefined real-workd-like actions (see Figure 1), resulting in CFFs that involve fewer actions but modify more features and lead to more improvement.
Although most low-level CFE generators operate on single-agent basis (Karimi al., 2022; Verma et al., 2024), recent studies (Pedapati et al., 2020; Rawal & Lakkaraju, 2020; Kanamori et al., 2022; Ley et al., 2023; Carrizosa et al., 2024) have introduced approaches that can produce CFFs for multiple agents. Most closely related to our work is the approach by Kanamori et al. (2022), which learns a decision-tree- global CFE generator that, once learned, can generate CFEs for multiple agents. However, unlike Kanamori et al. (2022), our data-driven CFE generators generate CFFs with real-world-like actions. Moreover, solving the mixed-integer lincar programs and the overall Counterfactual Explanation Tree can be computationally prohibitive for scenarios with large action spaces, making our supervised learning approach a more scalable and efficient alternative.
Unlike low-level CFE generators that require, at a minimum, query access to the classifier and knowledge of the cost and impact of each action on stale features (Shavit & Moses, 2019; Pedapati et al., 2020; Rawal & Lakkaraju, 2020; Naumann & Ntoutsi, 2021; Verma et al., 2022; Kanamori et al., 2022; De Toni et al., 2023; Ley al., 2023; Carrizosa et al., 2024), without explicit access to this information, our data-driven CFE generators leverage access to agents and their optimal CFEs to generate CFEs described by real-world-like actions and costs.
12

---

Published in Transactions on Machine Learning Research (06/2025)
While in some ways, the proposed data-driven CFE generators similar to reinforcement learning-basee CFE generation tools (Shavit & Moses, 2019; Naumann & Ntoutsi, 2021; De Toni et. al., 2023), our proposed approach offers a more resource-efficeet and exact solution alternative to the often high computational and approximate solutions. Notably, our approach is closest to that of Verma et al. (2022). While our method is akin to learning an optimal policy in a large but deterministic family of Markov decision processes (MDPs); Verma et al. (2022) focuses on learning optimal policies within smaller, stochastic MDP settings.
Finally, our work also relates to data-driven algorithm design (Gupta & Roughgarden, 2016; Balcan et al., 2018; Balcan, 2020), where models learn from training data instances to generalize to the testing data. We introduce novel data-driven CFE generators that address the question: Can we, by learning from training agent CFE dala (i.e., instances of agents and their optimal CFEs), develop a CFE generalor that quickly provides optimal CFEs for new agents? Our proposed approach excels in generating CFEs for new agents, is computationally efficient and scalable, and functions effectively under varied informational access settings.
9 Conclusion
In this work, we propose three forms of recourse where actionable insights align closcly with real-world actions and investigale settings where CFEs can be generated by analyzing the similarities ween nega- tively classified agents using data-driven approaches. Our findings show that compared low-level CFEs, both hl-continuous and hl-discrete CFEs require fewer actions, modify more features, and result in higher improvements. Additionally, the CFEs are fairer across sensitive groups and are easier to examine, compare, and personalize than low-level CFFs. Lastly, we empirically show that the proposed data-driven CFE, gen- erators are accurate, resource-efficient, and perform effectively under various information access constraints, including limited or restricted access to classifier parameters and training data.
Acknowledgments
We are deeply grateful to Alexandra DeLucia for their insightful and comprehensive feedback all drafts of the manuscript. We would also like to thank Harvineet Singh and the anonymous reviewers for their valuable feedback on our work. This work was supported in part by the National Science Foundation under grants CCF-2212968 and ECCS-2216899, and by the Simons Foundation under the Simons Collaboration on the Theory of Algorithmic Fairness.
References
Akshay Agrawal, Robin Verschueren, Steven Diamond, and Stephen Boyd. A rewriting system for convex optimization problems. Journal of Control and Decision, 5(1):42 60, 2018. 19
Chris Awram Food Nutritional Values Dataset, 2024.
URL
https://data.worldd
food-natritionall-allee 7, 19
Maria-Florina Balcan. Data-Drived Algorithm Design. In Tim Roughgarden (ed.), Beyond the Worst-
Analysis of Algorithms, pp. 626 645. Cambridge University Press, 2020. doi: 001007799881086337435. URL https: http:////oooooo...///o wtw.:////./////........ /11////0010///78811888778883 036. 13
Maria-Florina. Balcan, Travis Dick, and Colin White. Data-Driven Clustering via Parameter- ized Lloyd's Families. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grau- man, Nicolò Cesa-Bianchi, and Roman Garnett (eds.), Advances in Neural Information Process- ing Systems, pp. 10664-10674, 2018. URL https: / http://oooeeeiss..eeeee nourips. hwtrippscccpappe/0188//// 008a00-4773027744444444444400000 Abstract......t..........aa..... html. 13
Solon Barocas, Andrew D. Selbst, and Manish Raghavan. The hidden assumptions behind counterfactual explanations and principal reasons. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT "20, pp. 80-89, New York, NY, USA, 2020. Association for Computing Machin- ery. ISBN 9781450369367. doi: 10.1455/3551995.372888 URL https: otceiioorg/.... 1145/331095. 3372830. 4
13

---

Published in Transactions on Machine Learning Research (06/2025)
Leopoldo Bertossi. An ASP-Based Approach to Counterfactual Explanations for Classification. In Rules and Reasoning: 4th International Joinl. Conference, RuleMM.+RR 2020, Oslo, Norway, June 29 July 1, 2020, Proceedings, pp. 70 81, Berlin, Heidelberg, 2020. Springer- Verlag. ISBN 978-3-030-57976-0 doi: 10.1007/978 9-779888--0000777777777777 URL attps://doi.. org/ 1007/9..100//9 --77778-7-00--77777777777--- 12
Nader H. Bshouty and Philip M. Long. Lincar classifiers are nearly optimal when hidden variables have diverse elfects. Machine Learning, 86(2):209 231, February 2012. ISSN 1573-0565. doi: 10.1007/
s10994-011-5262-7 URL https:///oi.o. 10. http://.....r.110110100000940000
12
Rocco Caputo. Calories Ledger Food Prices, 2023. URL https://ggtthbb.cmmmcaappp
caloorieeladgeer/oooo//ssseeeeee prices. Accessod: 2024-08-01.
Emilio Carrizosa, Jasone Rambrez-Ayerbe, and Dolores Romero Morales. Generating collective counterfactual
explanations scoro-based classification via mathematical optimization. Expert Systems with Applications: An International Jourmal., 238(PD). March 2024. ISSN 0957-4174. doi: 00.1016/j.cswaa22223 1219.1 URL
https:://doii.or////// j016/ eswa. 2023. 121954.
12
CDC. National Health and Nutrition Examination Survey (NHANES).
nhanes/Defauut..spx. 1999. Accessed: 2024-08-01.
Centers for Disease Control and Prevention. Behavioral Risk Factor Surveillance System (BRFSS). https:
//www. wwwgle. cessed: 2024-08-01. 7
sys.a///eesstttecoooeeeeee 2024. Ac-
Zhicheng Cui, Wenlin Chen, Yujic He, and Yixin Chen. Optimal Action Extraction for Random Forests and Boosted Trees. In Proceedlings of the ACM SIGKDD International Conference Knowledge Discovery and Data Mining (KDD), pp. 179-188, 188, 2015. ISBN 9781450336642. doi: 10. 10.1145/778325822788881 URL https://doi. org/10../... 1111//113///.1..1 2783281. 12
Susanne Dandl, Christoph Molnar, Martin Binder, and Bernd Bischl. Multi-Objoctive Counterfactual Explanations. In Proceedings of the International Conference on Porallel Problem Solving from Na- ture (PPSN), pp. 448-469, 1698-49,,60 2020. ISBN 978- 978-3-030-58111-4 doi: (00007998-3-000-58888 31. URL
attps://áoi.ow oot...rgg////. 1007
2
Giovanni De Toni, Bruno Lepri, and Andrea Passerini. Synthesizing Explainable Counterfactual Policies for Algorithmic Recourse with Program Synthesis. Machine Learning, 112(4):1389 1409, 2023. ISSN 0885-6125. doi: 0..1077/1109944-02-002299 URL https://doi. ott.://...000//00044-----0009990 12, 13
SciPy Developers. seipy.slatskkendalaau, SciPy v1.11.3 Manual. https://àocs wwwars.ccpppooo crriipy.org/oo//ccpp///// reference//generated scipy. stats. kendalltau. htm,, 2023. Accessed: 2024-08-01. 28
Steven Diamond and Stephen Boyd. CVXPY: A Python-embedlded modeling language for convex optimiza-
tion. Journal of Machine Learning Research, 17(83):1-5, 2016. 19
European Parliament and Council of the EU. Regulation (EU) 2016/679 of the European Parliament and of the Council, 2016. URL https: /dtta. httpa////aaaaa.uup...eeoooo ouzo2a..v//e1/rrg/0066///////
Europcan Union. The EU Artificial Intelligence Act, Up-to-date developmennss and analyses of the EU AI Act, 2025. URL
http://ccccccceeeeaaaaccee////// Accessed: 2025-04-11. 2
Rishi Gupta and Tim Roughgarden. PAC Approach to Application-Speeific Algorithm Seloction. In Proceedings of the ACM Conference on Innovations in Theoretical Computer Science (ITCS), pp. 123-134 2016. ISBN 9781450340571. doi: 00.1155/8447288.840777 URL https://doi. 1ttp//00.11////47222222 2840766. 13
Vivek Gupta, Pogah Nokhiz, Chitradeep Dutta Roy, and Suresh Venkatasubramanian. Equalizing Recourse across Groups. ArXiv, abs/ 0bs/1999.03366 2019. URL https://apps semanticscholas httpaaaiiiicccaaaooooooorrppllaa
202537227. 12, 31
14

---

Published in Transactions on Machine Learning Research (06/2025)
ICPSR at the University of Michigan. Current Population Survey Scries. https://www..cpsr umich. edu/ web/IcPSR/series/S5 2024. Accessed: 2024-08-01. 7
Junqi Jiang, Francesco Leofante, Antonio Rago, and Francesca Toni. Robust counterfactual explanations in machine learning: survey. In Proceedings of the Thirty- Third International Joint Conference Artificial Intelligence, IJCAI '24, 2024. ISBN 978-1-956992-041. doi: 1144663/jcai.222// URL
https://doi.org: 24963/ijcaa 2024/894. 2, 27
Shalmali Joshi, Oluwasanmi Koyejo, Warut Vijitbenjaronk, Been Kim, and Joydeep Ghosh. Towards Re- alistic Individual Recourse and Actionable Explanations in Black- Box Decision Making Systems. arXiv preprinl ar Nio:1907.0961, 2019. URL http://arxiv. http://aaxiiv.orr/b////////
wtgp/b///////.....////// htt.ab/9.19.91....... 2
Kentaro Kanamori, Takuya Takagi, Ken Kobayashi, and Yuichi lke. Counterfactual Explanation Trees: Transparent and Consistent Actionable Recourse with Decision Trees. In Gustau Hamps-Valle Francisco J. R. Ruiz, and Isabel Valera (ods.), Proceedings of The 25th International Conference on Artificial Intelli- gence and Slafistics, volume 151 of In Proceedings of Machine Learning Research, pp. 1846 1870. PMLR, March 2022. URL https: /ttp::////cccce.....m.m .mlg....r.... wttpa///111//aaaarm222222 html. 4, 12
Amir-Hossets Karimi, Bernhard Schölkopf, and Isabel Valera. Algorithmic Recourse: from Counterfactual Explanations to Interventions. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transporency, FAceT '21, pp. 353 362, New York, NY, USA, 2021. Association for Computing Machin- URL https://doi.org: 1144/3441/88
ery. ISBN 9781450383097. doi: 10.1145/4442188.3442 3445899. 2
Amir-Hossein Karimi, Gilles Barthe, Bernhard Schölkopf, and Isabel Valera. A Survey of Algorithmic Recourse: Contrastive Explanations and Consequential Recommendations. ACM Computing Surveys, 55 (5), 2022. ISSN 0360-0300. doi: 10. 10.1145/3527848. URL https://doi. 1145/3527748.. 2, 4, 12
Diederik P. Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In Yoshua Bengio and Yann LeCun (cds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL 6.7
Dan Ley, Saumitra Mishra, and Daniele Magazzeni. GLOBE-CE: translation based approach for global counterfactual explanations. In Proceedings of the 40th International Conference on Machine Learning,
JCML'23. JMLR.org, 2023. 4, 12
Ping Li. Min-Max Kernels, 2015. URL /ttp:::/aap..sematttie ottp://wwwnnaitttooooooooorrrraa
12
Qiang Liu, Zhaocheng Liu, Haoli Zhang, Yuntian Chen, and Jun Zhu. Mining Cross Features for Financial Credit Risk Assessment. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, CIKM '21, pp. 1069 1078, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450884469. doi:
3482371.
URL https:/ //oii http:///oo.ooo..../.// 1145/
0459637.348233 12
Xinghan Liu and Emiliano Lorini. A unilied logical framework for explanations in classifier systems. Journal of Logic and Computation, 33(2):485-5)) 515, 2023. doi: 10.1093 /log////oo/////oo /exgco0/eeac122. exac112. 12
Joao Marques-Silva. Logic-Based Explainability: Past, Present and Future. In Leveraging Applications of Formal Methods, Verification and Validation. Software Engineering Methodologies: 12th International Symposium, ISoLA 2024, Crete, Greece, October 27 31, 2024, Proceedings, Part 1V, pp. 181 204, Berlin, Heidelberg, 2024. Springer-Verlart Verlag. ISBN 978-3-081-75386-22 doi: 10.1007/9978-3-031--53887 12. URL https:///oi..rgg/ 10.
12. 12
Ramaravind K. Mothilal, Amit Sharma, and Chenhao Tan. Explaining machine learning classifiers through diverse counterfactual explanations. In Proceedings of the 2020 Conference on Fairness, Accountabllity, and Transparency, FAT* '20, pp. 607-617, New York, NY, USA, 2020. Association for Computing Machin- URL https: http:///doooooo.o///// 1145/3351999
ery. ISBN 9781450369367. doi: 3372850. 2
15

---

Published in Transactions on Machine Learning Research (06/2025)
Philip Naumann and Eirini Ntoutsi. Consequenco-Aware Sequential Counterfactual Generation. In Pro- ceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML PKDD), pp. 682 698, 2021. ISBN 978-3-030--6519--1 doi: 10.1007/ 078-3-030-86520-77 978-3003--86220777ttttumddde textunderscoro... URL http:://dl..org/1... 0t0.:////......00/0700-----000-0
12, 13
Martin Pawelczyk, Chirag Agarwal, Shalmali Joshi, Sohini Upadhyay, and Himabindu Lakkaraju. Exploring counterfactual explanations through the lens of adversarial examples: theoretical and empirical analysis. In International Conference on Artificial Intelligence and Statistics, pp. 4574-4594. PMLR, 2022.
Tejaswini Pedapali, Avinash Balakrishman, Karthikeyan Shanmugan, and Amit Dhurandhar. Learning global transparent models consistent with local contrastive explanations. In Proceedings of the 94th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Goutham Ramakrishnan, Yun Chan Lee, and Aws Albarghouthi. Synthesizing Action Sequences for Modify- ing Model Decisions. In Proceedings of the AAAI Conference on Artificial Intelligence, 34(00))5540-54400 Apr. 2020. doi: 10.1609 /aaaaaaiii.... v34004..99 view/5996. 12
URL https://oje aaai. aetai..or//ineee
Peyman Rasouli and Ingrid Chieh Yu. CARE: coherent actionable recourse based on sound counterfactual
explanations. International Journal of Data Science and Analytics, 17(1):13 38, April 2024. ISSN 2364- 4168. doi:
000-10000002200005----1 2 022-00365-66 URL https://doi.org/10..000 0ttp:////........//////000000000
Kaivalya Rawal and Himabindu Lakkaraju. Beyond individualized recourse: interpretable and interac- tive summaries of actionable recourses. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20 Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN
9781713829546. 4, 12
Cosma Shalizi. Extending Linear Classifiers with Nonlinear Features (Lecture 8). https://www. stat. cmu. och/-cshalizi/dhe http://aa//a////eeeeeee///////ee Spring 2020. 12
html, February 2020. Lecture notes for 04-462/662
Yonadav Shavit and William S. Moses. Extracting Incentives from Black-Box Decisions. orXiv preprint arXiv:1910.0566 2019. URL http://arxiv.. wwtggaa///1/.....//oo 12, 13
Alex Teboul. Diabetes Health Indicators Dataset. https://ww. httgl//wwwwgg.ee.eeee// comm/ataaett//leetebblll
2024. Accessod: 2024-08-01. 7, 20
USDA, Agricultural Research Service, Nutrient Data Laboratory. USDA National Nutrient Database for Standard Reference, Release 28 (Slightly Revised). httt::/www.cr ars. www.da...uddaa............. usda. gov/ba./hnrc/nd, May 2016.
Version Current: May 2016. 7
Berk Ustun, Alexander Spangher, and Yang Liu. Actionable Recourse in Linear Classification. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* '19, pp. 10-19, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450361255. doi: URL https: http:///doo.o//...///// 10000..145/// 10111111113151 3287566. 2, 3, 4, 10, 12, 21, 27
Sahil Verma, Keegan Hines, and John P. Dickerson. Amortized Generation of Sequential Algorithmic Re- courses for Black-Box Models. In Proceedings of the AAAT Conference on Artificial Intelligence, 36(8): 8512 8519, June 2022. doi: 10.1609/ ///aaaaaiv.33.. v36i8.20824 URL https://ojs. http:////.aaaaa.......aaaa aadi..or//ndee. whppppppptt.a.a
article/viee/20828. 12, 13
Sahil Verma, Varich Boonsanong, Minh Hoang, Keegan Hines, John Dickerson, and Chirag Shah. Counterfac- tual Explanations and Algorithmic Recourses for Machine Learning: A Review. ACM Computing Surveys., 56(12), October 2024. ISSN 0360-0300. doi: 10.1145/3677110 URL https://dot http:///oo...oo///// org/10. 1145/3677199 2, 4, 10, 12
16

---

Published in Transactions on Machine Learning Research (06/2025)
Sandra Wachter, Brent D. Mittelstadt, and Chris Russell. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR. Harvard Journal of Law and Technology, (2): 841 887, 2018. 2
Jacques Wainer. Comparison of 14 different families of classification algorithms 115 binary datasets. CoRR, abs/1606.00930, 2016. URL http:///arav..orraab///600000000 12
WebMD. Body BMI Calculator. 2024-08-01. 19, 20
2024. Accessed:
Wikipedia contributors. Waist hip ratio Wikipedia. 93hip_ratio, 2024. Accessed: 2024-08-01. 19
17

---

Published in Transactions on Machine Learning Research (06/2025)
A The hl-discrete CFE vs. hl-continuous CFE: Supplementary Details
We further claborate on the distinction between hl-contimuous and hl-discrete CFEs and potentially inter- esting future direction.
Diffcrentiation between hl-discrete and hl-contimuous CFEs. The key distinction between hl- discrete and hl-continuous CFEs how they affect features, where the former caters to feature eligibility and the latter numerically modifies feature values. For instance, consider the feature healthBMT in App2, Figure 1. While hl-discrete CFE would ensure BMI past desired threshold, thus ensuring healthBMI 1, the hl-contimuous CFE would affect the actual feature values, such reducing BMI from 30 to 22.3.
It is important to note that threshold defining feature eligibility can stem from any classifier type. Contin- uing with the healthBMI example, the binary eligibility (40, is determined based on whether the feature crosses a predefined threshold what qualifies as a healthy BMI, as dictated by the classification model's set threshold.
These two forms of CFEs are suited for different contexts. The hl-discrete CFEs are particularly effective in rule-based systems, such as eligibility checks, wellness evaluations, or quality and safety assessments. In contrast, hl-continuous CFEs are better suited for scenarios where fine-grained numerical adjustments are meaningful, typically in settings where low-level CFEs are effective.
While it might be easier to go from hl-contimoous CFEs to hl-discrete CFEs, the reverse is less trivial and may require additional considerations.
Integration of high-level actions into existing CFE generation methods. While may be feasible incorporate the hl-continuous or hl-discrete actions into existing CFE generation methods workflows, it's currently unelear which methods are best suited, how readily they be adapted, or what challenges might emerge. For example, in evolutionary algorithm-based approaches, generating exact CFEs under this new paradigm could be computationally intensive, particularly in large action spaces, thus potentially only generating approximate or Pareto-optimal CFEs. We hope future research explores how to effectively adapt existing CFE generation methods to this new paradigm and uncover any associated interesting challenges.
18

---

Published in Transactions on Machine Learning Research (06/2025)
B Datasets: Supplemental Details
This section describes the supplemental details about the datasets used in the experiments. We conducted all experiments on laptop with a CPU featuring the following hardware specifications: a 2.6 GHz 6-Core Intel Core 17 processor, 16 GB of 2400 MHz DDR4 RAM, and an Intel UHD Graphics 630 with 1536 MB of video memory. In all cases where we implement Equations 1, 2 and 3, we use the CVXPY Python package (Diamond & Boyd, 2016; Agrawal et al., 2018).
B.1 Real-world Datasets Extraction and Preprocessing
First, we describe the extraction and preprocessing of real-world datasets: Foods, BMI, WHR, and BRFSS. Then, we describe the creation of semi-synthetic agent-hl-contimoous CFE, agent-hl-discrete CFE, and agent hl-id CFE datasets.
B.1.1 Foods, Body Mass Index (BMI), and Waist-to-Hip Ratio (WHR) Datasets
Intersectional nutritional features. After extracting the datasets for Foods, BMI, and WHR and re- moving features with missing values in the Foods dataset, we selected an intersectional subset of nutritional value uratures in the Foods and BMT datasets and the Foods and WHR datasets. This subset consisted of 20 features, including: 'protein (gm)', 'carbolyyraat (gm)', 'dietary fiber (gm)', 'calcium (mg)', " 'tron (mg)', "magnesium (mg)', 'phosphorus (mg)', 'potassium (mg)', 'sodium (mg)', 'zine (mg)', 'copper (mg)', 'sele- nium (meg)', 'vilamén C (mg)'; 'niacin (mg)', 'vilamin B6 (mg)', 'tolal folate (meg)', 'vilamin B12 (mag)'; 'total saturated fatty acids (gm)', 'total monounsaturated fatty acids (gm)², and 'total polyunsaturated fatty acids (gm)'.
Foods dataset preprocessing. The Foods dataset from Awram (2024) initially contained 53 features.
After finding the intersectional subset of nutritional value features and removing datapoints with missing values, the dataset had 27 features. These included the following: 'NDB No', 'Shrt Desc', 'Gm Wt. 1', [Refuse_Pat', Pet', along with the 20 nutritional features de- scribed above. To add costs to the dataset, we web-scraped the average USD prices and extracted caloric prices for cach food item given their name specified in the 'Shrt Desc feature. Out of 3901 food items, we successfully extracted USD prices for 3871 food ilems and caloric prices for 3125 food items. Therefore, when using USD prices as costs, there were 3871 possible hl-continuous actions, while using caloric prices mcant 3125 possible hl-continuous actions.
'Gm "Dfs__ 'Gm Wt__', 2", 'Gm Wt_ DDsc_' and 'Refuse_
BMI dataset preprocessing. The body mass index (BMI) dataset originally had 57 features. After removal of features with at least 20% null values and selecting the above nutritional features, except the feature 'total folate (mag)', we had 23 features including: 'gender', 'age', 'race', and 'body mass inder (kg/m** **2)' We selected agents whose age greater than equal to 20 at the time of surveys. Using the features 'body mass inder (kg/m***) for each agent as either healthy (1) BMI or unhealthy (0) (WebMD, 2024). We then removed the feature 'body mass inder (kg/m **2)' and all the duplicates datapoints. At the end of data preprocessing, we did the 80/20 train/test data split resulting in 40734 data points in the predictive training set and 10184 in the predictive testing set.
and 'age', we computed the target label (binary class variable)
WHR dataset preprocessing. Unlike the BMI dataset, there were fewer datapoints with 'waist-to-hip hip ratio' (WHR) information among the NHANES body measurement surveys (for years 1999 to prepandemic 2020) we scraped. First, we removed all features with at least 20% null values. Then using the features 'waist circumference (em)', 'hip cireumference (em)' and 'gender', we created the binary class variable whr-class (Wikipedia contributors, 2024), indicating healthy (1) or unhealthy (0) WHR. After preprocessing, we had 23 features, including the 20 nutritional features described above and the demographic features: 'gender', 'age', and 'race'. Lastly, we removed the duplicates and split the dataset 80/20, creating 7296 data points in the predictive training set and 1824 in the predictive testing set.
19

---

Published in Transactions on Machine Learning Research (06/2025)
B.1.2 Behavioral Risk Factor Surveillance System (BRFSS) Dataset
The initial BRESS dataset comprised 253680 rows and 22 features, each detailing various health and demo- graphic attributes of agents (Teboul, 2024).
First, we removed all data points where 'Age' = denoting an age range of 18-24 bocause computation a new variable which relied on age being equal The new variable was called BMT, an adult health BMT classification value (WebMD, 2024) from the feature ' 'BMI. Next, we transformed the existing features, which were predominantly binary, into new features where the 1 represents desirable condition and 0 otherwise. We focused particularly on features we deemed actionable and renamed them to enhance their intuitiveness, specific to satisfiability. For instance, we renamed the feature HiighBP' which indicated high blood pressure (0= = =n= 1 = =yss, to 'LowBP'; {I = -yes (lowBP), 0- = no (h) Additionally, we removed six features 'CholCheck,' Diabetes 012," 'Ser,' 'Age," "Education,' 2 and 'nnoome'' and remained with 16 features.
above 20 years, which reduced the dataset to 247, 980 rows.
These final 16 binary features included the following: 'LowBP': {1 yes (lowBP), 0 no (highBP)): )
'LwwChol: (1 = yes (lowChol), 0 = no (highChol)). The feature 'HealthBMI's {1 =yes (healthy), 0
no (unhealthy), 'NoSmoke': {1 = yes, 0 = no), 'NoStroke': {1 = yes, 0 = no}, 'NoCHD'; {1 = yes, 0
= =o),, PPhys {1 = yes, 0 = no), 'Fruits': {1 = =ys, 0 = no). 'Veggies': {1 = yes, 0 = no):
Light&lcohooConssngg {1 = yes, 0 = no), 'AnyHealthcare': {1= yes, 0 = no). 'DocbcCost': {1 = yes, 0
no); 'GoudGenHitth' {1 = excellent (1,2,3),
bad (4,5)), 'GoddMenHHlhh {1 = (1 = good (< 2), 0
bad (2 2)}, GGooPPhysHHic {1 = good (< 2), 0 bad (2 2)}, and 'NoDifWWllk: {1 yes, 0 = no).
Since we consider the setting where t- 11;; of the remaining data points, 8392 were considered to have desirable outcome (no health risk) because all their features met the respective feature thresholds. Lastly, after removing the duplicate health risk agents and splitting the whole dataset 80/20, we had 11039 data points in the prodictive training set and 2760 in the predictive testing set.
B.2 Single-agent CFE Generation and Semi-synthetic agent-CFE Datasets
For all datasets, to determine which agents require CFEs (negatively classified agents), we use the classifica- tion models. Specifically, we trained logistic regression models on the BMI and WHR training sets, tuning the solver and mar_ iter hyperparameters using GridSearchCV. The best-performing models achieved test accuracies of 72.78% on the BMT dataset and 85.18% the WHR dataset. For the BFRSS dataset, the threshold classifier t = 116 achioved 100% test accuracy. We then used these models to determine the clas- sifier parameters needed for single-agent CFE generation (see Equations 1, 2, and 3) and the specific agents requiring CFEs in the BMI, WHR, and BRESS train/test datasets.
Below are details on the actionable features for each of the datascts. Refer to Appendix B.1.1 and Ap- pendix B.1.2 for a detailed description of the meaning of the features.
BMI actionable features. For BMI agents states, we considered the following 19 actionable features: 'protein (gm)', 'carbohydrate (gm)', 'dictary fiber (gm)', 'calcium (mg)', 'iron (mg)', 'magnesium (mg)', "phosphorus (mg)', 'polassium (mg)", 'sodium (mg)', 'zinc (mg)., 'copper (mg)', 'selenium (meg)', 'vilamón C (mg)', "niacin (mg)', "vitamin B6 (mg), 'vitamin B12 (meg),, 'total saturated fatty acids (gm), 'total monounsaturuted falty acids (gm)', and "total polyunsaturated fally acids (gm).
WHR actionable features. For the generation of recourse for WHR agents, we use the following 20 actionable fcatures: 'protein (gm)", 'carbohydrate (gm), 'dietary fiber (gm)', 'calcium (mg)', 'iron (mg)', "magnesium (mg)', "phosphorus (mg)', "polassium, (mgg), 'sodium (mg)', 'zine (mg)', 'copper (mg)', 'sedenium (meg)', 'vitamnn C (mg)', 'niacin (mg)', 'vitamin B6 (mg)'. 'total folate (meg)', 'vitamán B12 (mag)', 'total saturated fatty acids (gm)', 'total monounsaturated fatty acids (gm)), and 'total polyunsaturated fatty acids (gm).
BRESS actionable features. Lastly, for the BRESS agent states, we considered the following 16 actionable features: Achvvict 'Fruits', Veeggees., ' 'AnyHealtheare', 'LowBP', 'NoSmoke', 'LowChol, "HellthBMI', 'NoStroke', 'NoCHD', "LighttlcoholConsumpp) 'DocbeCost', 'GoodGenHlthi, 'GoodMentHtth" "GoodPhysHlhh; and Walk?
20

---

Published in Transactions on Machine Learning Research (06/2025)
B.2.1 The Low-level CFEs
Given negalively classified agents in the training and lesting BMI, WHR and BRESS datasets, and the actionable features for the corresponding datasets, generate low-level CFEs using the low-level CFE generator (actionable recourse) (Ustun et al., 2019) described in Equation 1. Figure 4 illustrates examples of the generated low-level CFEs for the BMI, WHR and BRFSS datasets.
Features Change
Current Value
Required Value
Protein (gm)
253.51
14. 639999999999986
Calcium (mg)
1327
116
34.84200003000000066
Potassium (mg)
6163
+
6370.51888499999/
Selenium (mcg)
313.90957589999977
Total monounsaturated fatty acids (gm)
154.24
88.88112600000001
(a) for a BMI agent state
Features Change
Current Value
Required Value
Selenium (meg)
45
327.319
Total monounsatursted fatty acids (gm)
12.392
89.34236700000017
Total saturated acids
10.077
2.8045000000000004
Vitamin B12 (meg)
1.21
0.1200000000000001
Total (meg)
1179773800000000003
Vitamin B6 (mg)
0.482
0217949903389999998
Niacin (mg)
8.755
85.10721500000001
Vitamin
35.7
0.000000000000000142
Copper (mg)
0.425
0,212950000000001
Features Change Current Value to Required Value
(mg)
2.61
1.3895
PhysActivity
Sodium (mg)
1326
626.65
Fruits
Potassium (mg)
994
6520.500000000004
Veggies
Phospharus
488
217
Magnesium (mg)
96
57
NoSmoke
(mg)
4.08
42.72946000000002
LowChol
Calcium (mg)
309
113
HealthBMI
Total polyunsaturated fatty acids (gm)
13.999
4.40896
NoStroke
Carbohydrate
109.45
43.378000000000000
GoodPhysHlth
Diatary fiber (gm)
4.1
50.11390000000022
NoDirWaak
(b) for WHR agent state
(c) for a BRFSS agent state
Figure 4: Given negatively classified BMI agent with actionable features presented in the order specified in Appendix B.2 and values [253.51, 153.51,332.770 76, 48.2, 1327., 29.61, 1204., 3966., 61 63.,., 5890.0,90000 0, 44.19, 19, 7.903, 275.1,30 109.198, 3.492, 2.3, 59.686, 154.24, 113.429], the low-level CFE generator (cf. Equation 1) generates CFE (a) to help them become positive. On the other hand, given a negatively classified WHR agent with action- able features [29.03, 109.45, 4.1, 309., 4.08, 96., 488., 994., 1326., 2.61, 0.425, 45., 35.7,8.755, 0.482, 172., 1.21, 10.077, 12.392, 13.999] in the order as described in Appendix B.2, the low-level CFE generator generates CFE (b). Lastly, the low-level CFE generator generates CFE (c) or an agent negatively classified based on their BRFSS features, with values 0,, 1, O, 0, 1, 0). All the low-level CFFs ((a), (b) and (c)) are beature-based and precisely describe which features to change and by how much.
21

---

Published in Transactions on Machine Learning Research (06/2025)
B.2.2 The hl-continuous CFEs and agent-hl-continuous CFE Datasets
Below, we describe the singlo-agent hl-continuous CFE generation and the creation of the four semi-syuthetic, seeett-h-contimuout CFE datasets from the training/testiing BMT and WHR datasets and the 2 forms of hi- continuous actions: monetayy costs and Foodicaloric costs actions. Figure 6 shows examples of the generated hllcontinuous CFEs for BMI and WHR agents.
For each negatively classified BMT and WHR train/test set agent, we generated hl-contimmous CFFs using two types of actions: monctary costs and Foodo-caloric costs. Leveraging the respective classifier parameters and the ILP formulation (Equation 2), we computed two distinct CFEs for each agent, one optimized for monetary cost and the other for caloric cost, each specifying an optimal set of food items.
As result, we produced four unique ggent-hl-contimeg training/ iestin BMI agenl, one for each form of hl-continuous actions, yielding 40692 agent CFE pairs for training and 10167 pairs for testing in each case. Similarly, for the WHR dataset, we generated 6387 training and 1603 testing agent- CFE pairs for both Foods monetary and t costs.
CFE datascts. We generated two CFEs for cach
B.2.3 The hl-discrete CFEs and agent-hl-discrete CFE Datasets
Here, we outline the single-agent hl-discrete CFE generation and the process of generating the agent hl- discrete CFE dataset, which consists of negatively classified BRFSS training/ trainingttesting testing agents and their corre- sponding optimal synthetic hl-discrete CFEs. Figure 6 shows example of the generated hl-discrete CFE for BRESS agent.
First, we generated 100 synthetic 16-dimensional, binary hl-discrete actions. The probability Pa of an action satisfying feature cligibility was set to 0.5. The cost of satisfying feature eligibility was randomly prodefined and remained uniform across all actions and agents. The total cost of an action was the sum of the costs associated with satisfying each feature's eligibility.
Next, employed a threshold classifier with t 116 where agent is classified as not at health risk if x; 2 t, vi E [n], and health risk otherwise. This classifier achieved perfect test accuracy 100.00% the BRFSS dataset, allowing us to accurately identify agents requiring CFEs in the training and testing datasets.
Using the identified agents, the synthetic hl-discrete actions, and the threshold classifier = 1,, we applied Equation generate the agent- hl-discrrte CFE dataset. As a result, we obtained 11, 039 agent-CFF pairs in the training dataset and 2, 760 in the testing agent hl-discrete CFE dataset, where each CFE comprised optimal hl-discrete synthetic actions.
B.2.4 The agent-hl-id CFE Datasets and other Variants
After creating the agent hl-continuous CFE, agent hl-discrete CFE, beatiingtessting datasets, we generate other agent CFE dataset variants from them.
The agent-hh-id CFE datasets. Given the -llontiol-on CFE and -ilei-hhlddeeeeeee CFE datasets described in Sections B.2.2 and B.2.3, we created corresponding agent-hl-id CFE datasets. This process involves encoding each CFE in the agent CFE dataset with unique identifier that distinguishes it from all other possible CFEs in that dataset. For example, given instances of agents hl-discree CFEs, generate unique identifiers for all the hl-discrete CFEs to generate corresponding hl-id CFEs. At the end, we had 5 agent hl-id CFE training/ /esting datasets.
The semi-syuthetic varied frequency of CFEs agent-Cre CFE datasets. For each of the generated agent-hl-continoous CFE, agent-hl-discrete CFE, and the agent-hl-id CFE training/testing datascts de- scribed above, we generate three frequency of CFE dataset variants: all (including all data), >0 (more than 10 agents per CFE), and >40 (more than 40 agents per CFF).
22

---

Published in Transactions on Machine Learning Research (06/2025)
B.3 The Fully-synthetic agent-CFE Datasets
We created five kinds of fully-syuthetic agent- hl-discrete CFE datasets: varied dimension, frequency of CFFs, information access, feature satisfiability, and actions access. We provide statistical detailed information about the five variations of the agent hl-discrete CFE datasets in Table 2 and Figure 5.
B.3.1 Varied Dimensions agent-CFE Datasets
We created 20-, 50- and 100-dimensional agent states datasets by varying the number of actionable features (n 20, 50, 100) and keeping Py 0.68 the same for all datasets. We consider unit vector threshold of length n. The cost associated with satisfying a feature's eligibility was predefined randomly and the same across all actions and agents. Each action was of length n, Pa was 0.5, and action cost was the sum of the cost for each features the action fulfills. To create the 20-, 50- and 100-dimensional agent hl-discrete CFE datasets, we computed the hl-discrete CFEs for each varied dimensional agent states datasets using the information above and the ILP defined Equation 3.
B.3.2 Varied Frequency CFEs agent-CFE Datasets
To investigate the effect of frequency of CFEs the agent CFE training set on the performance of the data-driven CFE generator, we create three varied frequency of CFEs agent CFE datasets. For each of the varied dimensions agent hi-discrete CFE datasets described in Appendix B.3.1, before the train/tes split, created three frequency- baased agent- CFE datasets: all, where all data is included, >10, where ensure frequency of more than 10 agents per hl-discrete CFE, and >40 with insurance of a frequency of more than 40 agents per hl-discrete CFE.
B.3.3 Varied Information Access agent-CFE Datasets
We construct varied information access agent-CFF datasets for synthetically generated agents and their corresponding fully-synthetic hl-discrete CFEs. That is, for each of the 20-, 50- and 100-dimensional agent hl- discrete CFE datasots and their corresponding frequency-basee datasets (all, >10, and >40), we created three varied information access datasels: agent-hh-ddisceete CFE dataset where the original hil-diccrete CFE remains unchanged, the agent rediscrree-nameen CFE dataset where unique name encodes each hl-discrete action in the hl-discrete CFE, and the agent hl-discrete-id CFE dataset where a unique identifier denotes the entire hl-discrete CFE. For example, consider agent [0. 0, O, 0, 1] and their corresponding hl-discrete CFF given by 0, 1, 0), j0, 1, 0, 0), 0,0,0) The hi-discrete-namee CFE {a, b, c} where each hl- discrete action has a name (e.g,, a) that uniquely identifies a specific hl-discrete action (e.g., [0, 1, 0J) among all hl-discrete actions. On the other hand, a unique name, say 2, denotes the hl-discrete-id CFE, where z uniquely represents this specific hl-discrete CFE among all the hl-discrete CFEs.
Dataset name
Dataset size
Onc-action CFEs
Two-action CFEs
Threc-action CFEs
20-dimensional dataset
71125
23687
44858
2576
50-dimensional dataset
98966
1262
96770
934
100-dimensional dataset.
99728
0
45515
54213
manual groups
73484
13480
56653
3351
probabilistic groups
70226
44661
20258
5307
First10
74524
61794
12046
39
First5
74594
60656
6005
Last10
74401
53822
19952
Last5
74565
66068
644
Mid5
74594
63530
3010
Table 2: Statistics of some of the fully-synthetic agent -l-discreeeee CFE datasets used in the experiments. Each hil-discrete CFE for each agent in all datasets has atmost 3 hl-discrete actions.
23

---

Published in Transactions on Machine Learning Research (06/2025)
This setting aims to study the effectiveness of the data-driven CFE generators under various information access constraints within agent-CFF training set, for example, (1) full access to hl-discrete actions and their effects on features (hl-discrete CFE), (2) access only to the names of hi-discrete actions without any information on how each action affects features (hl-discrcte-nmmed CFE), and (3) minimal information access, where only hl-discrete-id CFEs are known, with no explicit knowledge of the corresponding hl-discrete actions or their impact on features.
Given the agent-hl-dissen CFE generator to generate hi-discrete CFEs, data-driven hl-continuous CFE generator for hl-discrete-namun CFEs, and data-driven hl-id CFE generators for hl-discrete-id CFEs.
CFE varied information access datasets,
the data-driven hl-continuous
B.3.4 Varied Feature Satisfiability agent-CFE Datasets
Using the ILP formulation defined Equation with n= 20, and following the same agent and hl-diserete generation approach as in Appendix B.3.1 while varying the feature satisfiability for the threshold- based binary classifier (differing in which features are classifier-active (non-zero)), we generated five agent hl- discrete CFE datasets. For the dataset Last5, the threshold vector set as = [15 zeros, 5 ones], while for the dataset First5, it is set t= [5 ones, 15 zeros]. The third dataset, First10, has threshold vector of == [10 ones, 5 zeros], and the dataset Last10 has [10 zeros, 10 ones]. Finally, the dalasel Mid5 has all features set to zero except for the five middle features set to one.
These varied feature satisfiability agent-hll-dicrete CFE datasets are specifically created to investigate the effect of feature satisfiability on the nature of the hil-dicrrete CFEs and the effectiveness of the data-driven hllcontinuous CFE generator at generating CFEs for new agents.
B.3.5 Varied Access to Actions agent-CFE Datasets
Lastly, consider two settings where grouped agents have restricted access set of actions: 1) manual groups where actions generated with the same probability Pa 0.5 and agents randomly assigned restricted subset of actions; and 2) probabilistic groups where agents are assigned to groups and cach group has ils actions generated by different probabilities Ps = [0.4, 0.5, 0.6, 0.7, 0.8). See Figure 5 for the statistics of the datasets.
We designed the varied access to actions hildisceee CFE datasets to empirically investigate fairness in CFE generation. Specifically, we examine the impact of restricting access of a group of agents some actions the nature of hl-discrete CFEs, such as CFE costs and the variations in accuracy of data-driven hl-continuous CFE generators across different groups.
1-action CFEs
2-action CFEs
3-action CFEs
1-action CFEs
2-action CFEs
3-action CFEs
Number of Agents
10a
320ze
12320
12372
Number Agents
104
7146
3252
2150
2003
103
10a
127
391
102
102
101
10a
Group 0
Group
Group
Group
Group
Group
Group 1
Group
Group
Group 4
(a) manual groups
(b) probabilistic groups
Figure 5: Statistics on the varied access to actions agent hl-discrete CFE datasels for manual groups and probabilistic groups. In the probabilistic groups, the action probability Pa varies as follows: Group (p. 0.4), Group 1 (pa 0.5), Group 2 (pa = 0.6), Group (pa = 0.7), and Group 4 (pa 0.8). While the manual groups exhibit more balanced distribution in terms of the number of actions taken by agents, the probabilistic groups introduce disparities where agents in certain groups have access only to expensive and limited hl-discrete actions compared to others.
24

---

Published in Transactions on Machine Learning Research (06/2025)
action-1
action-2
PhysActivity
Fruits
Veggies
AnyHealtheare
0
LowBP
NoSmoke
1
LowChol
0
HealthBMI
NoStroke
NoCHD
DocbeCost
GoodGen
GoodMentHlthh
GoodPhysHlth
NoDiffWalk
(a) for a BRESS agent state
action-1
action-2
action-3
(gm)
Protein (gm)
0.000
19.440
Carbokydrate (gm)
3.740
61.600
0.100
Carhohydrate (gmn)
01.500
0.000
Distary (gm)
Dietary (pm)
Calcium (ing)
51.000
8,000
18,000
Calcium (mgg)
8.000
17.000
(mg)
3.720
0.300
Tron (mg)
8,720
1,880
Magnesium (mg)
81.000
2.000
11.000
Magnesium (mg)
Phosphorux (mg)
46.000
5.000
114.000
Phoephorus (mgg)
5.000
139.000
Potassium (mag)
379.000
16600.000
149.000
Potassium (mg)
165000000
179.000
Sodium (mg)
213.000
52.000
216.000
Sodium (mg)
62,000
247.000
Zine (mg)
0.420
0.000
Zine (mg)
0.360
0.420
0.100
Copper (ang)
0.195
0.000
Copper (mg)
Selenium (enag)
70.600
Selesium (meg)
0.900
0.200
4.100
Vitamin (mg)
0.000
0.000
Vitamin (mg)
30.000
0.000
1.000
Niacio (mg)
0.000
10.136
Niscin (mg)
0.400
0.000
0.180
Vitamin 136 (mx))
0.000
0.319
B6 (mg)
0.089
0.000
0.010
(mag)
4,000
Vitamin B12 (megg)
0.000
0.000
5.000
Vitamin B12 (mag)
0.000
2.550
Total seturded futky ucids (gm)
0.000
0.000
0.002
Total saturated fasty acids (gm)
0.000
0
Total monomesturntedd fatty acids (gm)
0.010
0.000
0.002
monoussat
(gm)
Total polyunsaturated fatty acide (gm)
0.070
0.000
0.006
Total polyurasturatedd fatky acids (gm)
0.000
0.277
(b) for BMI agent state
(c) for WHR agent state
Figure 6: For an agent negatively classified based on their BRFSS fcatures, with values 1,0,0,1,,,0,,00 0, 1, 1,,,,, 1,0,, in order similar to (a), the hi-discrete CFE generator recommends hl-discrete CFE(a) with hl-discrete actions, action-1 and action-2. Additionally, for negatively classified BMI agent, given their ac- tionable features with values [253.51, 352.76, 48.2, 1327., 29.61, 1204., 3966., 6163., 5890.0, 44.19, 7.903, 275.1, 30., 109.198. 3.492, 2.3, 59.686, 154.24, 113.429), arranged in the same order as features shown in (b), the hl- continuous CFE generalor recommends CFE(b) containing the following hl-continuous actions: action-1: take Swiss chard, raw, action-2: take leavening agents: cream of tartar), and action-3: take clams, mized species, canned, in liquid. Similarly, for a negatively classified WHR agent with actionable feature values ordered as features in (c) [29.03, 109.45, 4.1, 309., 4.08, 96., 488., 994., 1326., 2.61, 0.425, 45,,45.,.. 35.7, 8.755, 0.482, 172., 1.21, 10.077, 12.392, 13.999) the hil-continouus CFE generator recommends CFE(c) with the following hllcontinuous actions: action-1: take leavening agents: cream of tartar and action-2: take fish, tuna, light, canned in water, drained solids.
25

---

Published in Transactions on Machine Learning Research (06/2025)
C Data-driven CFE Generators: Supplemental Details
This section includes supplemental details about the architectures of the data-driven CFE generators, details about other baseline models, and future works. Although we do not explicitly create separate validation set during the initial 80/20 data split for training and testing, we use "validation split" when training all the generator models.
C.1 The Data-driven hl-continuous CFE Generator
The neural-network hl-continnous CFE generator we use in these experiments is susceptible to imbalance and overfitting Therefore, we weight. and regularize the loss function LиC in Equation 4 as follows:
M Σ m=1
Dic PWCHC +
1mmm am mmll1
(8)
M
The weighting factor weights Lue by scaling the contribution of each agent to the loss function. The term a M Em=1 ||ām am II regularizes the model, thus preventing overfitting by nudging the model towards producing hl-contimuous CFEs closer am distribution. We, on average chose the values of a from the set (0.05, 0.1, 0.07) and Pw from (0.05, 0.1, 0.07).
C.2 The Data-driven hi-discrete CFE Generator
Below the architecture of the neural-network based data-driven hl-discrete CFE generator described in the main paper.
C.3 The Hamming Distance Data-driven CFE Generator
To produce hl-discrete-id CFEs (refer to Appendix B.3.3) for new agents, we mainly used the data-driven hl-id CFE generator. However, we wanted to investigate the effect of model complexity on the accuracy of CFE generation. Therefore, we compare the more complex data-driven hl-id CFE generator (refer Section 4.3) with a basic model, e.g., Hamming distance-based CFE generator, whose choice is due to the agent features being binary for this setting. Below is description of the Hamming distance hl-discreto-id CFE generator.
Encoder
Decoder
Internal state
O
0000
0000
1
000
000
00
Input: x E 10, 1]"
Output: E {0, 10,11xx0
Figure 7: An encoder-decoder data-driven hl-discrete CFE generator, where is the data dimension and is the number of hl-discrete actions in the generated hl-discrete CFE L.
26

---

Published in Transactions on Machine Learning Research (06/2025)
Xtr
0
X18
O
1 0 0 0
0
Hamming Distance: (Xtr: Xts)
Figure 8: Hamming distance belween the agent CFE training set agent Xer and a lesting set agent Xts-
Given a negatively classified new agent Xts: we compute the Hamming distance (see Figure 8) between them and each of the agents Xtr the agent hi-discrete-id CFE training set. Then, based on these distances, we choose the k nearest training set agents and their associated hi-discrete-id CFEs. We then use the most common hl-discrete-id CFE the hl-discrete-id CFE for the new agent Xts- We experimented with varied number of nearest neighbors: 5, 10 and 15, for the 20,, 50- and 100-dimensional agent hl-discrete-id CFE datasets, respectively.
C.4 Extensions of Data-driven CFE Generation
Robust data-driven CFE generators. CFE generators, whether single-agent, global, data-driven, can be explicitly or implicitly compromised by changes in the underlying classifier, especially if that classifier is inaccurate or evolves with time. Low-level, crature-based CFEs, such those proposed by Ustun et al. (2019), are particularly vulnerable: their specificity makes them highly sensitive to even minor modifications in the classification model. Data-driven CFE generators are also at risk of model drift and classification errors, primarily due to their reliance on agent- CFF datascts, implicitly or explicitly shapod by aggregation methods that depend the current. classifier.
To address issues related to classification model errors and changes, recent robust CFE generation approaches (see (Jiang et al., 2024)) account for model and distribution drift during the CFE generation process, which would improve the overall performance of all CFE generators, whether single-agent, global, or data-driven.
A key direction for future research to investigate the robustness of data-driven CFE generators under model drift. particular, it would be valuable to assess whether high-level CFEs offer improved resilience model changes or errors compared to feature-based, low-level CFEs. Furthermore, future work could focus valuing agent-CFE data instances and developing robust aggregation and data-sharing strategics (c.g.. federated learning) to enhance the overall performance of data-driven CFE generators.
Extension to multiple valid CFEs generation. In the agent-CFE dataset, each agent is associated with a single valid CFE of minimal cost. The probability of agents possessing multiple unique valid CFEs with identical minimal costs was negligible because we assigned unique feature eligibility costs, resulting in varied overall hl-discrete action costs and uniquely optimal CFEs. In the rare cases where such ties occurred, we excluded the corresponding agents from the dataset.
Although ensure each agent has unique optimal CFE in the agent-CFF. dataset, other valid CFEs that flip the prediction may exist but incur marginally higher costs. This scenario is more prevalent when generating sets of actions, whether hl-discrete or hl-continuous, as different combinations may achieve the same classification outcome at a higher cost.
Our current evaluation function (Equation 6) overly strict, disproportionately penalizing valid CFEs that incur higher costs. A promising avenue for future work involves extending data-driven CFE generators produce sets of valid CFFs rather than single optimal ones. Instead of training the generators agent -FF pairs (each agent mapped to unique minimal-cost. CFE), train the generators agent-validCFE sets, where each agent is associated with multiple valid CFEs. During inference, evaluate each generated CFE for prediction flip and relative cost. This approach broadens the CFE search space, potentially enhancing robustness of the CFE generators.
27

---

Published in Transactions on Machine Learning Research (06/2025)
D Evaluation and Comparative Analysis Metrics: Supplemental Details
Here, we provide additional details on the evaluation metrics used to compare low-level CFEs with both hl-continuous and hl-discrete CFEs, as well as to assess the effectiveness of data-driven CFE generators.
D.1 Comparison Metrics
We evaluate each key variable (v), such as the number of actions taken, features modified, agents using the same CFE, and agent improvement, when agents follow low-level CFE vs. high-level CFE (hl-continuous, or hl-discreto). Specifically, using the general Equation 9, we define comparison metric 6, that compares each variables when an agent follows a low-level CFE versus an hl-continuous or hl-discrete CFE.
(P. = P. Qu
(9)
Where P and Q denote two CFEs under consideration, e.g., P may correspond to a low-level CFE and Q to an hi-diicrett CFE. The terms P. and Qu denote the variable value, such as the number of actions taken, following the execution of each CFE.
For all variables, positive & indicates that the low-level CFE variable value is higher than the compared CFE, while negative 6, indicates the opposite. The magnitude of 8, reflects the extent of this difference. Below are the specific 5, metries.
Difference in number of actions taken. The metric Sactions(:, (Equation 10) quantifies the difference in the number of actions taken when an agent executes a low-level CFE versus an hl-contimuous or hl-discrete CFE.
Oactions (P. Pactions Qactions
(10)
Here, P and Q represent the low-level CFE and the hl-continuous hl-discrete CFE, respectively, while Pactions and Qactions denote the number of actions taken when executing each.
Difference in agent improvement. The metric eumprovement (---) (Equation 11) quantifies the difference in the agent improvement earned when an agent execules a low-level CFE versus an hl-continuous or hl-discrete CFE.
Smmprovemnnt (P. Q) Pimprovement Qimprovement
(11)
Here, P and Q represent the low-level CFF and the hl-continuous hl-discrete CFE, respectively, while Pimprovement and Qimprovement denote the agent improvement earned when executing each. Specifically,
Pimprovement ||x' -x|1
(12)
where P is the CFE (aken and x' is the resultant agent state after taking the CFE from x, which is the initial agent state. Ideally high improvement (less proximate), that is, x' more distant from x is preferred.
Difference in number of features modified. The metric Steatures (Equation 13) quantifies the difference in the number features modified when an agent executes a low-level CFE versus an hilcontinuous or hi-discrete CFE.
8ieatures (P,Q)= Pieatures Queatures
(13)
Here, P and Q represent. the low-level CFE and the hi-continuous hi-discreee CFE, respectively, while P Preatures and Queatures denote the number of features modified when executing each.
D.2 Statistical Significance between Variables
Given the different variables, e.g., list of the number of actions taken, number of modified features, and improvement achieved with each CFE: hl-contimooss, hl-discrete, and low-level, compute the statistical significance of the differences. We use the Scipy stats tool (Developers, 2023) to compute the Kendall tau and v-vuu to assess the statistical significance of the relationship between the two variables at time.
28

---

Published in Transactions on Machine Learning Research (06/2025)
E
Experimental Results: Supplemental Details
In this section, we provide additional and thorough empirical evidence demonstrating the strong performance of the proposed data-driven CFE generators in producing optimal CFEs for new agents. We also highlight the strong and desirable characteristics of the hl-continuous and hl-discrete CFFs the low-level CFEs. Lastly, analyze how various constraints, such varied data dimensions, the frequeney of CFEs, decision- makers information access, feature satisfiability, and restrictions on agents' access to actions, affect the CFE data distribution and the effectiveness of data-driven CFE generators.
(a)
low-level CFE
hi-continnons CFE
hi-discrete CFE
9.1
9.1
Average number of actions
6,7
6.1
6.1
4.1
1.9
2.0
WHRemonylary costs
Whi+ealoric costs
BMI-monntary costs
BMI+caloric costs
BRFSS
(b)
low-level CFE
hl-continuour CFE
hi-diserete CFE
16
18.8
15.9
18.5
14
12.1
12
Average number
9:1
9.1
9.5
WHi+monetarr costs
WHR+caloric costs
BMll-monetary costs
BM/rcaloric costs
BRFSS
(c)
low-leved CFE
hi-comlinuous CFE
hi-discrete CFE
12764.6
14811.5
10
7490.7
7589.3
Average improvement
4484.5
4464.5
1744-2
to²
10²
101
2.6
3.6
WHR-monntarr
costs
BMI-monnlary costs
BMI+calorie
Figure 9: A comparison of hl-contimuous CFEs consisting of a set of monetary or Food-ca cost hl- continuous actions for WHR and BMT datasets, alongside hl-discrete CFEs on the BRFSS dataset, evaluated against low-level CFEs for their respective datasets. All annotations up one decimal place, low-level CFEs require (a) actions but lead to (b) fewer feature modifications and (c) result in less improvement (i.e., closer resultant agent states) compared to hl-discrete and hllcontinuous CFEss
29

---

Published in Transactions on Machine Learning Research (06/2025)
Mear: -1200 07
Dev:
Bid Dev: 3799.34
Density
Density
10
25000
-20000
-15000
-10000
-5000
foannes (lvw- CFE, MI-continuous CFE)
Duspr
(low-Ievel (loowweelll CFE, hl-continuous CFE)
Difference mumher of modified features
(b) Difference agent improvement
Figure 10: Given WHR negatively classified agents and the low-level and hl-continuous CFEs they took, a computation of Ommprovenent (P, Q) (Equation 11)} and óteasures (P, Q) (Equation 13) where P denotes taking a low-level CFE and Q denotes taking an hl-continuous CFE, shows that most of the density is negative implying that when agents take hl-continuous CFEs, higher number of their features is modified (a) and resultant improvement is significantly higher (b) than if they took low-level CFEs.
for negatively classified Individual,
Uao -eeet CFE) modifies more features than ( (M- continouu (FE)
for a negatively classifid individual [Aoe -freel (PE) leads more improvement (M--onstanonss
(k contteeeven (FE) modifies more features than (low lenel CFE)
(M contesow more improvement then CFF)
5
Dfeatures (P, Q)
Omnprovmmnnt (P, Q)
-5.000
-10000
15000
-10
20000
-15
-25000
1000 1500
2500
3500
500
1000
1500
2000
2500
3000
3500
Negatively classified individuals
Negatively classified individuals
(a) Difference in number of modified features
(b) Difference in improvennent achieved
Figure 11: A comparative analysis of the (a) difference in number of modified features (freatures (P,Q) (Equa- tion 13)) and (b) difference in agent improvement (Samprovement (P,Q) (Equation 11)) when each negatively classified WHR agent takes a low-level CFE (P) versus an hllcontinuous CFE (Q), shows that hl-continuous
CFEs modify more features (a) and lead significantly higher improvement (b) than low-level CFEs.
E.1 High-level CFEs Result in Higher Improvement and More Feature Modifications
Unlike low-level CFEs, high-level CFEs (hl-continuous and hi-discrete CFEs) involve fewer actions on average (see Figure (a)) and results in higher improvements (Figures 9(c), 10(b), and 11(b)) and simultaneously modify multiple features (see Figures 9(b), 10(a), and 11(a)).
While low-level CFEs exhibit a perfect correlation between the number of actions taken and the number of features modified, hl-continuous and hi-discrete CFEs show a positive but weaker relationship (Figure 12(b)). Additionally, hl-discrete and low-level CFEs have a strong positive correlation (r = 0.708) in the number of modified features (see BRFSS dataset in Figure 12(a)). In contrast, hl-continuous CFEs show a weak negative correlation with low-level CFEs in the number modified features and actions taken (see BMI and WHR datasets in Figure 12(a), -0.2684 and -0.233 respectively).
30

---

Published in Transactions on Machine Learning Research (06/2025)
1.00
WHR (Low-lese))
0.368
1.000
0075
WHR
-0.018
0.542
-0.233
0.50
WHR((b-contnuoer)
0.062
0.722
0.50
0.25
Kendall value
BMI (fow-level)
0.561
1.000
0.25
Kerdaal value
BMI
-0.217
0.396
-0.264
0.00
-0.138
0.752
-0.25
-0.25
-0.50
BRFSS (Jow-Ievel)
1.000
1.000
-0.50
BRFSS
0.433
0.478
0.708
-0.75
BRFSS (hl-discrete)
0,613
0.501
Actions
Proximity
Features
#Actions Improvement
#Actions #Features
(a)
(b)
Figure 12: In (a), we illustrate the correlations for three different aspects: (1) between the number of actions taken with CFEs P and Q, (2) between the number of features modified with CFEs and Q, and (3) between the improvement achieved after taking CFEs P and Q. For the BMI and WHR datasets, P and Q represent low-level and hl-continuous CFEs, respectively. For the BRFSS dataset, P and Q denote low-level and hl- discrete CFEs, respectively. On the other hand, (b) shows the correlation between the number of actions laken and the number of modified features and between the number of actions taken and improvement achieved for each CFE and dataset. In general, low-level CFEs have a perfect positive relationship between the number actions and modified features
E.2 High-level CFEs are Easier to Personalize and Lead Fairer Outcomes
Fairness in CFE generation has primarily been studied along the dimension of equalizing the recourse costs across different groups (e.g., (Gupta et al., 2019)). In this work, we extend the analysis by exploring several dimensions of fairness in CFE gencration.
First, we investigate how agents across sensitive groups using the same CFE generator (same kind of CFFs) experience differences in how much they improve, the number of actions taken, the number of modified features, and the costs incurred. Second, explore the effects of limiting agents to subset of actions (varied access to actions) on the distribution of agent-CFF, datasets and the accuracy of data-driven CFE generators across groups. Lastly, we examine variations in feature satisfiability (differences in what features need to be satisfied) across agent groups, influences the distribution of the agent CFE dataset, and the performance of data-driven CFE generators in generating CFEs for different agent groups.
In addition to fairness, we also investigate the personalization of CFE generation along two dimensions. Agents may be interested in subset. of actions (varied access actions) and thus restricted to CFEs that involve only specific actions. 2) Agents might prioritize different costs in the CFE generation process (varied cost preferences) and thus prefer CFE generators that optimize those specific costs in CFE generation, e.g. caloric costs over monetary ones.
E.2.1 Fairness Based on Variability of CFEs Execution Outcome
We analyze variations in costs incurred, actions taken, features modified, and agent improvement across sensitive groups to assess the fairness of low-level CFEs compared to hl-continuous and hl-discrete CFEs.
Variability in improvement, number modified features, and number actions taken. To quantify differences agent experiences across sensitive groups, we compute the coefficient of variation for three key variables: improvement, number of modified features, and number of actions taken. Figure 13 illustrates that in the WHR dataset, low-level CFFs exhibit substantial variation across sensitive groups
31

---

Published in Transactions on Machine Learning Research (06/2025)
Intersectional Group Hispanic, 61+,
Hispanic, 21 400
Non-Hisparic,
Hapanic, 21-40, Male
Non-Hispario, 21-40, Female
Non-ffeparic, 41-60, Male
Hispanic, 41-80, Female
Non-Hispanic, 21-40, Male
Non-Hisparic, 61+, Male
Hispanic, 41-60, Male
17.5
1708
43,29
Average number aetons
15.0
Coefficient of Ivrriation
12.5
10.0
1040
30
27.48
7.5
10
2.5
1.50
0.0
low-level CFE
hi-continnous CFE
Meootihuuus CFE (b) Variability in #actions
Low- develevel CFE
(a) Average number of actions taken across groups
Intersectiona Group
Hépanic,
Non-fisparic,
Hispanic, 21-40, Maie
Non-Hispanic, 21-40, Female Non-Hapanni, 21-40, Male
Non-Hisparic, 41 -50, Male
Hapanic, 41-80, Female
Non-Hlispanic, 61 Male
41-60, Malc
Average number of features
Coefficient of variation (%)
43,29
15.0
12.5
15.35
12:03
11:00
8g
7.5
5.0
12,88
Jar
10
2.5
0.0
low-level CFE
hl-continuous CFE
Ai-continuons CFE (d) Variability in #features
low-lesel CFE
(c) Average number of features modified across groups
Intersectional Group
Hispanic, 21-40, Famate
Male Non-Htispani, 21-40, Famala Non-ttisparic,
Non-Hisparic, 41-60,
Hispanic, 21-40, Male
Non-Hisparic, 41-60, Male Non-Hisparic,
Hispanic, Female
Hispanic, 41-60, Male
16000
27.53
Average inprovveeett
14000
25
22,67
Coefficient variation
12000
20
10000
8000
15
10
4000
2000
low-level
hl-continmous CFE
low-level CFE M-conthnous CFE
(c) Average improvement achieved across groups
(1) Variability improvement
Figure 13: comparative analysis the average number actions taken, the number of features modified, and the improvement achieved by agents across sensitive groups when they take low-level CFEs versus hl- continuous CFEs. (a), (c), and (e) show the raw distributions for these variables across sensitive groups while (b), (d), and (f) present the coefficients of variation that concisely illustrate the extent of dispersion around the mean for each variable. In summary, low-level CFEs are less fair than hl-continuous CFEs, which exhibit lower coefficients of variation, ensuring more comparable outcomes for agents from different groups.
32

---

Published in Transactions on Machine Learning Research (06/2025)
Intersectional Group Hiepenic, 61+, Female
Hispanic, 21-40. 2emale Female
Non-Hispanic, 41-60, Female
Hispanic, 21-40, Male
Hispanic, 61+, Male
Non-Hispanic, 41-60, Male
Hispanic, 41-60, Female
Non-Hispanic, 21-40, Female Non-Hispanii, 21-40, Male
Non-Hispanic, 61+, Female
Hispanic, 41-60, Male
Non-Hispanic, 81+, Male
41,16
Coefficient variation (%)
40
Average CFE coets
35
30
6
25
20
481
337
2.40 3.92
336
15
2
10
5.60
low-level CFE
hl-continuous CFE
low-level CFE hi-continuous CFE
(a) BMT: average costs incurred across groups
(b) BMT: variability in costs
Intersectional Group
Hispanic, 21-40, Female
Hispanic, 61+, Male
Non-Hlspanic, 41-60, Female
Hispanic, 21-40, Male
Non-Hispanic, 21-40, Female Non-Hispanic, 21-40, Male
Non-Hispanic, 41-60, Male
Hispanic, 41 41-60, Female Hispanic, 41 11-00, Male
Non-Hispanic, 61+, Male
80
01.11
79,55
Coefficient of variation (%)
80
70
70
60
60
Average CFE
50
50
40
40
37.61
30
20.43
30
23.12
20
20
10
521
258
288
a21 242 352
10
low-level CFE
hl-continuous CFE
low-level CFE
hi-continuous CFE
(c) WHR: average cost incurred across groups
(d) WHR: variability in costs
Figure 14: A comparative analysis of average cost variations among agents in sensitive groups when taking low-level CFEs versus high-level continuous CFEs. Although not comparable across CFEs, (a) and (c) show the distribution of costs between groups for each CFE, and (b) and (d) show the coefficient of variations indicating how variable around mean the average costs in groups are. Costs across sensitive groups vary more when agents take low-level CFEs than when they take hl-continuous CFEs.
regarding agent improvement, actions taken, and features modified. Specifically, the coefficient of variation for agent improvement was 27.53% with low-level CFEs versus 22.67% otherwise. The variation in the 27.48%, and for modified features, 43.29% compared
number of actions taken was 43. 13.29% compared to 12.88%. These findings indicate that the benefits of low-level CFEs are not distributed equally across sensitive groups, potentially favoring some over others, thus raising fairness concerns in CFE generation.
Variability in costs incurred across sensitive groups. Although the costs agents incur by taking low-level CFEs cannot be directly compared with taking hl-continuous CFEs because they are contextually different (feature-based vs act ion-bses),,, we study how the costs of executing the same kind of CFEs varies across agents in different sensitive groups.
Our results show that the costs incurred in taking low-level CFEs vary more widely across various sensitive groups than in taking hllcontinuous CFFs. For example, in Figure 14, the coefficient. of variation for taking
33

---

Published in Transactions on Machine Learning Research (06/2025)
low-level CFEs is 41.16% and 79.55% versus 5.60% and 37.61% with taking hl-contimuous CFEs, BMI and WHR datasets, respectively. Therefore, compared to taking hl-continuous CFFs, taking low-level CFFs is more biased and more likely to favor some sensitive groups.
Features to Change
Current Value
Required Value
Salenam
3227220
Total mensunsmurtted acids
19.3723700000007
25004600000000000
maa
Nacin uri)
Vitarin img)
35.7
Copper img)
0.426
0210900000000000
Zino mmgg
2.81
18866
Sodium imgg
1926
628 e5
Poteaakmm img)
504
00500000000000000
Proephonic umg)
400
P17
Magneskum ing)
67
imgi
4.05
62/2240000000000
Calcumn img)
Tatal yunsaturated
18.099
440896
igmé
5780000000c0005
igmé
(a) low-level CFE
actiom-2
action-1
action-2
(pmn
Protein (gm)
19:440
Carbohydrate (gm)
3.350
Carbokydrate (g(n)
Distary fiber (gm)
0.200
Distary (gm)
Calcium (mg)
50 50,000
8.000
Calcium (ing)
17,000
Iroo (mg)
0.830
3.720
(mg)
1.630
Magnestum (mg))
15.000
2.000
Magnesium (mag)
2,000
23.000
Phosphorus (mg)
26.000
5,000
Phosphoras (mng)
5,000
139.000
Putasium (mg)
314.000
16600.000
Potassium (mn))
16900.000
179.000
Sodium (mg)
22.000
52.000
Sodium (mg)
32,000
247.000
Zine (mg)
0.790
0 02020
Xinc (mng
0.400
0.480
Copper (mgg)
0.099
0.190
Copper (mg)
0.195
0.050
Selenium (mon))
0,300
0.200
Selentum (meg)
0.200
70,000
Vitamin C (mg)
6,600
0,000
Vitamin (1sg)
0,000
0,000
Niscin (mg))
0.400
0.000
Niscin (mn)
0,000
10.136
Vitamin B6 (mg)
0.020
0.000
Vitamin Bd (irgg
0,000
0.519
Total folate (meg)
142,000
0.000
Total folate (mmg)
0,000
4,000
Vitamin B12 (meg)
0.000
Vitamin B12 (meg)
0.000
2.300
Total saturated acids (gm)
0.048
Total saturated fasty acids (gm)
0,000
Gally ucidss (pm)
Total polyunstursted acids
Total
polyunesurstte fatky scids (pm)
0,000
(b) hl-contimmous CFE with caloric costs
(c) lll-continoous CFE with monelary costs
Figure 15: When given actionable features values 29,03, 109.45, 4.1, 309., 4.08, 96., 488., 994., 1326., 2.61, 0.425, 45., 35,7, 8.755, 0.482, 172., 1.21, 10.077, 12.392, 13.999), in the same order as shown (b) and (c), for negatively classified WHR agent, the low-level CFE generator recommends CFE (a) with cost of 56.588. This CFE was unique to the agent. In contrast, the hl-continnous CFE generator generates two CFFs optimized for different agent's preferences. When optimizing for caloric. cost, the CFE generator generates CFE (a) with a cost of 2.750. This CFE, which was also optimal for other 25 negatively classified agents, includes action-l ( (consumee endive, raw) and action-2 (consume leavening agents: cream of tartar). When optimizing for monetary cost, the CFE generator produces a CFE (b) of cost 4.010, This CFE, also optimal for other 105 agents, consists of action-1 (consume leavening agents: cream of tartar) and action-2 (consume fish, tuna, light, conned in water, drained solids). Lastly, while the low-level CFE (a) involves 19 actions but. modifies 19 features and improve by 5679.95, the hl-continuous CFEs both involve actions but modify 19 features and lead to an improvement of 16815.04 (b) and 16682.62 (c).
34

---

Published in Transactions on Machine Learning Research (06/2025)
E.2.2 Varied Costs Preferences
We model two types of hllcontinuous CFEs: set of foods--monetar costs hi-contimuous actions and set of Foooi--caloric costs hl-continuous actions (see Appendix B.1.1 and B.2.2). In a setting where negatively classified agents care more about monetary costs over caloric costs, and vice versa, the CFE generator adapts to these preferences and recommends the corresponding optimal CFE, as demonstrated in Figure 15.
Additionally, regardless of whether monetary or caloric costs were the desired costs by the agent, we consis- tently observed that taking hl-continuous CFEs involved fewer actions, resulted in more feature modifications and higher improvement when compared to low-level CFFs (see Figures 13 and Figure 15 ). Future research could investigate the data-driven CFE generation at the intersection of various settings. For instance, this could involve exploring Pareto-optimal solutions where agents soek to simultancously optimize multiple fac- tors, such as monetary and caloric costs.
E.2.3 Varied Feature Satisfiability
In general, as shown in Table 2, compared to the unit threshold datasets: 20- 50- and 100-dimensional agent- CFE datasets, agents in the varied binary feature satisfiability datasets described in Appendix B.3.4 required fewer action due to the fewer features to satisfy to get a desirable classification.
Our results show that without explicit knowledge of the varied feature satisfiability when given testing set agents, the data-driven hildiscrete CFE generator trained on instances of a mixture of varied feature satisfiability agent-hl-ding hl-disccreee CFE datasets successfully generates the right hl-discrete CFEs for the new agents. The data-driven hl-discrete CFE generator achieves accuracy of 99.683% First10, 99.496% on Last10, 100% on First5, 100% on Mid5, and 100% on Last5, dataset variants.
E.2.4 Varied Access to Actions
The manual groups agent-hl-discree CFE datasets (described in Appendix B.3.5) are more balanced in terms of the number of actions agents take (see Figure 5(a)). The reason is agents have access to the same distribution of hi-discrete actions, i.e., although agents in each group have access to only a selected group of hilddscreee actions, all the hl-discrete actions for all groups were generated with the same probability, Pa 0.5.
However, for the probabilistic groups agent-hl-discne CFEs datasets (described in Appendix B.3.5), Figure 5(b) shows that as the probability of hl-discrete capabilities Po decreases, the number of hl-discrete agents require to get all the necessary capabilities to transform their states to gel a positive model outcome increases. In other words, agents in certain groups only have access to more expensive and limited hl-discrete actions compared to others. For instance, agents in the probabilistic groups Group 0 face more difficulty (due limited capabilities and more costly hl-discrete actions) in achieving positive classification outcomes than those in the Group 4.
Manual Groups
Probabilistic Groups
Group
Accuracy
Group
Accuracy
Group 0
0.881 + 0.01200
Group (0.4)
0.880 + 0.04400
Group 1
0.871 + 0.01260
Group 1 (0.5)
0.771 + 0.02081
Group 2
0.875 + 0.01249
Group 2 (0.6)
0.802 + 0.01571
Group 3
0.847 + 0.01359
Group 3 (0.7)
0.873 + 0.01241
Group 4
0.886 + 0.01212
Group 4 (0.8)
0.931 + 0.00947
Table 3: Group-wise accuracy of the data-driven hl-discrete CFE generator on manual groups & probabilistic groups (see Appendix B.3.5). While the accuracy on the manual groups within the same range (87%), it greatly varied across the probabilistic groups.
35

---

Published in Transactions on Machine Learning Research (06/2025)
1-action
2-action
3-action
Number of agents
10s
90 %
1
83. %
1000
94.6 9
79.6 %
966
20.4 %
10t
55 9%
16.9
101
10
3.8 96
12.9 %
53.6 53.5 %
6,9 %
16,5 5.5
60 %
5.4 %
40%%
0.6 %
10
10T
1100
100
0
0
(a) hl-discrete-id CFEs
(b) Ellicrete-namer CFEs
(c) di-dircrete CFEs
Figure 16: The data-driven hl-id CFE generator for the (a) hl-discrete-id CFEs, the data-driven hl-contimuous CFE generator for the (b) hl-discrete-namee CFEs, and the data-driven hi-discrete CFE generator for the (c) hl-discrete CFEs, achieved strong performance on the 20-dimensional all agent hl-discrete CFE, varied information access, test datasels (new agents for the respective variants).
Performance of the data-driven CFE generators all
>10
>40
20-dimensional
0.969 0.00284
0.984 + 0.00208
0.993 0.00141
50-dimensional
0.744 0.00608
0.838 + 0.00534
0.915 0.00458
100-dimensional
0.354 = 0.00664
0.630 = 0.00778
0.856 = 0.00772
Table 4: comparative analysis of the performance of the data-driven hl-discrete CFE generator across agent hil-diccrete CFE datasels with varied dimensions (20-, 50- and 100-dimensional) and varied frequency of the CFEs (all, >10, and >40). Results indicate that accuracy declines as dimensionality increases and CFE frequency decreases. Notably, the 20-dimensional >40 dataset, which has the lowest dimensionality and highest CFE frequency, achieved the highest accuracy.
Since the agents in the manual groups agent hl-discrete CFE datasets had more balanced access to hl- discrete actions as depicted in Figure 5(a), the data-driven hl-discrete CFE generators had almost similar accuracy (~87%) in the generation of CFFs across all agents in different manual groups, shown in Table 3 (left). On the other hand, since the agents in the probabilistic groups had access to varied hl-discrete actions, the accuracy of the dala-driven hildiscrete CPE generator varied greatly across the groups, as shown in Table 3 (right). For instance, expected, the CFEs for probabilistic groups Group 4 agents with onc-action hl-discrete CFEs were more accurately generated with an accuracy of 93.06% as compared to Group and Group agents, generated at an accuracy of 88.04% and 77.09%, respectively.
E.3 Data-driven Generators are Accurate, Confident and Approximate when needed
Our results show that the data-driven CFE generators are accurate and confident information-specific CFE generators. Additionally, unlike low-level CFE generators that sometimes fail to produce a CFE entirely for an agent, our data-driven CFE generators generate approximately good CFEs instead of no CFEs at all, The supplemental results in this appendix subsection are mainly for the fully-synthetic datasets.
E.3.1 Accuracy and Confidence
The proposed data-driven CFE generators are evidenced to perform strongly on the varied datascts. As shown Figure 16, on the 20-dimensional all agent- CFE dataset variants, the CFE generators achieved high accuracy at generating hi-discrete CFEs, hl-discrete-id CFFs, and hl-discrete-id CFFs. All the generators
36

---

Published in Transactions on Machine Learning Research (06/2025)
perform best the single-action CFE agents. Furthermore, with strong confidence, i.c. low margin rates (see Table 4); the proposed data-driven CFE generators performed well on all datasets regardless of the data dimension or frequency of CFEs. Notably, they excelled on high-frequency datasets, that is to say, >40 datasets regardless of the data dimensions, as seen Table 4.
E.3.2 Approximation
Unlike ILP-based low-level CFE generators, which do not generate CFEs for agents when the ILP solu- tion is sub-optimal or infeasible, our data-driven CFE generators alternatively produce valid CFE mistakes when suboptimal (see Figure 17). For example, of the 1.58%, 16.23% and 37.00% mistakes the hl-id gener- ator makes on the 20-, 50-, and 100-dimensional >10 87.29%, respectively, were valid CFE mistakes. Similarly, the majority of the mistakes of the hl-discrete CFE generators were valid, e.g., on the 20-dimensional >10 agent hl-discrete CFE dataset, of the 10.8% mistakes the generator makes, 63.10% were valid.
CFE datasets, 100%, 99.23%, and
Additionally, the likelihood of the ILP-based low-level CFE generator's failure at generating CFEs (i.e., returns no CFFs) increases with the number of actionable features (data dimensions). On the hand, the percentage of valid mistakes from our proposed CFE generators decreases with the frequency of CFEs in the agent-CFE training set, c.g., the percentage of valid mistakes is 87.29% on the >10 dataset and 57.83% on the 100-dimensional dataset.
200
200
180
cosls
CFE costs
180
140
160
120
140
100
PredCFE cost TrueCFE cost
TrueCFE cost (b) Valid CFE mistakes
PredCFE cost
(a) Invalid CFE mistakes
Figure 17: A generated CFE is mistake if the CFE doesn't match the true CFF. A valid CFE mistake transforms the agent's initial state to get desirable model outcome. An invalid CFE mistake does not lavorably transform the agent state. Distribution of costs of generated and true CFEs for (a) invalid and (b) valid CFE mistakes the data-driven hi-id CFE generator makes on 20-dimensional all agent hl-discrete-id datasot. Valid CFE mistakes are, by definition, more expensive than the truc CFEs, while invalid CFE mistakes are cheaper than the true CFEs.
E.4 Data-driven CFE Generators are Easier to Scale and the CFEs are more Interpretable
Our results show that data-driven CFE generators, including hllcontinuous, lil-discrete, and hl-id, are more scalable than low-level CFE generators. In addition, the costs and actions of hl-continuous and hl-discrete CFEs are interpretable and transparent, simplifying validation and comparison.
E.4.1 Scalability
Unlike the overly specific feature-based actions in low-level CFEs (see Figure 15(a)), actions in hl-continuous and hl-discrete CFEs are more general, increasing the likelihood of their optimality for agents with closely similar profiles. As shown in Figure 18, while low-level CFEs were typically unique to each agent, hl- continuous and hi-discrete CFFs were often simultaneously optimal for multiple agents (see also Figure 15).
37

---

Published in Transactions on Machine Learning Research (06/2025)
low-level CFE
hi-continuou CFE
hd-discrete CFE
Average Rindividuals per CFE
297
152
10"
20
17
10"
10"
WHR+-ooelary
costs BMIlmmnntaryy costs
callocc
Figure 18: The average number of agents with the same CFE when agents take hl- hl-contimuous CFEs (as a set of Food + monecary and Foods+ccaloris costs hllcontimuous actions) for WHR and BMI datasets, as well as hl-discrete CFEs on the BRFSS dataset, compared to when they take low-level CFEs for the respective datasets. Regardless of the dataset considered, on average, while low-level CFFs were unique to given agent (individual), hl-continuous and hi-discrete CFEs were simultaneously optimal to multiple agents.
Furthermore, while our data-driven CFE generators efficiently produce CFEs for new agents without re- quiring re-oplimization, low-level CFE generators operate on a single-agent basis, making them significantly more desource-intensin
E.4.2 Interpretability
The hl-continuous and hl-discrete CFEs comprise real-world-like actions (see Figures 15(b) and 15(c)), in contrast to low-level CFEs, which rely on overly specific, feature-based actions (sce Figure 15(a)). Con- sequently, hl-contimuous and hl-discrete CFEs are more intuitive, interpretable, and casier to execute and compare since they align more closely with executable actions. Moreover, the costs associated with these actions are more transparent and easier to comprehend, given a general understanding of how they were derived, an essential factor in ensuring clarity and trust in following the recommended CFE.
Looking ahead, conducting comprehensive user study would be valuable avenue for future work. The human-subject study could empirically validate the transparency and interpretability of the proposed high- level CFEs, serving Lest of the underlying hypotheses.
E.5 Performance under various Information Access Constraints
In addition to other information access constraints, we investigate the effectiveness of the data-driven CFE generators under two more information access constraints. From the original agent-hl-discrete CFE datasets, created two more information access variants, the agent agent-hil-liscreee--mnndd CFE dataset, and agent-hl- discrete-id CFE dataset as described in Appendix B.3.3. Given the agent hl-discrete CFE information access datasets, we use the data-driven hl-discrete CFE generators for the hi-discrete CFEs, hllcontinuous CFE generators for tetiisrrteenaamee CFEs, and data-driven hl-id CFE generators for hl-discrete-id CFEs.
In general, all the data-driven CFE generators, regardless of information access constraints described Appendix B.3.3, generate single-action CFEs more accurately than multi-action CFEs. For example, the hl-discrete CFE generator, seen in Figure 16(c), generates one-action CFEs at accuracy of 94.6%, two-action CFEs at an accuracy of 79.6%, and threc-action CFEs at an accuracy of 60.0%.
However, in general, data-driven hl-id CFE generators were shown in Figure 16(a) to 16(c) and Table 5 to be more accurate and need less CFE frequency in the training set than the hl-contimuons and hl-discrete CFE generators. For example, on the 20-dimensional all dataset, the data-driven hl-id CFE generator had an accuracy of 96.9%, compared 85.4% with hl-continnous CFE generator and 83.9% with hi-discrete CFE generator.
38

---

Published in Transactions on Machine Learning Research (06/2025)
Performance on 20-dimensional datasets
all
>10
>40
Data-driven hilid CFE generator
0.969 0.00284
0.984 + 0.00208
0.993 + 0.001
Data-driven hl-continuous CFE generator
0.854 0.00581
0.886 + 0.00531
0.940 = 0.00411
Data-driven hl-discrete CFE generator
0.839 + 0.00605
0.892 + 0.00518
0.937 = 0.00420
Table 5: Accuracy of the CFE generators on 20-dimensional: all, >10, and >40 datasets. All the data- driven CFE generators demonstrate consistently high accuracy across all datasets, with particularly strongest performance on high CFE frequency datasets (>40).
E.6 Challenges and Proposed Solutions in Designing Data-driven CFE Generators
We identify several challenges in data-driven CFE generators: the infrequent occurrence of CFEs, the large number of actionable features, and the significant dependence on the complexity of the CFE generator model. In this work, we thoroughly examine these challenges, propose plausible solutions, and suggest. avenues for future research to explore these issues greater depth.
E.6.1 Negatively Affected by High Number of Actionable Features
As the number of actionable features increases, agents CFEs have more actions (see Table 2). For instance, in the 100-dimensional agent-hl-discrete CFE dataset, 54.4% of agents' hl-discrete CFEs had three actions, and in the 20-dimensional datasct, 33.3% of agents needed only one action in their CFE and only 3.6% had three.
Beyond an increase in the number of actions in the CFEs, the uniqueness of CFEs also rises as the number of actionable features grows. The average frequency of CFEs in the all -h-eetth-diseretee CFE training set dropped from 46.64% in the 20-dimensional dataset to 21 .5% in the 50-dimensional dataset and further to 8.09% in the 100-dimensional dataset. Additionally, 18. 115%, 20.797%, and 31.072% of CFEs in the 20-, 50-, and 100-dimensional all datasets, respectively, were unique (i.e., optimal for only one agent). This low frequency of CFEs in the agent- CFE datasets led to discrepancies after train train/test test splits, where some CFEs appeared in one split but not the other. Specifically, for the 20-, 50- and 100-dimensional datascts, there were 52, 154, and 708 unique CFEs in the testing set that were absent from the training set.
As result, data-driven CFE generators become less accurate as the number of actionable features increases. As shown in Table 4, the data-driven hl-id CFE generator consistently performed worse on higher-dinensional datasets. For example, while it achieved 96.9% accuracy on the 20-dimensional all dataset, its accuracy dropped to 74.4% on the 50-dimensional all dataset and was lowest on the 100-dimensional all dataset.
E.6.2 Negatively Affected by Low Frequency of CFEs
We created the varied frequency of CFFs agent-CFF datasets: all, >10, and >40 (see Appendix B.3.2), to examine how CFE frequency in the agent hi-discrete CFE dataset impacts the robustness of data-driven CFE generators. After the train/teer /test split, the >40 dataset ensured that at least 20 agents shared the same CFE in the training set. By definition, the >40 dataset had the highest CFE frequency, while all had the lowest. This frequency also varied with data dimensionality, as illustrated in Appendix E.
The low frequency of CFEs in the alest-hle CFE training sets negatively impacted CFE generation across all datasets, regardless of data dimensionality. However, this effect became more pronounced data dimensions increased. For instance, as shown in Table 4, the accuracy of CFE generators on the 20-dimensional datasel. was highest when CFEs had a frequency of at least. 20 in the training set (>40) and lowest on the all dataset, where some CFEs appeared in the test set but not in the training set. Specifically, the data-driven hl-id CFE generator achieved an accuracy of 99.3% on the 20-dimensional >40 dataset, compared 96.9% on the 20-dimensional all dataset. In contrast, CFE generation accuracy on the 20-dimensional dataset. was significantly higher than the 100-dinensional dataset. This difference
39

---

Published in Transactions on Machine Learning Research (06/2025)
highlights that the negative impact of low CFE frequency in the training set becomes severe as data dimensionality increases.
Additionally, the minimum frequency of CFEs required for a strong CFE generator increases with the number of actionable features. While the frequency of at least 20 the training set ensured an accuracy of 99.3% of the CFF generator on the 20-dimensional dataset (see Table 4) higher frequency needed for the 50- and 100-dimensional datasets (see Table and Figure 19).
100-dimensional + 50-dimensional
100
90
80
Accuracy
70
60
50
40
30
>40
>100
>15
>200
Varied frequency CFEs the training agent CFE dataset
Figure 19: Impact of CFE frequency in agent CFE training datasets for 50 and 100-dimensional datasets on the accuracy of the data-driven CFE generators. As the froquency of CFEs (the number of agents sharing the same optimal CFE) increases, the accuracy of dala-driven CFE generators improves. This improvement occurs more rapidly in the lower-dimensional (50-dimensional) dataset.
Data augmentation. We examine the impact of increasing the frequency of CFEs through data aug- mentation (Algorithm 1) on the performance of the data-driven hil-diccrete CFE generator. Algorithm 1 is specific for the -i-iscete CFE datasets with all kinds of threshold classifiers. To generate agents for which given hl-discrree CFE is the most optimal, we ensure that no other hl-discrete CFE within the complete set of CFEs can achieve the transformation at lower cost.
Therefore, given an agent state, we find all possible worse-off agent states, such that the current optimal hl-ddicreee CFE still the best CFE for the worse-off agent states. Worse-off agent states are those such that the features where the hl-discrete CFE adds more capabilitics than required to transform the agent state favorably use in the experiments, hl-discrete CFE is adding more capabilities than required to feature of if by after the action, the transformed feature of is such that x; > 4. The derived worse-off agent state (augment) aug is valid if x's hl-discrete CFE is also the optimal CFE.
made worse, i.e., for such that x; > 4i, aug, < di. Specific to the threshold classifier
Algorithm 1:
The agent hl-discrete CFE dataset augmentation
Input: an agent x and their hl-discrete CFE T, and the threshold classifier
Output: valid derived augmentations agent x, Xaugs with the same CFE
Data: indices of features ids where the hl-discrete CFE when taken, adds more than needed
capabilities to x
augs iddal possible worsc-off agents; foreach aug in augs do
if aug is valid then
Xaugs t Xaugs U {aug):
end
end
40

---

Published in Transactions on Machine Learning Research (06/2025)
Effect of data augmentation
20-dimensiioall
50-dimensional
100-dimensional
Before data augmentation
0.969 + 0.00284
0.744 0. 0.00608
0.354 0.00664
After AG1
0.965 = 0. 0.00303
0.760 0. 0.00595
0.505 + 0.00694
After AG2
0.982 + 0.00218
0.845 = 0.00504
0.790 + 0.00565
Table 6: Effects of AG1 and AG2 augmentation on the accuracy of the data-driven CFE Generator. Data augmentation mitigates the negative impact of low frequency of CFEs and improves the accuracy of data- driven CFE generators on the 20-, 50-, and 100-dimensional: all datasets.
Data augmentation reduces negative impact of low frequency of CFEs. Using Algorithm 1, we augment the agent-hl-discrete CFE training set to ensure that each CFE appears at least twice (AG1) and to increase the frequency of CFEs with fewer than 20 occurrences (AG2). As a result, the number of hl-discrete CFFs with fewer than 20 agents significantly decreased from 813 to 638, 2676 to 2005, and 9043 to 7144 for the 20- 50- and 100-dimensional datasets, respectively.
Experimental results show an improvement in the accuracy the CFE generators on the test samples after data augmentation, e-g., the 100-dimensional dataset, the accuracy of the data-driven CFE generator increases from 35.37% before data augmentation to 50.54 after AG1, and 78.99% after AG2 (see Table 6). The findings show that data augmentation can improve the robustness of CFE generators in cases where the frequency of CFEs in the agent-CUr CFF training datasets is low.
E.7 Performance Heavily Depends Complexity of CFE Generator Model
Given the agent- -l-ni-hh--iscrte--idd CFE 20-dimensional, >40 dataset variant, we compare the effectiveness of the neural network- based CFE generator against the Hamming distance-based CFE generator. As shown in Figure 20, the neural network-based CFE generator demonstrates greater accuracy in generating CFEs for new agents. Interesting for future works is an exploration the effectiveness of CFE generators based on more advanced and alternative methods, c.g., multi-chain neural networks, reinforccment learning, and transformer models.
1-action
2-action
3-acctoo
105
9
Number of agents
99 1 933
09:30
11100
% 669
62.9
%
$0,99 96
47.15 1
104
%
%
53.4 %%
103
9
102
0.50
10
%
0.3
100
aa) Neural network-based
(b) Hamming distance-based
Figure 20: A comparison of accuracy of two data-driven hi-discrete CFE generators the 20-dimensional >40 dataset. The neural network model consistently performs better than the hamming distance model.
41


Reviews:
[{"review_id": "8z96TFrmGS", "reviewer": "Reviewer_rVLP", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n- Addresses an interesting problem with potential practical importance.\n\nWeaknesses:\n- Proposed methods appear to be limited to certain problems e.g., model must be a binary linear classifier, requires prior knowledge of the set of possible high-level actions. \n- Notation are not defined clearly and appear to overcomplicate the ideas. \n- Experiments do not fully support the claims e.g., on resource-efficiency.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper considers algorithmic recourse, where the goal is provide users with actionable insights on how to change their features in order to change a model's prediction (e.g. flipping negative to positive). These are known as \"counterfactual explanations\" or CFE. Previous work focused on low-level CFE requiring precise adjustments to feature values which can be impractical. In contrast, this work considers high-level CFE corresponding to real-world implementable actions. \n\nThe paper proposes three types of high-level CFEs: hl-continuous, hl-discrete, and hl-id. It then describes how to obtain the CFEs for a single example (or \"agent\" in the paper's phrasing) by formulating integer programs for hl-continuous and hl-discrete. Lastly, it describes a data-driven approach to learn a model that maps an agent's current feature to the optimal CFE. \n\nExperiments were conducted on health-related datasets including BMI (body mass index) and WHR (waist-to-hip ratio), and using Foods dataset to define the food items as the real-world implementable actions.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Major issues\n- The title is misleading. The phrase \"large state spaces\" occurs nowhere else in the paper. \"State space\" has a very specific meaning in machine learning and reinforcement learning, and the authors are encouraged to reconsider the wording. \n- In Sec 3 when introducing \"Single-agent CFE Generators\", it is unclear where the action space comes from. Later in experiments it seems to be derived from some auxiliary dataset, but this may need to be explained earlier in the paper. \n- Sec 3 only introduces \"Single-agent hl-continuous CFE Generation\" and \"Single-agent hl-discrete CFE Generation\". What about hl-id? \n- Eqn (2)(3) are written in a different format compared to Eqn (1) which makes it confusing to interpret. \n  - They also use new notation that are not introduced in the definitions of hl-continuous action and hl-discrete action (Def 1 and 2). \n  - What are the optimization variables of these optimization problem? This is not specified in the equation. \n- Eqn (2) assumes a linear classifier, whereas Eqn (3) assumes a threshold classifier. However, before Sec 3, the framing of the paper is addressing algorithmic recourse in binary classification (Sec 2) without placing assumption on the model. This discrepancy means the proposed approach does not fully address the problem setting considered. \n- Def 3 hl-discrete action only allows changing a binary feature from 0 to 1, but does not discuss the opposite, nor does it discuss categorical features (the paper only says \"the formulation is extensible to more general case\" without explaining which cases and how. \n- Experiments:\n  - In CFE generation the minimize problem only returns one optimal, but in reality there might be multiple actions that all achieve the desired outcome (of flipping predictions). The current evaluation on accuracy does not seem to reflect this. \n  - In Sec 6.1 & Fig 3(a), there is no comparison on the cost achieved by low-level vs high-level CFE, it only compares the number of actions, but each low-level action and high-level action costs are different so they are not directly comparable. \n  - Fig 3(b) compares across sensitive groups but did not specify what sensitive groups were used. \n  - Sec 6.2 & Table 1 shows results for the data-driven CFE, but does not show any data for it being \"resource-efficient\". \n  - Comparing single agent low-level CFE to data-driven high-level CFE is unfair. Why can't we make data-driven low-level CFE? \n  - The paper also makes claims about the proposed approach being \"more transparent and easier to interpret\" \"making them cheaper and more desirable\" (Page 11) but this is only based on the three examples shown in Fig 1. Such conclusions would require user study to justify. \n\nMinor issues\n- Incorrect citation style on page 1: \"A popular form of recourse, actionable recourse Ustun et al. (2019), ...\"\n- Page 2: \"Finally, we conduct extensive experiments on 11+ agent–CFE datasets\" - What does 11+ datasets mean? There should be an exact number of datasets used in the experiments.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "M8b9qxVu6L", "reviewer": "Reviewer_Vkih", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper proposes high-level recourse methods that are actionable and interpretable, and further introduces techniques to estimate the optimal recourse for each agent. More interestingly, it presents a data-driven method that learns a direct mapping from the features to the CFEs, which is generalizable across different agents. The proposed approach has been extensively evaluated on multiple real-world and synthetic datasets.\n\nHowever, the requirements for training both the single-agent CFE generators and the data-driven CFE generator are stringent. The single-agent CFE generators depend on the existence of some accurate classifiers. In the simulation experiments, the authors first trained a classifier before training the single-agent generator. Since it is usually difficult to obtain a perfect classifier (in the examples of BMI and WHR in the paper, the test accuracies are 72.78% and 85.18%), the performance of the generator is likely to be compromised. Although the data-driven CFE generator is claimed not to rely on direct access to the classifier, it depends on the existence of the true optimal CFEs, which is a stronger assumption than that required by a classifier. In the simulation experiments, the true label is obtained through the single-agent generators. However, since these generators are not perfect, the data-driven generators are likely to be further compromised. This long chain (training a classifier, then a single-agent generator, and finally a data-driven generator) may be fragile. I wonder whether there exists a method that directly finds the actions to generate the desired outcome without the intermediary steps.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper defines three high-level counterfactual explanation (CFE) or recourse methods, i.e. hl-continuous, hl-discrete, and hl-id, that provide recommendations for individuals or agents to modify their input features to achieve a desired outcome. Compared to low-level CFEs, the high-level CFEs are more actionable and interpretable, without requiring users to understand precise numerical adjustments. The authors formulate single-agent CFE generation as optimization problems and further propose data-driven generators that learn from historical agent–CFE pairs to rapidly produce recourses for new agents. Experimental results on multiple datasets demonstrate that the single-agent CFE generators require few actions while yielding greater improvements compared to low-level CFEs, and that the data-driven generators accurately replicate the optimal recourses computed by the single-agent generator while maintaining computational efficiency.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "The writing would benefit from greater clarity, as certain aspects of the problem challenges and the details of the proposed methods remain somewhat unclear.\n- Could the authors explicitly discuss the difference between low-level and high-level CFEs? Figure 1 was helpful in comparing them through concrete examples, but it is not immediately clear why estimating high-level CFEs is challenging and needs specific techniques. Technically, is this because multiple actions are selected simultaneously in a high-level CFE? \n- According to Figure 1 and the following descriptions, the ultimate goal is to provide a descriptive recommendation using natural language. However, these recommendations still need to be represented as numerical values when learning the recourse. How is a numerical action translated into a descriptive sentence?\n- The definitions for single-agent hl-continuous actions and hl-discrete actions are not clearly presented. What are the range and dimensions of these actions? If I understand correctly, a hl-continuous action is $\\in \\mathbb{R}^{J}$ for some high-dimensional $J$, and a hl-discrete action is $\\in \\{0, 1\\}^{J}$. Besides, what is the form of the cost and effect, also $\\in \\mathbb{R}$? Further, examples are usually not presented within the definition itself but serve as explanations that follow the definition.\n- For the single-agent hl-continuous CFE generator in Section 3.1, what is the range of $v_j$? Further, it should be clearly stated that $v_j$ represent the change in the feature $x$. Additionally, it is better to clearly define \"negatively classified agents\" as the $x$ with $c^T x + b < 0$.\n- For the single-agent hl-discrete CFE generator in Section 3.2, it is mentioned that each action is represented by a vector $v_j$, yet $v_j$ does not appear in equation (3). Furthermore, I assume that a hl-discrete action only operates on binary states $x$, and that a value of one indicates a better health state than a value of zero in $x$. This has not been explained explicitly.\n- A discussion of the differences between the problem settings for hl-continuous and hl-discrete actions should be provided. For example, if I understand correctly, the definition of the state $x$, the action $v$, and the form of the classifier are all different.\n- Why does equation (4) use a cross-entropy loss, while equation (5) is computed only over positive identifications? Additionally, in Section 4.2, why is the objective function defined as a mean-squared error, if—according to my understanding—this should still be treated as a classification problem?\n- Could the authors explain why the coefficient of variation for the high-continuous CFE is much lower in Figure 3(b)? Specifically, which part of the proposed method (perhaps the constraint in equation (2)?) guarantees this lower variation?\n- Section 6.1 discusses single-agent generators, but this is not explicitly stated in the text.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "rJu9jxpC78", "reviewer": "Reviewer_hnCE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper addresses an important problem (actionable resource that is robust, fair, etc.) and performs extensive empirical evaluations.\nHowever, in my opinion, it suffers from a few limitations:\n- Only linear classifiers are considered\n- Knowledge of pre-defined actions is assumed. It remains unclear to me how fair the comparison is to methods that do not have access to those pre-defined actions.\n\nHowever, it could also be the case that I did not understand this correctly. Happy to discuss this. Please see next section for a more detailed review.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes new types of recourse relying on pre-defined actions that are supposed to improve actionability as well as other important aspects such as robustness, fairness, etc. The recourse is generated by a learned function that maps individuals (agents) to recourse actions.\n\nWith the paper addressing the issue of actionable recourse, I think its topic would be valued by the XAI community.\nIn particular, two points are interesting:\n1. Replacing optimization-based recourse generators by a learned model. This has many benefits as outlined by the authors. However, it may be difficult to build in practice.\n2. Highlighting the usefulness and potential of considering pre-defined action sets. In particular, the impact on robustness and fairness.\n\nThe contributions are interesting, yet I see a few limitations (please see the next sections).\nRegarding the supportive evidence, I feel a bit mixed. On the one hand, extensive empirical evaluations support the claims, but on the other hand, the paper is limited to linear classifiers, and I also have some concerns about the comparison to existing methods (please see the next sections)", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I have three major concerns that need to be discussed and potentially addressed/changed:\n1. It seems that the entire work focuses on linear classifiers. While linear classifiers do exist in the real world, other non-linear classifiers such as tree-based models and deep neural networks are very common as well. I think this limitation should be stated more clearly, and it should be stated how well the proposed methods and concepts could be extended to non-linear classifiers. Because of the empirical focus of the paper, just considering linear classifiers looks a bit weak to me.\n2. The main benefit seems to come from more interpretable and actionable actions (with the side effect of being more robust, etc.). This is done by assuming knowledge of all actions. Is this really realistic? What if we only know about a subset of actions? Is recourse guaranteed for every individual/query?\n3. I am a bit unsure how fair the comparison to existing generators is. The proposed methods rely on manually created actions. I am wondering if those could be made available to existing generators.\n\nA few other comments and suggestions that I had while I was reading the paper:\n\nIntroduction:\n- The GDPR is already \"quite old\". You may want to consider citing the more recent EU AI act (https://artificialintelligenceact.eu) -- I found this website to be very informative and helpful for navigating the AI act.\n- Better explain \"recourse\". Personally, I am very familiar with counterfactuals and (computational) recourse, but I think the paper could be made more accessible by briefly explaining what is meant by \"recourse\" and how it relates to counterfactual explanations.\n- Figure 1 looks very appealing; however, it seems that the proposed recourse generator comes with a lot of prior (domain) knowledge and assumptions (e.g., knowing about the Google AI residency program and how it affects the attributes). I think this should be made clear because it seems to be very domain-specific.\n\nBackground:\n- Such crazy specific recommendations (e.g., \"43.376000000000005\") are unrealistic in practice. In practice, people would round numbers instead of reporting that many decimal places. I get the author's argument, but rounding is not the problem here.\n- \"Moreover, low-level CFE generation becomes very limited, sometimes impractical, in scenarios where access to critical information, such as sufficient training data, classifier parameters, or a comprehensive list of actions and their costs, is restricted or inaccessible.\" <- this statement is a bit provocative and needs references. While those aspects can be an issue for some generators, there also exist generators that do not need any training data or access to model internal parameters. I recommend being a bit more specific here and less general.\n- \"Lastly, low-level CFE generators operate on a single-agent basis\" this is not completly true since group counterfactuals and multi-instance counterfactual exists, which are valid for more than a single individual. For instance, Kanamori et al. (2022) proposed counterfactual explanation trees, which (after they have been learned) can be applied to any given instance. \n\nSection 3:\n- Eq. 2: You assume a linear binary classifier, while Eq. does not make this assumption. Please state this clearly in the beginning; right now it is a bit hidden at the bottom.\n- The same for Eq. 3. But here the classifier itself is a bit unusual -- can you give examples of such a threshold classifier?\n- Are Eq. 2 and 3 really novel? To me looks like applying an action vector to a given input, which is quite common in counterfactual reasoning. Distinguishing between discrete and continuous domains does not seem that novel to me. Can you please clarify?\n\nSection 4:\n- How does it relate to Section 3. To me it looks like that you \"just\" learn a generator for computing counterfactuals. Besides assuming a training data set of counterfactuals, it remains unclear if you can state any guarantees on the computed actions. How do you ensure feasibility? Are you still restricted to linear classifiers? Shouldn't this be possible for arbitrary classifier types?\n\nExperiments:\n- Why only a single train-test split? I recommend to use smth. like cross validation to get statistically meaningful results.\n- Eq. 6: I am unsure how reasonable it is to compare \"the ground truth counterfactual\". Since counterfactuals are known to be not unique, I was wondering why this is not considered here. Intuitively, I would just check for the correctness (i.e., change in prediction) and maybe some other quantities. Can you please clarify this?\n\nLimitations:\n- Only linear classifiers are considered in this work. As already stated, I consider this a major limitation that should be discussed as well.\n\nMinor:\n- GitHub repository: Please add a REQUIREMENTS.txt listing all needed packages -- the Prerequisites listed in the README are not sufficient\n- Appendix: Add line numbers to Algorithm 1\n- Please carefully check the references. For instance, \"Sandra Wachter, Brent D. Mittelstadt, and Chris Russell. Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR.\" has been published in a journal which should be cited instead of the arXiv preprint.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The reviewers collectively agree, after a rich discussion with the authors that this paper is adequately prepared for publication. The authors were very engaged in the process and submitted timely revisions of the paper in addressing reviewer comments. As such, it is felt that this paper is ready for publication as is. There are no outstanding major issues that the authors need to address before the camera ready deadline. 

I would recommend that the authors do take a final fine-grained pass through the reviews and their agreed upon changes to the paper to be sure that they've included everything.
Decision: Accept (poster)