Explaining Confident Black-Box Predictions
Authors: Evan Yao
Retsef Levi
Assaf Avrahami
Abraham Meidan
Venue: Accepted by TMLR ()

Abstract: Interpretability is crucial for leveraging predictive machine learning for decision-making, but the strongest performing models are often black-boxes in that they are difficult to understand. For binary classification models, a growing body of literature seeks to find \textit{model-agnostic} explanations by treating a model as a list of 0/1 predictions and identifying patterns for when a model predicts $1$ over $0$ (or vice versa). While such explanations are
useful for understanding when a model predicts 1 over 0, they do
not consider the confidence (i.e., the probability) behind predictions, a critical piece of information provided by most classification models. Since the 0/1 predictions of a model depend on the choice of a subjective threshold for discretizing predicted probabilities, as one changes the threshold, the resulting explanations may change despite the underlying model staying the same. In contrast, this work proposes model-agnostic explanations that treat a black-box model as a \textit{ranking} across a dataset from lowest predicted probability of $1$ to highest, rather than a list of 0/1 predictions. Under this ranking, a useful explanation should capture broadly when a model \textit{confidently} predicts $1$ (i.e., highly ranked data points). Since highly confident predictions are often correlated with predictions that are more accurate and actionable, understanding when a model predicts confidently is often quite valuable to a practitioner.

This work builds explanations based on rule lists (i.e., a collection of if-then rules) as well as a novel special case called checklists. A strong rule list or checklist is satisfied by a large number of data points that are ranked highly by the model. This criteria is measured by the traditional metric of support (i.e., the number of data points an explanation applies to), the \textit{average} ranking of those data points, which we call the Average Black-Box Ranking (ABBR), as well as the sparsity of the explanation (e.g., number of rules in the rule list, among others). Given these metrics, this work develops a local-search based optimization methodology for finding explanations based on rule lists and checklists that maximize ABBR for a user-specified support and sparsity constraint. The methodology leverages a local search approach where an initial rule list is chosen greedily from a pool of candidate rules, then slowly perturbed by swapping rules from the rule list with those in the candidate pool. This approach is evaluated on 6 real world datasets in application areas ranging from healthcare to criminal justice and finance. Empirical results suggest that this methodology finds rule lists of length at most 5 with ABBR within 7.4\% of the optimal ABBR of any explanation, while checklists provide greater interpretability for a small cost in performance.

Published in Transactions on Machine Learning Research (08/2025)
Explaining Confident Black-Box Predictions
Evan Yao
ceanyoo@mmil.cm edu
Operations Research Center
Massachusetts Institute of Technology
Retsef Levi
retsefümit..o edu
Sloan School of Monagement
Massachusetts Institute of Technology
Assaf Avrahami Wizsoft
on@wiizoff..co com
Abraham Meidan
abraham abruham@wizsoft.com
Wizsoft
Reviewed on OpenReview:
https: // optpr/veew..eee... net/ forum? id- SAuZpgKJcc
Abstract
Interpretability is crucial for leveraging predictive machine learning for decision prcision-making, making, but the strongest performing models are often black-boxes in that they are difficult understand. For binary classification models, a growing body of literature seeks to find model-agnostic explanations by treating a model as a list of 0/1 predictions and identifying patterns for when a model predicts over (or vice versa). While such explanations are useful for understanding when a model predicts over 0, they do not consider the confidence (i.e., the probability) behind predictions, a critical piece of information provided by most classification models. Since the 0/1 predictions of model depend the choice of subjective threshold for discretizing predicted probabilities, one changes the threshold, the resulting explanations may change despite the underlying model staying the same. In contrast, this work proposes model-agnostic explanations that treat a black-box model as a ranking across a dataset from lowest predicted probability of to highest, rather than a list of 0/1 predictions. Under this ranking, a useful explanation should capture broadly when model confulently predicts (i.e., highly ranked data points). Since highly confident predictions are often correlated with predictions that more accurate and actionable, understanding when model predicts confidently is often quite valuable to practitioner.
This work builds explanations based rule lists (i.e., a collection of if-then rules) well as novel special case called checklists. A strong rule list checklist is satisfied by large number of data points that are ranked highly by the model. This criteria is measured by the traditional metric of support (i.e., the number of data points an explanation applies to), the average ranking of those data points, which we call the Average Black-Box Ranking (ABBR), as well as the sparsity of the explanation (e.g., number of rules the rule list, among others). Given these metrics, this work develops local-search based optimization methodology for finding explanations based on rule lists and checklists that maximize ABBR for user-speeified support and sparsity constraint. The mothodology leverages local scarch approach where an initial rule list is chosen greedily from a pool of candidate rules, then slowly perturbed by swapping rules from the rule list with those in the candidate pool. This approach is ovaluated on 6 real world datasots in application areas ranging from healthcare to criminal justice and finance. Empirical results suggest that this methodology finds rule lists of length at most 5 with ABBR within 7.4% of the optimal ABBR of any explanation, while checklists provide greater interpretability for small cost in performance.

---

Published in Transactions on Machine Learning Research (08/2025)
1 Introduction
As digitizatiin increases the amount of available data,, an increasing number of human decisions are aided by predictive machine learning models. However, algorithms with the strongest predictive performance are often also the least interpretable, for example random forests [Beiman (2001) and neural networks Goodfellow et al. (2016). A growing body of literature seeks to understand such black-box models through model-agnostic explanations. For binary classification problems, such approaches treat a model as simply list of 0/1 predictions (i.e., model-agnostic), and discover patterns that elucidate when the model predicts 1 over 0 (or vice versa). One common type of explanation involves finding simple if-then rules that are (1) consistent with a model's binary predictions and (2) apply to large mumber of instances in the dataset (i.e., high support). For example, for a model predicting whether a patient will develop diabetes, consider the following if-then rule:
If BMT > 32 and High Blood Pressure = 1, then predict to diabetes
This rule provides a useful explanation of when the model predicts 1 by identifying a sizable subset of the population (i.e., those with BMT > 32 and high blood pressure), most of whom the black-box model also predicts 1 (i.e., the rule's prediction of 1 is consistent with the model's prediction). Broadly speaking, an explanation is statement about the model's prediction an interpretable subset of the dataset.
While such model-agnostic explanations are useful in understanding when and why model predicts 1 (or 0), they suffer from at least two drawbacks. First, since most classification models output a list of probabilities, the consistency of such explanations with the black-box model varies greatly depending on a 'subjective' threshold used to convert the probability into 0/1 predictions. Related to this first drawback, the second drawback is that considering only 0/1 prodictions, existing approaches overlook the heterogencity in confidence across predictions. A prediction of 1 can be "confident" (i.e., high probability) or just a relatively weak guess, but existing approaches do not distinguish between these two cases. For a well-trained model, the confidence of its predictions is correlated with the model's accuracy and thus provides valuable information when the model is most reliable in practice. Arguably, it is more useful for practitioners to understand why model makes confident predictions over weak predictions as confident predictions are more actionable. For example, a patient strongly predicted to develop diabetes might be put on preventative treatment, whereas a patient with a slightly elevated risk does not require immediate action.
Given these shortcomings, this paper develops model-agnostic explanations that treat a black-box model as ranking of probabilities across the datasct, rather than a discrete list of 0/1 predictions. To the best of our knowledge, this is the first work that provides model-agnostel explanations of the probabilities predicted by black-box models. Our work makes two main contributions.
First, this work proposes a natural way of evaluating the quality of an explanation with respect to a ranking. Explanations are evaluated based on the size of the data points that satisfy the explanation (i.e., support) and the average ranking of those data points, a novel metric called the Average Black- Box Ranking (ABBR). Explanations that marimize ABBR provides interpretability for when a black-box model prodicts 1 with high confidence. Naturally, one can simply flip the class labels 0 and 1 to find similar explanations for the class. The ABBR metric can be justified through a natural connection to the traditional consistency metric used to evaluate an explanation on discrete 0/1 predictions. While any probability ranking can be discretized into 0/1 predictions by choosing a threshold, an explanation with high ABBR is highly consistent with a model's predictions of 1, averaged across all possible thresholds.
Second, given the ABBR metric, this work develops algorithms RuleListSearch and ChecklistSeareh which seek to find explanations based on rule lists and checklists respectively. Both algorithms attempt to find explanations with high ABBR given desired target support. A rule list Angelino et al. (2017 2018 Letham et al.] (2015) is the union (logical OR) of one or more if-then rules; a data point satisfies rule list if it isties any of the rules in it. This work also proposes explanat ions using checklists, which as the name suggests is a collection of K conditions and a threshold such that a data point satisfies the checklist if it satisfics at least t conditions. Figure shows an example of a rule list with M 4 and Figure [T shows a checklist with K- = 4 and t = 3. Both algorithms search for explanations based on local search. Start by generating large pool of rules or conditions, filtering them down to rules with high ABBR, and searching
2

---

Published in Transactions on Machine Learning Research (08/2025)
If any of the following rules are satisfied:
If at least 3 of the following conditions are satisfied:
(Age > 50 and BMI > 35)
Age > 60
(BMI > 30 and High Blood Pressure)
BMI > 30
(Heart Disease and BMI > 28 and Smoker)
High Blood Pressure
(Age > 65 and BMI > 28 and Exercise <, 20)
Exercise < 20
then predict for diabetes.
then predict for diabetes,
Rule List
Checklist Figure 1: Examples of a Rule List and a Checklist
the space of rule lists or checklists by randomly making swaps with the pool using simulated annealing Kirkpatrick et al. 1983). Both algorithms are also versatile in that a user can control the sparsity of resulting explanation. For rule lists, one can set the number rules M and the maximum number of conditions in each rule, while for checklists one can set the number of conditions K. When evaluated on 6 real-world datasets, RuleListSearch generates rule lists consisting of up to 5 rules whose ABBR is within 8% of the best possible ABBR, while checklist rules perform slightly worse, but in return provide higher degree of interpretability.
Reproducibility. Code and datasets will be released on GitHub upon acceplance. Pseudo-code is available in Appendix
2 Related Work
Interpretable Rule-Based Classificrs. The simplest logical building block of any classifier are if-then association rules. Such rules serve as the basis for many interpretable classification algorithms, including decision trees Quinlan (1966) Breiman et al. (2017); Safavian & Landgrebe (1991), rule lists (i.e., an ordered
list of if-then statements where earlier rules prioritized) Liu et al (1998) Li et al. (2001): Wang & Rudin
(2015): Chen & Rudin (2018) ;
Pan et al. (2020): Dash et al. (2018); Letham et al. (2015) or rule sets (i.c,
an unordered set of rules where predictions are made by a weighting scheme) Angelino el al. (2017 2018 |);
Lakkaraju et al. (2016). Such rule based classifiers are often constructed by first mining for a large pool of association rules using association rule-mining algorithus such as Apriori Agrawal et al. (1996) or FPGrowth Han et al. (2004), or using commercially available software like WizWhy Meidan (2005). While such classifiess are highly interpretable, optimizing for them can be challenging due to the discrete nature of association
rules.
Explanations of Black-Box Models. Recognizing the importance of interpretabilty in practice, growing body of literature develops techniques for explaining black- box models. One type of explanation is local explanation, which takes a particular prediction and seeks to understand how small perturbations affect the prediction. This can be done by approximating the prediction locally with a linear model Ribeiro el al.] (2016); Yoon et al. (2019); SShankaranarayana & Runjel (2019); Zafar & Khan (2021) or classification trees Guidotti useful in understanding counterfactuals (c.g., what is the bost way for
et al. (0018). Such explanations patient to reduce their risk of diabetes). On the other hand, global explanations seek to find overall patterns in how the model predicts, often by approximating the model with an interpretable surrogate Guidotti et al. (2022): Lakkaraju et al. (2016) 2019). Global explanations also cover visualizations for understanding feature importance such as partial dependence plots (PDP) Friedman (2001) or individual continuous expectation (ICE) Goldstein et al. (2015).
Some work also straddles the line between local and global. Shapley scores (SHAP) Lundberg & Lee (2017 can be assigned to an individual prediction, detailing the contribution of each feature to that prediction, but also be viewed in aggregate across the whole dataset. Recent work globally consistent rule-based explanations Rudin & Shaposhnik] (2023) seck to explain local prediction by finding a rule with the largest support that is perfectly consistent with the black-box model's 0/1 predictions. An interesting way of adding interpretability to a black- box model is through a 2-stage classifier, where rules are first used to make some predictions, then a black-box model is used for the rest Wang & Lin (2021): Ferry et al. (2023). Such rescarch has shown that part of a black- box model can often be replaced by an interpretable model at little cost performance.
3

---

Published in Transactions on Machine Learning Research (08/2025)
3 Problem Setup and Notation
Denote the input data as (Xn; Un) m= where X,, E RD is a D-dimensional feature vector and yo E {0, 1 is the larget outcome of interest. Let b: _R_ + [0, 1] be binary classification model that assigns a predicted probability that the target is for any data point in the feature space. No assumptions are made on the model that yielded these probabilitics. Without loss of generality, assume that (X,) in descending order based b, i.e,
b(x)) b(Xx-1).. ((x)) 2 b(X1)
(ties are broken arbitrarily). For any data point Xa, we will refer to n as the rank of Xn with respect to b.
3.1 Rule Lists and Checklist Rules
We now define rule lists and checklists, the structures of the explanations studied in this paper. To build up to defining a rule list, we first define a condition and a rule.
3.1.1 Conditions.
A condition = (d, v) consists of a feature d E [D), an operator o (S,=,2) and value E R, for example "age 2 50' or "gender = male'. Let c(X,) E {0,1} be the indicator of whether data point Xn satisfies the condition c.
3.1.2 Rule.
A rule r= (cI...., CL) - z of order L is a conjunction (logical AND) of L conditions c1,...,Cy conditions along with a predicted target outcome E {0, For example, the statement "If age 2 50 and BMI 2 30, then diabetes = 1' is a rule of order 2. Since this work seeks to explain when the black-box model predicts with high confidence, this paper assumes that all rules r predict z = 1; hence, we define rule simply by (cI....ccL) CL).
A data point X, satisfies the rule if c(X,)) = 1 for all (ei....,ct) ct. (i.e., the data point satisfies all conditions of the rule). Define r(Xn) as the indicator of Xn satisfying rule r.
r(x,) -(1
Ve, :r::
(1)
otherwise
3.1.3 Rule List.
A rule list (ri...., rM) of length M is disjunction (logical OR) of M rules. X, satisfics the rule list if it satisfies least one of the rules ri,..r, TM- Let ((Xa) be the indicator of whether Xn satisfies rule list l.
{
3r, E (xx))
e(x,))
(2)
otherwise
For notational convenience, let [N] C [N] be the set of data points that satisfy l.
[N]e = {n e [M] f(x,)= 1}
(3)
Note that since the index n [N] also the ranking under the black-box model, [N]e contains the ranking of the data points that are satisfied by l. Since any rule can be considered a rule list of length 1, also denote [N], as the set of data points that satisfy
3.1.4 Checklist.
A chocklist k; = (cI...., .x) defined by list of K conditions and a threshold E N. A data point X,, satisfies ke if it satisfies at least out of the K conditions. Let (xx,,, be an indicator for whether X,, satisfies k
4

---

Published in Transactions on Machine Learning Research (08/2025)
14
13
18
20
2
15
[
8
17
9
Figure 2: An example of [N]e for rule list for N 20, where the ranking of each point is marked. The
support is supp(e) = 5/20 0.25 and the ABBR is abbr(f) 20 avg( (14, 15, 16, 18, (0,) = 0.83
[1
K 3 2t
kX,,,)
(4)
otherwise
A checklist spocial case of rule list by listing out (*) rulcs, one for each of the (*) ways that conditions out of K can be satisfied.
3.2 Evaluating a Rule List: Support, ABBR and Sparsity
A rule list (of which a checklist rule is special case) provides useful explanation of confident (i.e., highly ranked) black-bor box predictions if (1) [N] is large in size, and (2) [N] contains data points with large rank average, and (3) is sparse (i.e., interpretable). We define each of the three criteria more formally.
3.2.1 Support.
The support of f is defined as the proportion of data points in [N] (i..., data points that satisfy E) Definition 1 (Support).
supp( () N |[N]
(5)
A rule list with higher support explains the model's behavior across more data points, and thus is more desirable.
3.2.2 Average Black-Box Ranking
The ABBR of a rule list is simply the average (avg) value of the ranks [N]e, normalized by & to give a number between 0 and 1.
Definition 2 (Average Black-Box Ranking).
1 abbr(() = N (Vg((
(6)
In addition to rule lists, Support and ABBR are defined on a rule by treating it as rule list of length 1. An example calculation of support and ABBR is shown in Figure 2 Note that both high support and ABBR are nceded for a usefuul rule- a rule that is satisficd by single data point can have high ABBR, while a vacuous rule like "If (Age 2 0)* can easily be satisfied by the entire dataset.
3.2.3 Sparsity.
For a rule list or checklist to be easily interpretable, sparsity is crucial. For a rule list, sparsity is measured in two ways: (1) the number of rules the rule list, and (2) the maximum order (number of conditions) each of those rules. For checklists, sparsity is measured by the number of conditions in the checklist. The algorithms developed in this paper will allow user to explicitly set limit these measures of sparsity.
5

---

Published in Transactions on Machine Learning Research (08/2025)
3.3 Connection between ABBR and Consistency.
There is natural connection between ABBR, novel metric calculated the rankings of the model, and consistency, traditional metric calculated discrete 0/1 prodictions. Under 0/1 predictions, rule list e that explains when model predicts 1 over 0 can be evaluated based on its consistency, the proportion of [N] that are predicted as 1. For any m E [M], the black-box's predictions can be discretized at rank m, such that b(X,) = 0 if n < m and b(X,)) = for n 2 m. Equivalently, this is the proportion of [N]e with rank at least m.
Definition 3 (Consistency with Threshold m). The consistency of to the predictions of with threshold chosen at rank m is:
consistenncy(..n m) [N] n {m, m + ....
[N]
Rather than subjectively choosing a particular threshold m to discretize the black-box's predictions into 0 and 1, ABBR captures the average consistency across all possible thresholds. The following Lemma states this connection between ABBR and consistency.
Lemma 3.1 (Equivalence to Average Consistency). For a rule list t, an equivalent definition of the Average Black-Bo Ranking from Definition given by:
abdr(() = Σ
consistency((,
A proof of this lemma can found Appendix
B
While ABBR is an intuitive metric generalizing the notion of consistency, we highlight the value of ABBR in Appendix A quantitatively by showing that rules which have higher ABBR tend to have better generalizability to unseen data, compared to rules that are just optimized for consistency with respect to an arbitrarily chosen threshold.
4 Optimizing for Rule Lists and Checklists
This section presents RuleListSearch and ChecklistSearch, algorithms that search for rule lists and checklists that maximize ABBR and support while satisfying user-defined criteria. The algorithms explicitly constrain the sparsity of the resulting rule list and checklists, as explained in Section 3.2.3. Section 4.1
formally defines this objective, while Section ion 4.2 presents an aliic framework of the two approaches.
Details of the two algorithms are presented in Sections and 1.4
4.1 Objective: Maximize ABBR Given Support and Sparsity
This paper presents algorithms that take as input (1) sparsity constraints, and (2) a targel support s & [o, 1] and seek to find rule lists or checklists with high ABBR that satisfy both the sparsity constraints and has support of s. Using a specific support target s, it possible to compare different algorithms; Given two rule lists or checklists with the same support, the one with the highest ABBR is preferred. However, technical issue that it is typically infeasible to find rule list whose support is eractly due to the discrete nature of rule list. It is unlikely that there exists a rule list satisfied by exactly N data points (that is, not single data point more fewer). Furthermore, even if one could find such a rule with support dataset, its support on unseen data will certainly not be which complicates out-of out-of-sample evaluation.
particular
In light of this, let abbr(f, s) be the ABBR of = (r1,..., TM) interpolated at the target support s for A supp(f). Let lm = (F1...., Fm) be a sub rule list consisting of the first m rules of E. Define this m such that:
supp(Cm-) < supp(emm-t))
(9)

---

Published in Transactions on Machine Learning Research (08/2025)
ABBR
Swaps
Support
Support
Condition
Condition Candidates
Checklist
Rule Extraction
Rule Candidates
Rule List
Phase
Phase
Phase
Phase 2
ChecklistSearch. Conditions č are extracted from a RuleListSearch. A large set of rule R is extracted each feature d E [D] by discrelizing continuous features, using Random Forest (shown) or WizWhy (not shown), then filtered to C based on support and ABBR to gener- then filtered to R. based support and ABBR to gener- ate set of condition candidates. Then, local search is ate a set of rule candidates. Local search is performed performed by randomly swapping out a condition from by randomly swapping rules from the rule list with the checklist and condition from the set of condition from the set of rule candidates.
candidates.
Figure 3: Local Search Algorithmic Framework.
This m² is chosen such that support of points satisfying the first m rules is less than but adding those that satisly +m++1 causes the support to exceed
Intuitively, abbr(e, s) is calculated by taking the average ranking of set with support exactly s, one that is induced by the ordering of the rules in f. This set is constructed by taking all the points in [N] (whose support is supp(fm.)) and arbitrary subset of the points in [N] \ [N]er such that this subset has support supp(em-)). Across all choices of this arbitrary subset, the average ranking of this subset is equal to the average of the entire set [_Vc,,.+ \ [N]e, Let a == supp(("n) be the proportion of this reuslting set with support that came from [N] instcad of [N]em.+4 \NN]e.. The average ranking of this set of support exactly given by the following convex combination.
abbr(E, = N N..aav([[]e,, + - (0) a))ggg)N))mm) \ (N)e..)
4.2 Algorithmic Framework
The algorithms RuleListSearch and ChecklistSearch seek to find rule lists or checklist rule he that maximize abbr(-.s) s) as defined in Equation equation 4.1 Both RuleListSearch and ChecklistSearch follow the same 2 phase algorithmic framework based on local search.
Phase generates a set of rule candidates R or condition condidates C, which are the building blocks for selecting strong rule lists or checklists, respectively. This involves first generating a large set of rules R or conditions C, then filtering them down manageable size by only keeping the rules or conditions that excel in support and ABBR.
Phase performs local search to maximize abbr(-.s). Starting with an initial rule list or checklist, rules from the rule list conditions from the checklist are randomly swapped out with those in the candidate set. Simulated anncaling Kirkpatrick et al. (1983) used to effectively balance exploration (that is, searching the space of possible rule lists or checklists) and exploilation (that is, making changes that improve the objective function).
Figure 3a and 3b provides visual overview of Phase 1 and Phase 2 of RuleListSearch and ChecklistSearch. We dive into the details of the implementation below.
Since these nested rule lists have increasing supports, 0 supp((1) supp((2) ... supp((s) this m² is well defined. edge cases, define &o as rule satisfied by no data points (l.e., supp(fo) abbr(fo) 0) and add dummy rule TM-1 that satisfied by all data points so that supp(fxr41) a,abbr(fg41) 0.5.

---

Published in Transactions on Machine Learning Research (08/2025)
4.3 Details of RuleListSearch
RuleListSearch takes as parameters: (1) E [o, 1), the target suppori, (2) The maximum order of each rule L and (3) The maximum length of the resulting rule list M.
Phase of RuleListSearch first builds a large pool of rules R. which is then filtered to a refined set of candidate rules R.
4.3.1 Phase 1: Generating R.
We generate a large pool of rules R in the following way.
Case = Use WizWhy Meidan (2005): software tool which exhaustively searches a dataset for rules of order up to I and support at least ς. WizWhy relies on the Apriori Algorithm Agrawal al., (1996), a well-known algorithm that was developed to efficiently frequent item-sets in market basket analysis (e.g., if consumer buys bread and peanut butter, then they likely to buy jelly). For higher order rules, exhaustive enumeration is typically computationally intractable.
Case 2. L23 3. Higher order rules are generated from leaves of Classification and Regression Trees (CART). n
n 1N That is, train a Random Forest Regressor on the dataset {Xns N Jn=1, treating N as a continuous dependent variable. The maximum depth of each tree the random forest. set as L,, the minimum proportion of data points in leaf node set as 0.02 and fix the number of trees 50. Splits in cach tree are chosen minimize mean-squared error for predicting rank, which will separate data points with high and low rank. For each of the 50 trees, extract a rule from each leaf node by noticing that any path from the root node to leaf defines list of conditions (c.g., Age 60, BMI < 35). Choosing L as the maximum depth of cach tree ensures all rules extracted have order at most L, while choosing y as the minimum proportion of data points in a leaf ensures all rules have support at least. 7.
4.3.2 Phase 1: Filtering R R.
The rules in R are filtered to a more manageable size R. Since R was generated cither exhaustively or via a heuristic, R likely contains many rules with low ABBR. Intuitively, a rule list. with high ABBR and support is composed of individual rules which also have high ABBR and support P Since Phase 2 chooses rules from R to build rule lists, filtering R into improves the local search's chances of finding better rule lists.
The filtering criteria defined below:
R: = fre E R: :r'r E R such that supp(r') >> supp(r) and abbr(r') > 1.1 a11-a0 abbr(rr))
(10)
We add a rule r E R to R if it is approzimately Pareto optimal, that is there does not exist another rule I' E R with strictly higher support and ABBR that is at least 1.1 times higher. A Pareto optimal rule is one that cannot be dominated in both support and ABBR, but our approximate Pareto optimal definition allows ABBR to be dominated by up to factor of 1.1.
4.3.3 Phase 2: Local Search
A local search procedure based on simealated annealing Kirkpatrick et al. (1983) is used to find a rule list with high abbr(e, s). Simulated annealing local search paradigm that effectively balances between exploration and exploitation. A starting rule list g(0) is constructed by taking the M rules from R. with the highest ABBR. Over T = 500 iterations, in iteration E [T], candidate neighbor e' is generated by swapping out a randomly chosen rule from g(i-l) and a rule from R. To decide whether to update g(i) to l' or keep g(s) at (i,1),, simulated annealing compares abbr((", s) and abbr(((i-1). s).
If abbrrff, (f', abbr(((i-1), (i.e., L' is an improvement over ((i-1)), then update {0) to be L.
Note that although we defined ABBR and support for rule lists in Section the same metrics and be applied individual rule condition by considering rule rule list. of length and condition rule of order 1.
8

---

Published in Transactions on Machine Learning Research (08/2025)
If abbr s) abbr(e(i-1), s), then with some probability update g(9) = f", otherwise keep ( gli-1), This probability of an update depends on (1) The difference in the objective function abbr(eli-1,,. .) - abbr((",s) (i.e., how much worse e" is compared to g(i-1) and (2) the value of i relative to T (i.c., how close the algorithm is to termination). For small values of i, simulated annealing is willing to explore and moves to neighbors l' even if abbr ,) S abbr (bbr((((-1).; but as approaches T, the algorithm will exploit by only take steps that strictly increase abbr (bbr(e((-1)ia))
See Appendix D for details the Simulated Annealing algorithm.
4.3.4 Final Rule List.
Let g(T) ( MM) be the output of the Simulated Anncaling algorithm. Note that g(T) always has M rules regardless of s, but some rules may be removed without affecting the ABBR. As a final step, RuleListSearch removes rules from the end of ((T) while keeping ABBR the same. Let m be define in equation equation 9 which states that abbr(s, (()) only depends on e and ((T) mm+ we can remove all the rules +2.+2:.... T' without changing the ABBR at support s. The final output of RuleListSearch is (2) mm"41"
4.4 Details of ChecklistSearch
ChecklistSearch takes as input (1) the target support E [0, 1], and (2) the number of conditions K in the checklist. ChecklistSearch follows the same 2-phase algorithmic framework as RulelistSeerch except. that it works with conditions instead of rules. Phase 1 builds a large pool of conditions and filters down to C, while Phase 2 performs local scarch for checklist using C generate neighbors.
4.4.1 Phase 1: Generate C
A pool of conditions is generated by going through each of the D features. Recall that we assumed each feature is numerical (categorical variables have already been encoded).
If a discrete feature, add the conditions (d)=,)) for all possible values v feature (e.g., Race
1, Race et..e.c.).
If a numerical feature, break the feature into 5 intervals based on the quantiles of the empirical distribution. Let Q1, 92, 93, q4 be the 1/5, 2/5, 3/5 and 4/5 quantiles of the values {Xnd) We add the conditions (d, 2, 91), (d. 2. 9)....... .. (d, S. 92). (d. 2,4
4.4.2 Phase 2: Filtering C to C.
For each condition, treat it as a rule with single condition and evaluate support and ABBR. We filter for conditions with a support least 0.02 and no more than 0.02 0.98 and ABBR of at least 0.5.
C= EcecC C supp(c) > 0.02, (c) < 0.98, abbr(c) 2 0.5)
(11)
Conditions satisfied by too few data points (e.g., Age > 95) too many data points (e.g, Age 95) are lypically not useful in checklist. Since ChecklistSearch seeks checklists with high ABBR, it removes conditions with ABBR less than 0.5 as those are correlated with predictions of 0 instead of 1.
4.4.3 Phase 3. Local Search
Perform local search using simulated annealing identical to that from RuleListtearch.. A starting checklist k is chosen by be the top conditions from C based on ABBR and computing threshold maximize the objective(iee,,
arg maXve(K] abbr(ker,s)
For each iteration i E [T], similar to the local search for rule lists, generate a neighbor hy by randomly swapping one of the conditions in 1 (i-1) with from C and choosing the appropriate threshold that maximizes ABBR

---

Published in Transactions on Machine Learning Research (08/2025)
Datasct Name
Example Features
Target Outcome
Black-Box AUC
Adults
30,000
14 Gender, age, ethnicity, edu- cation, marital status, occu- pation
Income >50K
0.900
Diabetes
30,000
21
BMI, age, smoker, high blood pressure, high choles- terol
Patient develops diabetes
0.817
FICO (Loan)
10,459
23 Risk estimate, month since last transaction, income
Default on the loan in next years
0.788
Recidivism
25,835
48 gender, race, age at release, gang affiliated, education level
Commits a crime again in the next 3 years
0.764
Schizophrenia
12,380
Gene SPEN Pos 16262471 Gene NBPF1 Pos 16905788
Patient is diagnosed with Schizophrenia
0.675
Readmission
30,000
hospital ID, admission type, length of stay, treatments re- ceived
Discharged patient was readmitted to the hospital within 30 days
0.659
Overview of Datasets used in Empirical Experiments. This tables shows the number of rows N, features D, examples of some features, the target outcome of interest and the AUC of the black-box
Table 1:
Random Forest Classifier seek explain. Note that datasets with N 30, 000 have been sampled from
larger dataset to reduce computational time. Sources for the 6 datasets are follows: Dua & Graff (2019a).
Dua & Graff (2019b), FICO 20188) ProPublica 2016). for Biotechnology Information (2024), Dua & Graff
(209cc
at support 8, that is 7 arg maxyre[]] abbr(ke, 8). Simulated annealing is used to determine whether to keep x(()) k. (t-1) update it to x(() kz-
5 Empirical Results
This section presents empirical results for RuleListSearch and ChecklistSearch on 6 real-world datasets for binary classification shown. All datasets are binary classification tasks chosen from variety of application areas, including healthcare, finance, economics, and criminal justice. The 6 datasets are summarized in Table
Our results are organized into two subsections:
Section 5.1
Experiments that show our RuleListSearch and ChecklistSeerch algorithms generate strong rules that achieve ncar-optimal ABBR on all 6 datascts. Ablation studies are performed explain the value of higher order rules, longer rules, and interpretability.
Scction 5.2
Comparison of RuleLLstSearch a statc-of-the-art method from Rudin & Shaposhnik 22023) highlighting value of Phase 1 (candidate rule generation) of RuleListSearch.
5.1 Achieving Near-Optimal ABBR
Setup. Rule lists generated with L E (2, (maximum order of each rule) and M {3,5) (maximum number of rules in the rule list). Checklists generated with K E {5, 7} (number of conditions). The target support s is chosen from 10 0.2}. The black-box predictions (b(X.)) Forest Classifier trained with 500 estimators. All empirical results are reported out-of-sample on a holdout. set of data not used finding the rule list or checklist. Experiments were run on MIT's Engage Compute Cluster over 100 train splits, taking around hour of compute time when parallelizod with 24 cores. Each instance of RuleListSearee or ChecklistSearch take no more than seconds to run on the 6 datasets in Table 1 Results shown the average of the test performance across the 100 train test splits.
are generated from a Random
10

---

Published in Transactions on Machine Learning Research (08/2025)
Target Support
10%
Target Support 20%
Rule Lists
Checklists
Rule Lists
Checklists
Dataset
Best
K=5
K=7
L=3
Best
K=5
K=7
M=3
-6.6%
-3.5%
0.1%
M=3
-6.2%
-.99
-3.5%
Recidivism
92.4%
90.4%
91.0%
91.6%
85.4%
89.8%
M=5
-6.5%
-3.9%
X
M=5
-6.5%
-2.8%
M=3
-9.1%
-3.5%
-0.4%
M-3
-8.3%
-2.9%
-1.8%
FICO
92.8%
91.7%
92.7%
94.1%
92.3%
92.4%
M=5
-9.7%
-3.3%
x
M=5
-7.6%
-2.0%
-0.1%
-2.9%
-1.5%
Adults
98.2%
94.0%
95.0%
95.9%
90.8%
93.5%
M=5
-2.6%
-0.2%
M=5
-0.4%
0.0%
*
M-3
-9.8%
-3.7%
-0.3%
M-3
-8.2%
-0.5%
-0.2%
Diabetes
96.6%
95.6%
95.6%
94.8%
94.1%
97.0%
M=5
-9.7%
-3.4%
*
M=5
-7.1%
-0.3%
M-3
-4.7%
-1.3%
2.4%
Mw3
-4.2%
-2.1%
-1.7%
Schizo
97.2%
96.5%
98.9%
84.7%
88.8%
89.2%
M=5
-0.3%
M=5
-2.0%
-0.2%
M=3
9..4%
4.4%
0.4%
M=3
9.11%
2.8%
0.2%
Readmission
86.3%
81.0%
85.5%
87.0%
86.0%
87.0%
M=5
-8.8%
-4.0%
x
M-5
-9.7%
-1.9%
x
Table Optimality ratio of various methods for 8 = 0.10 (left) and & == 0.20 (right). each table, rule lists are generated at M E 5) and E {2, 3, 4), with the best optimality ratio (which is always L= 4, 5 5) shown in the "Best" column. Shaded cells represent the absolute difference in optimality ratio from the "Best' column. For checklists, the optimality ratio is shown for K 5 and K 7. The best algorithm in each row is shown in blue text.
For each generated rule list or checklist, we report its optimality ratio, defined to be the ratio between its improvement over naive baseline versus the inprovement an optimal rule relative to a naive baseline. A rule list with support can have maximum possible ABBR of 1- s, which is achieved if the rule list is satisfied by eractly the top percentile of the dataset (i.e., those with ranking - s) N through N, for
ABBR of 1 - s). A naive baseline for ABBR is 0.5 randomly chosen subset of the data will have ABBR on average 0.5. Hence, the following definition:
Optimality Ratio(E, 8) abbr (f, s) 0.5
(12)
8 0.5
Table presents the main results of this empirical study, with s 0.10 shown in the left table and 0.20 shown in the right table. Results for RuleListSearch are reported under "Rule Lists'. which varies L E (2,3,4) 4) and M E {3,5} in matrix. The value of L and M that generates the top optimality ratio is marked with a star and its optimality ratio is reported in the column "Best', while the colored cells represent the absolute delta with the optimality ratio of "Best". For ChecklistSearch, the two columns under "Checklists" show the optimality ratio for K and K = 7. In both tables, the algorithm with the highest optimality ratio is highlightod bluc.
Tt is clear that RuleListSearch and ChecklistSearch exhibit strong performance across 6 datasets. Across == 0.10 and 20, the best rule list achieves average optimality ratio of 0.926 (an optimality gap of 7.4%), while the best checklist achieves an average optimality ratio of 0.923 (an optimality gap of 7.7%). Some examples of the resulting rule lists and checklists for each dataset are shown in Figures 6 and in Appendix
5.1.1 Sparsity vs. ABBR Trade-off in Rule Lists
Although larger values of L and M result in less sparsity and lower interpretabilty in a rule list, they give rise to rules with higher ABBR for any support. This trade-off between sparsity and performance is evident.
11

---

Published in Transactions on Machine Learning Research (08/2025)
L Rule
Support
ABBR
2
If Heart Discase 1 and High Cholesterol
0.07
0.902
3
If Difficulty Walking = 1, High Cholesterol = 1 and BMI > 29
0.05
0.943
4
Tf BMT > 29, General Health < 3, High Blood Pressure and Heart Disease
0.03
0.964
Table 3: Rules in R with the Highest ABBR for L E 32,3,4) (Diabeles Dataset)
L=2
L-3
L=4
ABBR
0.80
0.00
0.00
0.75
0.75
0.75
0.70
0.70
0.70
0.09
0.10
0:35
0.05
0.10
0.15
0.05
0.10
0.15
Support
Support
Support
R for L 2
b R for =
C R for L =
Figure 4: Support and ABBR of Rules in R for various L. in the FICO Dataset Larger value of L lead to rules in R with higher ABBR and thus leader to higher ABBR when combined into rule lists.
in Table 2 as the optimality ratio is generally increasing in both L and M, with the best optimality ratio achieved at L = and M 5.
Value of Higher Order Rules. First, we study the impact rules of higher order (increasing L). Higher order rules are able identify more specific subsets of the data, including ones where all data points are ranked more highly by the black- box model. This can be seen by examining the pool of rules that RuleListSearch uses to construct rule lists. Figure 4a 4b and 4c plot the support (æ-axis) and ABBR (y-axis) of the rules in R. for L 2,3 and 4 in the diabetes dataset. As L increases, more rules appear in the high ABBR but lower support region (i.e., top left region in the figures). Table 3 shows the rule with the highest ABBR in each of the 3 rule candidate sets, which is marked with star. While the rule for L= 2 is fairly broad (i.c., there are many patients with both heart discase and high cholestcrol), the rule for L. == 4 is much more specific and thus has lower support, but captures a subset of the population which on average are very high predicted risk of diabetes. Intuitively, it is unlikely that a rule with just 2 conditions can identify set of patients, all of which confidently data point due the existence of numerous risk factors, and hence explaining such data points requires higher order rules that can filter for patients with multiple risk factors. Since RR for I, = 4 has more rules with high ABBR than L = 2,3, it is clear that by combining such rules together, we can get rule lists with better ABBR.
at high risk of diabetes. Black box model typically prodicts
Value of Longer Rule Lists M. Given that rules with higher ABBR tend to have lower support, the length of rule list M also crucial. As M increases, the rule list can fit more rules with low support and high ABBR, whereas small values of M force each rule of the rule list to have higher support and in turn lower ABBR. This is evident in Table 2 as the benefit of M 5 over M = 3 is
prominent for 0.20 over = 0.10. To achieve target support of 0.20 with M = 3 rules, cach rule must have average a support of 0.066, thus compromising its ABBR. On the other hand, when M = 5, each rule must only have a 4% coverage on average, allowing us to choose rules with lower support and higher ABBR. Similarly, the optimality ratios of the best rule lists are higher for M = when s = 0.10 over s = 0.20 for the same reasoning. Given a fixed M, lower target support allows the rule list to use low-support, but high ABBR rules, while larger target support precludes a rule list. from using such rules.
12

---

Published in Transactions on Machine Learning Research (08/2025)
5.1.2 Improving Interpretability with Checklists
Although rule lists perform well, rule lists with M = 5 rules and L = 4 conditions per rule may be difficult to interpret. Checklists are designed with greater interpretability in mind. Table Table@showe shows that while checklists tend to slightly under-perform rule lists This can be explained by discussing the tradcoffs between rule lists and checklists.
0.10, they can sometimes match or exceed rule lists at s = 0.20.
Advantage of Checklists. The main advantage of checklists over rule lists is its compact form and interpretability. A rule list with M = 5 and L = 4 can have 5 rules with 4 conditions each, resulting in a verbose list involving up to 20 conditions. On the other hand, the compact representation of a checklist allows it to implicitly define rule lists involving many rules without being difficult to interpret. For example, a checklist with K = 7 and T = 5 implicitly defines a rule list with (2) = 14 rules. While the rule lists in our experiment are constrained to maximum of M = 5 rules, checklist can bypass this constraint as it can implicitly define rule lists with many more rules. This is especially advantageous when the target support is high, hence why checklists perform better at = 0.20 than 0.10.
Disadvantage of Checklists. The main disadvantage of checklists is that by assuming an equally weighted linear relationship between each of its K = 5 conditions, the checklist may inadvertently capture combination of conditions that have low ABBR. For example, consider a checklist f3 = (c1, (21,02, C2 C3 G4); a checklist with 3. This chocklist is equivalent to a rule list with the following 4 rules rı- (cI, (2, (z), 12 (2, c4); ra (c1, ca, c4) and r4 (c2, It may be the case that while abbr(ri), (bbr(r2) and abbr(ra) are high, abbr(ra) is relatively low. Intuitively, this means that e1 is very crucial condition that should be weighted more than the other conditions. While rule list would be able to explicitly list out rules r2,72 and r's checklist is forced to also include r4, thereby dragging down the overall ABBR.
K 4 conditions and threshold
At low target support (i.e., s= 0.10), the disadvantage of checklists outweight the advantages as rule lists can better capture a specific list of rules with high ABBR. On the other hand, higher target supports like 0.20 start to favor checklists for some datasets longer rule lists needed for rule lists to achieve this support, but rule lists are constrained by M = 5.
5.2 Comparison to Baseline Method
In this scction, we propose natural bascline algorithm to compare against RuleLlstSearch, First, recall that the RuleListSearch algorithm consists of two broad steps:
1. Mining for a set of candidate rules through Random Forest. Candidate rules are chosen to optimize ABBR while satisfying a minimum support.
2. Combining the rules together to form a Rule List, using simulated annealing to optimize for the in-sample ABBR at a target support.
Note that in the body of the paper, we refer to Step 1 above as Phase (mining) and Phase 2 (filtering rules), while we referred to Step 2 as Phase 3 (optimization).
5.2.1 Benchmark Description
We compare RuleListSearch with a benchmark where we replace Step 1. above with a statc-of-the-art. method of generating a set of rules, in particular Rudin & Shaposhnik (2023)
Rudin & Shaposhnik 2'2 work Globally- Consistent Learning Models constructs local explanations for a given data point by finding a rule that is globally consistent, i.e. matches the predictions of a black-box algorithm entirely. For example, if a patient where a black-box model predicts the patient will develop diabetes, a globally consistent explanation for this patient. is an if-then rule under which all patients are predicted by the model to develop diabetes.
Hummary-EEplaancee
for Machine
While Rudin & Shaposhnik (0233) generates local explanations for a given data instance, we can aggregate these rules into global explanation using Step 2. Our benchmark approach is to replace Step above with
13

---

Published in Transactions on Machine Learning Research (08/2025)
rule-mining procedure where take the predictions of black-box algorithm, discretize them at E [0, 1], then construct a globally consistent local explanation for every data point for which the black-box model's s predicted probability is greater than We feed this large set of rules into Step 2, which outputs rule list while optimizing for ABBR. This is unchanged from still RuleListSearch.
5.2.2 Results and Discussion
We evaluate this baseline using two discretization thresholds: p 0.5 (a natural threshold) and p 0.9 (an aggressive threshold). We set minimum support of 0.1 for any rule in the pool of rules. Table 4 reports confidence intervals for the A ABBR of the final rule list after simulated annealing, comparing the baseling against RuleListSearch. Note that we were unable to generate results the benchmark in the = 0.9 case under a reasonable time limit, duc to a limitation discussed below.
Dataset Name
Our Algorithm
Benchmark (p= 0.5)
Benchmark (p 0.9)
Adults
0.941 = 0.031
0.697 = 0.062
N/A
Diabetes
0.933 + 0.010
0.912 + 0.007
N/A
FICO
0.913 = 0.020
0.862 = 0.018
N/A
Readmission
0.880 + 0.003
0.861 + 0.008
N/A
Recidivism
0.910 + 0.008
0.886 + 0.012
N/A A
Schizo
0.935 0.014
0.903 = 0.011
N/A
Table 4: Performance comparison between our algorithm and benchmark models with different values.
The results reveal fundamental limitations of approaches that rely solely binary prediction alignment:
Natural Threshold (p- = 0.5): When using a moderate cutoff, Rudin & Shaposhnik (0023
method successfully generates globally consistent rules, but these rules achieve significantly lower ABBR scores compared to RuleListSearch This occurs because global consistency with binary predictions fails to distinguish between instances with vastly different model confidence levels. For instance, patient with 51% predicted probability of diabetes is treated identically to a patient with 99% predicted probability, despite the substantial difference in model certainty.
A Aggressive Threshold (p= 0.9): With a high threshold, Rudin Shaposhnik (2023) approach fails to find globally consistent rules altogether. The requirement for 100% consistency becomes nearly impossible to satisfy when targeting only very high-confidence predictions, real-world datascts rarely contain perfect decision boundaries that achieve complete separation. [udin & Shaposhnik (2023) leverages an integer optimization program to find globally consistent rules involving up to k features, which becomes intractable when k is large.
This benchmark experiment demonstrates that existing approaches focusing on alignment with 0/1 predictions ignore the confidence of those predictions, leading to either (1) rules that don't 't't explain confidence, or (2) the lack of any rules at all. RuleListSearch fixes both of these by focusing rules that explain high-confidence predictions and mining for such rules using heuristic (i.c., Random Forcst) rather than an optimization approach.
6 Conclusion
This paper presented novel evaluation metric for model-agnostic post-hoc explanations that emphasizes when model predicts confidently. Rather than evaluate an explanation on how consistent it is with the binary predictions of black-box model, the ABBR metric rewards rules that capture large number of confident predictions. This paper argues that explaining confident predictions more useful than just explaining when a model prediction, which can possibly be a guess. With this goal in mind, this paper developed methodologies for finding rule lists and chocklists, both of which were based on a local search heuristic. Empirical results show that rule lists with up to rules achieve strong performance, attaining ABBR
14

---

Published in Transactions on Machine Learning Research (08/2025)
within 7.4% of the optimal possible ABBR average, while checklists can achieve better interpretability with slight reduction in performance.
References
Rakesh Agrawal, Heikki Mannila, Ramakrishnan Srikant, Hannu Toivonen, A Inkeri Verkamo, et al. Fast. discovery of association rules. Advances in knowledge discovery and data mining, 12(1):307-328, 1996.
Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Scltzer, and Cynthia Rudin. Learning certifiably optimal rule lists. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 35-45 44, 2017.
Elaine Angelino, Nicholas Larus-Stone Daniel Alabi, Margo Seltzer, and Cynthia Rudin. Learning certifiably optimal rule lists for categorical data. Journal of Machine Learning Research, 18(234):1-78, 2018. URL http://www..rr.ppappppaaa/////// 1tmm
Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
Leo Breiman, Jerome Friedman, Charles J Stone, and Richard A Olshen. Classification and regression frees. Routledge, 2017.
Chaofan Chen and Cynthia Rudin. An optimization approach to learning falling rule lists. In International conference on artificial intelligence and statistics, pp. 604 612. PMLR, 2018
Sanjeeb Dash, Oktay Günlük, and Dennis Wei. Boolean decision rules via column generation. Tn Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, pp. 4660 4670, Red Hook, NY, USA, 2018. Curran Associates Inc.
Dheeru Dua and Casey Graff. Uci machine learning repository, 2019a. URL https://archive.iessuu edh/dataaset/22adu15
Dheeru Dua and Casey Graff. Uci machine learning repository, 2019b. URL https://ww.arrchive. uci hril.eduddtaasee http://awaaa.tet....c.aaaaaaaaaa
Dheeru Dua and Casey Graff. Uci machine learning repository, 2019c. URL https://archive.icssuu
Julien Ferry, Gabriel Laberge, and UIrich Aïvodji. Learning hybrid interpretable models: Theory, taxonomy, and methods. arXiv preprint arXiv:2803. 04437, 2023.
FICO. Explainable machine learning challenge, 2018.
URL httpp://commnity.ffic... com/s/
National Center for Biotechnology Information. Nhlbi go-esp: Lung cohorts exome sequencing project
2phs0000473.v..p2), 2024. URL https://www.ncbi nlm. nih, www...i.....//pp.eeee http://///oooett///////////////a
studyy_drrphs00477..222
Jerome H. Friedman. Greedy function approximation: A gradient boosting machine. Annals of Statistics, 29
(5):1189-1232, 2001. doi: 10.1214/a0s// 10on///011311111
Alex Goldstein, Adam Kapelner, Justin Bleich, and Emil Pitkin. Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation. Journal of Computational and Graphical Statistics, 24(1):44 65, 2015. doi: 10.1080 10000000611000.200. 2014.907955.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Dino Pedreschi, Franco Turini, and Fosca Giannotti. Local ruse- explanations of black box decision systems. ArXiv, abs/1805.10820 10820, 2018. URL https
7/api saapi.sematticccholaa..cc wtt.://ccooortooooooooooaaaaaaaa
15

---

Published in Transactions on Machine Learning Research (08/2025)
Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Francesca Naretto, Franco Turini, Dino Pedreschi, and Fosca Giannotti. Stable and actionable explanations of black-box models through factual and counterfactual rules. Data Mining and Knowledge Discovery, 2022. doi: 10.1007/6400114--22-00077777 URL https://doi.org org. 10. 1007/ 0tt/://11061102200807770
Jiawei Han, Jian Pei, Yiwen Yin, and Runying Mao. Mining frequent patterns without candidate generation:
A frequent-pattern tree approach. Dala mining and knowledge discovery, 8:53 87, 2004.
S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. Science, 220 671 680, 1983. ISSN 00368075. doi: 10.1126/science.22 220.4598.671. URL http://www.jstorr org/stable/ 1690046
Himabindu Lakkaraju, Stephen H. Bach, and Jure Leskovec. Interpretable decision sets: A joint framework for description and prediction. Tn Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 1675 1684, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342322. doi: 10. 02.1145/2939672.222 29399999999989997772 URL battps://don org/
10. 00.1145/2939977 2939874
Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovee. Faithful and customizable explanations of black box models. In Proceedings of the 2019 AAAT/ACM Conference on AT, Ethics, and Society, AIFS '19, pp. 131 138, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 4701420333 doi: 10.1145/3 40.1145/330666 035/5/3336661131122200
URL htttps://do... ottps:://dii.or/1100 1145/3306618..... 3314229
Benjamin Letham, Cynthia Rudin, Tyler H McCormick, and David Madigan. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. 2015.
Wei Li, Jiawei Han, and Jian Pei. Cmar: Accurate and efficient classification based on multiple dlass-sssociation rules. Proceedings 2001 IEEE International Conference on Data Mining, pp. 369-376, 2001.
Bing Liu, Wynne Hsu, and Yiming Ma. Integrating classification and association rule mining. In Proceedings of the fourth international conference on knowledge discovery and data mining (KDD), pp. 80-86. AAAI Press, 1998.
Scott M. Lundberg and Su-Tn Tee. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Newral Information Processing Systems, NIPS'17, pp. 4768 4777, Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
A. Meidan. Wizsoft's ' wizwhy. In O. Maimon and L. Rokach (eds.), Data Mining and Knowledge Discovery Handbook, pp. 1365 1369. Springer-Verlays New York, 2005. doi: 10. 00.1007/0-387-2565-5 X 67.
Danqing Pan, Tong Wang, and Satoshi Hara. Interpretable companions for black-box models. In International conference on artificial intelligence and statistics, pp. 2444-2454. PMLR, 2020.
ProPublica. Compas recidivism risk score data and analysis, 2016. URL aaraatorre/attattt/////s/rrroara
scwre- -nwaaattaaaaaaaaaaaaaaaaaaaaaaaa
J. Ross Quinlan. Induction of dccision trees. Machine learning, 1(1):81-106, 1986.
Marco Tulio Ribciro, Samcer Singh, and Carlos Guestrin. "why should i trust you?': Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pp. 1135 1144, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450344322. doi: 10.11 21144///23367222337777 URL https:///oi..rg
10. Dp.1145/2939672 2939778.
Cynthia Rudin and Yaron Shaposhnik. Globally rule- based nummary--xplanation for machine learning models: Application credit-risk evaluation. Journal of Machine Learning Research, 24(16):1 44, 2023. URL http:://www...orpppprssss//////e wttg//ppers////4//1--0882 http::///mmmnlroorrrprrsr/////// html
S. Rasoul Safavian and David Landgrebe. A survey of decision tree classifier methodology. In IEEE Transactions Systems, Man, and Cybernetics, volume 21, pp. 660 674. IEFE, 1991.
16

---

Published in Transactions on Machine Learning Research (08/2025)
Sharath M Shankaranarayana and Davor Runje. Alime: Autoencoder based approach for local interpretability. In Intelligent Data Engineering and Automated Learning SeariingIDDFAT, 2019: 20th International Conference, Monchester, UK, November 14-16, 2019, Proceedings, Part 20, 454 463. Springer, 2019.
Fulton Wang and Cynthia Rudin. Falling rule lists. In Artificial intelligence and statistics, pp- 1013 1022. PMLR, 2015.
Tong Wang and Qihang Lin. Hybrid predictive models: When an interpretable model collaborates with a black-box model. The Journal of Machine Learning Research, 22(1):6085-6122 2021.
Jinsung Yoon, Sercan O Arik, and Tomas Pfister. RI-lim: Reinforcement learning-based locally interpretable modeling. 2019.
Muhammad Rehman Zafar and Naimul Khan. Deterministic local interpretable model-agnostic explanations for stable explainability. Machine Learning and Knowledge Erlmaciinn 3(3):525-541) 2021.
17

---

Published in Transactions on Machine Learning Research (08/2025)
A Justification of ABBR as a Useful Metric
In this section, we justify the usefulness of ABBR as a metric of explainability by arguing that rules with higher ABBR lend to be more generalizable to unseen data. Generalizability is crucial because explanations that overfit to specific training examples may give incorrect insights about the model's true decision boundary, leading to incorrect conclusions downstream. We leverage consistency with the 0/1 predictions of a black-box model well-established baseline metric. We show that rules optimized for in-sample ABBR exhibit better generalization than rules optimized for consistency alone.
A.1 Experiment Setup
We define the consistency generakizalion gop of rule as:
Gap(r) Consistency(r,, Vtrain) Consistency(r, Stest)
where Verain and Ytest are binary predictions from black-box model with some threshold p. This gap is typically positive, rules selected based on their in-sample performance. Recall that the consistency of a rule r with respect to binary predictions is simply the fraction of the data points that applies to for which 1.
Our exact experimental procedure is as follows:
Generate 1000 random rules of up to 5 conditions by randomly sampling features, and choosing thresholds (continuous) or values (discrete) for those features. Only keep rules with a minimum support of least 2%.
Out these 1000 rules, let тa be the rule with the highest in-sample ABBR, and let re the rule with the highest in-sample consistency.
3. We report the following: the precent decrease in the gap of rasee relative to Tronsistency-
Gap (op((coosittencg) Gap(rator)
(13)
Cap(Tconsistenccy
A.2 Results and Discussion
Dataset Name
Gap(ratbr)
Gap
% Improvement
Adults
0.3% 0.1%
1.1% = 0.2%
72.7%
Diabetes
0.1% = 0.0%
0.5% + 0.0%
80.0%
FICO
1.0% + 0.3%
2.8% + 1.1%
64.3%
Readmission
0.7% = 0.1%
1.0% = 0.1%
30.0%
Recidivism
0.2% + 0.3%
1.1% + 0.3%
81.8%
Schizo
1.5% = 0.2%
1.0% = 0.2%
50.0%
Table 5: Generalization gap comparison between ABBR-Rules and Consistency Rules
Table 5 shows the 95% confidence interval of Gap(Faomeisteeey) and Gap(rawo)) across 500 test/trains splits. The last column shows the percent improvement of the ABBR-rules versus the consistency rules. Across the datasets, of them show better generalizability in the ABBR optimized rules compared the consistency optimized rules.
The key insight behind ABBR's superior generalizability lies in how binary classification thresholds create artificial boundaries. When discretizing predicted probabilities using a threshold p, instances near the decision boundary are arbitrarily classified. For instance, with = 0.9, an instance with probability 0.89 receives label while probability 0.91 reccives label 1, despite minimal difference model confidence. Rules that rely heavily on such boundary cases are prone to poor generalization. ABBR mitigates this by considering the underlying probabilities rather than just binary agreement.
18

---

Published in Transactions on Machine Learning Research (08/2025)
B Proof of Lemma 3.1
Proof. Proof of Lemma 3.1 We leverage well-known fact: For any random variable 2 which takes on non-negative values with cumulative distribution function Fz(z) = P(Z < z), one way of calculating the expectation Z given by
E| = =
(1 (1- (z())
(14)
Take to a discrete random variable that takes on a value from the set [N] uniformly at random. Then, E|Z] is simply the ABBR metric, whereas 1 f2(2) is simply consistency(E,,) z). Finally, noting that the integral from 0 to be replaced with summation due to the step-function nature of fz(=), obtain the desired result.
C Pseudo-Code for RuleListMiner and ChecklistMiner
Pseudo-code for RuleListSearch and ChecklistSearch are provided in Algorithm and Algorithm
2
below.
The function Simulated Annealing is defined in Equation equation 15 in Appendix D
Algorithm RuleListSearch
1: Input: (Xna)ne(N) [0, 1], E N(max number of rules), L E N(maximum order each rule) Output: Rule List = (ri....,T.. Tm) with < M and |r, L.
if L then
R (
Large Pool of Rules
6:
for tree E RandomForeestRggressse (f n n==1 L, 0.02) do
for leaf E tree do R R-RU Rule(eaa) end for 10: end for
11: else
12: R + WizWhy((X,JH) 0.02)
13: end
14:
15: R= = [}
Filtered Pool of Rules
16: for R do
17: if Vr' E R it is true that supp(r') I supp(r) abbr(r') 1.1 abbr(r) then
18:
RU
19: end if
20: end for
21: R - sort (R, abbr)
Descending order ABBR
22:
23: (0)) R|: N]
Take first. S rules to form rule list.
24: for E [T] do
25:
+ rand(1,2.....
26:
n2 + rand(1, R
27:
f + 40) ['ni]= t -R[ng]
Candidate Neighbor to Move To
28:
29:
(() + Simulated Annealing( abbr(((-1) s), abbr(b', s), i,
30: end for
31: Output.
(CT) 11 where m² is chosen such that supp((m.)) supp(em'+1).
19

---

Published in Transactions on Machine Learning Research (08/2025)
Algorithm ChecklistSearch
Input: [M) [0, 1]
Parameters: K N (number conditions)
Output: Checklist k CK).
5: C++p 0
> Large Pool of Conditions
6: for d E [D] do
7: Ō-ūu Conditions(d, Q)
Discretizo or Ono-Hot-Encode
end for
10: C= {}
Filtered Pool of Conditions
11: for c C do
12: if supp(c) 2 0.02 & supp(c) s 0.98 & abbr(c) 2 0.5 then
13:
C+e
14: end if
15: end for
16: C sort(C, abbr)
Descending order of ABBR
17:
18: & C]: .V. arg [a abbr(k(,ss
Take first K conditions to form checklist
19: for E [500] do
20:
- rand(1n (1, , K)
21:
n2 < rand( (1, 2..,.....2 (C))
22:
b
Candidate Neighbor to Move To
k[n] + Rinz]
23:
24:
T arg max abbrrk'',.l v' 8)
(i-1) A((1) SA(abbr (( 8), abbr abbr(k; s),
25:
26:
end for
27: Output.
(T)
20

---

Published in Transactions on Machine Learning Research (08/2025)
fa (
U U<eep . -f(x))) - f(((-1))]] where U ~ Unif(0,
(()) Simulated Annealing(f Ameelinn((for--)))(((()))) =
otherwise
(15)
Figure 5: Simulated Annealing Updating
D Details of Simulated Annealing
Simulated annealing is a probabilistic local search heuristic which optimizes an objective function through finding candidate neighbors and moving to them with probability. It is often used for optimizations with a large and discrete search space. Suppose we are trying to maximize a function f(x) where æ E X is in imput in some high-dimensional space. Simulated anncaling works by first starting with solution z(()) and slowly updating it over the course of T iterations. In iteration E [7], the algorithm considers (({-1) and generates neighbor z' by typically applying small perturbation to z(1-1). We compare f(2(-1)) with f(x') and decide to either (1) update z(a) to x' or (2) keep z equal to 28-1)) This decision is made probabilistically as shown in Figure 5
Here, we draw a value ~ Unif((, 1) and compare it to the quantity exp (-p-KK [f(x') f( 1))1, where K is a
This quantity can be dissected as follows:
If f(x') f(1(-11) exp (K 4. -.((x()) -((((11))) > I and we will always choose Zi()) x'.
O (i.e., x' is an improvement over I(i-1)); then
Otherwise - (()(-1)) < 0,, then we will choose z(() = x' with probability that is dependent on: (1) How negative f(x') - (=) is and (2) The quantity $. Intuitively, if f(x') is significantly worse than (= then we will choose x' with small probability. Also, as the iteration number increases, will chooses also choose move suboptimal candidates x' less frequently.
In our paper, we fix K = 0.03.
E Datasets Descriptions
We provide an overview of the datasets used in our empirical study.
Recidivism ProPublica 2016 Data collected using the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), an algorithm used aiding a judge or parole officer's decision in granting/ denying parole a defendant.
Soanting/eenyin
FICO (Loan Approval) FICO (2018): As part of an interpretable machine learning challenge, this dataset consists of Home Equity Line of Credit (HELOC) applications made by homcowners and whether they repaid the loan within 2 years. Such a model is used to make real-time decisions on whether to approve or reject loan applications.
Diabetes Dua & Grafß (2019b): Compiled by the Center for Disease Control (CDC) 2014, the Diabetes Health Indicators dataset is used find lifestyle and demographic information that correlate with getting diabetes.
Adults Income Dua & Graff (2019a). This is a classic dataset compiled by Barry Becker from the 1994 Census database. It used for variety of machine learning benchmarking research. Each row correspond a particular individual and the goal is to predict whether that individual's annual income is greater than $50,000.
Schizophrenia for Biotechnology Information (024).. The Sweden Schezpn-Sciizopheenia Population-Ban Basod Case- Control Exome Sequencing dataset, available through Database for Genotypes and Phenotypes
21

---

Published in Transactions on Machine Learning Research (08/2025)
If any of the following rules satisfied:
Prior Arrests 2 3.5, Drug Tests Positive 2 0.06, Jobs Per Year S 0.23, Age at Release S 48 Employed Σ 0.47, Prior Arrests 2 2, Gang Affiliated = 1, Prior Misdemeanors 2
Gang A Affiliated = 1, Prior Arrests 2 1, Drug Tests 2 0.04, Days Employed 0.47
then predict to recidivism
Recividism: Support 0.104, ABBR 0.913
If any of the following rules are satisfied:
External Risk Estimate I 65.5, Average Months in File < 58.5, Trades with Balance 2 80.5
External Risk Estimate < 70.5, Average Months in File < 59.5, Trades with Balance 2 79.5 and Months Since Oldest !
Percent Trades Never Delinquent 91.5, External Risk Estimate 66.5, Average Moths in File I 59.5
then predict to FICO loan default.
b FICO: Support 0.102, ABBR 0.910
If any of the following rules are satisfied: Relationship Husband, Education 2 14, Age 2 30 Married = 1, Capital Gain 2 5000
Sex = Male, Married = 1, Work Hours Per Week 2 42, Education 2 13
then predict to Income 2 50, 000.
с Adults: Support = 0.105, ABBR = 0.941
If any of the following rules satisfied:
High Blood Pressure = 1, High Cholesterol = 1, BMI 2 29.5, General Health 2 3
High Blood Pressure = 1, BMT 2 27.5, High Cholesterol = 1, Difficulty Walking = 1
High Blood Pressure, General Health 2 27.5, BMI 2 28, Heart Disease = 1
then predict to diabetes
Diabetes: Support 0.110, ABBR 0.927
If any of the following rules are satisfied:
Admission Type # 3, Insulin = Yes and Diagnosis = 428
Number Inpatient = 0, Admission Type and Diabetes Meds Insulin Yes, Admission Type 6 and Number Inpatient then predict to readmission
Readmission: Support 0.08, ABBR 0.83
Figure 6: Examples of Rule Lists for M = 3, T. =
(DbGaP) from the National Institute of Health (NIH) contains mutation-level genomic information for a cohort of patients with schizophrenia as well as a control group.
Readmission Dua & Graff 2019c). This dataset contains 10 years of clinical care 130 US hospitals for patients with diabetes. Each row represents hospitalized patient records diagnosed with diabetes. The goal is to determine whether the patient would be readmitted to the hospital within 30 days of discharge.
F Examples of Explanations Generated
See Figures P and not informative.
Note that examples are not shown for the Schizophrenia dataset as the gene names are
22

---

Published in Transactions on Machine Learning Research (08/2025)
If at least 4 of the following conditions are satisfied:
Gang Affiliated
Jobs Per Year 0
Arrests with PPV Violation >= 5
Arrests Property >= 2
Age at Release 48
then predict 1 for recidivism.
Recidivism: Support = 0.099, ABBR = 0.903
If at least 4 of the following conditions satisfied:
External Risk Estimate < 63
External Risk Estimate S 53
Trades with Balance 2 0.86
Average Months in File I 53
Number of Total Trades < 19
then predict 1 for FICO loan default.
b FICO: Support 0.095, ABBR 0.921 If at least 5 the following conditions are satisfied:
Education 13
Age 33
Relationship Husband Work Class is not Missing
Married =
then predict for diabetes.
Adults: Support = 0.102, ABBR = 0.980
If at least 4 of the following conditions satisfied: Heart Disease
BMI 2 32
High Cholesterol
High Blood Pressure
General Health
then predict 1 for Income 2 50K.
d Diabetes: Support = 0.104, ABBR 0.929
If least of the following conditions are satisfied:
Admission Type = 6
Number of Medications 2
Admission Source 2 7
Number Inpatietn 2
Insulin Yes then predict 1 for readmission.
Readmission: Support 0.135, ABBR 0.86
Figure 7: Examples of Checklists for K
23


Reviews:
[{"review_id": "l8ZM4EUnbq", "reviewer": "Reviewer_qZvC", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. The proposed method is efficient and scalable. The two-phase search runs efficiently and can produce rules on relatively large datasets.\n2. Experiments on various datasets show promising performance.\n3. The paper is well-written and easy to understand.\n\n**Weaknesses**\n1. The paper does not include comparisons with any baselines, making the empirical improvement unclear.\n2. There is no ablation study to clearly demonstrate the effectiveness of each component and the effect of hyperparameters.\n3. It would be beneficial to discuss why simulated annealing was adopted instead of other search algorithms such as Bayesian optimization and evolutionary computation.\n4. Figure 4 overlaps with Table 3.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper introduces a new method for model-agnostic, rule-based explanations that focus on confident predictions of a black-box binary classifier. It replaces the traditional \"explain the hard thresholded labels\" view with an explanation objective defined over the ranking of predicted probabilities. Experiments on six datasets show it reaches small gap to the optimum.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Add comparisons to state-of-the-art baselines.\n2. Include comprehensive ablation studies: (i) Analyze the contribution of each key component (ii) Evaluate the sensitivity to important hyperparameters.\n3. Provide thorough analysis of search algorithm choice.", "broader_impact_concerns": "No concerns identified.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "HndlSp673Y", "reviewer": "Reviewer_Yw4T", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n\nNovelty and Relevance\nIntroduces the ABBR metric, an innovative approach for evaluating interpretability, and proposes a clear, practical method for directly optimizing interpretability metrics linked to model confidence.\n\nPractical Impact\nThe developed algorithms are highly practical, easy to understand, and readily deployable. Demonstrated empirical validity across diverse, realistic datasets reinforces the usefulness of the approach for real‑world applications.\n\nMethodological Clarity\nEmpirical experiments are thorough, with a clear presentation of comparative performance and optimality gaps.\n\nBalance between Interpretability and Accuracy\nThe paper convincingly illustrates the trade‑offs between explanation sparsity (interpretability) and predictive performance (ABBR), enabling users to make informed decisions based on their needs.\n\nWeaknesses\n\nScalability\nGenerating candidate rule sets may become computationally infeasible for extremely large or high‑dimensional datasets.\n\nComparison with Baselines\nThe approach focuses primarily on rule lists and checklists; the experiments lack comparisons with other post‑hoc explanation techniques such as LIME (https://dl.acm.org/doi/pdf/10.1145/2939672.2939778) or SHAP (https://dl.acm.org/doi/pdf/10.5555/3295222.3295230).\n\nGeneralization to Multi‑Class or Regression Tasks\nThe current method addresses only binary classification. Extending ABBR or similar metrics to multi‑class classification or regression remains an open question.\n\nTypos\nA few typographical errors remain (see “Requested Changes”).", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In this paper, the authors propose a novel metric and algorithm to improve model‑agnostic explanations of binary classifiers. The new metric, Average Black‑Box Ranking (ABBR), considers the confidence ranks (i.e., predicted probabilities) assigned by models to their predictions across a dataset. Based on this measure, two algorithms—RuleListSearch and ChecklistSearch—are designed to maximize ABBR while accounting for the sparsity of the explanation (in other words, interpretability). The authors demonstrate the effectiveness of their approach empirically across six datasets spanning healthcare, criminal justice, and finance, highlighting a clear trade‑off between interpretability and predictive accuracy.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Typos and Formatting\n\nPage 6, last paragraph: change “taking taking” to “taking.”\n\nPage 10, Section 5, second paragraph, third line: change “he” to “The.”\n\nTable 2 caption: replace \"S∈{3,5}” with “𝑀∈{3,5}”\n\nTable 3 caption: the caption is not visible and covered by a figure\n\nComparison with Alternative Methods\nInclude at least one baseline post‑hoc explanation technique (e.g., LIME or SHAP link to papers are above) in the experimental comparison. \n\nExtension Beyond Binary Classification\nAdd a discussion of how ABBR—and the associated algorithms—could be adapted (or why they may not extend) to multi‑class classification and regression tasks. Highlight any theoretical (if possible) or practical obstacles.\n\nScalability With High‑Dimensional Data\nThe current datasets have 14–54 features. If feasible, run an additional experiment on a dataset with substantially more features (e.g., >100). If that is not possible, include a discussion of how the number of features affects candidate‑rule generation and explanation quality within your framework, and outline potential strategies to mitigate scalability issues.", "broader_impact_concerns": "N/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "xFuP0x7bR8", "reviewer": "Reviewer_cvZ2", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- Clear motivation:\n\n    Confidence scores naturally carry rich information about model behavior and are highly relevant to concepts such as reliability, which is a key concern in explainable AI.\n\n    Given a fixed support target, it is intuitive that explaining samples with higher confidence results in rules that better capture model behavior.\n\n**Weaknesses**\n\n- Evaluation:\n\n    The paper aims to demonstrate the effectiveness of ABBR-based explanations. However, the experiments primarily focus on showing that high ABBR values can be achieved, rather than **providing evidence for why ABBR-based explanations are superior**.\n\n    Table 2, the paper’s main empirical result, only demonstrates that high ABBR can be achieved. It does not empirically validate that achieving high ABBR leads to better explanations.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes improvements to rule-based, model-agnostic explanation methods for binary classification model predictions.\n\n1. While prior methods typically select rules that best explain binary (0/1) predictions, this paper selects rules and checklists that explain prediction patterns including confidence scores.\n\n2. To quantify how well a rule explains model behavior, the paper introduces a new metric called Average Black-Box Ranking (ABBR).\n\n3. To efficiently discover rules with high explanatory power, a local search-based optimization method is proposed.\n\n4. The method is evaluated on 6 real-world datasets, and demonstrated that the algorithm reached high ABBR values compared to optimal possible ABBR.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Critical to securing your recommendation:**\n\n- Evaluation Setting:\n\n    The evaluation should answer the following core question:\n    \n    > Given the same target support, do rules that have higher ABBR explain model behavior provide **better explanations**?\n\n    For example, the paper proposes confidence-based explanations instead of explanations based solely on binary predictions. But is there a **quantitative experiment** showing that the former is better than the latter? There needs to be a **metric independent of ABBR** that can justify the superiority of ABBR-based explanations. There are two kinds of idea for evaluation benchmark:\n\n    > [Idea 1] Suppose a method that simply maximizes support achieves support **x** on the train set and **y** on the test set. Now, if you use ABBR-maximizing rules (with target support **x**), does the resulting test set support **y′** become higher than **y**? If so, that could indicate the generalizability of ABBR-based rules.\n\n    > [Idea 2] You could sample many random rules $\\ell_k$ with the same support and show that **those with higher ABBR values tend to achieve higher test support**. That would empirically justify ABBR as a better criterion.\n\n    Evaluation is always one of the most challenging yet crucial aspects of interpretability research. I would appreciate if the authors could come up with a compelling evaluation strategy.\n\n---\n\n**Would strengthen the work:**\n\n- Theoretical justification for ABBR:\n\n    Although I’m not certain, ABBR reminds me of **Mann–Whitney U statistic** / **Wilcoxon rank-sum test**, from non-parametric, rank-based statistical methods. Is there a precise mathematical concept that connects to ABBR?\n\n    While ABBR is intuitively reasonable, the paper lacks a **formal justification for why it is a good metric for evaluating ranking quality**. The connection to consistency alone feels insufficient—especially since **consistency itself is not a strong enough metric**, which is the very motivation for introducing a ranking-based measure like ABBR.\n\n- Figure 4: LaTeX rendering errors.", "broader_impact_concerns": "N/A.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)