On Computing Probabilistic Explanations for Decision Trees
Abstract
Formal XAI (explainable AI) is a growing area that focuses on computing explanations with mathematical guarantees for the decisions made by ML models.
Inside formal XAI, one of the most studied cases is that of explaining the choices taken by decision trees, as they are traditionally deemed as one of the most interpretable classes of models.
Recent work has focused on studying the computation of sufﬁcient reasons , a kind of explanation in which given a decision tree T and an instance x , one explains the decision T ( x ) by providing a subset y of the features of x such that for any other instance z compatible with y , it holds that T ( z )
=
T ( x ) , intuitively meaning that the features in y are already enough to fully justify the classiﬁcation of x by T .
It has been argued, however, that sufﬁcient reasons constitute a restrictive notion of explanation.
For such a reason, the community has started to study their probabilistic counterpart, in which one requires that the probability of T ( z )
=
T ( x ) must be at least some value δ ∈ (0 , 1] , where z is a random instance that is compatible with y .
Our paper settles the computational complexity of δ -sufﬁcient-reasons over decision trees, showing that both (1) ﬁnding δ -sufﬁcient-reasons that are minimal in size, and (2) ﬁnding δ -sufﬁcient-reasons that are minimal inclusion-wise, do not admit polynomial-time algorithms (unless PTIME = NP).
This is in stark contrast with the deterministic case ( δ = 1 ) where inclusion-wise minimal sufﬁcient-reasons are easy to compute.
By doing this, we answer two open problems originally raised by Izza et al., and extend the hardness of explanations for Boolean circuits presented by Wäldchen et al.
to the more restricted case of decision trees.
On the positive side, we identify structural restrictions of decision trees that make the problem tractable, and show how SAT solvers might be able to tackle these problems in practical settings.
Introduction
Context.
The trust that AI models generate in people has been repetitively linked to our ability of explaining the decision of said models (Arrieta et al., 2020), thus suggesting the area of explainable
AI (XAI) as fundamental for the deployment of trustworthy models.
A sub-area of explainability that has received considerable attention over the last years, showing quick progress in theoretical and practical terms, is that of local explanations, i.e., explanations for the outcome of a particular input to an ML model after the model has been trained.
Several queries and scores have been proposed to 36th Conference on Neural Information Processing Systems (NeurIPS 2022). specify explanations of this kind.
These include, e.g., queries based on prime implicants (Shih et al., 2018) or anchors (Ribeiro et al., 2018), which are parts of an instance that are sufﬁcient to explain its classiﬁcation, as well as scores that intend to quantify the impact of a single feature in the output of such a classiﬁcation (Lundberg and Lee, 2017; Yan and Procaccia, 2021).
A remarkable achievement of this area of research has been the development of formal notions of explainability.
The beneﬁts brought about by this principled approach have been highlighted, in a very thorough and convincing way, in a recent survey by Marques-Silva and Ignatiev (2022).
A prime example of this kind of approach is given by sufﬁcient reasons , which are also known as prime implicant explanations (Shih et al., 2018) or abductive explanations (Ignatiev et al., 2021).
Given an ML model M of dimension n and a Boolean input instance x ∈ { 0 , 1 } n , a sufﬁcient reason for x under M is a subset y of the features of x , such that any instance z compatible with y receives the same classiﬁcation result as x on M .
In more intuitive words, y is a sufﬁcient reason for x under
M if the features in y sufﬁces to explain the output of M on x .
In the formal explainability approach, one then aims to ﬁnd sufﬁcient reasons y that satisfy one of the following optimality criteria: (a) they are minimum , i.e., there are no sufﬁcient reasons with fewer features than y , or (b) they are minimal , i.e., there are no sufﬁcient reasons that are strictly contained in y .
Problem.
The XAI community has studied for which Boolean ML models the problem of computing (minimum or minimal) sufﬁcient reasons is computationally tractable and for which it is computationally hard (see, e.g., (Barceló et al., 2020; Marques-Silva et al., 2020, 2021)).
It has been argued, however, that for practical applications sufﬁcient reasons might be too rigid , as they are speciﬁed under worst-case conditions.
That is, y is a sufﬁcient reason for x under M if every “completion” of y is classiﬁed by M in the same way as x .
As several authors have noted already, there is a natural way in which this notion can be relaxed in order to become more suitable for real-world explainability tasks: Instead of asking for each completion of y to yield the same result as x on M , we could allow for a small fragment of the completions of y to be classiﬁed differently than x (Izza et al., 2021; Wäldchen et al., 2021; Wang et al., 2021).
More precisely, we would like to ensure that a random completion of y is classiﬁed as x with probability at least δ ∈ (0 , 1] , a threshold that the recipient of the explanation controls.
We call y a δ -sufﬁcient reason for e under M .
The study of the cost of computing minimum δ -sufﬁcient reasons for expressive Boolean ML models based on propositional formulas was started by Wäldchen et al.
They show, in particular, that the decision problem of checking if x admits a δ -sufﬁcient reason of a certain size k under a model M , where M is speciﬁed as a CNF formula, is NP PP -complete.
This result shows that the problem is very difﬁcult for complex models, at least in theoretical terms.
Nonetheless, it leaves the door open for obtaining tractability results over simpler Boolean models, starting from those which are often deemed to be “easy to interpret”, e.g., decision trees (Gilpin et al., 2018; Izza et al., 2020a; Lipton, 2018).
In particular, the study of the cost of computing both minimum and minimal δ -sufﬁcient reasons for decision trees was initiated by Izza et al.
(2021, 2022), but nothing beyond the fact that the problem lies in NP has been obtained.
Work by Blanc et al.
(2021) has shown that it is possible to obtain efﬁcient algorithms that succeed with a certain probability, and that instead of ﬁnding a smallest (either cardinality or inclusion-wise) δ -sufﬁcient reason, ﬁnd δ -sufﬁcient reasons that are small compared to the average size of δ -sufﬁcient reasons for the considered model.
Our results.
In this paper we provide an in-depth study of the complexity of the problem of minimum and minimal δ -sufﬁcient reasons for decision trees.
We start by pinpointing the exact computational complexity of these problems by showing that, under the assumption that PTIME (cid:54) =
NP, none of them can be solved in polynomial time.
We start with minimum δ -sufﬁcient reasons and show that the problem is hard even if δ is an arbitrary ﬁxed constant in (0 , 1] .
Our proof takes as basis the fact that, assuming
PTIME (cid:54) = NP, the problem of computing minimum sufﬁcient reasons for decision trees is not tractable (Barceló et al., 2020).
The reduction, however, is non-trivial and requires several involved constructions and a careful analysis.
The proof for minimal δ -sufﬁcient reasons is even more difﬁcult, and the result more surprising, as in this case we cannot start from a similar problem over decision trees: computing minimal sufﬁcient reasons over decision trees (or, equivalently, minimal δ -sufﬁcient reasons for δ = 1 ) admits a simple polynomial time algorithm.
Our result then implies that such a good behavior is lost when the input parameter δ is allowed to be smaller than 1 .
To deal with the high computational complexity of the problems, we look for structural restrictions of it that, at the same time, represent meaningful practical instances and ensure that these problems can be solved in polynomial time.
The ﬁrst restriction, called bounded split number , assumes there is a constant c ≥ 1 such that, for each node u of a decision tree
T of dimension n , the number of features that are mentioned in both T u , the subtree of T rooted at u , and T − T u , the subtree of T obtained by removing T u , is at most c .
We show that the problems of computing minimum and minimal δ -sufﬁcient reasons over decision trees with bounded split number can be solved in polynomial time.
The second restriction is monotonicity .
Intuitively, a Boolean ML model M is monotone , if the class of instances that are classiﬁed positively by M is closed under the operation of replacing 0 s with 1 s. For example, if M is of dimension 3 and it classiﬁes the input (1 , 0 , 0) as positive, then so it does for all the instances in { (1 , 0 , 1) , (1 , 1 , 0) , (1 , 1 , 1) } .
We show that computing minimal δ -sufﬁcient reasons for monotone decision trees is a tractable problem.
(This good behavior extends to any class of monotone ML models for which counting the number of positive instances is tractable; e.g., monotone free binary decision diagrams (Wegener, 2004)).
In spite of the intractability results we obtain in the paper, we show experimentally that our problems can be solved over practical instances by using SAT solvers.
This requires ﬁnding efﬁcient encodings of such problems as conjunctive normal form ( CNF ) formulas, which we then check for satisﬁability.
This is particularly non-trivial for probabilistic sufﬁcient reasons, as it requires dealing with the arithmetical nature of the probabilities involved through a Boolean encoding.
Organization of the paper.
We introduce the main terminology used in the paper in Section 2, and we deﬁne the problems of computing minimum and minimal δ -sufﬁcient reasons in Section 3.
The intractability of the these problems is proved in Section 4, while some tractable restrictions of them are provided in Section 5.
Our Boolean encodings can be found in Section 6.
Finally, we discuss some future work in Section 7.
Background
An instance of dimension n , with n ≥ 1 , is a tuple x ∈ { 0 , 1 } n .
We use notation x
[ i ] to refer to the i -th component of this tuple, or equivalently, its i -th feature.
Moreover, we consider an abstract notion of a model of dimension n , and we deﬁne it as a Boolean function M : { 0 , 1 } n → { 0 , 1 } .
That is,
M assigns a Boolean value to each instance of dimension n , so that we focus on binary classiﬁers with Boolean input features.
Restricting inputs and outputs to be Boolean makes our setting cleaner while still covering several relevant practical scenarios.
A partial instance of dimension n is a tuple y ∈ { 0 , 1 , ⊥} n .
Intuitively, if y [ i ] = ⊥ , then the value of the i -th feature is undeﬁned.
Notice that an instance is a particular case of a partial instance where all features are assigned value either 0 or 1 .
Given two partial instances x , y of dimension n , we say that y is subsumed by x , denoted by y ⊆
x , if for every i ∈ { 1 , . . . , n } such that y [ i ] (cid:54) =
⊥ , it holds that y
[ i ] = x [ i ] .
That is, y is subsumed by x if it is possible to obtain x from y by replacing some undeﬁned values.
Moreover, we say that y is properly subsumed by x , denoted by y (cid:40)
x , if y ⊆ x and y (cid:54) =
x . Notice that a partial instance y can be thought of as a compact representation of the set of instances z such that y is subsumed by z , where such instances z are called the completions of y and are grouped in the set C OMP ( y ) .
A binary decision diagram (BDD) of dimension n is a rooted directed acyclic graph M with labels on edges and nodes such that: (i) each leaf (a node with no outgoing edges) is labeled with true or false ; (ii) each internal node (a node that is not a leaf) is labeled with a feature i ∈ { 1 , . . . , n } ; and (iii) each internal node has two outgoing edges, one labeled 1 and the other one labeled 0 .
Every instance x ∈ { 0 , 1 } n deﬁnes a unique path π x = u 1 · · · u k from the root u 1 to a leaf u k of M such that: if the label of u i is j ∈ { 1 , . . . , n } , where i ∈ { 1 , . . .
, k − 1 } , then the edge from u i to u i + is labeled with x [ j ] .
Moreover, the instance x is positive, denoted by M ( x ) = 1 , if the label of u k is true ; otherwise the instance x is negative, which is denoted by M ( x ) = 0 .
A BDD M is free if for every path from the root to a leaf, no two nodes on that path have the same label.
A decision tree is simply a free BDD whose underlying directed acyclic graph is a rooted tree.
Probabilistic Sufﬁcient Reasons
Sufﬁcient reasons are partial instances obtained by removing from an instance x components that do not affect the ﬁnal classiﬁcation.
Formally, ﬁx a dimension n .
Given a decision tree T , an instance x , and a partial instance y with y ⊆ x , we call y a sufﬁcient reason for x under T if T ( x ) =
T ( z ) for every z ∈ C OMP ( y ) .
In other words, the features of y that take value either 0 or 1 explain the decision taken by T on x , as T ( x ) would not change if the remaining features (i.e., those that are undeﬁned in y ) were to change in x , thus implying that the classiﬁcation T ( x ) is a consequence of the features deﬁned in y .
We say that a sufﬁcient reason y for x under T is minimal , if it is minimal under the order induced by ⊆ , that is, if there is no sufﬁcient reason y (cid:48) for x under T such that y (cid:48) (cid:40)
y .
Also, we deﬁne a minimum sufﬁcient reason for x under T as a sufﬁcient reason y for x under T that maximizes the value | y | ⊥ :
= |{ i ∈ { 1 , . . . , n } | y
[ i ] = ⊥}| .
It turns out that minimal sufﬁcient reasons can be computed efﬁciently for decision trees with a very simple algorithm, assuming a sub-routine to check whether a given partial instance is a sufﬁcient reason (not necessarily minimal) of another given instance.
As shown in Algorithm 1, the idea of the algorithm is as follows: start with a candidate answer y which is initially equal to x , the instance to explain, and maintain the invariant that y is a sufﬁcient reason for x , while trying to remove deﬁned components from y until no longer possible.
It is not hard to see that one can check whether a partial instance y is a sufﬁcient reason for an instance x in linear time over decision trees (Audemard et al., 2021).
This algorithm is well known (see e.g., (Huang et al., 2021)), and relies on the following simple observation, tracing back to Goldsmith et al.
Observation 1.
For any class of models C , if a partial instance y of dimension n is a sufﬁcient reason for an instance x under a model M ∈ C , but not a minimal sufﬁcient reason, then there is a partial instance ˆ y which is equal to y except that ˆ y
[ i ] = ⊥ , y
[ i ] (cid:54) = ⊥ for some i ∈ { 1 , . . . , n } which is also a sufﬁcient reason for x under M .
Input: Decision tree T and instance x , both of dimension n Output: A minimal sufﬁcient reason y for x under T .
do
←
The following theorem shows a stark contrast between the complexity of computing minimal and minimum sufﬁcient reasons over decision trees.
Theorem 1 (Barceló et al.
Assuming PTIME (cid:54)
= NP , there is no polynomial-time algorithm that, given a decision tree T and an instance x of the same dimension, computes a minimum sufﬁcient reason for x under T .
Arguably, the notion of sufﬁcient reason is a natural notion of explanation for the result of a classiﬁer.
However, such a concept imposes a severe restriction by asking all completions of a partial instance to be classiﬁed in the same way.
To overcome this limitation, a probabilistic generalization of sufﬁcient reasons was proposed by Wäldchen et al.
(2021) and Izza et al.
More precisely, this notion allows to settle a conﬁdence δ ∈ (0 , 1] on the fraction of completions of a partial instance that yield the same classiﬁcation.
false true false true
T p ( ∅ ) =p ( { 1 } ) =p ( { 2 } ) =p ( { 3 } ) =p ( { 1 , 2 } ) =p ( { 1 , 3 } ) =p ( { 2 , 3 } ) =p ( { 1 , 2 , 3 } ) =
Deﬁnition 1 (Probabilistic sufﬁcient reasons) .
Given a value δ ∈ (0 , 1] , a δ -sufﬁcient reason ( δ -SR for short) for an instance x under a decision tree T is a partial instance y such that y ⊆ x
and
Pr z [ T ( z ) =
T ( x ) | z ∈ C OMP ( y )] := |{ z ∈ C OMP ( y )
| T ( z ) =
T ( x ) }| 2 | y | ⊥ ≥ δ.
Minimal and minimum δ -sufﬁcient reasons are deﬁned analogously as in the case of minimal and minimum sufﬁcient reasons.
Example 1.
Consider the decision tree T over features { 1 , 2 , 3 } shown in Figure 1 and the input instance x
= (1 , 1 , 1) .
Notice that T ( x ) = 1 .
For each X ⊆ { 1 , 2 , 3 } , we show the probability p ( X ) :
=
Pr z [ T ( z ) = 1 | z ∈ C OMP ( y X )] , where y X is the partial instance that is obtained from x by ﬁxing y X
[ i ] = ⊥ for each i (cid:54)∈ X .
We observe, for instance, that x itself is neither a minimum nor a minimal 1 -SR for x under T , as y { 1 , 3 }
= (1 , ⊥ , 1) is also a 1 -SR.
In turn, y { 1 , 3 } is both a minimal and a minimum 1 -SR for x under T .
The partial instance y { 1 , 3 } is not, however, a minimal or a minimum 3 / 4 -SR for x under T , as y { 1 } = (1 , ⊥ , ⊥ ) is also a 3 / 4 -SR.
Example 2.
Consider the decision tree T over features { 1 , 2 , 3 } shown in Figure 2 and the input instance x
= (1 , 1 , 1) .
Notice that T ( x ) = 1 .
Exactly as in Example 1, we display as well the probabilities p ( X ) for each X ⊆ { 1 , 2 , 3 } .
Interestingly, this example illustrates that Observationdoes not hold when δ < 1 .
Indeed, consider that y { 1 , 2 , 3 } is a 5 / 8 -SR which is not minimal, as y ∅ is also a 5 / 8 -SR, but if we remove any single feature from y { 1 , 2 , 3 } , we obtain a partial instance which is not a 5 / 8 -SR.
As illustrated on Example 2, it is not true in general that if y (cid:48) ⊂
y then
[Equation]
which means that standard algorithms for ﬁnding minimal sets holding monotone predicates (see e.g., (Marques-Silva et al., 2013)) cannot be used to compute minimal δ -SRs.
The problems of computing minimum and minimal δ -SR on decision trees were deﬁned and left open by (Izza et al., 2021, 2022).
These problems are formally deﬁned as follows.
PROBLEM:
[Equation]
decision tree T of dimension n , an instance x of dimension n and δ ∈ (0 , 1]
[Equation]
minimum δ -SR for x under T
[Equation]
INPUT
[Equation]
A decision tree T of dimension n , an instance x of dimension n and δ ∈ (0 , 1] OUTPUT
[Equation]
A minimal δ -SR for x under T
The Complexity of Probabilistic Sufﬁcient Reasons on Decision Trees
In what follows, we show that neither Compute-Minimum-SR nor Compute-Minimal-SR can be solved in polynomial time (unless PTIME = NP).
We ﬁrst consider the problem
false true false true   true true false
T p ( ∅ ) =p ( { 1 } ) =p ( { 2 } ) =p ( { 3 } ) =p ( { 1 , 2 } ) =p ( { 1 , 3 } ) =p ( { 2 , 3 } ) =p ( { 1 , 2 , 3 } ) =
Compute-Minimum-SR , and in fact prove a stronger result by considering the family of problems δ - Compute-Minimum-SR where δ ∈ (0 , 1] is assumed to be ﬁxed.
More precisely, we obtain as a corollary of Theorem 1 that 1 - Compute-Minimum-SR cannot be solved efﬁciently.
Moreover, a non-trivial modiﬁcation of the proof of this theorem shows that this negative result continues to hold for every ﬁxed δ ∈ (0 , 1] .
Theorem 2.
Fix δ ∈ (0 , 1] .
Then assuming that PTIME (cid:54) = NP , there is no polynomial-time algorithm for δ - Compute-Minimum-SR .
Let us now look at the problem Compute-Minimal-SR .
When δ = 1 , this problem can be solved in polynomial time as stated in Theorem 1.
However, it is conjectured by Izza et al.
(2021) that assuming
PTIME (cid:54) = NP, this positive behavior does not extend to the general problem Compute-Minimal-SR , in which δ is an input conﬁdence parameter.
Our main result conﬁrms that this conjecture is correct.
Theorem 3.
Assuming that PTIME (cid:54) = NP , there is no polynomial-time algorithm for
Compute-Minimal-SR .
Proof sketch.
We ﬁrst show that the following decision problem, called Check-Sub-SR , is NP-hard.
This problem takes as input a tuple ( T, x ) , for T a decision tree of dimension n , and x ∈ { 0 , 1 , ⊥} n a partial instance, and the goal is to decide whether there is a partial instance y (cid:40) x with Pr z [ T ( z ) =
| z ∈ C OMP ( y )] ≥ Pr z [ T ( z )
= 1 | z ∈ C OMP ( x )] .
We then show that if Compute-MinimalSR admits a polynomial time algorithm, then Check-Sub-SR is in PTIME, which contradicts the assumption that PTIME (cid:54) = NP.
The latter reduction requires an involved construction exploiting certain properties of the hard instances for Check-Sub-SR .
To show that Check-Sub-SR is NP-hard, we use a polynomial time reduction from a decision problem over formulas in CNF , called Minimal-Expected-Clauses , and which we also show to be
NP-hard.
Both the NP-hardness of Minimal-Expected-Clauses and the reduction from MinimalExpected-Clauses to Check-Sub-SR may be of independent interest.
We now deﬁne the problem Minimal-Expected-Clauses .
Let ϕ be a CNF formula over variables
X = { x 1 , . . .
, x n } .
Partial assignments of the variables in X , as well as the notions of subsumption and completions over them, are deﬁned in exactly the same way as for partial instances over features.
For a partial assignment µ over X , we denote by E ( ϕ, µ ) the expected number of clauses of ϕ satisﬁed by a random completion of µ .
We then consider the following problem for ﬁxed k ≥(recall that a k - CNF formula is a CNF formula where each clause has at most k literals):
PROBLEM : k -Minimal-Expected-Clauses
INPUT : ( ϕ, σ ) , for ϕ a k - CNF formula and σ a partial assignment
OUTPUT :
Yes , if there is a partial assignment µ (cid:40) σ such that E ( ϕ, µ ) ≥ E ( ϕ, σ ) and No otherwise
We show that k -Minimal-Expected-Clauses is NP-hard even for k = 2 , via a reduction from the well-known clique problem.
Finally, the reduction from k -Minimal-Expected-Clauses to
Check-Sub-SR builds an instance ( T, x ) from ( ϕ, σ ) in a way that there is a direct correspondence between partial assignments µ ⊆ σ and partial instances y ⊆ x , satisfying that
[Equation]
where m is the number of clauses of ϕ .
This implies that ( ϕ, σ ) is a Yes -instance for k -MinimalExpected-Clauses if and only if ( T, x ) is a Yes -instance for Check-Sub-SR .
Tractable Cases
We now study restrictions on decision trees that lead to polynomial time algorithms for
Compute-Minimum-SR or Compute-Minimal-SR , hence avoiding the general intractability results shown in the previous section.
We identify two such restrictions: bounded split number and monotonicity .
5.1 Bounded split number
Let T be a decision tree of dimension n .
For a set U of nodes of T , we denote by F ( U ) the set of features from { 1 , . . . , n } labeling the nodes in U .
For each node u of T , we denote by N ↓ u the set of nodes appearing in T u , that is, the subtree of T rooted at u .
On the other hand, we denote by N ↑ u the set of nodes of T minus the set of nodes N ↓ u .
We deﬁne the split number of the decision tree T to be
[Equation]
Intuitively, the split number of a decision tree T is a measure of the interaction (number of common features) between the subtrees of the form T u and their exterior.
A small split number allows us to essentially treat each subtree T u independently (in particular, the left and right subtrees below any node), which in turn leads to efﬁcient algorithms for the problems Compute-Minimum-SR and
Compute-Minimal-SR .
Theorem 4.
Let c ≥ 1 be a ﬁxed integer.
Both Compute-Minimum-SR and Compute-Minimal-SR can be solved in polynomial time for decision trees with split number at most c .
Proof Sketch.
It sufﬁces to provide a polynomial time algorithm for Compute-Minimum-SR .
(The same algorithm works for Compute-Minimal-SR as a minimum δ -SR is in particular minimal.)
In turn, using standard arguments, it is enough to provide a polynomial time algorithm for the following decision problem Check-Minimum-SR : Given a tuple ( T, x , δ, k ) , where T is a decision tree of dimension n , x ∈ { 0 , 1 , ⊥} n is a partial instance, δ ∈ (0 , 1] , and k ≥ 0 is an integer, decide whether there is a partial instance
y ⊆
x such that n − | y | ⊥ ≤ k and Pr z [ T ( z ) =
| z ∈ C OMP ( y )] ≥ δ .
In order to solve Check-Minimum-SR over an instance ( T, x , δ, k ) , where T has split number at most c , we apply dynamic programming over T in a bottom-up manner.
Let Z ⊆ { 1 , . . .
, n } be the set of features deﬁned in x , that is, features i with x [ i ] (cid:54) =
⊥ .
Those are the features we could eventually remove when looking for y .
For each node u in T , we solve a polynomial number of subproblems over the subtree T u .
We deﬁne
[Equation]
∩ Z. In other words, Int ( u ) are the features appearing both inside and outside T u , while New ( u ) are the features only inside T u , that is, the new features introduced below u .
Both sets are restricted to Z as features not in Z play no role in the process.
Each particular subproblem is indexed by a possible size s ∈ { 0 , . .
. , k } and a possible set J ⊆
Int ( u ) and the goal is to compute the quantity:
[Equation]
where C u,s,J is the space of partial instances
y ⊆ x with n − | y | ⊥ ≤ s and such that y
[ i ] = x
[ i ] for i ∈ J and y
[ i ] = ⊥ for i
∈ Int ( u ) \ J .
In other words, the set J ﬁxes the behavior on Int ( u ) (keep features in J , remove features in Int ( u ) \ J ) and hence the maximization occurs over choices on the set New ( u ) (which features are kept and which features are removed).
The key idea is that p u,s,J can be computed inductively using the information already computed for the children u 1 and u 2 of u .
Intuitively, this holds since the common features between T u 1 and T u 2 are at most c , which is a ﬁxed constant, and hence we can efﬁciently synchronize the information stored for u 1 and u 2 .
Finally, to solve the instance ( T, x , δ, k ) we simply check whether p r,k, ∅ ≥ δ , for the root r of T .
5.2 Monotonicity
Monotonic classiﬁers have been studied in the context of XAI as they often present tractable cases for different explanations, as shown by Marques-Silva et al.
The computation of minimal sufﬁcient reasons for monotone models was known to be in PTIME since the work of Goldsmith et al.
We show that this is also the case for computing minimal δ -SRs under a mild assumption on the class of models.
Let us deﬁne the ordering (cid:22) for instances in { 0 , 1 } n as follows:
[Equation]
We can now deﬁne monotonicity as follows.
A model M of dimension n is said to be monotone if for every pair of instances x , z ∈ { 0 , 1 } n , it holds that x (cid:22) z
= ⇒ M ( x ) ≤ M ( z ) .
We now prove that the problem of computing minimal probabilistic sufﬁcient reasons can be solved in polynomial time for any class C of monotone Boolean models for which the problem of counting positive completions can be solved efﬁciently.
Formally, the latter problem is deﬁned as follows: given a model M ∈ C of dimension n and a partial instance y ∈ { 0 , 1 , ⊥} n , compute |{ x ∈
C OMP ( y ) | M ( x ) = 1 }| .
We call this problem C - Count-Positive-Completions
Theorem 5.
Let C be a class of monotone Boolean models such that the problem
C - Count-Positive-Completions can be solved in polynomial time.
Then Compute-Minimal-SR can be solved in polynomial time over C .
Proof sketch.
Consider a partial instance y of dimension n and i ∈ { 1 , . .
. , n } .
Suppose y [ i ] (cid:54) = ⊥ .
We write y \ { i } for the partial instance y (cid:48) that is exactly equal to y save for the fact that y (cid:48)
[ i ] = ⊥ .
We make use of the following lemma, which is a probabilistic counterpart to Observation 1.
Lemma 1.
Let C be a class of monotone models, M ∈ C a model of dimension n , and x ∈ { 0 , 1 } n an instance.
Consider any δ ∈ (0 , 1] .
Then if y ⊆ x is a δ -SR for x under M which is not minimal, then there is a partial instance y \ { i } , for some i ∈ { 1 , . . . , n } , that is also a δ -SR for x under M .
With this lemma we can prove Theorem 5.
In fact, it can be shown then that a slight variant of
Algorithm 1 computes in polynomial time a minimal δ -SR for x under M .
As a corollary, the computation of minimal probabilistic sufﬁcient reasons can be carried out in polynomial time not only over monotone decision trees, but also over monotone free BDDs.
Corollary 1.
The problem of computing minimal δ -SRs can be solved in polynomial time over the class of monotone free BDDs.
SAT to the Rescue!
Despite the theoretical intractability results presented earlier on, many NP-complete problems can be solved over practical instances with the aid of SAT solvers .
By deﬁnition, any NP-complete problem can be encoded as a CNF satisﬁability (SAT) problem, which is then solved by a highly optimized program, a SAT solver .
In particular, if a satisfying assignment is found for the CNF instance, one can translate such an assignment back to a solution for the original problem.
In fact, this paradigm has been successfully used in the literature for other explainability problems (Ignatiev et al., 2021, 2022; Izza et al., 2020b).
The effectiveness of this approach is highly dependent on the particular encoding being used (Biere et al., 2009), where the aspect that arguably impacts performance the most is the size of the encoding, measured as the number of clauses of the resulting CNF formula.
In this section we present an encoding that uses some standard automated reasoning tools (e.g., sequential encondings (Sinz, 2005)) combined with ad-hoc bit-blasting (Biere et al., 2009), where the arithmetic operations required for probabilistic reasoning are implemented as Boolean circuits and then encoded as CNF by Tseitin transformations.
The appendix describes experimentation both over synthetic datasets and standard datasets (MNIST) and reports empirical results.
A recent report by Izza et al.
(2022) also uses automated reasoning for this problem, although through an SMT solver.
Let us consider the following decision version of the Compute-Minimum-SR problem.
PROBLEM: :
Decide-Minimum-SR
INPUT :
A decision tree T of dimension n , an instance x of dimension n , an integer k ≤ n , and δ ∈ (0 , 1]
OUTPUT :
Yes if there is a δ -SR y for x under T with | y | ⊥ ≤ k , and
No otherwise.
This problem is NP-complete.
Membership is already proven by Izza et al.
Hardness follows directly from Theorem 2, as if one were able to solve Decide-Minimum-SR in polynomial time, then a simple binary search over k would allow solving Compute-Minimum-SR in polynomial time.
6.1 Deterministic Encoding
Let us ﬁrst propose an encoding for the particular case of δ = 1 .
First, create Boolean variables
f i for i ∈ { 1 , . . . , n } , with f i representing that
y [ i ] (cid:54) = ⊥ , where y is the desired δ -SR.
Then, for every node u of the tree T , create a variable r u representing that node u is reachable by a completion of y , meaning that there exists a completion of y that goes through node u when evaluated over T .
We then want to enforce that:
r ROOT = 1 , where ROOT is the root of T .
This means that the root is always reachable.
The desired δ -SR y satisﬁes n − | y | ⊥ ≤ k , meaning that (cid:80) ni =
1 f i ≤ k . 3.
If T ( x ) = 1 , and F is the set of false leaves of T , then r (cid:96) = {+
+} for every (cid:96) ∈ F .
This means that if we want to explain a positive instance, the completions of the explanation y must all be positive (recall we assume δ = 1) , and thus no false leaf should be reachable.
Conversely, If T ( x ) = 0 , and G is the set of true leaves of T , then r (cid:96) = {+
+} for every (cid:96) ∈ G . 4.
The semantics of reachability is consistent : if a node u is reachable, and its labeled with feature i , then if f i = 0 , meaning that feature i is undeﬁned in y , both children of u should be reachable too.
In case f i = 1 , only the child along the edge corresponding to x
[ i ] should be reachable.
Let us analyze the size of this encoding.
Condition 1 requires a single clause to be enforced.
Condition 2 can be enforced with O ( nk ) variables and clauses using the linear encoding of Sinz (2005).
Condition 3 requires at most one clause per leaf of T and thus at most O ( | T | ) clauses.
Condition 4 can be implemented with a constant number of clauses per node.
We thus incur in a total of O ( nk + | T | ) variables and clauses, which is pretty much optimal considering Ω( n + | T | ) is a lower bound on the representation of the input.
Note as well that from a satisfying assignment we can trivially recover the desired explanation y as
[Equation]
, ⊥ otherwise. An efﬁcient encoding supporting values of δ different than 1 is signiﬁcantly more challenging and involved, and thus we only provide an outline here.
An exhaustive description, together with the implementation, is provided in the supplementary material.
6.2 Probabilistic Encoding
In order to encode that Pr z [ T ( z ) =
T ( x ) | z ∈ C OMP ( y )] ≥ δ , we will directly encode that |{ z ∈ C OMP ( y ) | T ( z )
=
T ( x ) }| ≥ (cid:6) δ 2 | y | ⊥ (cid:7) , where we assume for simplicity that the ceiling can be take safely, in order to have a value we can represent by an integer.
This assumption is not crucial.
As before, we will have f i variables representing whether y
[ i ] (cid:54) = ⊥ or not, and enforce that (cid:80) ni =
f i ≤ k .
Now deﬁne variables c j for j ∈ { 0 , . . . , n } , such that c j is true exactly when (cid:80) ni
=1 f i = j .
This can again be done efﬁciently via a linear encoding.
Let t i for i ∈ { 0 , . . . , n } a series of variables that represent the bits of an integer t .
The integer t will correspond to the value of (cid:6) δ 2 | y | ⊥ (cid:7) .
Note that as | y | ⊥ can only take n + 1 different values, there are only n + 1 different values that t can take.
Moreover, the value of the c j variables completely determines the value of t , and thus we can manually encode how the c j variables determine the bits t i .
We can now assume that we have access to t through its bits t i .
Our goal now is to build a binary representation of
[Equation]
so that we can then implement the condition α ≥ t with a Boolean circuit on their bits.
In order to represent α , we will decompose the number of instances z according to the leaves of T as follows.
If L is the set of leaves of T whose label matches T ( x ) , then
[Equation]
( z ) (cid:32) (cid:96) ) }| , where notation T ( z ) (cid:32) (cid:96) means that the evaluation of z under T ends in the leaf (cid:96) .
For a given leaf (cid:96) , we can compute its weight w (cid:96) : = |{ z ∈ C OMP ( y ) | T ( z ) (cid:32) (cid:96) ) }| as described next.
Let F (cid:96) be the set of labels appearing in the unique path from the root of T down to (cid:96) .
Now let u (cid:96) be the number of undeﬁned features of y along said unique path, and thus u (cid:96) can be deﬁned as follows: u (cid:96) : = | F (cid:96) \ { i | f i = 1 }| .
The sets F (cid:96) depend only on T and thus can be precomputed.
Therefore we can use a linear encoding again to deﬁne the values u (cid:96) .
It is simple to observe now that w (cid:96) = 2 u (cid:96) .
which means that by representing the values u (cid:96) we can trivially represent the values w (cid:96) in binary (they will simply consist of a 1 in their ( u (cid:96) + 1) -th position right-to-left), and then implement a
Boolean addition circuit to compute α = (cid:80) (cid:96) w (cid:96) .
This concludes the outline of the encoding.
Its number of variables clauses can be shown to be at most O ( n 2 | T | + nk ) .
Conclusions and Future Work
The results proven in this paper suggest that minimal (or minimum) explanations might be hard to obtain in practice even for decision trees, especially in problems where the feature space has large dimension.
A way to circumvent the limitations proven in our work is to relax the guarantee of minimality, or introduce some probability of error, as done in the work of Blanc et al.
A promising direction of future research is to better understand the kind of guarantees and settings in which is still possible to obtain tractability.
Our work leaves open some interesting technical questions: (1) What is the parameterized complexity of computing minimum δ -SRs over decision trees, assuming that the parameter is the size of the explanation one is looking for?
It is not hard to see that W[2] hardness follows from our proofs (i.e., they are parameterized reductions all the way to Set Cover), but membership in any class is fully open.
(2) Does Theorem 2 continue to hold for monotone decision trees?
That is, is it the case that computing minimum δ -SRs over monotone decision trees is hard for every ﬁxed δ ∈ (0 , 1] ?
(3) Is it the case that the hardness of computation for minimal sufﬁcient reasons holds for a ﬁxed δ ∈ (0 , 1] ?
If so, does it hold for every ﬁxed such a δ , or only for some?
(4) Is it possible to extend the positive behavior of decision trees with bounded split number to free BDDs?
Acknowledgments
Barceló is funded by Fondecyt grant 1200967.
Arenas and Barceló have been funded by ANID - Millennium Science Initiative Program - Code ICN17002.
Barceló has also been funded by
National Center for Artiﬁcial Intelligence CENIA FB210017, Basal ANID.
Romero is funded by
Fondecyt grant 11200956, the Data Observatory Foundation, and the National Center for Artiﬁcial Intelligence CENIA FB210017, Basal ANID.
Subercaseaux is supported by NSF awards CCF2015445, CCF1955785, and CCF2006953.