Original: Probabilistic Sufﬁcient Explanations
Revised: Probabilistic Sufﬁcient Explanations

Original: Abstract
Revised: Abstract

Original: Understanding the behavior of learned classiﬁers is an important task, and various black-box explanations, logical reasoning approaches, and modelspeciﬁc methods have been proposed.
Revised: Understanding the behavior of learned classiﬁers is an important task, and various black-box explanations, logical reasoning approaches, and modelspeciﬁc methods have been proposed.

Original: In this paper, we introduce probabilistic sufﬁcient explanations , which formulate explaining an instance of classiﬁcation as choosing the “simplest” subset of features such that only observing those features is “sufﬁcient” to explain the classiﬁcation.
Revised: In this paper, we introduce probabilistic sufﬁcient explanations , which formulate explaining an instance of classiﬁcation as choosing the “simplest” subset of features such that only observing those features is “sufﬁcient” to explain the classiﬁcation.

Original: That is, sufﬁcient to give us strong probabilistic guarantees that the model will behave similarly when all features are observed under the data distribution.
Revised: That is, sufﬁcient to give us strong probabilistic guarantees that the model will behave similarly when all features are observed under the data distribution.

Original: In addition, we leverage tractable probabilistic reasoning tools such as probabilistic circuits and expected predictions to design a scalable algorithm for ﬁnding the desired explanations while keeping the guarantees intact.
Revised: In addition, we leverage tractable probabilistic reasoning tools such as probabilistic circuits and expected predictions to design a scalable algorithm for ﬁnding the desired explanations while keeping the guarantees intact.

Original: Our experiments demonstrate the effectiveness of our algorithm in ﬁnding sufﬁcient explanations, and showcase its advantages compared to Anchors and logical explanations.
Revised: Our experiments demonstrate the effectiveness of our algorithm in ﬁnding sufﬁcient explanations, and showcase its advantages compared to Anchors and logical explanations.

Original:  Introduction
Revised:  Introduction

Original: Machine learning models are becoming ubiquitous, and are being used in critical and sensitive areas such as medicine, loan applications, and risk assessment in courts.
Revised: Machine learning models are becoming ubiquitous, and are being used in critical and sensitive areas such as medicine, loan applications, and risk assessment in courts.

Original: Hence, unexpected and faulty behaviors in machine learning models can have signiﬁcant negative impact on people.
Revised: Hence, unexpected and faulty behaviors in machine learning models can have signiﬁcant negative impact on people.

Original: As a result, there is much focus on explaining and understanding the behavior of such models.
Revised: As a result, there is much focus on explaining and understanding the behavior of such models.

Original: Explainable AI (or XAI) is an active area of research that aims to tackle these issues.
Revised: Explainable AI (or XAI) is an active area of research that aims to tackle these issues.

Original: There have been many approaches toward explaining an instance of a classiﬁcation (called a local explanation) from different perspectives, including logic-based [Shih et al. , 2018;
Revised: There have been many approaches toward explaining an instance of a classiﬁcation (called a local explanation) from different perspectives, including logic-based [Shih et al. , 2018;

Original: Ignatiev et al. , 2019a; Darwiche and Hirth, 2020] or modelagnostic approaches [Ribeiro et al. , 2016; Lundberg and Lee, 2017; Ribeiro et al. , 2018].
Revised: Ignatiev et al. , 2019a; Darwiche and Hirth, 2020] or modelagnostic approaches [Ribeiro et al. , 2016; Lundberg and Lee, 2017; Ribeiro et al. , 2018].

Original: Each of these methods have their pros and cons; some focus on scalability and ﬂexibility, and some focus on providing guarantees.
Revised: Each of these methods have their pros and cons; some focus on scalability and ﬂexibility, and some focus on providing guarantees.

Original: In this work, we strive to probabilistically explain an instance of classiﬁcation.
Revised: In this work, we strive to probabilistically explain an instance of classiﬁcation.

Original: Explanations are partial examples, where we treat the features not in the explanation as missing values.
Revised: Explanations are partial examples, where we treat the features not in the explanation as missing values.

Original: We aim for our explanations to be as simple as possible while providing the following sufﬁciency guarantee: given only the features in the explanation, with high probability under the data distribution Pr( X ) , the classiﬁer makes the same prediction as on the full example.
Revised: We aim for our explanations to be as simple as possible while providing the following sufﬁciency guarantee: given only the features in the explanation, with high probability under the data distribution Pr( X ) , the classiﬁer makes the same prediction as on the full example.

Original: Simplicity and sufﬁciency are often at odds with each other, hence balancing them is a challenging act.
Revised: Simplicity and sufﬁciency are often at odds with each other, hence balancing them is a challenging act.

Original: We brieﬂy overview other approaches to local explanations and discuss their pros and cons in the framework of sufﬁciency and simplicity.
Revised: We brieﬂy overview other approaches to local explanations and discuss their pros and cons in the framework of sufﬁciency and simplicity.

Original: Broadly, model-agnostic methods are more scalable and ﬂexible but tend to be not as reliable as logic-based methods in providing sufﬁciency.
Revised: Broadly, model-agnostic methods are more scalable and ﬂexible but tend to be not as reliable as logic-based methods in providing sufﬁciency.

Original: On the other hand, logical explanation methods tend to sacriﬁce simplicity for deterministic guarantees of sufﬁciency.
Revised: On the other hand, logical explanation methods tend to sacriﬁce simplicity for deterministic guarantees of sufﬁciency.

Original: We then motivate how probabilistic notions of sufﬁciency can overcome these shortcomings as the foundation for sufﬁcient explanations.
Revised: We then motivate how probabilistic notions of sufﬁciency can overcome these shortcomings as the foundation for sufﬁcient explanations.

Original: Next, we introduce the probabilistic reasoning tools needed to quantify sufﬁciency and deﬁne sufﬁcient explanations: the
Revised: Next, we introduce the probabilistic reasoning tools needed to quantify sufﬁciency and deﬁne sufﬁcient explanations: the

Original: Same-Decision Probability (SDP) and Expected Prediction (EP).
Revised: Same-Decision Probability (SDP) and Expected Prediction (EP).

Original: We use probabilistic circuits (PCs)
Revised: We use probabilistic circuits (PCs)

Original: [ Choi et al. , 2020 ] to model the probability distribution Pr( X ) over the features due to their expressivity and tractability in answering complex probabilistic queries.
Revised: [ Choi et al. , 2020 ] to model the probability distribution Pr( X ) over the features due to their expressivity and tractability in answering complex probabilistic queries.

Original: We also explore connections between SDP and expected prediction and show why expected prediction is better suited for providing the probabilistic guarantees for sufﬁcient explanations.
Revised: We also explore connections between SDP and expected prediction and show why expected prediction is better suited for providing the probabilistic guarantees for sufﬁcient explanations.

Original: We then formalize the desired properties of sufﬁcient explanations and motivate our choices for sufﬁciency and simplicity.
Revised: We then formalize the desired properties of sufﬁcient explanations and motivate our choices for sufﬁciency and simplicity.

Original: Sufﬁciency leads to maximizing the expected prediction, while simplicity leads to constraining the size of explanations as well as choosing the subset that also maximizes the marginal probability.
Revised: Sufﬁciency leads to maximizing the expected prediction, while simplicity leads to constraining the size of explanations as well as choosing the subset that also maximizes the marginal probability.

Original: We capture all these with an optimization problem for ﬁnding sufﬁcient explanations.
Revised: We capture all these with an optimization problem for ﬁnding sufﬁcient explanations.

Original: Then, we design a scalable algorithm for ﬁnding the most likely sufﬁcient explanation by leveraging tractability of expected prediction.
Revised: Then, we design a scalable algorithm for ﬁnding the most likely sufﬁcient explanation by leveraging tractability of expected prediction.

Original: We provide experiments showcasing the empirical advantage of sufﬁcient explanations and the effectiveness of our search algorithm.
Revised: We provide experiments showcasing the empirical advantage of sufﬁcient explanations and the effectiveness of our search algorithm.

Original: Our advantages include: (i) compared to
Revised: Our advantages include: (i) compared to

Original: Anchors we get better and more accurate sufﬁciency guarantees and (ii) our method ﬁnds simpler explanations and is more scalable than logical methods.
Revised: Anchors we get better and more accurate sufﬁciency guarantees and (ii) our method ﬁnds simpler explanations and is more scalable than logical methods.

Original: Lastly, we show the tradeoffs between sufﬁciency and simplicity, and show that slightly reducing the guarantee can lead to simpler explanations with higher likelihood.
Revised: Lastly, we show the tradeoffs between sufﬁciency and simplicity, and show that slightly reducing the guarantee can lead to simpler explanations with higher likelihood.

Original: Notation We use uppercase letters ( X ) for features (random variables) and lowercase letters ( x ) for their value assignments.
Revised: Notation. We use uppercase letters ( X ) for features (random variables) and lowercase letters ( x ) for their value assignments.

Original: Analogously, we denote sets of features in bold uppercase ( X ) and their assignments in bold lowercase ( x ).
Revised: Analogously, we denote sets of features in bold uppercase ( X ) and their assignments in bold lowercase ( x ).

Original: We denote the set of all possible assignments to X as X .
Revised: We denote the set of all possible assignments to X as X .

Original: Concatenation XY denotes the union of disjoint sets.
Revised: Concatenation XY denotes the union of disjoint sets.

Original: We focus on discrete features unless otherwise noted.
Revised: We focus on discrete features unless otherwise noted.

Original: We represent a probabilistic predictor as f : X → [0 , 1] and its decision function as C :
Revised: We represent a probabilistic predictor as f : X → [0 , 1] and its decision function as C :

Original: X → { 0 , 1 } , with T denoting the decision threshold.
Revised: X → { 0 , 1 } , with T denoting the decision threshold.

Original: Hence, C ( x ) = (cid:74) f ( x ) ≥ T (cid:75) .
Revised: Hence, C ( x ) = (cid:74) f ( x ) ≥ T (cid:75) .

Original: Sometimes we want to directly deal with log-odds instead of probabilities, in which case we use the log-odds predictor
Revised: Sometimes we want to directly deal with log-odds instead of probabilities, in which case we use the log-odds predictor

Original: O : X → R which is deﬁned as O ( x ) = log f ( x ) 1 − f ( x ) .
Revised: O : X → R which is deﬁned as O ( x ) = log f ( x ) 1 − f ( x ) .

Original:  Related Work
Revised: 2. Related Work

Original: Computing explanations of classiﬁers has been studied from many different perspectives, including logical reasoning, black-box methods, and model-speciﬁc approaches.
Revised: Computing explanations of classiﬁers has been studied from many different perspectives, including logical reasoning, black-box methods, and model-speciﬁc approaches.

Original: Some try to explain the learned model globally, making it more interpretable [ Guidotti et al. , 2018; Liang and Van den Broeck, 2019 ] , while others focus more locally on explaining its prediction for a single instance.
Revised: Some try to explain the learned model globally, making it more interpretable [ Guidotti et al. , 2018; Liang and Van den Broeck, 2019 ] , while others focus more locally on explaining its prediction for a single instance.

Original: Next, we go over some local explanation methods, discuss their pros and cons, and motivate how our framework might solve those issues.
Revised: Next, we go over some local explanation methods, discuss their pros and cons, and motivate how our framework might solve those issues.

Original: Model Agnostic Approaches
Revised: Model Agnostic Approaches

Original: These methods treat the classiﬁer as a black box.
Revised: These methods treat the classiﬁer as a black box.

Original: Given an input instance to explain, they perturb the instance in many different ways and evaluate the model on those perturbed instances.
Revised: Given an input instance to explain, they perturb the instance in many different ways and evaluate the model on those perturbed instances.

Original: Then, they use the results from the perturbations to generate an explanation.
Revised: Then, they use the results from the perturbations to generate an explanation.

Original: Two popular methods under this umbrella are Lime
Revised: Two popular methods under this umbrella are LIME

Original: [ Ribeiro et al. , 2016 ] and SHAP [ Lundberg and Lee, 2017 ] .
Revised: [ Ribeiro et al. , 2016 ] and SHAP [ Lundberg and Lee, 2017 ] .

Original: The main difference between these methods is the heuristics used to obtain perturbed instances and how to analyze the predictions on these local perturbations.
Revised: The main difference between these methods is the heuristics used to obtain perturbed instances and how to analyze the predictions on these local perturbations.

Original: Most provide feature attributions, which are real-valued numbers assigned to each feature, to indicate their importance to the decision and in what direction.
Revised: Most provide feature attributions, which are real-valued numbers assigned to each feature, to indicate their importance to the decision and in what direction.

Original: A beneﬁt of these methods is that they can be used to explain any model and are generally more ﬂexible and scalable than their alternatives.
Revised: A beneﬁt of these methods is that they can be used to explain any model and are generally more ﬂexible and scalable than their alternatives.

Original: On the other hand, the downsides are that they can be very sensitive to the choice of local perturbations and might produce over-conﬁdent results [ Ignatiev et al. , 2019b] or be fooled by adversarial methods [Slack et al. , 2020; Dimanov et al. , 2020].
Revised: On the other hand, the downsides are that they can be very sensitive to the choice of local perturbations and might produce over-conﬁdent results [ Ignatiev et al. , 2019b] or be fooled by adversarial methods [Slack et al. , 2020; Dimanov et al. , 2020].

Original: One of the main reasons for these downsides is that the distribution of the local perturbations tends to be different from the data distribution the classiﬁer was originally trained on.
Revised: One of the main reasons for these downsides is that the distribution of the local perturbations tends to be different from the data distribution the classiﬁer was originally trained on.

Original: Hence, these approaches do not beneﬁt from the intended generalization guarantees of machine learning models.
Revised: Hence, these approaches do not beneﬁt from the intended generalization guarantees of machine learning models.

Original: Moreover, some of the perturbations might be low probability or even impossible inputs, and we might not care as much about their classiﬁcation outcome.
Revised: Moreover, some of the perturbations might be low probability or even impossible inputs, and we might not care as much about their classiﬁcation outcome.

Original: Additionally, feature attribution methods treat each feature independently and cannot easily capture interactions between bigger subsets of features such as when two features cancel each other’s effects.
Revised: Additionally, feature attribution methods treat each feature independently and cannot easily capture interactions between bigger subsets of features such as when two features cancel each other’s effects.

Original: We refer to Camburu et al.
Revised: We refer to Camburu et al.

Original: [2020] for more such examples and discussion on pros/cons of attribution based explanations.
Revised: [2020] for more such examples and discussion on pros/cons of attribution based explanations.

Original: Logical Reasoning Approaches
Revised: Logical Reasoning Approaches

Original: These methods provide explanations with some principled guarantees by leveraging logical reasoning tools.
Revised: These methods provide explanations with some principled guarantees by leveraging logical reasoning tools.

Original: Some approaches use knowledge compilation and tractable Boolean circuits [Shih et al. , 2018; Darwiche and Hirth, 2020; Shi et al. , 2020], some adopt the framework of abductive reasoning [Ignatiev et al. , 2019a; Ignatiev et al. , 2019b], and some tackle a speciﬁc family of models such as linear models [Marques-Silva et al. , 2020], decision trees [Izza et al. , 2020], or tree ensembles [Devos et al. , 2020].
Revised: Some approaches use knowledge compilation and tractable Boolean circuits [Shih et al. , 2018; Darwiche and Hirth, 2020; Shi et al. , 2020], some adopt the framework of abductive reasoning [Ignatiev et al. , 2019a; Ignatiev et al. , 2019b], and some tackle a speciﬁc family of models such as linear models [Marques-Silva et al. , 2020], decision trees [Izza et al. , 2020], or tree ensembles [Devos et al. , 2020].

Original: The main beneﬁt of these approaches is that they guarantee provably correct explanations, that is they guarantee a certain prediction for all examples described by the explanation.
Revised: The main beneﬁt of these approaches is that they guarantee provably correct explanations, that is they guarantee a certain prediction for all examples described by the explanation.

Original: On the other hand, one downside is that they are generally not as scalable (in the number of features) as black-box methods.
Revised: On the other hand, one downside is that they are generally not as scalable (in the number of features) as black-box methods.

Original: Another downside is that they need to completely remove the uncertainty from the classiﬁer to be able to use logical tools and therefore become more rigid.
Revised: Another downside is that they need to completely remove the uncertainty from the classiﬁer to be able to use logical tools and therefore become more rigid.

Original: In particular, in order to guarantee a certain outcome with absolute certainty, it is often necessary to include many of the features into the explanation, making the explanation more complex.
Revised: In particular, in order to guarantee a certain outcome with absolute certainty, it is often necessary to include many of the features into the explanation, making the explanation more complex.

Original: Sufﬁcient Reasons
Revised: Sufﬁcient Reasons

Original: [ Shih et al. , 2018; Darwiche and Hirth, 2020 ] is one example of these methods that selects as an explanation a minimal subset of features guaranteeing that, no matter what is observed for the remaining features, the decision will stay the same.
Revised: [ Shih et al. , 2018; Darwiche and Hirth, 2020 ] is one example of these methods that selects as an explanation a minimal subset of features guaranteeing that, no matter what is observed for the remaining features, the decision will stay the same.

Original: Sufﬁcient reasons, as well as related logical explanations, ensure minimality and deterministic guarantees in the outcome, while as we see later our sufﬁcient explanations ensure probabilistic guarantees instead.
Revised: Sufﬁcient reasons, as well as related logical explanations, ensure minimality and deterministic guarantees in the outcome, while as we see later our sufﬁcient explanations ensure probabilistic guarantees instead.

Original: For a recent and more comprehensive comparison of logicbased vs. model-agnostic explanation methods, we refer to
Revised: For a recent and more comprehensive comparison of logicbased vs. model-agnostic explanation methods, we refer to

Original: Ignatiev et al.
Revised: Ignatiev et al.

Original: ; Ignatiev [2019b; 2020].
Revised: ; Ignatiev [2019b; 2020].

Original:  Motivation and Problem 
Revised:  Motivation and Problem Statement

Original: Statement

Original: We will overcome the limitations of both the model-agnostic and logic-based approaches by building local explanation methods that are aware of the distribution over features
Revised: We will overcome the limitations of both the model-agnostic and logic-based approaches by building local explanation methods that are aware of the distribution over features

Original: Pr( X ) .
Revised: Pr( X ) .

Original: We assume for now that we have access to an accurate model for the feature distribution, and discuss in Section 4.1 how it can be obtained.
Revised: We assume for now that we have access to an accurate model for the feature distribution, and discuss in Section 4.1 how it can be obtained.

Original: This distribution will allow us to (i) reason about the classiﬁer’s behavior on realistic input instances and (ii) provide probabilistic guarantees on the veracity of the explanations.
Revised: This distribution will allow us to (i) reason about the classiﬁer’s behavior on realistic input instances and (ii) provide probabilistic guarantees on the veracity of the explanations.

Original: We thus take a principled probabilistic approach in explaining an instance of classiﬁcation.
Revised: We thus take a principled probabilistic approach in explaining an instance of classiﬁcation.

Original: Intuitively, given an instance x and the classiﬁers outcome
Revised: Intuitively, given an instance x and the classiﬁers outcome

Original: C ( x ) , we would like to choose a subset of features z ⊆ x as the “simplest sufﬁcient explanation.”
Revised: C ( x ) , we would like to choose a subset of features z ⊆ x as the “simplest sufﬁcient explanation.”

Original: Firstly, we want it to be sufﬁcient, in that it provides strong probabilistic guarantees about the outcome of the classiﬁer when only features z are observed.
Revised: Firstly, we want it to be sufﬁcient, in that it provides strong probabilistic guarantees about the outcome of the classiﬁer when only features z are observed.

Original: To this end, we want to maximize some sufﬁciency metric F ( z , x , f, Pr) describing the behavior of our predictor when only the explanation z is observed.
Revised: To this end, we want to maximize some sufﬁciency metric F ( z , x , f, Pr) describing the behavior of our predictor when only the explanation z is observed.

Original: However, naively maximizing this metric can run into the pitfall of building more complex explanations in order to squeeze out tiny improvements in the sufﬁciency metric.
Revised: However, naively maximizing this metric can run into the pitfall of building more complex explanations in order to squeeze out tiny improvements in the sufﬁciency metric.

Original: To address this, we have our second metric of simplicity S ( z , Pr) , which we use to deﬁne a constraint when maximizing the sufﬁciency metric.
Revised: To address this, we have our second metric of simplicity S ( z , Pr) , which we use to deﬁne a constraint when maximizing the sufﬁciency metric.

Original: Putting these together, we call a subset z of an instance x a sufﬁcient explanation if it maximizes some sufﬁciency metric
Revised: Putting these together, we call a subset z of an instance x a sufﬁcient explanation if it maximizes some sufﬁciency metric

Original: F under some simplicity constraint.
Revised: F under some simplicity constraint.

Original: With this current deﬁnition of sufﬁcient explanations, there is still the possibility that two subsets of a given instance, one a subset of the other, are both simple and have the same degree of sufﬁciency.
Revised: With this current deﬁnition of sufﬁcient explanations, there is still the possibility that two subsets of a given instance, one a subset of the other, are both simple and have the same degree of sufﬁciency.

Original: This motivates us to further deﬁne a minimal sufﬁcient explanation z , i.e. no subset of z is also a sufﬁcient explanation.
Revised: This motivates us to further deﬁne a minimal sufﬁcient explanation z , i.e. no subset of z is also a sufﬁcient explanation.

Original: This condition is exactly the meaning of “prime” in prime implicants as they are used in logical explanations.
Revised: This condition is exactly the meaning of “prime” in prime implicants as they are used in logical explanations.

Original: We will see in Section 5 how we can use the feature distribution to ensure this minimality property.
Revised: We will see in Section 5 how we can use the feature distribution to ensure this minimality property.

Original:  Probabilistic Notions of Sufﬁciency
Revised:  Probabilistic Notions of Sufﬁciency

Original: We next introduce two notions of sufﬁciency using probabilistic reasoning tools and discuss their pros and cons.
Revised: We next introduce two notions of sufﬁciency using probabilistic reasoning tools and discuss their pros and cons.

Original: Then, we discuss their connections.
Revised: Then, we discuss their connections.

Original: Finally, we use these tools in
Revised: Finally, we use these tools in

Original: Section 5 to formalize our deﬁnition and formulate ﬁnding the desired subsets of features as an optimization problem.
Revised: Section 5 to formalize our deﬁnition and formulate ﬁnding the desired subsets of features as an optimization problem.

Original: 4. Probabilistic Reasoning Tools
Revised: 4. Probabilistic Reasoning Tools

Original: Probabilistic reasoning is a hard task in general, so we need to choose our probabilistic model for Pr( X ) carefully.
Revised: Probabilistic reasoning is a hard task in general, so we need to choose our probabilistic model for Pr( X ) carefully.

Original: We choose probabilistic circuits (PCs), which given some structural constraints enable tractable and exact computation of probabilistic reasoning queries such as marginals [ Choi et al. , 2020 ] .
Revised: We choose probabilistic circuits (PCs), which given some structural constraints enable tractable and exact computation of probabilistic reasoning queries such as marginals [ Choi et al. , 2020 ] .

Original: Moreover, they do so without giving up much expressivity.
Revised: Moreover, they do so without giving up much expressivity.

Original: Another advantage of PCs is that we can learn their structure and parameters from data, allowing us to avoid the exponential worst-case behavior of other probabilistic models.
Revised: Another advantage of PCs is that we can learn their structure and parameters from data, allowing us to avoid the exponential worst-case behavior of other probabilistic models.

Original: The two main probabilistic reasoning tools that we use for our explanations are the Same Decision Probability (SDP)
Revised: The two main probabilistic reasoning tools that we use for our explanations are the Same Decision Probability (SDP)

Original: [ Chen et al. , 2012 ] and Expected Prediction (EP)
Revised: [ Chen et al. , 2012 ] and Expected Prediction (EP)

Original: [ Khosravi et al. , 2019b ] .
Revised: [ Khosravi et al. , 2019b ] .

Original: We introduce them next and explore their tradeoffs and connections.
Revised: We introduce them next and explore their tradeoffs and connections.

Original: First we have SDP [ Chen et al. , 2012 ] which, intuitively, gives us the probability that our classiﬁer has the same output as C ( x ) given only some subset of observed features z .
Revised: First we have SDP [ Chen et al. , 2012 ] which, intuitively, gives us the probability that our classiﬁer has the same output as C ( x ) given only some subset of observed features z .

Original: Deﬁnition 1 (Same Decision Probability) .
Revised: Deﬁnition 1 (Same Decision Probability) .

Original: Given a classiﬁer C , a distribution Pr( X ) over features, a partition ZM of features X , and an assignment x to X (and corresponding z ⊆ x ), the same decision probability (SDP) of z w.r.t.
Revised: Given a classiﬁer C , a distribution Pr( X ) over features, a partition ZM of features X , and an assignment x to X (and corresponding z ⊆ x ), the same decision probability (SDP) of z w.r.t.

Original: x is
Revised: x is

Original: SDP C , x ( z ) =
Revised: SDP C , x ( z ) =

Original: [Equation]
Revised: [Equation]

Original: The higher the SDP, the better guarantee we have that the partial example z will be classiﬁed the same way as the full example x .
Revised: The higher the SDP, the better guarantee we have that the partial example z will be classiﬁed the same way as the full example x .

Original: SDP and related notions have been successfully used in applications such as trimming Bayesian network classiﬁers [Choi et al. , 2017] and robust feature selection [Choi and Van den Broeck, 2018].
Revised: SDP and related notions have been successfully used in applications such as trimming Bayesian network classiﬁers [Choi et al. , 2017] and robust feature selection [Choi and Van den Broeck, 2018].

Original: Renooij [2018] introduced various theoretical properties and bounds on the SDP.
Revised: Renooij [2018] introduced various theoretical properties and bounds on the SDP.

Original: Other explanation methods that provide sufﬁciency guarantees can be ﬁt into our framework, using SDP as the sufﬁciency metric.
Revised: Other explanation methods that provide sufﬁciency guarantees can be ﬁt into our framework, using SDP as the sufﬁciency metric.

Original: Notably, Anchors [Ribeiro et al. , 2018] can be thought of as an empirical approximation of SDP sufﬁciency, as they aim to provide sufﬁciency guarantees based on sampling local perturbations instead of relying on the data distribution.
Revised: Notably, Anchors [Ribeiro et al. , 2018] can be thought of as an empirical approximation of SDP sufﬁciency, as they aim to provide sufﬁciency guarantees based on sampling local perturbations instead of relying on the data distribution.

Original: Logical explanations also ﬁt in this framework, as they completely prioritize having deterministic guarantees, i.e. SDP=1, over the need for simplicity.
Revised: Logical explanations also ﬁt in this framework, as they completely prioritize having deterministic guarantees, i.e. SDP=1, over the need for simplicity.

Original: Another example of a method that uses probabilistic sufﬁciency can be found in
Revised: Another example of a method that uses probabilistic sufﬁciency can be found in

Original: Khosravi et al.
Revised: Khosravi et al.

Original: [2019b], where they explain logistic regression models w.r.t.
Revised: [2019b], where they explain logistic regression models w.r.t.

Original: Naive Bayes data distributions.
Revised: Naive Bayes data distributions.

Original: Expected Prediction is another probabilistic reasoning task that has been shown to be successful in handling missing values in classiﬁcation [Khosravi et al. , 2019a; Khosravi et al. , 2019b; Khosravi et al. , 2020].
Revised: Expected Prediction is another probabilistic reasoning task that has been shown to be successful in handling missing values in classiﬁcation [Khosravi et al. , 2019a; Khosravi et al. , 2019b; Khosravi et al. , 2020].

Original: It provides a promising alternative for SDP in explanations.
Revised: It provides a promising alternative for SDP in explanations.

Original: Intuitively, given some partial observation, expected prediction can be thought of as trying all possible ways of imputing the remaining features and computing an average of all subsequent predictions weighted by the probability of each imputation.
Revised: Intuitively, given some partial observation, expected prediction can be thought of as trying all possible ways of imputing the remaining features and computing an average of all subsequent predictions weighted by the probability of each imputation.

Original: In many cases, our classiﬁers directly output a probability and in those cases we can compute expected prediction as follows:
Revised: In many cases, our classiﬁers directly output a probability and in those cases we can compute expected prediction as follows:

Original: Deﬁnition 2 (Expected Prediction) .
Revised: Deﬁnition 2 (Expected Prediction) .

Original: Given a probabilistic predictor f , a distribution Pr( X ) over features, a partition
Revised: Given a probabilistic predictor f , a distribution Pr( X ) over features, a partition

Original: ZM of features X , and an assignment z to Z , the expected prediction of f on z is
Revised: ZM of features X , and an assignment z to Z , the expected prediction of f on z is

Original: [Equation]
Revised: [Equation]

Original: For some tasks, we may care more about odds rather than probabilities.
Revised: For some tasks, we may care more about odds rather than probabilities.

Original: In those cases, the predictor usually outputs the log-odds O ( x ) , so we also deﬁne expected log-odds:
Revised: In those cases, the predictor usually outputs the log-odds O ( x ) , so we also deﬁne expected log-odds:

Original: Deﬁnition 3 (Expected Log-Odds) .
Revised: Deﬁnition 3 (Expected Log-Odds) .

Original: Given a log-odds predictor O , a distribution Pr( X ) over features, a partition ZM of features X , and an assignment z to Z , the expected log-odds of f on z is
Revised: Given a log-odds predictor O , a distribution Pr( X ) over features, a partition ZM of features X , and an assignment z to Z , the expected log-odds of f on z is

Original: [Equation]
Revised: [Equation]

Original: In this paper, unless otherwise noted, expected prediction denoted as EP ( z ) could refer to both cases.
Revised: In this paper, unless otherwise noted, expected prediction denoted as EP ( z ) could refer to both cases.

Original: 4. Connections between SDP and EP
Revised: 4. Connections between SDP and EP

Original: The choice between using SDP and expected prediction as a sufﬁciency metric is important, as it will decide what kind of guarantees our explanations provide and how efﬁcient they are to compute.
Revised: The choice between using SDP and expected prediction as a sufﬁciency metric is important, as it will decide what kind of guarantees our explanations provide and how efﬁcient they are to compute.

Original: Here we explore connections between SDP and expected prediction and provide intuition on some advantages of using expected prediction in deﬁning sufﬁcient explanations.
Revised: Here we explore connections between SDP and expected prediction and provide intuition on some advantages of using expected prediction in deﬁning sufﬁcient explanations.

Original: Oneofthe main differences between SDP and expected prediction is their computational complexity.
Revised: Oneofthe main differences between SDP and expected prediction is their computational complexity.

Original: While SDP is an appealing criteria to use for selecting explanations, computing the SDP exactly is computationally hard.
Revised: While SDP is an appealing criteria to use for selecting explanations, computing the SDP exactly is computationally hard.

Original: In particular, it is PP PP -hard on Bayesian networks [Choi et al. , 2012].
Revised: In particular, it is PP PP -hard on Bayesian networks [Choi et al. , 2012].

Original: Even on a simple Naive Bayes model for both Pr and the classiﬁer, computing SDP is NP-hard
Revised: Even on a simple Naive Bayes model for both Pr and the classiﬁer, computing SDP is NP-hard

Original: [Chen et al. , 2013].
Revised: [Chen et al. , 2013].

Original: On the other hand, expected predictions can be tractably computed for many different pairs of discriminative and generative models.
Revised: On the other hand, expected predictions can be tractably computed for many different pairs of discriminative and generative models.

Original: For example, it is known to be tractable for the following cases
Revised: For example, it is known to be tractable for the following cases

Original: : (i) logistic regression using a conformant
Revised: : (i) logistic regression using a conformant

Original: naive Bayes distribution [Khosravi et al. , 2019b], (ii) decision trees w.r.t.
Revised: naive Bayes distribution [Khosravi et al. , 2019b], (ii) decision trees w.r.t.

Original: probabilistic circuits (PCs) [Khosravi et al. , 2020], (iii) discriminative circuits w.r.t.
Revised: probabilistic circuits (PCs) [Khosravi et al. , 2020], (iii) discriminative circuits w.r.t.

Original: PCs [Khosravi et al. , 2019a], and (iv) when both the feature distribution and predictor are deﬁned by the same PC distribution Pr .
Revised: PCs [Khosravi et al. , 2019a], and (iv) when both the feature distribution and predictor are deﬁned by the same PC distribution Pr .

Original: In the latter case, the predictor is the conditional probability Pr( c | X ) , and the feature distribution is Pr( X ) .
Revised: In the latter case, the predictor is the conditional probability Pr( c | X ) , and the feature distribution is Pr( X ) .

Original: Then, expected prediction can be reduced to probabilistic marginal inference in PCs which is tractable for decomposable circuits [Choi et al. , 2020].
Revised: Then, expected prediction can be reduced to probabilistic marginal inference in PCs which is tractable for decomposable circuits [Choi et al. , 2020].

Original: Another difference comes from how SDP and expected prediction handle the two distinct uncertainties that arise from the feature distribution and the classiﬁer.
Revised: Another difference comes from how SDP and expected prediction handle the two distinct uncertainties that arise from the feature distribution and the classiﬁer.

Original: Although both are aware of the uncertainty from the feature distribution Pr , SDP ignores the uncertainty in the classiﬁer since it only deals with the decision function C ( x ) .
Revised: Although both are aware of the uncertainty from the feature distribution Pr , SDP ignores the uncertainty in the classiﬁer since it only deals with the decision function C ( x ) .

Original: For example, SDP cannot distinguish between cases when the classiﬁer has low conﬁdence f ( x )
Revised: For example, SDP cannot distinguish between cases when the classiﬁer has low conﬁdence f ( x )

Original: =
Revised: =

Original: T + (cid:15) (only slightly above decision threshold) and cases when the classiﬁer has high conﬁdence such as f ( x ) = 0 . 99 .
Revised: T + (cid:15) (only slightly above decision threshold) and cases when the classiﬁer has high conﬁdence such as f ( x ) = 0 . 99 .

Original: In many domains this distinction is vital.
Revised: In many domains this distinction is vital.

Original: For example, doctors care more about the odds of a patient having cancer rather than binary decisions.
Revised: For example, doctors care more about the odds of a patient having cancer rather than binary decisions.

Original: For these reasons, we choose to use expected predictions to deﬁne and optimize for our explanations.
Revised: For these reasons, we choose to use expected predictions to deﬁne and optimize for our explanations.

Original: The good news is that optimizing EP also allows us to maximize a lower bound on SDP.
Revised: The good news is that optimizing EP also allows us to maximize a lower bound on SDP.

Original: The next theorem provides a theoretical lower bound for SDP using expected predictions.
Revised: The next theorem provides a theoretical lower bound for SDP using expected predictions.

Original: Theorem 1.
Revised: Theorem 1.

Original: Given a predictor f (or O ), its thresholded classiﬁer C , a positively classiﬁed instance x (i.e. C ( x ) = 1 ), a distribution Pr , and some subset of the features z ⊆ x , we have:
Revised: Given a predictor f (or O ), its thresholded classiﬁer C , a positively classiﬁed instance x (i.e. C ( x ) = 1 ), a distribution Pr , and some subset of the features z ⊆ x , we have:

Original: [Equation]
Revised: [Equation]

Original: where EP ( z ) refers to expected prediction, and U ( z ) is an upper bound for the predictor after ﬁxing z , i.e. ∀ m U ( z ) ≥ f ( zm ) .
Revised: where EP ( z ) refers to expected prediction, and U ( z ) is an upper bound for the predictor after ﬁxing z , i.e. ∀ m U ( z ) ≥ f ( zm ) .

Original: Moreover, if U ( z ) is a tight bound, then Equationis also tight.
Revised: Moreover, if U ( z ) is a tight bound, then Equationis also tight.

Original: Proof.
Revised: Proof.

Original: The proof is included in the appendix.
Revised: The proof is included in the appendix.

Original: Theorem 1 gives us two ways of computing a lower bound for SDP: using a probabilistic predictor f or a log-odds predictor O .
Revised: Theorem 1 gives us two ways of computing a lower bound for SDP: using a probabilistic predictor f or a log-odds predictor O .

Original: Additionally, the theorem can be easily generalized to cases when C ( x ) = 0 .
Revised: Additionally, the theorem can be easily generalized to cases when C ( x ) = 0 .

Original: See appendix for a more detailed discussion on which bound is better.
Revised: See appendix for a more detailed discussion on which bound is better.

Original:  Probabilistic Sufﬁcient Explanations
Revised:  Probabilistic Sufﬁcient Explanations

Original: In this section, we use the aforementioned sufﬁciency metrics and introduce some desirable constraints to ensure simplicity.
Revised: In this section, we use the aforementioned sufﬁciency metrics and introduce some desirable constraints to ensure simplicity.

Original: Then, we put everything together to formalize sufﬁcient explanations and introduce an optimization problem for ﬁnding them.
Revised: Then, we put everything together to formalize sufﬁcient explanations and introduce an optimization problem for ﬁnding them.

Original: Finally, we introduce a search algorithm for ﬁnding sufﬁcient explanations by modifying beam search and leveraging tractability of expected predictions.
Revised: Finally, we introduce a search algorithm for ﬁnding sufﬁcient explanations by modifying beam search and leveraging tractability of expected predictions.

Original: To simplify the deﬁnitions, we assume without loss of generality that the instance we want to explain is positively classiﬁed, i.e. C ( x )
Revised: To simplify the deﬁnitions, we assume without loss of generality that the instance we want to explain is positively classiﬁed, i.e. C ( x )

Original: = 1 .
Revised: = 1 .

Original: In Section 4.2, we saw two candidates that we can use for probabilistic sufﬁciency guarantees.
Revised: In Section 4.2, we saw two candidates that we can use for probabilistic sufﬁciency guarantees.

Original: The ﬁrst was SDP, which focuses on the ﬁnal decision of the model.
Revised: The ﬁrst was SDP, which focuses on the ﬁnal decision of the model.

Original: The second was expected prediction, which takes the conﬁdence of the model into account.
Revised: The second was expected prediction, which takes the conﬁdence of the model into account.

Original: Because of this, along with its computational tractability, we choose to use expected prediction for our sufﬁciency guarantees.
Revised: Because of this, along with its computational tractability, we choose to use expected prediction for our sufﬁciency guarantees.

Original: More speciﬁcally, we want to maximize the expected prediction of our explanation to ensure our model is conﬁdent in its classiﬁcation.
Revised: More speciﬁcally, we want to maximize the expected prediction of our explanation to ensure our model is conﬁdent in its classiﬁcation.

Original: In addition, maximizing the expected prediction also maximizes a lower bound for SDP, so in practice we will still get good
Revised: In addition, maximizing the expected prediction also maximizes a lower bound for SDP, so in practice we will still get good

Original: SDP guarantees.
Revised: SDP guarantees.

Original: Having chosen a suitable sufﬁciency metric, we must now choose a simplicity constraint.
Revised: Having chosen a suitable sufﬁciency metric, we must now choose a simplicity constraint.

Original: There are multiple candidates to choose from for this as well.
Revised: There are multiple candidates to choose from for this as well.

Original: For example, we can impose a cardinality constraint or require explanations to have a high enough likelihood.
Revised: For example, we can impose a cardinality constraint or require explanations to have a high enough likelihood.

Original: We choose the cardinality constraint as it is easy to decide on a threshold on the explanation size.
Revised: We choose the cardinality constraint as it is easy to decide on a threshold on the explanation size.

Original: We can now formalize the notion of sufﬁcient explanations.
Revised: We can now formalize the notion of sufﬁcient explanations.

Original: Deﬁnition 4 (Sufﬁcient Explanations) .
Revised: Deﬁnition 4 (Sufﬁcient Explanations) .

Original: Given a predictor f , distribution Pr , and a positively classiﬁed instance x (i.e.
Revised: Given a predictor f , distribution Pr , and a positively classiﬁed instance x (i.e.

Original: C ( x ) = 1 ), the set of sufﬁcient explanations for x is deﬁned as the solution of the following optimization problem:
Revised: C ( x ) = 1 ), the set of sufﬁcient explanations for x is deﬁned as the solution of the following optimization problem:

Original: [Equation]
Revised: [Equation]

Original: which we denote as SE k ( x ) .
Revised: which we denote as SE k ( x ) .

Original: Having deﬁned sufﬁcient explanations, and following the same line of thought as before, we are now interested in choosing sufﬁcient explanations which are minimal.
Revised: Having deﬁned sufﬁcient explanations, and following the same line of thought as before, we are now interested in choosing sufﬁcient explanations which are minimal.

Original: For logical explanations, any minimal sufﬁcient subset can be chosen.
Revised: For logical explanations, any minimal sufﬁcient subset can be chosen.

Original: However, we can be more selective in our choices using the feature distribution.
Revised: However, we can be more selective in our choices using the feature distribution.

Original: A natural choice is to choose the most likely sufﬁcient explanations, as these are the most realistic.
Revised: A natural choice is to choose the most likely sufﬁcient explanations, as these are the most realistic.

Original: By maximizing the marginal probability of the explanation, we also ensure the desired minimality.
Revised: By maximizing the marginal probability of the explanation, we also ensure the desired minimality.

Original: This is because, for subsets z 1 and z 2 of an instance x , if z 1 ⊆
Revised: This is because, for subsets z 1 and z 2 of an instance x , if z 1 ⊆

Original: z 2 then
Revised: z 2 then

Original: Pr( z 1 ) ≥ Pr( z 2 ) .
Revised: Pr( z 1 ) ≥ Pr( z 2 ) .

Original: We thus arrive at the following deﬁnition.
Revised: We thus arrive at the following deﬁnition.

Original: Deﬁnition 5 (Most Likely Sufﬁcient Explanations) .
Revised: Deﬁnition 5 (Most Likely Sufﬁcient Explanations) .

Original: Given a predictor f , a distribution Pr , and an instance x , the most likely sufﬁcient explanations for x are given by:
Revised: Given a predictor f , a distribution Pr , and an instance x , the most likely sufﬁcient explanations for x are given by:

Original: [Equation]
Revised: [Equation]

Original: Next, we devise a search algorithm for ﬁnding sufﬁcient explanations.
Revised: Next, we devise a search algorithm for ﬁnding sufﬁcient explanations.

Original: In Section 6, we show that our algorithm works well in practice, ensuring both sufﬁciency and simplicity.
Revised: In Section 6, we show that our algorithm works well in practice, ensuring both sufﬁciency and simplicity.

Original: Finding Probabilistic Sufﬁcient Explanations
Revised: Finding Probabilistic Sufﬁcient Explanations

Original: To ﬁnd the most likely sufﬁcient explanations, we use a beam search algorithm to greedily search the space of potential explanations.
Revised: To ﬁnd the most likely sufﬁcient explanations, we use a beam search algorithm to greedily search the space of potential explanations.

Original: We do so by keeping track of the top b candidates (the beam) for explanations for each cardinality based on their expected predictions and marginal probabilities.
Revised: We do so by keeping track of the top b candidates (the beam) for explanations for each cardinality based on their expected predictions and marginal probabilities.

Original: In more detail, we begin at level zero with the empty candidate, i.e. no features selected.
Revised: In more detail, we begin at level zero with the empty candidate, i.e. no features selected.

Original: For each subsequent level, we expand the top b candidates of the previous level by considering all feature subsets with one more feature than in the previous level, formed by adding a previously unselected feature to each candidate.
Revised: For each subsequent level, we expand the top b candidates of the previous level by considering all feature subsets with one more feature than in the previous level, formed by adding a previously unselected feature to each candidate.

Original: We then select the new top b candidates by ranking the expanded states based on EP, breaking ties using Pr , and keeping the top b .
Revised: We then select the new top b candidates by ranking the expanded states based on EP, breaking ties using Pr , and keeping the top b .

Original: The search stops after level k .
Revised: The search stops after level k .

Original: At the same time we keep track of the most likely candidate satisfying the current maximum EP.
Revised: At the same time we keep track of the most likely candidate satisfying the current maximum EP.

Original: Due to the nature of the search, each level of the beam search is highly parallelizable, which can be leveraged to speed up the search.
Revised: Due to the nature of the search, each level of the beam search is highly parallelizable, which can be leveraged to speed up the search.

Original: Additionally, by running the algorithm for k levels, we can also keep track of the best sufﬁciency guarantee at each level between 1 and k .
Revised: Additionally, by running the algorithm for k levels, we can also keep track of the best sufﬁciency guarantee at each level between 1 and k .

Original: Hence, with a little extra book keeping we can keep track of explanations with different sizes, and thus degrees of simplicity, and their corresponding sufﬁciency guarantees.
Revised: Hence, with a little extra book keeping we can keep track of explanations with different sizes, and thus degrees of simplicity, and their corresponding sufﬁciency guarantees.

Original: Note that the search framework can also be easily adapted for different use cases.
Revised: Note that the search framework can also be easily adapted for different use cases.

Original: For example, we can provide a ﬁxed sufﬁciency constraint
Revised: For example, we can provide a ﬁxed sufﬁciency constraint

Original: and then ﬁnd the simplest and most likely explanation with an expected prediction higher than the sufﬁciency threshold.
Revised: and then ﬁnd the simplest and most likely explanation with an expected prediction higher than the sufﬁciency threshold.

Original: However, choosing a good threshold for expected prediction is not always straightforward.
Revised: However, choosing a good threshold for expected prediction is not always straightforward.

Original:  Experiments
Revised:  Experiments

Original: In this section, we provide several experiments to showcase the effectiveness of our search algorithm in ﬁnding sufﬁcient explanations.
Revised: In this section, we provide several experiments to showcase the effectiveness of our search algorithm in ﬁnding sufﬁcient explanations.

Original: Additionally, we aim to highlight the advantages of our method in comparison with Anchors and logical explanation methods.
Revised: Additionally, we aim to highlight the advantages of our method in comparison with Anchors and logical explanation methods.

Original: More speciﬁcally, we would like to answer the following questions:
Revised: More speciﬁcally, we would like to answer the following questions:

Original: – Can we ﬁnd explanations with good sufﬁciency guarantees?
Revised: – Can we ﬁnd explanations with good sufﬁciency guarantees?

Original: How do they compare with Anchors?
Revised: How do they compare with Anchors?

Original: – Does relaxation from logical to probabilistic guarantees lead to much simpler explanations?
Revised: – Does relaxation from logical to probabilistic guarantees lead to much simpler explanations?

Original: – What are the tradeoffs between different sufﬁciency levels and explanation complexity?
Revised: – What are the tradeoffs between different sufﬁciency levels and explanation complexity?

Original: We use the adult and MNIST datasets [ Kohavi,
Revised: We use the adult and MNIST datasets [ Kohavi,

Original: Yann et al. , 2009 ] for our experiments.
Revised: Yann et al. , 2009 ] for our experiments.

Original: For each dataset, we model the feature distribution by learning a probabilistic circuit (PC) using the open source Juice library [ Dang et al. , 2021 ] .
Revised: For each dataset, we model the feature distribution by learning a probabilistic circuit (PC) using the open source Juice library [ Dang et al. , 2021 ] .

Original: We choose decision forests learned by XGBoost
Revised: We choose decision forests learned by XGBoost

Original: [ Chen and Guestrin, 2016 ] as our classiﬁer, as they are popular and because expected prediction is tractable for forests w.r.t.
Revised: [ Chen and Guestrin, 2016 ] as our classiﬁer, as they are popular and because expected prediction is tractable for forests w.r.t.

Original: PCs [ Khosravi et al. , 2020 ] .
Revised: PCs [ Khosravi et al. , 2020 ] .

Original: For more detailed information on the datasets, preprocessing steps, learned models, and computing infrastructure, please refer to the appendix.
Revised: For more detailed information on the datasets, preprocessing steps, learned models, and computing infrastructure, please refer to the appendix.

Original: 6. Comparison with Anchors
Revised: 6. Comparison with Anchors

Original: To demonstrate the scalability of our method and showcase some advantages of our method in comparison with Anchors, we ran our algorithm on the binarized MNIST dataset with a binary classiﬁcation task of distinguishing between 3s and 5s.
Revised: To demonstrate the scalability of our method and showcase some advantages of our method in comparison with Anchors, we ran our algorithm on the binarized MNIST dataset with a binary classiﬁcation task of distinguishing between 3s and 5s.

Original: Some images are shown in Figure 1 along with a comparison between explanations found using our method and Anchors.
Revised: Some images are shown in Figure 1 along with a comparison between explanations found using our method and Anchors.

Original: For Anchors, we used an SDP (precision) threshold of 0.95, a tolerance ( δ ) of 0.05, and a beam size of 5.
Revised: For Anchors, we used an SDP (precision) threshold of 0.95, a tolerance ( δ ) of 0.05, and a beam size of 5.

Original: On average it took Anchors 454s to generate explanations.
Revised: On average it took Anchors 454s to generate explanations.

Original: Our algorithm with the same beam size and cardinality constraint k =took 347s using 16 threads; the sufﬁcient explanations with
Revised: Our algorithm with the same beam size and cardinality constraint k =took 347s using 16 threads; the sufﬁcient explanations with

Original: [Figure]
Revised: [Figure]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: the same size as Anchors took 45s.
Revised: the same size as Anchors took 45s.

Original: See appendix for more details on the run-times.
Revised: See appendix for more details on the run-times.

Original: The last image of each row in Figure 1 is the explanation found using our algorithm with cardinality constraint k = 30 .
Revised: The last image of each row in Figure 1 is the explanation found using our algorithm with cardinality constraint k = 30 .

Original: We see that these explanations were able to pick up on certain features we would naturally use to distinguish between 3s and 5s.
Revised: We see that these explanations were able to pick up on certain features we would naturally use to distinguish between 3s and 5s.

Original: In particular the chosen pixels were mostly in the upper portion of each image.
Revised: In particular the chosen pixels were mostly in the upper portion of each image.

Original: This makes sense as both 3s and 5s have a similar arch shape in their lower portions, so the upper portion would be more useful for distinguishing between the two.
Revised: This makes sense as both 3s and 5s have a similar arch shape in their lower portions, so the upper portion would be more useful for distinguishing between the two.

Original: Additionally, the explanations contain not only some white pixels showing an outline of the predicted number, but also some black pixels, where a number of the opposite label may be present.
Revised: Additionally, the explanations contain not only some white pixels showing an outline of the predicted number, but also some black pixels, where a number of the opposite label may be present.

Original: Finally, by looking only at the explanation in the rightmost column of the last two rows we can guess that the classiﬁer will misclassify those examples.
Revised: Finally, by looking only at the explanation in the rightmost column of the last two rows we can guess that the classiﬁer will misclassify those examples.

Original: Table 1 provides data for the expected log-odds, SDP, and marginal probabilities for generated explanations using Anchors and our method in a few different scenarios.
Revised: Table 1 provides data for the expected log-odds, SDP, and marginal probabilities for generated explanations using Anchors and our method in a few different scenarios.

Original: Since SDP is intractable to compute exactly, we estimate it by computing the SDP on 10000 samples drawn from the probabilistic circuit conditioned on the explanation.
Revised: Since SDP is intractable to compute exactly, we estimate it by computing the SDP on 10000 samples drawn from the probabilistic circuit conditioned on the explanation.

Original: One advantage of using PCs as our generative model is that drawing conditional samples from Pr( · | z ) is very fast.
Revised: One advantage of using PCs as our generative model is that drawing conditional samples from Pr( · | z ) is very fast.

Original: For example, generating 10K samples takes 1 second.
Revised: For example, generating 10K samples takes 1 second.

Original: We see that the Anchors are quite overconﬁdent, giving explanations with much lower sample SDPs compared to the desired 0.95.
Revised: We see that the Anchors are quite overconﬁdent, giving explanations with much lower sample SDPs compared to the desired 0.95.

Original: This trend was also observed in Ignatiev et al.
Revised: This trend was also observed in Ignatiev et al.

Original: [2019b].
Revised: [2019b].

Original: We plot the sample SDPs for our explanations, along with lower bounds calculated using Theorem 1, in Figure 2.
Revised: We plot the sample SDPs for our explanations, along with lower bounds calculated using Theorem 1, in Figure 2.

Original: The green line shows that, even when optimizing the EP of our explanations, the SDP still tends to be very high.
Revised: The green line shows that, even when optimizing the EP of our explanations, the SDP still tends to be very high.

Original: Moreover, the blue line shows that the simple and efﬁcient SDP lower bound can also provide this guarantee for some of the larger explanations with high EP.
Revised: Moreover, the blue line shows that the simple and efﬁcient SDP lower bound can also provide this guarantee for some of the larger explanations with high EP.

Original: The red line is yet another way to estimate the SDP lower bound.
Revised: The red line is yet another way to estimate the SDP lower bound.

Original: See appendix for more details.
Revised: See appendix for more details.

Original: 6. Comparison with Logical Explanations
Revised: 6. Comparison with Logical Explanations

Original: We also compared explanations found using our method to logical explanations, i.e. minimal explanations with SDP = 1 .
Revised: We also compared explanations found using our method to logical explanations, i.e. minimal explanations with SDP = 1 .

Original: We used the adult dataset, which has much fewer features, for this task in order for the logical explanation computation to become feasible.
Revised: We used the adult dataset, which has much fewer features, for this task in order for the logical explanation computation to become feasible.

Original: We removed some features to allow for brute force computation of minimum cardinality logical explanations.
Revised: We removed some features to allow for brute force computation of minimum cardinality logical explanations.

Original: We chose to use brute force because, as far as we know, there is no method for ﬁnding logical explanations for our use case.
Revised: We chose to use brute force because, as far as we know, there is no method for ﬁnding logical explanations for our use case.

Original: In total we removed 3 features, leaving us with 11 features.
Revised: In total we removed 3 features, leaving us with 11 features.

Original: From our brute force search on some test examples, we found that logical explanations needed on average 39% of the features in order reach an SDP of one.
Revised: From our brute force search on some test examples, we found that logical explanations needed on average 39% of the features in order reach an SDP of one.

Original: By using our algorithm of maximizing the expected prediction, we found that in most cases selecting only 18% of the features was already enough to guarantee an SDP of 0.95 on average.
Revised: By using our algorithm of maximizing the expected prediction, we found that in most cases selecting only 18% of the features was already enough to guarantee an SDP of 0.95 on average.

Original: More detailed numbers are provided in Table 2.
Revised: More detailed numbers are provided in Table 2.

Original: We see that the strict requirement imposed by logical explanation methods can lead to selecting more features leading to more complex explanations.
Revised: We see that the strict requirement imposed by logical explanation methods can lead to selecting more features leading to more complex explanations.

Original: We expect this gap between the complexities of logical and probabilistic explanations to widen for datasets with more features or more complex models.
Revised: We expect this gap between the complexities of logical and probabilistic explanations to widen for datasets with more features or more complex models.

Original: [Table]
Revised: [Table]

Original: 6. Tradeoffs between Sufﬁciency and Simplicity
Revised: 6. Tradeoffs between Sufﬁciency and Simplicity

Original: Finally, we examine the tradeoff between maximizing sufﬁciency and explanation simplicity.
Revised: Finally, we examine the tradeoff between maximizing sufﬁciency and explanation simplicity.

Original: We compare the expected predictions with marginal likelihoods of explanations.
Revised: We compare the expected predictions with marginal likelihoods of explanations.

Original: Figure 3 shows a scatter plot comparing these two quantities for different explanations generated for 50 MNIST images.
Revised: Figure 3 shows a scatter plot comparing these two quantities for different explanations generated for 50 MNIST images.

Original: As we see, the general trend is that by enforcing less strict sufﬁciency requirements we can get more likely (and also smaller) explanations.
Revised: As we see, the general trend is that by enforcing less strict sufﬁciency requirements we can get more likely (and also smaller) explanations.

Original: In particular, the trend is very steep around EPs of 0 and 1, meaning that making the guarantees even slightly probabilistic will lead to signiﬁcant simpliﬁcation of the explanations, thus validating our probabilistic approach.
Revised: In particular, the trend is very steep around EPs of 0 and 1, meaning that making the guarantees even slightly probabilistic will lead to signiﬁcant simpliﬁcation of the explanations, thus validating our probabilistic approach.

Original:  Conclusion
Revised:  Conclusion

Original: We introduced a new framework for reasoning about the local behavior of classiﬁers.
Revised: We introduced a new framework for reasoning about the local behavior of classiﬁers.

Original: We formulated the problem as ﬁnding simplest and most likely explanations that maximize a probabilistic guarantee, and discussed advantages of our framework compared to model agnostic and logical explanation methods.
Revised: We formulated the problem as ﬁnding simplest and most likely explanations that maximize a probabilistic guarantee, and discussed advantages of our framework compared to model agnostic and logical explanation methods.

Original: We provided experiments to validate our claims.
Revised: We provided experiments to validate our claims.

Original: We conclude that probabilistic sufﬁcient explanations are a valuable addition to the arsenal of local explanation methods.
Revised: We conclude that probabilistic sufﬁcient explanations are a valuable addition to the arsenal of local explanation methods.

Original: Acknowledgments
Revised: Acknowledgments

Original: The authors would like to thank YooJung Choi for helpful discussions regarding SDP.
Revised: The authors would like to thank YooJung Choi for helpful discussions regarding SDP.

Original: This work is partially supported by NSF grants #IIS-1943641, #IIS-1633857, #CCF-1837129,
Revised: This work is partially supported by NSF grants #IIS-1943641, #IIS-1633857, #CCF-1837129,

Original: DARPA grant #N66001-17-2-4032, a Sloan Fellowship, Intel, and Facebook.
Revised: DARPA grant #N66001-17-2-4032, a Sloan Fellowship, Intel, and Facebook.

Original: A. Proof of Theorem
Revised: A. Proof of Theorem

Original: Proof.
Revised: Proof.

Original: We provide the proof for the case of using EP =
Revised: We provide the proof for the case of using EP =

Original: EP f , and U ( z ) providing an upper bound on the probabilistic predictor f ( zM ) .
Revised: EP f , and U ( z ) providing an upper bound on the probabilistic predictor f ( zM ) .

Original: The proof is mostly identical in the case of having EP =
Revised: The proof is mostly identical in the case of having EP =

Original: EP O
Revised: EP O

Original: and U ( z ) being an upper bound for the log-odds predictor O ( zM ) .
Revised: and U ( z ) being an upper bound for the log-odds predictor O ( zM ) .

Original: Let T be the decision threshold.
Revised: Let T be the decision threshold.

Original: Then we have
Revised: Then we have

Original: SDP C , x ( z ) =
Revised: SDP C , x ( z ) =

Original: Pr( f ( zm ) ≥ T ) and EP f ( z ) =
Revised: Pr( f ( zm ) ≥ T ) and EP f ( z ) =

Original: E [ f ( zm )] where m ∼ Pr( M | z ) .
Revised: E [ f ( zm )] where m ∼ Pr( M | z ) .

Original: Thus,
Revised: Thus,

Original: EP f ( z ) =
Revised: EP f ( z ) =

Original: E [ f ( zm )]
Revised: E [ f ( zm )]

Original: =
Revised: =

Original: E [ f ( zm ) | f
Revised: E [ f ( zm ) | f

Original: ( zm ) < T ] · Pr( f ( zm ) < T ) + E [ f ( zm ) | f ( zm ) ≥ T ] · Pr( f ( zm ) ≥ T ) <
Revised: ( zm ) < T ] · Pr( f ( zm ) < T ) + E [ f ( zm ) | f ( zm ) ≥ T ] · Pr( f ( zm ) ≥ T ) <

Original: T (1 − Pr( f ( zm ) ≥ T ))
Revised: T (1 − Pr( f ( zm ) ≥ T ))

Original: + U ( z )
Revised: + U ( z )

Original: Pr( f ( zm ) ≥ T ) =
Revised: Pr( f ( zm ) ≥ T ) =

Original: T + ( U ( z ) − T )
Revised: T + ( U ( z ) − T )

Original: Pr( f ( zm ) ≥ T ) =
Revised: Pr( f ( zm ) ≥ T ) =

Original: T + ( U ( z ) − T )
Revised: T + ( U ( z ) − T )

Original: SDP C , x ( z ) .
Revised: SDP C , x ( z ) .

Original: Rearranging the terms leads to Equation 1.
Revised: Rearranging the terms leads to Equation 1.

Original: Next we show how to construct a distribution for f ( zM ) to demonstrate the tightness of the bound.
Revised: Next we show how to construct a distribution for f ( zM ) to demonstrate the tightness of the bound.

Original: Assume we are given
Revised: Assume we are given

Original: EP f ( z ) =
Revised: EP f ( z ) =

Original: F , U ( z ) =
Revised: F , U ( z ) =

Original: U , and a decision threshold T with
Revised: U , and a decision threshold T with

Original: F < T < U .
Revised: F < T < U .

Original: Then let (cid:15) > 0 and consider the distribution given by
Revised: Then let (cid:15) > 0 and consider the distribution given by

Original: [Equation]
Revised: [Equation]

Original: otherwise where A is some constant which makes EP f ( z )
Revised: otherwise where A is some constant which makes EP f ( z )

Original: =
Revised: =

Original: F and
Revised: F and

Original: A < T so that SDP C , x ( z ) =
Revised: A < T so that SDP C , x ( z ) =

Original: F − T U − T + (cid:15) .
Revised: F − T U − T + (cid:15) .

Original: We now show how to solve for A .
Revised: We now show how to solve for A .

Original: We start by computing expected prediction (Deﬁnition 2), we have:
Revised: We start by computing expected prediction (Deﬁnition 2), we have:

Original: EP
Revised: EP

Original: [Equation]
Revised: [Equation]

Original: z
Revised: z

Original: [Equation]
Revised: [Equation]

Original: =
Revised: =

Original: [Equation]
Revised: [Equation]

Original: U
Revised: U

Original: [Equation]
Revised: [Equation]

Original: (cid:19)
Revised: (cid:19)

Original: =
Revised: =

Original: F and then solve for A :
Revised: F and then solve for A :

Original: [Equation]
Revised: [Equation]

Original: U
Revised: U

Original: [Equation]
Revised: [Equation]

Original: ) (cid:15)
Revised: ) (cid:15)

Original: [Equation]
Revised: [Equation]

Original: U
Revised: U

Original: [Equation]
Revised: [Equation]

Original: ) −
Revised: ) −

Original: [Equation]
Revised: [Equation]

Original: ( U − T ) (cid:15)
Revised: ( U − T ) (cid:15)

Original: U − F − ( U − T ) (cid:15) =
Revised: U − F − ( U − T ) (cid:15) =

Original: T
Revised: T

Original: Thus we have EP f ( z ) =
Revised: Thus we have EP f ( z ) =

Original: F and, since A < T , SDP C , x ( z ) =
Revised: F and, since A < T , SDP C , x ( z ) =

Original: F − T
Revised: F − T

Original: U − T + (cid:15) .
Revised: U − T + (cid:15) .

Original: A. Beam Search Algorithm
Revised: A. Beam Search Algorithm

Original: Input: instance x , feature distribution Pr( z ) , expected prediction function EP ( z ) , max features k , beam size b 1: MLSE ← ∅ 2: beam ← {∅} 3: for i = 1 ...
Revised: Input: instance x , feature distribution Pr( z ) , expected prediction function EP ( z ) , max features k , beam size b 1: MLSE ← ∅ 2: beam ← {∅} 3: for i = 1 ...

Original: k do 4: candidates ← (cid:83) c ∈ beam expand ( c ) 5: compute EP ( c ) and Pr( c ) for c ∈ candidates 6: beam ← top b ( candidates ) 7: update MLSE 8: end for
Revised: k do 4: candidates ← (cid:83) c ∈ beam expand ( c ) 5: compute EP ( c ) and Pr( c ) for c ∈ candidates 6: beam ← top b ( candidates ) 7: update MLSE 8: end for

Original: In the above pseudocode, the expand function generates a new set of candidates, each formed by adding one yet unobserved feature to the input candidate.
Revised: In the above pseudocode, the expand function generates a new set of candidates, each formed by adding one yet unobserved feature to the input candidate.

Original: The top b function selects the b candidates with the highest EP, with ties broken using Pr .
Revised: The top b function selects the b candidates with the highest EP, with ties broken using Pr .

Original: Finally, the MLSE is updated if an explanation with a higher EP is found, or if one has the same EP as the previous
Revised: Finally, the MLSE is updated if an explanation with a higher EP is found, or if one has the same EP as the previous

Original: MLSE but a higher Pr .
Revised: MLSE but a higher Pr .

Original: Computational Complexity
Revised: Computational Complexity

Original: When using PCs to model the feature distribution, the runtimes for tractable expected prediction (EP) and marginal ( Pr ) algorithms do not depend on how many features are observed.
Revised: When using PCs to model the feature distribution, the runtimes for tractable expected prediction (EP) and marginal ( Pr ) algorithms do not depend on how many features are observed.

Original: Pr can be computed in linear time w.r.t.
Revised: Pr can be computed in linear time w.r.t.

Original: circuit size.
Revised: circuit size.

Original: Linearity of decision forest classiﬁcation also allows for expected prediction computations in time
Revised: Linearity of decision forest classiﬁcation also allows for expected prediction computations in time

Original: linear w.r.t.
Revised: linear w.r.t.

Original: circuit size.
Revised: circuit size.

Original: Thus we treat EP and Pr as oracles and measure our algorithm’s runtime by the number of calls to them.
Revised: Thus we treat EP and Pr as oracles and measure our algorithm’s runtime by the number of calls to them.

Original: In each level of the beam search, we expand the previous level’s top b candidates, giving us at most nb unique states, where n is the number of features.
Revised: In each level of the beam search, we expand the previous level’s top b candidates, giving us at most nb unique states, where n is the number of features.

Original: We then call EP and Pr once for each state.
Revised: We then call EP and Pr once for each state.

Original: Since the search stops after level k , overall we make in total O ( nkb ) calls to the EP and Pr oracles.
Revised: Since the search stops after level k , overall we make in total O ( nkb ) calls to the EP and Pr oracles.

Original: A. More Experiment Details Computing Infrastructure
Revised: A. More Experiment Details Computing Infrastructure

Original: All experiments were run on a Linux server with 40 CPU cores and 500 GB of RAM, albeit not all the memory or
Revised: All experiments were run on a Linux server with 40 CPU cores and 500 GB of RAM, albeit not all the memory or

Original: CPU cores where needed for generating one explanation.
Revised: CPU cores where needed for generating one explanation.

Original: Our method is highly parallizable as we see adding more threads speeds up the process substantially.
Revised: Our method is highly parallizable as we see adding more threads speeds up the process substantially.

Original: No GPUs were used in this paper.
Revised: No GPUs were used in this paper.

Original: Although we did not use any GPUs, it should be easy to re-implement some parts of our method to support GPUs.
Revised: Although we did not use any GPUs, it should be easy to re-implement some parts of our method to support GPUs.

Original: Some of the main ingredients of our method such as computing marginal probabilities using Probabilistic Circuits already support GPUs and enjoy huge speed ups.
Revised: Some of the main ingredients of our method such as computing marginal probabilities using Probabilistic Circuits already support GPUs and enjoy huge speed ups.

Original: Furthermore, computing expected predictions and the search at each level should be amenable to GPU parallization.
Revised: Furthermore, computing expected predictions and the search at each level should be amenable to GPU parallization.

Original: So, overall we should be able to speed up the algorithm by 1-2 orders of magnitude.
Revised: So, overall we should be able to speed up the algorithm by 1-2 orders of magnitude.

Original: This allows us to scale to even more complicated models with more features in the future.
Revised: This allows us to scale to even more complicated models with more features in the future.

Original: Datasets and Preprocessing Steps
Revised: Datasets and Preprocessing Steps

Original: The MNIST [Yann et al. , 2009] dataset consists of 60, grayscale images ( 28 × 28 pixels) of handwritten digits (09).
Revised: The MNIST [Yann et al. , 2009] dataset consists of 60, grayscale images ( 28 × 28 pixels) of handwritten digits (09).

Original: We limit the dataset into digits of 3 and 5 to ensure we have a binary classiﬁcation task.
Revised: We limit the dataset into digits of 3 and 5 to ensure we have a binary classiﬁcation task.

Original: The dataset is already split between train and test images, so we used the same split, selecting from each digits of 3s and 5s.
Revised: The dataset is already split between train and test images, so we used the same split, selecting from each digits of 3s and 5s.

Original: Since our probabilistic circuit library currently only supports binary feature values, we also binarized pixel values.
Revised: Since our probabilistic circuit library currently only supports binary feature values, we also binarized pixel values.

Original: indepedently for each image, by applying a threshold at 0.05 standard deviations above the mean.
Revised: indepedently for each image, by applying a threshold at 0.05 standard deviations above the mean.

Original: The classiﬁcation task for the adult dataset [ Kohavi, 1996 ] is to determine whether a given individual makes over $50 , 000 per year.
Revised: The classiﬁcation task for the adult dataset [ Kohavi, 1996 ] is to determine whether a given individual makes over $50 , 000 per year.

Original: Features include age, sex, working class, hours worked per week, education level, nationality, etc.
Revised: Features include age, sex, working class, hours worked per week, education level, nationality, etc.

Original: We perform discretization of continuous features by applying standard binning.
Revised: We perform discretization of continuous features by applying standard binning.

Original: We then perform one-hot-encoding for all features.
Revised: We then perform one-hot-encoding for all features.

Original: Again, this is because the probabilistic circuit library only supports binary features.
Revised: Again, this is because the probabilistic circuit library only supports binary features.

Original: We then used an 80split for our training and test data.
Revised: We then used an 80split for our training and test data.

Original: In general, our method should be straightforward to extend to multi-category features and also continuous features.
Revised: In general, our method should be straightforward to extend to multi-category features and also continuous features.

Original: Source code for processing the datasets will be included in the ﬁnal version of paper as supplementary.
Revised: Source code for processing the datasets will be included in the ﬁnal version of paper as supplementary.

Original: Details on models
Revised: Details on models

Original: For our MNIST experiments, we used a PC with 10124 nodes, 15640 edges, and 5916 parameters.
Revised: For our MNIST experiments, we used a PC with 10124 nodes, 15640 edges, and 5916 parameters.

Original: It took approximatelyminutes to train, stopping after 410 iterations.
Revised: It took approximatelyminutes to train, stopping after 410 iterations.

Original: For our classiﬁer, we trained a decision forest with 6 trees, each with a maximum depth of 6.
Revised: For our classiﬁer, we trained a decision forest with 6 trees, each with a maximum depth of 6.

Original: The ﬁnal model has a total of 297 leaves.
Revised: The ﬁnal model has a total of 297 leaves.

Original: The classiﬁcation accuracy on our test images is 98 .
Revised: The classiﬁcation accuracy on our test images is 98 .

Original: For our adult experiments, we used a PC with 22899 nodes, 44152 edges, and 14828 parameters.
Revised: For our adult experiments, we used a PC with 22899 nodes, 44152 edges, and 14828 parameters.

Original: It took approximately 17 minutes to train, stopping after 190 iterations of structure learning.
Revised: It took approximately 17 minutes to train, stopping after 190 iterations of structure learning.

Original: For our classiﬁer, we trained a decision forest with the same constraints as above.
Revised: For our classiﬁer, we trained a decision forest with the same constraints as above.

Original: The ﬁnal model has a total of 255 leaves.
Revised: The ﬁnal model has a total of 255 leaves.

Original: The classiﬁcation accuracy on our test dataset is 84 .
Revised: The classiﬁcation accuracy on our test dataset is 84 .

Original: Metric Calculation Details
Revised: Metric Calculation Details

Original: The classiﬁers used in our experiments were decision forests, which allow for efﬁcient computation of expected log-odds.
Revised: The classiﬁers used in our experiments were decision forests, which allow for efﬁcient computation of expected log-odds.

Original: Classiﬁcation using these models is done by summing the weights of one leaf from each tree, where the chosen leaf from each tree has a path from the root not contradicting with the instance to be classiﬁed.
Revised: Classiﬁcation using these models is done by summing the weights of one leaf from each tree, where the chosen leaf from each tree has a path from the root not contradicting with the instance to be classiﬁed.

Original: This additive model allows us to utilize linearity of expectation to compute expected log-odds [Khosravi et al. , 2020].
Revised: This additive model allows us to utilize linearity of expectation to compute expected log-odds [Khosravi et al. , 2020].

Original: This model does not, however, allow for efﬁcient computation of expected predictions, where prediction refers to taking the sigmoid of the classiﬁer output.
Revised: This model does not, however, allow for efﬁcient computation of expected predictions, where prediction refers to taking the sigmoid of the classiﬁer output.

Original: This is due to the nonlinear nature of the sigmoid function.
Revised: This is due to the nonlinear nature of the sigmoid function.

Original: Instead, when we give expected prediction values, we use a ﬁrst order approximation by taking the sigmoid of the expected log-odds.
Revised: Instead, when we give expected prediction values, we use a ﬁrst order approximation by taking the sigmoid of the expected log-odds.

Original: In Figure 2 we show a plot with SDP lower bounds computed using Theorem 1.
Revised: In Figure 2 we show a plot with SDP lower bounds computed using Theorem 1.

Original: Graphing of the blue curve requires computation of an upper bound U ( z ) on the predictions of a decision forest.
Revised: Graphing of the blue curve requires computation of an upper bound U ( z ) on the predictions of a decision forest.

Original: The upper bound used is a loose one which we computed by adding the weights of one leaf from each tree in the forest, where each leaf is the maximum weighted leaf in its tree whose path from the root does not contradict with the explanation.
Revised: The upper bound used is a loose one which we computed by adding the weights of one leaf from each tree in the forest, where each leaf is the maximum weighted leaf in its tree whose path from the root does not contradict with the explanation.

Original: This bound is a loose once since paths for leaves from different trees may contradict.
Revised: This bound is a loose once since paths for leaves from different trees may contradict.

Original: For the red curve, since our model outputs log-odds, we used the ﬁrst order approximation mentioned above.
Revised: For the red curve, since our model outputs log-odds, we used the ﬁrst order approximation mentioned above.

Original: Effects of Parallelism on Performance
Revised: Effects of Parallelism on Performance

Original: One advantage of our method is that we can parallelize computation of expected predictions at each level of the beam search.
Revised: One advantage of our method is that we can parallelize computation of expected predictions at each level of the beam search.

Original: The effects of can be seen in Figure 4, which plots the cumulative time taken to complete each level of the beam search for different numbers of threads.
Revised: The effects of can be seen in Figure 4, which plots the cumulative time taken to complete each level of the beam search for different numbers of threads.

Original: We see that more threads do indeed allow for much faster runtimes.
Revised: We see that more threads do indeed allow for much faster runtimes.

Original: We leave investigating beneﬁts of GPU acceleration for future work.
Revised: We leave investigating beneﬁts of GPU acceleration for future work.

Original: More on SDP Lowerbounds
Revised: More on SDP Lowerbounds

Original: In the main text we saw that Theorem 1 gives us two ways of computing a lower bound for SDP: using a probabilistic predictor f or a log-odds predictor O .
Revised: In the main text we saw that Theorem 1 gives us two ways of computing a lower bound for SDP: using a probabilistic predictor f or a log-odds predictor O .

Original: One of them might give a tigher bound, depending on how our predictor is deﬁned.
Revised: One of them might give a tigher bound, depending on how our predictor is deﬁned.

Original: Generally, the tighter bound we can ﬁnd for U ( z ) the tighter bound we get for SDP.
Revised: Generally, the tighter bound we can ﬁnd for U ( z ) the tighter bound we get for SDP.

Original: For example, in the case of having a probabilistic predictor, the decision threshold is usually T = 0 .
Revised: For example, in the case of having a probabilistic predictor, the decision threshold is usually T = 0 .

Original: 5 , and we can easily get a trivial upper bound of U ( z )
Revised: 5 , and we can easily get a trivial upper bound of U ( z )

Original: = 1 .
Revised: = 1 .

Original: Depending on the model family we might be able to get a tighter bound for
Revised: Depending on the model family we might be able to get a tighter bound for

Original: U ( z ) , and hence a tighter lower bound for SDP.
Revised: U ( z ) , and hence a tighter lower bound for SDP.

Original: For the case of a log-odds predictor, we usually take T = 0 , but getting an upper-bound U ( z ) might not be trivial.
Revised: For the case of a log-odds predictor, we usually take T = 0 , but getting an upper-bound U ( z ) might not be trivial.

Original: However, in many cases an upper bound U ( z ) can be computed efﬁciently for log-odds predictor, but might not be tight.
Revised: However, in many cases an upper bound U ( z ) can be computed efﬁciently for log-odds predictor, but might not be tight.

Original: Limitations
Revised: Limitations

Original: Some knowledge of probabilities is needed to fully interpret the results.
Revised: Some knowledge of probabilities is needed to fully interpret the results.

Original: However, as seen with the MNIST example, visual representations can be designed to give users a feel for what the model is doing.
Revised: However, as seen with the MNIST example, visual representations can be designed to give users a feel for what the model is doing.

Original: More MNIST examples
Revised: More MNIST examples

Original: Figure 5 provides some extra examples on sufﬁcient explanations for MNIST.
Revised: Figure 5 provides some extra examples on sufﬁcient explanations for MNIST.

Original: We see examples for both correctly and incorrectly classiﬁed images, and see the effects of cardinality constraints on the explanations.
Revised: We see examples for both correctly and incorrectly classiﬁed images, and see the effects of cardinality constraints on the explanations.

Original: [Figure]
Revised: [Figure]