Original: Decision Trees with Short Explainable Rules
Revised: Decision Trees with Short Explainable Rules

Original: Abstract
Revised: Abstract

Original: Decision trees are widely used in many settings where interpretable models are preferred or required.
Revised: Decision trees are widely used in many settings where interpretable models are preferred or required.

Original: As conﬁrmed by recent empirical studies, the interpretability/explainability of a decision tree critically depends on some of its structural parameters, like size and the average/maximum depth of its leaves.
Revised: As conﬁrmed by recent empirical studies, the interpretability/explainability of a decision tree critically depends on some of its structural parameters, like size and the average/maximum depth of its leaves.

Original: There is indeed a vast literature on the design and analysis of decision tree algorithms that aim at optimizing these parameters.
Revised: There is indeed a vast literature on the design and analysis of decision tree algorithms that aim at optimizing these parameters.

Original: This paper contributes to this important line of research: we propose as a novel criterion of measuring the interpretability of a decision tree, the sparsity of the set of attributes that are (on average) required to explain the classiﬁcation of the examples.
Revised: This paper contributes to this important line of research: we propose as a novel criterion of measuring the interpretability of a decision tree, the sparsity of the set of attributes that are (on average) required to explain the classiﬁcation of the examples.

Original: We give a tight characterization of the best possible guarantees achievable by a decision tree built to optimize both our new measure (which we call the explanation size) and the more classical measures of worst-case and average depth.
Revised: We give a tight characterization of the best possible guarantees achievable by a decision tree built to optimize both our new measure (which we call the explanation size) and the more classical measures of worst-case and average depth.

Original: In particular, we give an algorithm that guarantees O (ln n ) -approximation
Revised: In particular, we give an algorithm that guarantees O (ln n ) -approximation

Original: (hence optimal if
Revised: (hence optimal if

Original: P (cid:54) = NP ) for the minimization of both the average/worst-case explanation size and the average/worst-case depth.
Revised: P (cid:54) = NP ) for the minimization of both the average/worst-case explanation size and the average/worst-case depth.

Original: In addition to our theoretical contributions, experiments with 20 real datasets show that our algorithm has accuracy competitive with CART while producing trees that allow for much simpler explanations.
Revised: In addition to our theoretical contributions, experiments with 20 real datasets show that our algorithm has accuracy competitive with CART while producing trees that allow for much simpler explanations.

Original:  Introduction
Revised:  Introduction

Original: Machine learning models and algorithms appear more and more frequently in systems that make decisions with an impact in our lives.
Revised: Machine learning models and algorithms appear more and more frequently in systems that make decisions with an impact in our lives.

Original: Thus, it is highly desirable that the output of these methods are interpretable so that we can use them more comfortably or, eventually, question its applicability [13].
Revised: Thus, it is highly desirable that the output of these methods are interpretable so that we can use them more comfortably or, eventually, question its applicability [13].

Original: Decision trees are used in many settings as a tool to provide explainability.
Revised: Decision trees are used in many settings as a tool to provide explainability.

Original: However, their explainability greatly depends on the depths of its leaves, as empirically demonstrated by [46].
Revised: However, their explainability greatly depends on the depths of its leaves, as empirically demonstrated by [46].

Original: In fact, based on empirical data from a survey with 98 questions answered by 69 respondents, the authors conclude that “question depth” (the depth of the deepest leaf that is required when answering questions about a tree) turns out to be the most important parameter.
Revised: In fact, based on empirical data from a survey with 98 questions answered by 69 respondents, the authors conclude that “question depth” (the depth of the deepest leaf that is required when answering questions about a tree) turns out to be the most important parameter.

Original: Essentially, users prefer trees where the information about the most common items are given at the top of the tree.
Revised: Essentially, users prefer trees where the information about the most common items are given at the top of the tree.

Original: Minimizing the average/worst-case depth indeed has been a classic goal for decision tree algorithms (see the Related Work section below).
Revised: Minimizing the average/worst-case depth indeed has been a classic goal for decision tree algorithms (see the Related Work section below).

Original: However, another very important component for explainability is having decision rules that are sparse, namely, that use as few different attributes as possible to classify an object or make a prediction.
Revised: However, another very important component for explainability is having decision rules that are sparse, namely, that use as few different attributes as possible to classify an object or make a prediction.

Original: For decision trees, this means that the path to any given leaf should test only a small number of different attributes.
Revised: For decision trees, this means that the path to any given leaf should test only a small number of different attributes.

Original: Figure 1 shows two trees for the Sensorless dataset [7] with similar accuracy.
Revised: Figure 1 shows two trees for the Sensorless dataset [7] with similar accuracy.

Original: While the trees have the same size, the rightmost one gives much more concise classiﬁcation rules.
Revised: While the trees have the same size, the rightmost one gives much more concise classiﬁcation rules.

Original: As a spoiler, the left tree was constructed using CART and the right one using the algorithm we propose in this paper.
Revised: As a spoiler, the left tree was constructed using CART and the right one using the algorithm we propose in this paper.

Original: To exemplify their differences, let us consider the leaves that are marked in the ﬁgure with a thick rectangle.
Revised: To exemplify their differences, let us consider the leaves that are marked in the ﬁgure with a thick rectangle.

Original: Both are assigned the same class and cover the maximum number of examples in the training set (approximately 11.500 example each).
Revised: Both are assigned the same class and cover the maximum number of examples in the training set (approximately 11.500 example each).

Original: Despite this similar behaviour, the explanations for their classiﬁcations are quite different (the D
Revised: Despite this similar behaviour, the explanations for their classiﬁcations are quite different (the D

Original: i ’s denote the attributes):
Revised: i ’s denote the attributes):

Original: CART:
Revised: CART:

Original: D 11 ∈
Revised: D 11 ∈

Original: [-0.11, 0.07] AND
Revised: [-0.11, 0.07] AND

Original: D 9 > -0.01 AND
Revised: D 9 > -0.01 AND

Original: D 10 ≤ 0.
Revised: D 10 ≤ 0.

Original: New algorithm: D 11 ∈
Revised: New algorithm: D 11 ∈

Original: Deﬁne the explanation size of a leaf as the number of distinct attributes tested on the path from the root to this leaf.
Revised: Deﬁne the explanation size of a leaf as the number of distinct attributes tested on the path from the root to this leaf.

Original: The above example shows that having trees whose leaves have small explanation size yields signiﬁcantly simpler (hence easier to interpret/explain) rules.
Revised: The above example shows that having trees whose leaves have small explanation size yields signiﬁcantly simpler (hence easier to interpret/explain) rules.

Original: However, to the best of our knowledge there is no prior work considering decision trees optimized with respect to the explanation size.
Revised: However, to the best of our knowledge there is no prior work considering decision trees optimized with respect to the explanation size.

Original: Our contributions.
Revised: Our contributions.

Original: In this work we propose the explanation size as a novel measure to capture the interpretability of a decision tree.
Revised: In this work we propose the explanation size as a novel measure to capture the interpretability of a decision tree.

Original: In particular, we initiate a principled study of decision trees with small explanation size by focusing on: (i) the trade-off that optimizing this criterion imposes on other desirable metrics (e.g., average/worst-case depth); (ii) the design of efﬁcient algorithms for building such trees that guarantee good performance in practice.
Revised: In particular, we initiate a principled study of decision trees with small explanation size by focusing on: (i) the trade-off that optimizing this criterion imposes on other desirable metrics (e.g., average/worst-case depth); (ii) the design of efﬁcient algorithms for building such trees that guarantee good performance in practice.

Original: Our ﬁrst result is that it is possible to essentially obtain a best-of-both worlds in terms of average/worstcase depth and explanation size: We show that there is always a binary decision tree that has the smallest average explanation size possible but also has average depth not much larger than that of the optimal tree for the latter metric (Item 2 of Theorem 1).
Revised: Our ﬁrst result is that it is possible to essentially obtain a best-of-both worlds in terms of average/worstcase depth and explanation size: We show that there is always a binary decision tree that has the smallest average explanation size possible but also has average depth not much larger than that of the optimal tree for the latter metric (Item 2 of Theorem 1).

Original: The same result also holds when considering worst-case explanation size and depth (Item 1 of Theorem 1).
Revised: The same result also holds when considering worst-case explanation size and depth (Item 1 of Theorem 1).

Original: Remarkably, the latter (worst-case bound) turns out to be tight.
Revised: Remarkably, the latter (worst-case bound) turns out to be tight.

Original: Theorem 2 shows it matches a necessary trade-off between optimizing explanation size and depth, i.e., improving the bound on the depth in Theorem 1 can only be attained at the cost of a logarithmic factor loss in the explanation size.
Revised: Theorem 2 shows it matches a necessary trade-off between optimizing explanation size and depth, i.e., improving the bound on the depth in Theorem 1 can only be attained at the cost of a logarithmic factor loss in the explanation size.

Original: Despite having strong theoretical guarantees, the construction to obtain these trees is too wasteful to be used in practice.
Revised: Despite having strong theoretical guarantees, the construction to obtain these trees is too wasteful to be used in practice.

Original: Thus, our second contribution is an algorithm that still yields a tree that provably approximates both optimal average/worst-case explanation size and depth (Theorem 3) but has enough ﬂexibility that it can be employed to obtain a good performance in practice.
Revised: Thus, our second contribution is an algorithm that still yields a tree that provably approximates both optimal average/worst-case explanation size and depth (Theorem 3) but has enough ﬂexibility that it can be employed to obtain a good performance in practice.

Original: To demonstrate the applicability of our proposal, we compare it against CART [11], a quite popular method to build decision trees, on 20 real classiﬁcation datasets.
Revised: To demonstrate the applicability of our proposal, we compare it against CART [11], a quite popular method to build decision trees, on 20 real classiﬁcation datasets.

Original: Our method leads to much more (resp.
Revised: Our method leads to much more (resp.

Original: more) interpretable trees in terms of explanation size (resp.
Revised: more) interpretable trees in terms of explanation size (resp.

Original: average depth), while having a performance similar to CART in terms of accuracy and speed.
Revised: average depth), while having a performance similar to CART in terms of accuracy and speed.

Original:  Related work
Revised:  Related work

Original: Our work can be connected with an active line of research that proposes interpretable models for machine learning [13], in particular more interpretable rule-based models (e.g. decision lists, sets, and trees) [37, 30, 21, 9].
Revised: Our work can be connected with an active line of research that proposes interpretable models for machine learning [13], in particular more interpretable rule-based models (e.g. decision lists, sets, and trees) [37, 30, 21, 9].

Original: Our interpretability metric is closely related to rule sizes considered in the rule learning literature, e.g., [31, 47].
Revised: Our interpretability metric is closely related to rule sizes considered in the rule learning literature, e.g., [31, 47].

Original: In addition, we can relate our work with those from the vast literature of methods with provable guarantees that are designed to build decision trees of “low complexity” (e.g. depth of leaves, number of nodes, etc.)
Revised: In addition, we can relate our work with those from the vast literature of methods with provable guarantees that are designed to build decision trees of “low complexity” (e.g. depth of leaves, number of nodes, etc.)

Original: More interpretable rule-based models.
Revised: More interpretable rule-based models.

Original: There is a body of work that aims to understand what makes a rule model more comprehensible via experiments with end users [4, 29, 46].
Revised: There is a body of work that aims to understand what makes a rule model more comprehensible via experiments with end users [4, 29, 46].

Original: The paper [4] compares the comprehensibility of classiﬁers that are learned by decision trees and rule-learning algorithms, based on subjective comparisons of classiﬁer pairs by 100 Computer Science students.
Revised: The paper [4] compares the comprehensibility of classiﬁers that are learned by decision trees and rule-learning algorithms, based on subjective comparisons of classiﬁer pairs by 100 Computer Science students.

Original: They conclude that decision trees are more comprehensible.
Revised: They conclude that decision trees are more comprehensible.

Original: In [29], based on experiments with business students, it is concluded that decision tables are more interpretable than decision trees and propositional “if-then” rules.
Revised: In [29], based on experiments with business students, it is concluded that decision tables are more interpretable than decision trees and propositional “if-then” rules.

Original: One potential limitation of this study is that these students had no experience with the representation formats, so it is not clear whether this conclusion can be extended for more experienced users.
Revised: One potential limitation of this study is that these students had no experience with the representation formats, so it is not clear whether this conclusion can be extended for more experienced users.

Original: The work of [46] reports a survey with 69 respondents that was carried out to understand what makes a decision tree more interpretable.
Revised: The work of [46] reports a survey with 69 respondents that was carried out to understand what makes a decision tree more interpretable.

Original: Among their ﬁndings is that the depth of the leaves required by users during the experiments had one of the biggest and most consistent impact on the usability of decision trees across multiple tasks (classify, explain, validate, and discover).
Revised: Among their ﬁndings is that the depth of the leaves required by users during the experiments had one of the biggest and most consistent impact on the usability of decision trees across multiple tasks (classify, explain, validate, and discover).

Original: There are some recent works that try to optimize interpretabilty metrics when building rule-based models [37, 10, 56, 30, 40, 9].
Revised: There are some recent works that try to optimize interpretabilty metrics when building rule-based models [37, 10, 56, 30, 40, 9].

Original: We brieﬂy discuss those that focus on decision trees.
Revised: We brieﬂy discuss those that focus on decision trees.

Original: The paper [30] contends that splits that have at least one of its parts/child nodes being class-homogeneous (roughly this means that most examples have the same class) are important for interpretability, and propose a splitting criterion that tries to balance this homogeneity and the depth of the leaves.
Revised: The paper [30] contends that splits that have at least one of its parts/child nodes being class-homogeneous (roughly this means that most examples have the same class) are important for interpretability, and propose a splitting criterion that tries to balance this homogeneity and the depth of the leaves.

Original: In contrast to our work, no provable guarantees are provided.
Revised: In contrast to our work, no provable guarantees are provided.

Original: The works [10, 56] employ Integer Linear Programming to build trees of a given maximum depth, while [40] employs Dynamic Programming techniques to develop an optimization framework that allows the construction of decision trees with few leaves that optimize a variety of objective functions such as F-score and AUC .
Revised: The works [10, 56] employ Integer Linear Programming to build trees of a given maximum depth, while [40] employs Dynamic Programming techniques to develop an optimization framework that allows the construction of decision trees with few leaves that optimize a variety of objective functions such as F-score and AUC .

Original: These methods, while interesting, are much more complex to implement than ours and they do not run in polynomial time, which may compromise their application on large datasets.
Revised: These methods, while interesting, are much more complex to implement than ours and they do not run in polynomial time, which may compromise their application on large datasets.

Original: Decision trees with provable guarantees.
Revised: Decision trees with provable guarantees.

Original: There is a vast literature dedicated to the problem of building decision trees with “low complexity”, where the complexity of a tree can be measured in different ways, including worst-case/average leaf depth and number of nodes, among others [12].
Revised: There is a vast literature dedicated to the problem of building decision trees with “low complexity”, where the complexity of a tree can be measured in different ways, including worst-case/average leaf depth and number of nodes, among others [12].

Original: It is known that building a decision tree that minimizes the worst-case or average leaf depth, among those that ﬁt the training data, does not admit an o (log n ) approximation unless P=NP [16, 36].
Revised: It is known that building a decision tree that minimizes the worst-case or average leaf depth, among those that ﬁt the training data, does not admit an o (log n ) approximation unless P=NP [16, 36].

Original: When the goal is minimizing the number of nodes, the problem is even harder to approximate [3, 26].
Revised: When the goal is minimizing the number of nodes, the problem is even harder to approximate [3, 26].

Original: Regarding upper bounds, several algorithms with the optimal (within constants) O (log n ) -approximation are known for minimizing the worst-case and average depth [23, 22, 17, 18, 8, 24, 32, 39, 44].
Revised: Regarding upper bounds, several algorithms with the optimal (within constants) O (log n ) -approximation are known for minimizing the worst-case and average depth [23, 22, 17, 18, 8, 24, 32, 39, 44].

Original: What distinguishes each method is the generality of its guarantee; for example, some consider scenarios that include tests with noisy outcomes [32], while others consider items/examples with non-uniform weights and tests with non-uniform costs [18, 44].
Revised: What distinguishes each method is the generality of its guarantee; for example, some consider scenarios that include tests with noisy outcomes [32], while others consider items/examples with non-uniform weights and tests with non-uniform costs [18, 44].

Original: We remark that the method we propose here also allows the use of non-uniform weights on the examples, which can be used, for instance, to prioritize models that yield simpler explanations for some classes of particular interest, by setting high weights for examples in these classes.
Revised: We remark that the method we propose here also allows the use of non-uniform weights on the examples, which can be used, for instance, to prioritize models that yield simpler explanations for some classes of particular interest, by setting high weights for examples in these classes.

Original: The key difference between our method and the existing ones is that ours is the only one that provides theoretical guarantees on the explanation size.
Revised: The key difference between our method and the existing ones is that ours is the only one that provides theoretical guarantees on the explanation size.

Original:  Model
Revised:  Model

Original: We describe the model used throughout for the theoretical analyses.
Revised: We describe the model used throughout for the theoretical analyses.

Original: For that, we adopt a terminology similar to that employed by some closely related works [1, 18].
Revised: For that, we adopt a terminology similar to that employed by some closely related works [1, 18].

Original: On an instance I , there is a set of ordinal attributes A and a set of objects O , where each object o ∈ O is described by the value it takes on each attribute a ∈ A ; we denote this value by a ( o ) .
Revised: On an instance I , there is a set of ordinal attributes A and a set of objects O , where each object o ∈ O is described by the value it takes on each attribute a ∈ A ; we denote this value by a ( o ) .

Original: Each object o also has a class c ( o ) in some set
Revised: Each object o also has a class c ( o ) in some set

Original: C .
Revised: C .

Original: In order to classify an object o , we are allowed to use threshold tests of the form “Is a ( o ) < t ?”
Revised: In order to classify an object o , we are allowed to use threshold tests of the form “Is a ( o ) < t ?”

Original: for some attribute a and threshold value t .
Revised: for some attribute a and threshold value t .

Original: A (threshold) decision tree T is a rooted tree where each internal node ν is associated with a threshold test “Is a ( o ) < t ?”
Revised: A (threshold) decision tree T is a rooted tree where each internal node ν is associated with a threshold test “Is a ( o ) < t ?”

Original: and the edges from a node to its children are associated to the two possible outcomes “ a ( o ) < t ” and “ a ( o ) ≥ t ”.
Revised: and the edges from a node to its children are associated to the two possible outcomes “ a ( o ) < t ” and “ a ( o ) ≥ t ”.

Original: We also refer to this test by the attribute/threshold pair ( a, t ) .
Revised: We also refer to this test by the attribute/threshold pair ( a, t ) .

Original: Each leaf (cid:96) of T is associated with a class in C. Given a decision tree T , we say that an object o reaches a node ν of T if it agrees with all outcomes associated with the path from the root of T to ν .
Revised: Each leaf (cid:96) of T is associated with a class in C. Given a decision tree T , we say that an object o reaches a node ν of T if it agrees with all outcomes associated with the path from the root of T to ν .

Original: For each o ∈ O, we use (cid:96) ( o ) to denote the unique leaf reached by object o .
Revised: For each o ∈ O, we use (cid:96) ( o ) to denote the unique leaf reached by object o .

Original: Finally, we consider the exact classiﬁcation model, namely a decision tree must correctly classify all objects of the instance, i.e., each object o ∈ O reaches some leaf associated to its correct class c ( o ) .
Revised: Finally, we consider the exact classiﬁcation model, namely a decision tree must correctly classify all objects of the instance, i.e., each object o ∈ O reaches some leaf associated to its correct class c ( o ) .

Original: We now formalize the measures of interpretability that were mentioned in the introduction.
Revised: We now formalize the measures of interpretability that were mentioned in the introduction.

Original: Depth and explanation size of a tree.
Revised: Depth and explanation size of a tree.

Original: We start recalling the classical notions of worst-case and average tree depth.
Revised: We start recalling the classical notions of worst-case and average tree depth.

Original: Given a decision tree T , the depth of a leaf (cid:96) is the number of tests/internal-nodes on the path from the root of T to (cid:96) , and is denoted by depth ( (cid:96) ) .
Revised: Given a decision tree T , the depth of a leaf (cid:96) is the number of tests/internal-nodes on the path from the root of T to (cid:96) , and is denoted by depth ( (cid:96) ) .

Original: The worst-case depth of the tree T is obtained by considering the maximum depth over its leaves, namely
Revised: The worst-case depth of the tree T is obtained by considering the maximum depth over its leaves, namely

Original: [Equation]
Revised: [Equation]

Original: To measure the average depth of the tree, in addition to the datum above, as part of the input each object o has a non-negative weight w ( o ) ∈ R + indicating its likelihood/importance.
Revised: To measure the average depth of the tree, in addition to the datum above, as part of the input each object o has a non-negative weight w ( o ) ∈ R + indicating its likelihood/importance.

Original: Letting w ( (cid:96) ) be the sum of the weights of all objects that reach leaf (cid:96) , the average depth of the tree T is then the weighted sum of the depth of its leaves, namely
Revised: Letting w ( (cid:96) ) be the sum of the weights of all objects that reach leaf (cid:96) , the average depth of the tree T is then the weighted sum of the depth of its leaves, namely

Original: [Equation]
Revised: [Equation]

Original: We now consider the novel measures of quality/interpretability of a tree using the notion of explanation size.
Revised: We now consider the novel measures of quality/interpretability of a tree using the notion of explanation size.

Original: The explanation size of a leaf (cid:96) , denoted by expl ( (cid:96) ) , is the number of different attributes on the tests on the path from the root of T to (cid:96) .
Revised: The explanation size of a leaf (cid:96) , denoted by expl ( (cid:96) ) , is the number of different attributes on the tests on the path from the root of T to (cid:96) .

Original: As an example, the explanation size of the leaf marked with a thick rectangle on the left tree of Figure 1 is 3.
Revised: As an example, the explanation size of the leaf marked with a thick rectangle on the left tree of Figure 1 is 3.

Original: The worst-case and average explanation size of a tree T are then given as before by the largest and the weighted sum of the explanation sizes of its leaves, respectively:
Revised: The worst-case and average explanation size of a tree T are then given as before by the largest and the weighted sum of the explanation sizes of its leaves, respectively:

Original: [Equation]
Revised: [Equation]

Original: Our goal is to obtain trees with as small worst-case/average depth and explanation size as possible.
Revised: Our goal is to obtain trees with as small worst-case/average depth and explanation size as possible.

Original: We use depth
Revised: We use depth

Original: [Equation]
Revised: [Equation]

Original: to denote the smallest possible worst-case depth of a decision tree that solves instance I , and deﬁne the optimal values expl ∗ wc , depth ∗ avg , and expl ∗ avg analogously.
Revised: to denote the smallest possible worst-case depth of a decision tree that solves instance I , and deﬁne the optimal values expl ∗ wc , depth ∗ avg , and expl ∗ avg analogously.

Original:  Tradeoff between depth and explanation size
Revised:  Tradeoff between depth and explanation size

Original: Our goal is to obtain trees that simultaneously have both small worst-case/average depth and explanation size.
Revised: Our goal is to obtain trees that simultaneously have both small worst-case/average depth and explanation size.

Original: However, is this even possible?
Revised: However, is this even possible?

Original: It is conceivable that in order to obtain trees with small depth, one may be required to use several different attributes along the paths to effectively classify the objects; but this would make the tree have a large explanation size.
Revised: It is conceivable that in order to obtain trees with small depth, one may be required to use several different attributes along the paths to effectively classify the objects; but this would make the tree have a large explanation size.

Original: Conversely, in a tree with small explanation size, the few attributes along a path may need to be used many times in order to correctly classify the objects, leading to a large tree depth.
Revised: Conversely, in a tree with small explanation size, the few attributes along a path may need to be used many times in order to correctly classify the objects, leading to a large tree depth.

Original: Perhaps surprisingly, we show that there are trees that simultaneously have optimal explanation size and almost optimal depth.
Revised: Perhaps surprisingly, we show that there are trees that simultaneously have optimal explanation size and almost optimal depth.

Original: Theorem 1.
Revised: Theorem 1.

Original: Given an instance of the classiﬁcation problem, the following holds:
Revised: Given an instance of the classiﬁcation problem, the following holds:

Original: (Worst-case metrics)
Revised: (Worst-case metrics)

Original: There exists a binary tree T for which simultaneously • expl wc ( T ) = expl ∗ wc • depth wc ( T ) ≤ 2 depth ∗ wc + log n. 2.
Revised: There exists a binary tree T for which simultaneously • expl wc ( T ) = expl ∗ wc • depth wc ( T ) ≤ 2 depth ∗ wc + log n. 2.

Original: (Average metrics)
Revised: (Average metrics)

Original: There exists a binary tree T for which simultaneously • expl avg ( T ) = expl ∗ avg • depth avg ( T ) ≤ 2 depth ∗ avg + W log n.,
Revised: There exists a binary tree T for which simultaneously • expl avg ( T ) = expl ∗ avg • depth avg ( T ) ≤ 2 depth ∗ avg + W log n.,

Original: where W is the sum of the weights of the object in the instance.
Revised: where W is the sum of the weights of the object in the instance.

Original: We remark that these additive bounds on the worst-case and average depth imply O (log n ) multiplicative approximations as well.
Revised: We remark that these additive bounds on the worst-case and average depth imply O (log n ) multiplicative approximations as well.

Original: Observation 1.
Revised: Observation 1.

Original: The bounds from Theorem 1 imply
Revised: The bounds from Theorem 1 imply

Original: [Equation]
Revised: [Equation]

Original: where c is the number of classes.
Revised: where c is the number of classes.

Original: While the proof of Theorem 1 is deferred to Appendix B, we give here the main ideas behind it.
Revised: While the proof of Theorem 1 is deferred to Appendix B, we give here the main ideas behind it.

Original: We will consider here only the worst-case metric (Item 1), since the proof is simpler and more transparent.
Revised: We will consider here only the worst-case metric (Item 1), since the proof is simpler and more transparent.

Original: To show the existence of our desired tree we make use of multiway trees, i.e., a decision tree where multiway tests are used rather than threshold tests.
Revised: To show the existence of our desired tree we make use of multiway trees, i.e., a decision tree where multiway tests are used rather than threshold tests.

Original: A multiway test associated with attribute a splits the objects based on all possible values of this attribute.
Revised: A multiway test associated with attribute a splits the objects based on all possible values of this attribute.

Original: As an example, if an attribute a takesdistinct values for the objects in the instance and we use a at the root of a multiway tree, then the root will have 5 children.
Revised: As an example, if an attribute a takesdistinct values for the objects in the instance and we use a at the root of a multiway tree, then the root will have 5 children.

Original: The starting point for the construction of the tree in Theorem 1 is the equivalence between optimal binary trees and optimal multiway trees in terms of worst-case explanation size.
Revised: The starting point for the construction of the tree in Theorem 1 is the equivalence between optimal binary trees and optimal multiway trees in terms of worst-case explanation size.

Original: While this is formally proved in Lemmas 2 and 3, for an intuitive view of this equivalence ﬁrst notice that there is a multiway tree M ∗ that is simultaneously optimal in terms of worst-case depth and worst-case explanation size, since each attribute only needs to be used once in a path.
Revised: While this is formally proved in Lemmas 2 and 3, for an intuitive view of this equivalence ﬁrst notice that there is a multiway tree M ∗ that is simultaneously optimal in terms of worst-case depth and worst-case explanation size, since each attribute only needs to be used once in a path.

Original: Also, this optimal multiway tree M ∗ has worst-case explanation size (equivalently worst-case depth) at most that of the best binary tree, namely depth wc ( M ∗ ) =
Revised: Also, this optimal multiway tree M ∗ has worst-case explanation size (equivalently worst-case depth) at most that of the best binary tree, namely depth wc ( M ∗ ) =

Original: expl wc ( M ∗ ) ≤ expl ∗ wc , since intuitively multiway tests are more informative than binary tests.
Revised: expl wc ( M ∗ ) ≤ expl ∗ wc , since intuitively multiway tests are more informative than binary tests.

Original: Conversely, we can transform an optimal multiway tree M ∗ into a binary decision tree T by simulating each multiway test on an attribute a by using multiple threshold tests “Is a ( o ) < t ” with varying t (but same attribute a ).
Revised: Conversely, we can transform an optimal multiway tree M ∗ into a binary decision tree T by simulating each multiway test on an attribute a by using multiple threshold tests “Is a ( o ) < t ” with varying t (but same attribute a ).

Original: Since explanation size only counts the number of distinct attributes used along a path, the tree T so created has exactly the same explanation sizes as M ∗ , and hence expl wc ( M ∗ ) =
Revised: Since explanation size only counts the number of distinct attributes used along a path, the tree T so created has exactly the same explanation sizes as M ∗ , and hence expl wc ( M ∗ ) =

Original: expl wc ( T ) ≥ expl ∗ wc .
Revised: expl wc ( T ) ≥ expl ∗ wc .

Original: Thus, we have the equivalence expl wc ( M ∗ ) =
Revised: Thus, we have the equivalence expl wc ( M ∗ ) =

Original: expl ∗ wc .
Revised: expl ∗ wc .

Original: To prove Theorem 1, we start with the optimal multiway tree M ∗ and convert it into a binary tree, as above.
Revised: To prove Theorem 1, we start with the optimal multiway tree M ∗ and convert it into a binary tree, as above.

Original: However, the conversion in the previous paragraph is not enough: while it preserves the explanation size, it may greatly increase the depth of the leaves when using multiple threshold tests to simulate a multiway test (possibly yielding depth (cid:29) depth ∗ wc ).
Revised: However, the conversion in the previous paragraph is not enough: while it preserves the explanation size, it may greatly increase the depth of the leaves when using multiple threshold tests to simulate a multiway test (possibly yielding depth (cid:29) depth ∗ wc ).

Original: The key idea is to use a much more efﬁcient simulation that is based on alphabetic codes, a classic notion from coding theory [28].
Revised: The key idea is to use a much more efﬁcient simulation that is based on alphabetic codes, a classic notion from coding theory [28].

Original: The following result from [2, Chp. 2, p. 341], rephrased in the terminology of decision trees, gives a sufﬁcient condition for the existence of such codes with prescribed code-lengths d i ’s.
Revised: The following result from [2, Chp. 2, p. 341], rephrased in the terminology of decision trees, gives a sufﬁcient condition for the existence of such codes with prescribed code-lengths d i ’s.

Original: Lemma 1.
Revised: Lemma 1.

Original: Consider an instance of the classiﬁcation problem with n objects but only 1 attribute.
Revised: Consider an instance of the classiﬁcation problem with n objects but only 1 attribute.

Original: Then for any positive integers d 1 , . . .
Revised: Then for any positive integers d 1 , . . .

Original: , d n such that (cid:80) nj =1 2 − d i ≤ 12 , there exists a binary threshold decision tree with n leaves at depths d 1 , . . .
Revised: , d n such that (cid:80) nj =1 2 − d i ≤ 12 , there exists a binary threshold decision tree with n leaves at depths d 1 , . . .

Original: d n such that the i -th object reaches the leaf at depth d i .
Revised: d n such that the i -th object reaches the leaf at depth d i .

Original: Then for each node ν of M ∗ (corresponding to an attribute a , and with children ch 1 , ch 2 , . . . ), we consider all objects that reach a child ch i as a “single object” (with the corresponding value in attribute a ) and applying the previous lemma we replace ν by a binary tree A where the leaf corresponding to the objects in ch i end up in a leaf at depth (in A ) (cid:96)
Revised: Then for each node ν of M ∗ (corresponding to an attribute a , and with children ch 1 , ch 2 , . . . ), we consider all objects that reach a child ch i as a “single object” (with the corresponding value in attribute a ) and applying the previous lemma we replace ν by a binary tree A where the leaf corresponding to the objects in ch i end up in a leaf at depth (in A ) (cid:96)

Original: i =
Revised: i =

Original: (cid:100) log n ( ν ) n ( ch i )
Revised: (cid:100) log n ( ν ) n ( ch i )

Original: (cid:101)
Revised: (cid:101)

Original: + 1 , where n ( node ) is the number of objects that reach a given node in M ∗ .
Revised: + 1 , where n ( node ) is the number of objects that reach a given node in M ∗ .

Original: The ﬁnal tree obtained, call it T , still has the same explanation sizes as M ∗ , so expl wc ( T ) = expl ∗ wc .
Revised: The ﬁnal tree obtained, call it T , still has the same explanation sizes as M ∗ , so expl wc ( T ) = expl ∗ wc .

Original: Moreover, in terms of worst-case depth, any root-to-leaf path P in T has a corresponding path P M ∗ =
Revised: Moreover, in terms of worst-case depth, any root-to-leaf path P in T has a corresponding path P M ∗ =

Original: ν 0 , ν 1 , ν 2 , . .
Revised: ν 0 , ν 1 , ν 2 , . .

Original: in M ∗ , and the length of P is at most
Revised: in M ∗ , and the length of P is at most

Original: [Equation]
Revised: [Equation]

Original: By looking at the longest such path we get
Revised: By looking at the longest such path we get

Original: [Equation]
Revised: [Equation]

Original: where the last inequality holds because of the optimality of M ∗ with respect to worst-case depth.
Revised: where the last inequality holds because of the optimality of M ∗ with respect to worst-case depth.

Original: This gives Item 1 of Theorem 1.
Revised: This gives Item 1 of Theorem 1.

Original: The average-case part of the theorem (Item 2) uses similar ideas, but in addition relies on entropybased calculations to argue about the average depth of the constructed tree.
Revised: The average-case part of the theorem (Item 2) uses similar ideas, but in addition relies on entropybased calculations to argue about the average depth of the constructed tree.

Original: Observation 2.
Revised: Observation 2.

Original: Theorem 1 is an existential result and the construction outlined above cannot be done in polytime, since it relies on the availability of an optimal multiway tree.
Revised: Theorem 1 is an existential result and the construction outlined above cannot be done in polytime, since it relies on the availability of an optimal multiway tree.

Original: However, one can obtain in polytime a tree that is simultaneously an O (log n ) -approximation for both worst-case (respectively average) explanation size and depth by replacing the optimal multiway tree by one that approximates within a factor of O (log n ) the worst-case (resp. average) depth, which can be found in polytime (see, e.g., [18] and references therein quoted).
Revised: However, one can obtain in polytime a tree that is simultaneously an O (log n ) -approximation for both worst-case (respectively average) explanation size and depth by replacing the optimal multiway tree by one that approximates within a factor of O (log n ) the worst-case (resp. average) depth, which can be found in polytime (see, e.g., [18] and references therein quoted).

Original: Although guaranteeing asymptotically the desired optimal approximation, the construction leading to such trees might be wasteful in practice as it involves the use of distinct alphabetic codes to turn each multiway tests into a short sequences of threshold tests.
Revised: Although guaranteeing asymptotically the desired optimal approximation, the construction leading to such trees might be wasteful in practice as it involves the use of distinct alphabetic codes to turn each multiway tests into a short sequences of threshold tests.

Original: Therefore, we present an alternative approach in Section 5 that achieves the same approximation guarantee and has also very good performance in practice.
Revised: Therefore, we present an alternative approach in Section 5 that achieves the same approximation guarantee and has also very good performance in practice.

Original: 4.1 Lower bound
Revised: 4.1 Lower bound

Original: Given these positive results, a natural question is whether it is possible to obtain a tree that is optimal for both depth and explanation size.
Revised: Given these positive results, a natural question is whether it is possible to obtain a tree that is optimal for both depth and explanation size.

Original: The next result answers this in the negative, and shows that in a way the worst-case bound in Theorem 1 cannot be improved.
Revised: The next result answers this in the negative, and shows that in a way the worst-case bound in Theorem 1 cannot be improved.

Original: Theorem 2.
Revised: Theorem 2.

Original: Fix c and n ≥ cst · c for a sufﬁciently large constant cst .
Revised: Fix c and n ≥ cst · c for a sufﬁciently large constant cst .

Original: Then for every α ∈ [ 1 2log
Revised: Then for every α ∈ [ 1 2log

Original: [Equation]
Revised: [Equation]

Original: , 12 ( log
Revised: , 12 ( log

Original: [Equation]
Revised: [Equation]

Original: ) log
Revised: ) log

Original: [Equation]
Revised: [Equation]

Original: − 1)] , there is a classiﬁcation instance with n examples and c classes such that every binary decision tree T for this instance has either
Revised: − 1)] , there is a classiﬁcation instance with n examples and c classes such that every binary decision tree T for this instance has either

Original: [Equation]
Revised: [Equation]

Original: or
Revised: or

Original: [Equation]
Revised: [Equation]

Original: Remark 1.
Revised: Remark 1.

Original: To get a more concrete idea for this lower bound, consider setting α at its upper limit, namely α ≈ 12 log n log c .
Revised: To get a more concrete idea for this lower bound, consider setting α at its upper limit, namely α ≈ 12 log n log c .

Original: In this case we obtain that every tree with depth wc ( T ) (cid:46) 14 log n log c depth ∗ wc must have expl wc ( T ) ≥ Ω((1 − log c log n ) log c ) expl ∗ wc , which is Ω(log n ) expl ∗ wc if we set c = log n 2 .
Revised: In this case we obtain that every tree with depth wc ( T ) (cid:46) 14 log n log c depth ∗ wc must have expl wc ( T ) ≥ Ω((1 − log c log n ) log c ) expl ∗ wc , which is Ω(log n ) expl ∗ wc if we set c = log n 2 .

Original: Comparing this against Theorem 1 and Observation 1 we see an interesting and subtle phenomenon: while you can have a tree with depth wc ( T ) ≤ 3log n log c depth ∗ wc and optimal expl wc ( T ) , if you require the approximation in the depth to be a constant factor smaller then you must lose a logarithmic factor in the approximation in the explanation size.
Revised: Comparing this against Theorem 1 and Observation 1 we see an interesting and subtle phenomenon: while you can have a tree with depth wc ( T ) ≤ 3log n log c depth ∗ wc and optimal expl wc ( T ) , if you require the approximation in the depth to be a constant factor smaller then you must lose a logarithmic factor in the approximation in the explanation size.

Original:  An efﬁcient and practical algorithm
Revised:  An efﬁcient and practical algorithm

Original: In this section we design an algorithm, which we name Short Explanaible Rules (SER-DT), that always yields trees of approximately optimal average/worst-case explanation size and depth.
Revised: In this section we design an algorithm, which we name Short Explanaible Rules (SER-DT), that always yields trees of approximately optimal average/worst-case explanation size and depth.

Original: Importantly, our method has enough ﬂexibility that it can be tuned to trade-off accuracy and interpretability, as shown in our computational experiments (Section 6).
Revised: Importantly, our method has enough ﬂexibility that it can be tuned to trade-off accuracy and interpretability, as shown in our computational experiments (Section 6).

Original: Similar to other algorithms in the area, ours chooses in each step a split that creates subtrees with “small impurity”.
Revised: Similar to other algorithms in the area, ours chooses in each step a split that creates subtrees with “small impurity”.

Original: However, unlike most such algorithms, it is not completely greedy and allows for the desired extra ﬂexibility in the choices.
Revised: However, unlike most such algorithms, it is not completely greedy and allows for the desired extra ﬂexibility in the choices.

Original: To describe the algorithm, consider a set of objects S ⊆ O and let S ( a, i ) (respectively S ( a, ≤ i ) and S ( a, > i ) ) be the set of objects in S with value equal (respectively at most and larger than) i on attribute a .
Revised: To describe the algorithm, consider a set of objects S ⊆ O and let S ( a, i ) (respectively S ( a, ≤ i ) and S ( a, > i ) ) be the set of objects in S with value equal (respectively at most and larger than) i on attribute a .

Original: Moreover, let w ( S ) :
Revised: Moreover, let w ( S ) :

Original: = (cid:80) o ∈ S
Revised: = (cid:80) o ∈ S

Original: w ( o ) denote its total weight.
Revised: w ( o ) denote its total weight.

Original: Let o, o (cid:48) be a pair of objects in S such that c ( o ) (cid:54)
Revised: Let o, o (cid:48) be a pair of objects in S such that c ( o ) (cid:54)

Original: =
Revised: =

Original: c ( o (cid:48) ) .
Revised: c ( o (cid:48) ) .

Original: We will refer to such a pair as a misclassiﬁed pair because if both o and o (cid:48) reach the same leaf in the decision tree then one of them will be surely misclassiﬁed.
Revised: We will refer to such a pair as a misclassiﬁed pair because if both o and o (cid:48) reach the same leaf in the decision tree then one of them will be surely misclassiﬁed.

Original: We use P ( S ) to denote the number of misclassiﬁed pairs in S .
Revised: We use P ( S ) to denote the number of misclassiﬁed pairs in S .

Original: This quantity can be thought of as a measure of the amount of work that is needed to reach a correct classiﬁcation of all objects in S and it has been previously used in [19, 22, 18].
Revised: This quantity can be thought of as a measure of the amount of work that is needed to reach a correct classiﬁcation of all objects in S and it has been previously used in [19, 22, 18].

Original: To take into account also the importance/weight of set S we deﬁne wpm ( S ) :
Revised: To take into account also the importance/weight of set S we deﬁne wpm ( S ) :

Original: = P ( S ) · w ( S ) as the weighted pair-wise misclassiﬁcation of S .
Revised: = P ( S ) · w ( S ) as the weighted pair-wise misclassiﬁcation of S .

Original: As a pre-processing step, before executing SER-DT, each weight w ( o ) smaller than w ( O ) /n 3 ) is replaced with w ( O ) / ( w min n 3 ) , where w min is the smallest positive weight among the objects in
Revised: As a pre-processing step, before executing SER-DT, each weight w ( o ) smaller than w ( O ) /n 3 ) is replaced with w ( O ) / ( w min n 3 ) , where w min is the smallest positive weight among the objects in

Original: O .
Revised: O .

Original: This idea (from [35]) is important to guarantee a logarithmic dependence on n instead of w ( O ) .
Revised: This idea (from [35]) is important to guarantee a logarithmic dependence on n instead of w ( O ) .

Original: After this preprocessing, SER-DT is called for the set of objects O .
Revised: After this preprocessing, SER-DT is called for the set of objects O .

Original: The pseudo-code description of SER-DT is presented in Algorithm 1, First SER-DT tries to use any balanced test that reduces the weighted pair-wise misclassiﬁcation of the current set of objects (in the worst case) by at least a 12 factor.
Revised: The pseudo-code description of SER-DT is presented in Algorithm 1, First SER-DT tries to use any balanced test that reduces the weighted pair-wise misclassiﬁcation of the current set of objects (in the worst case) by at least a 12 factor.

Original: In any path of the tree built by SER-DT the amount of these balanced tests is at most logarithmic, so they can be easily handled in our analysis.
Revised: In any path of the tree built by SER-DT the amount of these balanced tests is at most logarithmic, so they can be easily handled in our analysis.

Original: If no balanced test exists, then the algorithm ﬁnds an attribute a ∗ and value t ∗ such that in the ternary split S ( a ∗ , < t ) ,
Revised: If no balanced test exists, then the algorithm ﬁnds an attribute a ∗ and value t ∗ such that in the ternary split S ( a ∗ , < t ) ,

Original: S ( a ∗ , t ∗ ) , S ( a ∗ , > t ∗ ) , only the middle set S ( a ∗ , t ∗ ) has weighted pair-wise misclassiﬁcation larger than the desired 12 wpm ( S ) .
Revised: S ( a ∗ , t ∗ ) , S ( a ∗ , > t ∗ ) , only the middle set S ( a ∗ , t ∗ ) has weighted pair-wise misclassiﬁcation larger than the desired 12 wpm ( S ) .

Original: This 3-way partition is obtained by using two binary splits.
Revised: This 3-way partition is obtained by using two binary splits.

Original: Then, the algorithm recurses on each set.
Revised: Then, the algorithm recurses on each set.

Original: A critical issue is to show that in this case some progress is also achieved with the problematic subproblem on S ( a ∗ , t ∗ ) where wpm ( S ( a ∗ , t ∗ )) >
Revised: A critical issue is to show that in this case some progress is also achieved with the problematic subproblem on S ( a ∗ , t ∗ ) where wpm ( S ( a ∗ , t ∗ )) >

Original: 12 wpm ( S ) .
Revised: 12 wpm ( S ) .

Original: In fact, the choice of the attribute a ∗ is such that, the instance S ( a ∗ , t ∗ ) has the minimum weighted pair-wise misclassiﬁcation among all the attributes and other possible tripartitions.
Revised: In fact, the choice of the attribute a ∗ is such that, the instance S ( a ∗ , t ∗ ) has the minimum weighted pair-wise misclassiﬁcation among all the attributes and other possible tripartitions.

Original: As a result, we can employ a lower bound on the optimum ( Lemma 8 in appendix) that allows us to absorb the cost of the subtree for the subproblem S ( a ∗ , t ) in the logarithmic guarantee.
Revised: As a result, we can employ a lower bound on the optimum ( Lemma 8 in appendix) that allows us to absorb the cost of the subtree for the subproblem S ( a ∗ , t ) in the logarithmic guarantee.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: decision tree 4:
Revised: decision tree 4:

Original: Recurse on the children S ( a, ≤ t ) and S ( a, > t ) 5: else 6:
Revised: Recurse on the children S ( a, ≤ t ) and S ( a, > t ) 5: else 6:

Original: Let a ∗ be an attribute in argmin a { max i wpm ( S ( a, i )) } 7:
Revised: Let a ∗ be an attribute in argmin a { max i wpm ( S ( a, i )) } 7:

Original: Let t ∗ be the smallest value of the attribute a ∗ such that the “left child” S ( a ∗ , ≤ t ∗ ) satisﬁes
Revised: Let t ∗ be the smallest value of the attribute a ∗ such that the “left child” S ( a ∗ , ≤ t ∗ ) satisﬁes

Original: [Equation]
Revised: [Equation]

Original: Use two binary tests to simulate the 3-way split S ( a ∗ , < t ∗ ) , S ( a ∗ , t ∗ ) , S ( a ∗ , > t ∗ ) .
Revised: Use two binary tests to simulate the 3-way split S ( a ∗ , < t ∗ ) , S ( a ∗ , t ∗ ) , S ( a ∗ , > t ∗ ) .

Original: More precisely, at the root use a test on attribute a ∗ that splits S into the sets S ( a ∗ , < t ∗ ) and S ( a ∗ , ≥ t ∗ ) .
Revised: More precisely, at the root use a test on attribute a ∗ that splits S into the sets S ( a ∗ , < t ∗ ) and S ( a ∗ , ≥ t ∗ ) .

Original: Then, apply a test on the right child of the root, currently associated with S ( a ∗ , ≥ t ∗ ) , creating two new children with objects S ( a ∗ , t ∗ ) and S ( a ∗ , > t ∗ ) 9:
Revised: Then, apply a test on the right child of the root, currently associated with S ( a ∗ , ≥ t ∗ ) , creating two new children with objects S ( a ∗ , t ∗ ) and S ( a ∗ , > t ∗ ) 9:

Original: Recurse on each of the three leaf nodes in the current tree 10: end if
Revised: Recurse on each of the three leaf nodes in the current tree 10: end if

Original: The following is the promised guarantee for the average/worst-case depth and explanation size of the trees produced by the algorithm.
Revised: The following is the promised guarantee for the average/worst-case depth and explanation size of the trees produced by the algorithm.

Original: Theorem 3.
Revised: Theorem 3.

Original: Given an instance I , the algorithm SER-DT produces a tree T that satisﬁes
Revised: Given an instance I , the algorithm SER-DT produces a tree T that satisﬁes

Original: (Worst-case metrics) • depth avg ( T ) ≤
Revised: (Worst-case metrics) • depth avg ( T ) ≤

Original: O (log n ) depth ∗ avg , • expl avg ( T ) ≤ O (log n ) expl ∗ avg .
Revised: O (log n ) depth ∗ avg , • expl avg ( T ) ≤ O (log n ) expl ∗ avg .

Original: (Average metrics) • depth wc ( T ) ≤ O (log n ) depth ∗ wc , • expl wc ( T ) ≤ O (log n ) expl ∗ wc .
Revised: (Average metrics) • depth wc ( T ) ≤ O (log n ) depth ∗ wc , • expl wc ( T ) ≤ O (log n ) expl ∗ wc .

Original: We remark that, in the light of the inapproximability results from [36, 16] 2 this theorem says that
Revised: We remark that, in the light of the inapproximability results from [36, 16] 2 this theorem says that

Original: SER-DT (with the preprocessing step) guarantees the best possible approximation obtainable by a polynomial algorithm with respect to both the measures under consideration: worst/average depth and worst/average explanation size.
Revised: SER-DT (with the preprocessing step) guarantees the best possible approximation obtainable by a polynomial algorithm with respect to both the measures under consideration: worst/average depth and worst/average explanation size.

Original:  Experiments
Revised:  Experiments

Original: In this section, we report the experiments that were carried out to evaluate how our proposed algorithm
Revised: In this section, we report the experiments that were carried out to evaluate how our proposed algorithm

Original: SER-DT performs in practice.
Revised: SER-DT performs in practice.

Original: We considered the 20 datasets that appear on Column 1 of Table 1 (see Appendix E for their main characteristics).
Revised: We considered the 20 datasets that appear on Column 1 of Table 1 (see Appendix E for their main characteristics).

Original: For all of them, 70% of the examples were used for training and the remaining 30% for testing.
Revised: For all of them, 70% of the examples were used for training and the remaining 30% for testing.

Original: Moreover, all the examples (objects) were considered equally important (weight=1/size of training set).
Revised: Moreover, all the examples (objects) were considered equally important (weight=1/size of training set).

Original: During a preprocessing step we converted all categorical attributes into binary attributes via one-hot-encoding.
Revised: During a preprocessing step we converted all categorical attributes into binary attributes via one-hot-encoding.

Original: See also Appendix E for more details on the experimental setup.
Revised: See also Appendix E for more details on the experimental setup.

Original: Recall that in (Line 2) of algorithm SER-DT any test that splits the current set of objects into subsets of small enough wpm could be used.
Revised: Recall that in (Line 2) of algorithm SER-DT any test that splits the current set of objects into subsets of small enough wpm could be used.

Original: We use this ﬂexibility to select a test among these that should further help in obtaining small explanation sizes and high accuracy in practice.
Revised: We use this ﬂexibility to select a test among these that should further help in obtaining small explanation sizes and high accuracy in practice.

Original: To explain our selection, recall the Gini impurity measure employed by CART .
Revised: To explain our selection, recall the Gini impurity measure employed by CART .

Original: For a set of examples (objects) S , each of them labeled with a class in { 1 , . . .
Revised: For a set of examples (objects) S , each of them labeled with a class in { 1 , . . .

Original: , c } , the Gini impurity is given by
Revised: , c } , the Gini impurity is given by

Original: [Equation]
Revised: [Equation]

Original: where S i is the set of examples of class i .
Revised: where S i is the set of examples of class i .

Original: Moreover, the weighted Gini impurity Gini ( τ, ν ) induced by a test τ that divides the set of examples S that reach a node ν into S L and S R is given by
Revised: Moreover, the weighted Gini impurity Gini ( τ, ν ) induced by a test τ that divides the set of examples S that reach a node ν into S L and S R is given by

Original: [Equation]
Revised: [Equation]

Original: Let FactorExpl be a hyper-parameter in the range [0 , 1] .
Revised: Let FactorExpl be a hyper-parameter in the range [0 , 1] .

Original: Because we are interested in trees that induce accurate classiﬁers with short explanation rules, to expand a given leaf ν , in Line 2 of the algorithm we select, among the permissible tests 3 the test τ that minimizes
Revised: Because we are interested in trees that induce accurate classiﬁers with short explanation rules, to expand a given leaf ν , in Line 2 of the algorithm we select, among the permissible tests 3 the test τ that minimizes

Original: [Equation]
Revised: [Equation]

Original: where I ( τ, ν ) =
Revised: where I ( τ, ν ) =

Original: FactorExpl
Revised: FactorExpl

Original: if the attribute associated with test τ has already appeared in the path from the root to ν , and I ( τ, ν )
Revised: if the attribute associated with test τ has already appeared in the path from the root to ν , and I ( τ, ν )

Original: = 1 otherwise.
Revised: = 1 otherwise.

Original: Since FactorExpl is used to favour attributes that have already appeared in the path, when FactorExpl is set to a low value we expect to obtain trees with short explanations but also with lower accuracy, as the effect of the Gini impurity is reduced.
Revised: Since FactorExpl is used to favour attributes that have already appeared in the path, when FactorExpl is set to a low value we expect to obtain trees with short explanations but also with lower accuracy, as the effect of the Gini impurity is reduced.

Original: In terms of stopping rules, we do not expand a leaf ν if it is either located at depth 6 or if there is no test τ for which Gini ( τ, ν ) is smaller than Gini ( Examples thaat reach ν ) .
Revised: In terms of stopping rules, we do not expand a leaf ν if it is either located at depth 6 or if there is no test τ for which Gini ( τ, ν ) is smaller than Gini ( Examples thaat reach ν ) .

Original: As a post-processing step, whenever two sibling leaves are assigned the same class, we delete them both and leave their parent as a leaf.
Revised: As a post-processing step, whenever two sibling leaves are assigned the same class, we delete them both and leave their parent as a leaf.

Original: In our ﬁrst experiment we study how the accuracy and the interpretability measures of the trees produced by our algorithm behave when FactorExpl is varied.
Revised: In our ﬁrst experiment we study how the accuracy and the interpretability measures of the trees produced by our algorithm behave when FactorExpl is varied.

Original: Figure 2 shows the average accuracy (left image) and the average explanation size expl avg (right image) as a function of FactorExpl .
Revised: Figure 2 shows the average accuracy (left image) and the average explanation size expl avg (right image) as a function of FactorExpl .

Original: More precisely, for the left image, the y -value associated with a point x corresponds to the average accuracy on the testing set, calculated over the 20 datasets, when our algorithm is executed with
Revised: More precisely, for the left image, the y -value associated with a point x corresponds to the average accuracy on the testing set, calculated over the 20 datasets, when our algorithm is executed with

Original: FactorExpl
Revised: FactorExpl

Original: = x .
Revised: = x .

Original: For the right image the same logic holds.
Revised: For the right image the same logic holds.

Original: As expected, the larger the FactorExpl the larger the accuracy and the expl avg .
Revised: As expected, the larger the FactorExpl the larger the accuracy and the expl avg .

Original: The interesting ﬁnding , however, is that the accuracy increases relatively slow, for FactorExpl is close to 1, compared with the growth in expl avg (see the
Revised: The interesting ﬁnding , however, is that the accuracy increases relatively slow, for FactorExpl is close to 1, compared with the growth in expl avg (see the

Original: Table 5 in the appendix for experiments with FactorExpl in the range [0.95,0.99]).
Revised: Table 5 in the appendix for experiments with FactorExpl in the range [0.95,0.99]).

Original: This suggests that it is possible to obtain trees that are signiﬁcantly more interpretable without sacriﬁcing the accuracy.
Revised: This suggests that it is possible to obtain trees that are signiﬁcantly more interpretable without sacriﬁcing the accuracy.

Original: [Table]
Revised: [Table]

Original: In our second experiment we compare the results of our method, using FactorExpl = 0 .
Revised: In our second experiment we compare the results of our method, using FactorExpl = 0 .

Original: 97 with
Revised: 97 with

Original: CART [11].
Revised: CART [11].

Original: The value 0.97 is motivated by the above observation.
Revised: The value 0.97 is motivated by the above observation.

Original: To expand a leaf ν , recall that
Revised: To expand a leaf ν , recall that

Original: CART selects the test τ for which Gini ( τ, ν ) is minimum and it only expands ν if there is a test τ for which Gini ( τ, ν ) <
Revised: CART selects the test τ for which Gini ( τ, ν ) is minimum and it only expands ν if there is a test τ for which Gini ( τ, ν ) <

Original: Gini ( Examples that reach ν ) .
Revised: Gini ( Examples that reach ν ) .

Original: To provide a fair comparison with our algorithm we set the maximum depth to 6 and applied the same aforementioned post-processing.
Revised: To provide a fair comparison with our algorithm we set the maximum depth to 6 and applied the same aforementioned post-processing.

Original: Table 1 shows the accuracy on the testing set as well as the average explanation size expl avg and the worst-case explanation size expl wc for all datasets.
Revised: Table 1 shows the accuracy on the testing set as well as the average explanation size expl avg and the worst-case explanation size expl wc for all datasets.

Original: Each entry in this table is the average of ten runs, where in each of them a different seed is used to split a dataset into training and testing set.
Revised: Each entry in this table is the average of ten runs, where in each of them a different seed is used to split a dataset into training and testing set.

Original: We notice that the accuracy of our method is very close to that obtained by CART , while the gain in terms of the interpretability metrics is signiﬁcant.
Revised: We notice that the accuracy of our method is very close to that obtained by CART , while the gain in terms of the interpretability metrics is signiﬁcant.

Original: On 7 datasets (bold-faced on columns 2 and 3) we observe a difference larger than 1% on the accuracies; on 3 of them our algorithm outperforms CART while on the remaining 4, CART is better.
Revised: On 7 datasets (bold-faced on columns 2 and 3) we observe a difference larger than 1% on the accuracies; on 3 of them our algorithm outperforms CART while on the remaining 4, CART is better.

Original: In terms of the average explanation size, our algorithm is at least as good as CART for all datasets, and for 9 of them (bold-faced on columns 4 and 5) it improves the expl avg by at least 25% .
Revised: In terms of the average explanation size, our algorithm is at least as good as CART for all datasets, and for 9 of them (bold-faced on columns 4 and 5) it improves the expl avg by at least 25% .

Original: For the expl wc the gain is also clear.
Revised: For the expl wc the gain is also clear.

Original: For all datasets, but Banknote , our algorithm is at least as good as CART .
Revised: For all datasets, but Banknote , our algorithm is at least as good as CART .

Original: Moreover, for 3 of them (bold-faced on columns 6 and 7), it provides a gain of at least 25% .
Revised: Moreover, for 3 of them (bold-faced on columns 6 and 7), it provides a gain of at least 25% .

Original: Boxplots for the experiments in Table 1 are provided in Section
Revised: Boxplots for the experiments in Table 1 are provided in Section

Original: E.5 of the appendix.
Revised: E.5 of the appendix.

Original: In Appendix E we compare CART and SER-DT in terms of depth avg and depth wc .
Revised: In Appendix E we compare CART and SER-DT in terms of depth avg and depth wc .

Original: For depth avg ,
Revised: For depth avg ,

Original: SER-DT performs better than CART while for depth wc the results are similar.
Revised: SER-DT performs better than CART while for depth wc the results are similar.

Original: The result for depth wc is not surprising since we set 6 as the maximum depth in our experiments.
Revised: The result for depth wc is not surprising since we set 6 as the maximum depth in our experiments.

Original: Regarding running time, the algorithms present similar behaviour, as it can be veriﬁed in
Revised: Regarding running time, the algorithms present similar behaviour, as it can be veriﬁed in

Original: Appendix E. This is somehow expected since both consist of mostly a greedy split selection at each node.
Revised: Appendix E. This is somehow expected since both consist of mostly a greedy split selection at each node.

Original: In the appendix we also present additional experiments and analyses.
Revised: In the appendix we also present additional experiments and analyses.

Original: In particular, in Section E.8, we evaluate the impact of applying post-pruning to both CART and SER-DT.
Revised: In particular, in Section E.8, we evaluate the impact of applying post-pruning to both CART and SER-DT.

Original: Moreover, in Section
Revised: Moreover, in Section

Original: E.7, we show comparisons of SER-DT with one of the state of the art decision tree methods that optimize the average depth, namely the EC2 algorithm from [22]: the experiments suggest that
Revised: E.7, we show comparisons of SER-DT with one of the state of the art decision tree methods that optimize the average depth, namely the EC2 algorithm from [22]: the experiments suggest that

Original: SER-DT performs signiﬁcantly better than EC2 on all metrics.
Revised: SER-DT performs signiﬁcantly better than EC2 on all metrics.

Original:  Conclusion
Revised:  Conclusion

Original: In this work, we proposed the explanation size as a new metric to capture intrepretability of decision trees and initiated a principled study of it.
Revised: In this work, we proposed the explanation size as a new metric to capture intrepretability of decision trees and initiated a principled study of it.

Original: We presented upper and lower bound on the trade-off of simultaneously optimizing this new metric and metrics related to depths of the leaves.
Revised: We presented upper and lower bound on the trade-off of simultaneously optimizing this new metric and metrics related to depths of the leaves.

Original: We also proposed a practical algorithm that provably approximates the average explanation size and the average depth and showed, via experiments over 20 datasets, that it is competitive with the widely used CART algorithms in terms of accuracy while being much better in terms of producing trees with short explanation size.
Revised: We also proposed a practical algorithm that provably approximates the average explanation size and the average depth and showed, via experiments over 20 datasets, that it is competitive with the widely used CART algorithms in terms of accuracy while being much better in terms of producing trees with short explanation size.

Original: On the basis of both the theoretical analysis (approximation guarantee) and the performance demonstrated in the empirical studies, we believe that our algorithm (or some variation based on its ideas) can be used to generate accurate and highly interpretable trees for practical applications.
Revised: On the basis of both the theoretical analysis (approximation guarantee) and the performance demonstrated in the empirical studies, we believe that our algorithm (or some variation based on its ideas) can be used to generate accurate and highly interpretable trees for practical applications.

Original: Acknowledgments and Disclosure of Funding
Revised: Acknowledgments and Disclosure of Funding

Original: This study was ﬁnanced in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.
Revised: This study was ﬁnanced in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Original: The work of the second author is partially supported by
Revised: The work of the third author is partially supported by

Original: CNPq (grant 307572/2017-0), by FAPERJ, grant E- 26/202.823/2018 and by the Air Force Ofﬁce of
Revised: CNPq (grant 307572/2017-0), by FAPERJ, grant E- 26/202.823/2018 and by the Air Force Ofﬁce of

Original: Scientiﬁc Research under award number FA9550-22-1-0475.
Revised: Scientiﬁc Research under award number FA9550-22-1-0475.

Original: The third author is supported by Bolsa de Produtividade em
Revised: The fourth author is supported by Bolsa de Produtividade em

Original: Pesquisa #3 12751/2021-4 from CNPq, and FAPERJ grant Jovem Cientista do Nosso Estado.
Revised: Pesquisa #3 12751/2021-4 from CNPq, and FAPERJ grant Jovem Cientista do Nosso Estado.

Original: Appendix A Multiway trees and optimal explanation sizes
Revised: Appendix A Multiway trees and optimal explanation sizes

Original: In this section we prove a characterization of optimal explanation sizes via multiway trees.
Revised: In this section we prove a characterization of optimal explanation sizes via multiway trees.

Original: First recall that the only difference between these and threshold decision trees is that in multiway trees each test corresponds to an attribute a , and the answer to the test is the value a ( o ) that the desired object o has on attribute a ; each internal node ν is associated with such a test τ ν , and the edges from a node to its children are associated to the possible outcomes of τ ν .
Revised: First recall that the only difference between these and threshold decision trees is that in multiway trees each test corresponds to an attribute a , and the answer to the test is the value a ( o ) that the desired object o has on attribute a ; each internal node ν is associated with such a test τ ν , and the edges from a node to its children are associated to the possible outcomes of τ ν .

Original: The worst-case/average depth and explanation size of a multiway tree are deﬁned exactly as for threshold decision trees, see Section 3.
Revised: The worst-case/average depth and explanation size of a multiway tree are deﬁned exactly as for threshold decision trees, see Section 3.

Original: We start by proving this equivalence for the worst-case metrics.
Revised: We start by proving this equivalence for the worst-case metrics.

Original: Lemma 2 (Multiway trees and expl.
Revised: Lemma 2 (Multiway trees and expl.

Original: size, worst-case) .
Revised: size, worst-case) .

Original: Consider an instance of the classiﬁcation problem, and let M ∗ be a multiway tree of minimal worst-case depth for this instance.
Revised: Consider an instance of the classiﬁcation problem, and let M ∗ be a multiway tree of minimal worst-case depth for this instance.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Some quick notation:
Revised: Some quick notation:

Original: Given a multiway tree M and one of its nodes ν , let a ν be the attribute that is tested on this node.
Revised: Given a multiway tree M and one of its nodes ν , let a ν be the attribute that is tested on this node.

Original: Also let M ν,i be the subtree of M rooted at the child of ν reached by objects whose value for attribute
Revised: Also let M ν,i be the subtree of M rooted at the child of ν reached by objects whose value for attribute

Original: a ν is i .
Revised: a ν is i .

Original: We use int ( M ) to denote the set of internal nodes of a tree
Revised: We use int ( M ) to denote the set of internal nodes of a tree

Original: M .
Revised: M .

Original: We ﬁrst show that expl ∗ wc ≤ expl wc ( M ∗ ) ≤ depth wc ( M ∗ ) .
Revised: We ﬁrst show that expl ∗ wc ≤ expl wc ( M ∗ ) ≤ depth wc ( M ∗ ) .

Original: For that, we convert M ∗ into a threshold decision tree that has explanation size at most depth wc ( M ∗ ) by just simulating the multiway tests using multiple threshold tests in the natural way (see Fig.3 for a pictorial example of the following construction).
Revised: For that, we convert M ∗ into a threshold decision tree that has explanation size at most depth wc ( M ∗ ) by just simulating the multiway tests using multiple threshold tests in the natural way (see Fig.3 for a pictorial example of the following construction).

Original: More formally, let T a ν be binary search tree over the values of attribute a ν , that is, it is a threshold decision tree where there is a one-to-one correspondence between its leaves and the values of the attribute a ν (i.e. every object o reaches the unique leaf of T a ν corresponding to the value a ν ( o ) ).
Revised: More formally, let T a ν be binary search tree over the values of attribute a ν , that is, it is a threshold decision tree where there is a one-to-one correspondence between its leaves and the values of the attribute a ν (i.e. every object o reaches the unique leaf of T a ν corresponding to the value a ν ( o ) ).

Original: Then let T be the threshold decision tree obtained from M ∗ by (starting from the root and proceeding downwards) replacing each internal node ν of M ∗ with T a ν and identifying the leaf of T a ν corresponding to value a ν ( · ) =
Revised: Then let T be the threshold decision tree obtained from M ∗ by (starting from the root and proceeding downwards) replacing each internal node ν of M ∗ with T a ν and identifying the leaf of T a ν corresponding to value a ν ( · ) =

Original: i with the root of M ∗ ν,i .
Revised: i with the root of M ∗ ν,i .

Original: Consider a leaf (cid:96) M ∗ in M ∗ .
Revised: Consider a leaf (cid:96) M ∗ in M ∗ .

Original: Notice that the path P M ∗ from the root of M ∗ to (cid:96) M ∗ induces a path P T in the tree T that goes from its root to one of its leaves (cid:96) T (i.e., whenever the path P M ∗ goes from a node ν to a subtree M ∗ ν,i , the path P T goes from the root of the binary search tree T a ν to its leaf corresponding to value a ν ( · ) = i ).
Revised: Notice that the path P M ∗ from the root of M ∗ to (cid:96) M ∗ induces a path P T in the tree T that goes from its root to one of its leaves (cid:96) T (i.e., whenever the path P M ∗ goes from a node ν to a subtree M ∗ ν,i , the path P T goes from the root of the binary search tree T a ν to its leaf corresponding to value a ν ( · ) = i ).

Original: We can see that the set of attributes tested in the paths P M ∗ and
Revised: We can see that the set of attributes tested in the paths P M ∗ and

Original: P T is exactly the same, and thus the leaves (cid:96) M ∗ and (cid:96) T have the same explanation size.
Revised: P T is exactly the same, and thus the leaves (cid:96) M ∗ and (cid:96) T have the same explanation size.

Original: Looking at the largest explanation size of the leaves in these trees, we get that expl wc ( T ) =
Revised: Looking at the largest explanation size of the leaves in these trees, we get that expl wc ( T ) =

Original: expl wc ( M ∗ ) .
Revised: expl wc ( M ∗ ) .

Original: This gives
Revised: This gives

Original: [Equation]
Revised: [Equation]

Original: as desired.
Revised: as desired.

Original: Now we show that depth wc ( M ∗ ) ≤ expl ∗ wc , which together with (1) above gives the desired equalities in the lemma.
Revised: Now we show that depth wc ( M ∗ ) ≤ expl ∗ wc , which together with (1) above gives the desired equalities in the lemma.

Original: Let T be a threshold decision tree achieving the optimal explanation size expl ∗ wc .
Revised: Let T be a threshold decision tree achieving the optimal explanation size expl ∗ wc .

Original: We will show how to build a multiway decision tree M such that depth wc ( M ) =
Revised: We will show how to build a multiway decision tree M such that depth wc ( M ) =

Original: expl wc ( T ) =
Revised: expl wc ( T ) =

Original: expl ∗ wc (refer to Fig.
Revised: expl ∗ wc (refer to Fig.

Original: 4 for an example of the construction).
Revised: 4 for an example of the construction).

Original: We ﬁrst recursively turn each test of T into a multiway test on the same attribute in the natural way.
Revised: We ﬁrst recursively turn each test of T into a multiway test on the same attribute in the natural way.

Original: More precisely, this can be done in a recursive way: If T is a leaf, then deﬁne the multiway tree ˜
Revised: More precisely, this can be done in a recursive way: If T is a leaf, then deﬁne the multiway tree ˜

Original: M = T. Otherwise, let ( a, t ) be the test at root of T , and let T < and T ≥ be the subtrees of T rooted at the children of the root and associated with the two results of the root-test, i.e., T < (resp.
Revised: M = T. Otherwise, let ( a, t ) be the test at root of T , and let T < and T ≥ be the subtrees of T rooted at the children of the root and associated with the two results of the root-test, i.e., T < (resp.

Original: T ≥ ) is the subtree reached by the objects o such that a ( o ) < t (resp.
Revised: T ≥ ) is the subtree reached by the objects o such that a ( o ) < t (resp.

Original: a ( o ) ≥ t ).
Revised: a ( o ) ≥ t ).

Original: Then ˜ M is obtained by putting the multiway test on attribute a on its root r , and for each value i of the attribute a , if i < t (resp.
Revised: Then ˜ M is obtained by putting the multiway test on attribute a on its root r , and for each value i of the attribute a , if i < t (resp.

Original: i ≥ t ) the i -th child of the root r is the subtree obtained by recursively applying this construction to
Revised: i ≥ t ) the i -th child of the root r is the subtree obtained by recursively applying this construction to

Original: T < (resp.
Revised: T < (resp.

Original: T ≥ ).
Revised: T ≥ ).

Original: Since this construction simply converts each threshold test ( a, t ) in T into a multiway test on a , it is easy to see that
Revised: Since this construction simply converts each threshold test ( a, t ) in T into a multiway test on a , it is easy to see that

Original: [Equation]
Revised: [Equation]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: v': a(o)
Revised: v': a(o)

Original: [Figure]
Revised: [Figure]

Original: (cid:3)
Revised: (cid:3)

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: c(o)
Revised: c(o)

Original: [Figure]
Revised: [Figure]

Original: b(o)
Revised: b(o)

Original: [Figure]
Revised: [Figure]

Original: b
Revised: b

Original: [Figure]
Revised: [Figure]

Original: o)
Revised: o)

Original: [Figure]
Revised: [Figure]

Original: (cid:4) a(o)
Revised: (cid:4) a(o)

Original: [Figure]
Revised: [Figure]

Original: a(o)= (cid:5) (cid:5) a(o)
Revised: a(o)= (cid:5) (cid:5) a(o)

Original: [Figure]
Revised: [Figure]

Original: 1 a(o)
Revised: 1 a(o)

Original: [Figure]
Revised: [Figure]

Original: a(o)
Revised: a(o)

Original: [Figure]
Revised: [Figure]

Original: w: b(o)
Revised: w: b(o)

Original: [Figure]
Revised: [Figure]

Original: (cid:1) v''
Revised: (cid:1) v''

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: b(
Revised: b(

Original: [Figure]
Revised: [Figure]

Original: a(o)
Revised: a(o)

Original: [Figure]
Revised: [Figure]

Original: a(o)=2 a(o)=
Revised: a(o)=2 a(o)=

Original: Tx
Revised: Tx

Original: Ty
Revised: Ty

Original: Tx Ty Ty (cid:4) (cid:5) (cid:5)
Revised: Tx Ty Ty (cid:4) (cid:5) (cid:5)

Original: Tx Ty Ty we remove the redundancies (grey parts), by collapsing v'-x and v''-y w: b(o)?
Revised: Tx Ty Ty we remove the redundancies (grey parts), by collapsing v'-x and v''-y w: b(o)?

Original: v: a(o)?
Revised: v: a(o)?

Original: (cid:1) z: c(o)?
Revised: (cid:1) z: c(o)?

Original: (cid:2) (cid:3) c(o)= c(o)=2 b(o)= b(o)=2 (cid:4) a(o)= a(o)= a(o)= w: b(o)?
Revised: (cid:2) (cid:3) c(o)= c(o)=2 b(o)= b(o)=2 (cid:4) a(o)= a(o)= a(o)= w: b(o)?

Original: (cid:1)
Revised: (cid:1)

Original: b(o)= b(o)=
Revised: b(o)= b(o)=

Original: Tx (cid:5)
Revised: Tx (cid:5)

Original: Ty
Revised: Ty

Original: However, the multiway tree ˜
Revised: However, the multiway tree ˜

Original: M may test the same attribute multiple times on a root-to-leaf path, because T may do so, and so we can have depth wc ( ˜ M ) > expl wc ( ˜ M ) .
Revised: M may test the same attribute multiple times on a root-to-leaf path, because T may do so, and so we can have depth wc ( ˜ M ) > expl wc ( ˜ M ) .

Original: But in ˜ M these multiple tests are redundant, because a single multiway test reveals complete information about the attribute.
Revised: But in ˜ M these multiple tests are redundant, because a single multiway test reveals complete information about the attribute.

Original: So we now remove these redundancies to obtain a multiway tree M with depth wc ( M ) = expl wc ( ˜ M ) , as follows: If there is a node ν ∈ int ( ˜ M ) and a descendant ν (cid:48) in one of the child-subtree T ν,i that tests the same attribute as ν (i.e., a ν (cid:48)
Revised: So we now remove these redundancies to obtain a multiway tree M with depth wc ( M ) = expl wc ( ˜ M ) , as follows: If there is a node ν ∈ int ( ˜ M ) and a descendant ν (cid:48) in one of the child-subtree T ν,i that tests the same attribute as ν (i.e., a ν (cid:48)

Original: = a ν ), we replace the subtree T ν (cid:48) with the subtree T ν (cid:48) ,i .
Revised: = a ν ), we replace the subtree T ν (cid:48) with the subtree T ν (cid:48) ,i .

Original: Notice that since ν (cid:48) ∈ T ν,i , any object reaching node ν (cid:48) has a ( · ) = i , and thus would move next to the subtree
Revised: Notice that since ν (cid:48) ∈ T ν,i , any object reaching node ν (cid:48) has a ( · ) = i , and thus would move next to the subtree

Original: T ν (cid:48) ,i ; our replacement operation just bypassed the node ν (cid:48) , and so we can see that the multiway tree obtained remains equivalent to ˜ M for the classiﬁcation problem.
Revised: T ν (cid:48) ,i ; our replacement operation just bypassed the node ν (cid:48) , and so we can see that the multiway tree obtained remains equivalent to ˜ M for the classiﬁcation problem.

Original: Repeat this replacement operation until no root-to-leaf path contains two nodes that test on the same attribute.
Revised: Repeat this replacement operation until no root-to-leaf path contains two nodes that test on the same attribute.

Original: Let M be the multiway tree obtained at the end.
Revised: Let M be the multiway tree obtained at the end.

Original: Since no attribute is repeatedly tested in such paths, for every leaf (cid:96) ∈ leaf ( M ) we have depth ( (cid:96) )
Revised: Since no attribute is repeatedly tested in such paths, for every leaf (cid:96) ∈ leaf ( M ) we have depth ( (cid:96) )

Original: =
Revised: =

Original: expl ( (cid:96) ) , and hence depth wc
Revised: expl ( (cid:96) ) , and hence depth wc

Original: [Equation]
Revised: [Equation]

Original: M
Revised: M

Original: [Equation]
Revised: [Equation]

Original: = expl
Revised: = expl

Original: [Equation]
Revised: [Equation]

Original: M
Revised: M

Original: [Equation]
Revised: [Equation]

Original: ( ˜ M ) , the inequality following because M was obtained by deleting nodes of ˜ M .
Revised: ( ˜ M ) , the inequality following because M was obtained by deleting nodes of ˜ M .

Original: Together with inequality (2) and the fact that M ∗ is depth-optimal, this gives
Revised: Together with inequality (2) and the fact that M ∗ is depth-optimal, this gives

Original: [Equation]
Revised: [Equation]

Original: as desired.
Revised: as desired.

Original: This concludes the proof of the lemma.
Revised: This concludes the proof of the lemma.

Original: The same argument also shows the following analogous result about the average depth and explanation size.
Revised: The same argument also shows the following analogous result about the average depth and explanation size.

Original: Lemma 3 (Multiway trees and expl.
Revised: Lemma 3 (Multiway trees and expl.

Original: size, average) .
Revised: size, average) .

Original: Consider an instance of the classiﬁcation problem, and let M ∗ be a multiway tree of minimal average depth for this instance.
Revised: Consider an instance of the classiﬁcation problem, and let M ∗ be a multiway tree of minimal average depth for this instance.

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: B Proof of Theorem
Revised: B Proof of Theorem

Original: We prove the guarantees for the worst-case and average metrics (Items 1 and 2 of the theorem) separately.
Revised: We prove the guarantees for the worst-case and average metrics (Items 1 and 2 of the theorem) separately.

Original: As before, given a tree T and one of its nodes ν , we use T ν to denote the subtree rooted at ν .
Revised: As before, given a tree T and one of its nodes ν , we use T ν to denote the subtree rooted at ν .

Original: We also use n ( ν ) to denote the number of objects of the instance that reach node ν (and hence some leaf in the subtree rooted at ν ); sometimes we also use a subscript n T ( ν ) to make it clearer which tree we are talking about in order to avoid any confusion.
Revised: We also use n ( ν ) to denote the number of objects of the instance that reach node ν (and hence some leaf in the subtree rooted at ν ); sometimes we also use a subscript n T ( ν ) to make it clearer which tree we are talking about in order to avoid any confusion.

Original: B.1 Proof of Item 1: Worst-case metrics
Revised: B.1 Proof of Item 1: Worst-case metrics

Original: Recall that in this item we want a threshold tree T satisfying:
Revised: Recall that in this item we want a threshold tree T satisfying:

Original: [Equation]
Revised: [Equation]

Original: Using Lemma 2, let M ∗ be an optimal multiway tree with respect to both worst-case explanation size and depth, and hence satisfying depth wc ( M ∗ )
Revised: Using Lemma 2, let M ∗ be an optimal multiway tree with respect to both worst-case explanation size and depth, and hence satisfying depth wc ( M ∗ )

Original: =
Revised: =

Original: expl wc ( M ∗ ) = expl ∗ wc .
Revised: expl wc ( M ∗ ) = expl ∗ wc .

Original: We show how to transform
Revised: We show how to transform

Original: M ∗ into a binary tree T with expl wc ( T ) =
Revised: M ∗ into a binary tree T with expl wc ( T ) =

Original: expl wc ( M ∗ ) and depth wc ( T ) ≤ 2 depth wc ( M ∗ )+log n (which then will give (3)-(4)).
Revised: expl wc ( M ∗ ) and depth wc ( T ) ≤ 2 depth wc ( M ∗ )+log n (which then will give (3)-(4)).

Original: Consider an internal node ν of M ∗ , which tests an attribute a and has as children ch 1 , ch 2 , . . . .
Revised: Consider an internal node ν of M ∗ , which tests an attribute a and has as children ch 1 , ch 2 , . . . .

Original: Let d i :=
Revised: Let d i :=

Original: (cid:100) log n ( ν ) n ( ch i )
Revised: (cid:100) log n ( ν ) n ( ch i )

Original: (cid:101)
Revised: (cid:101)

Original: + 1 and notice that
Revised: + 1 and notice that

Original: [Equation]
Revised: [Equation]

Original: the last equation following because (cid:80) i n ( ch i ) = n ( ν ) .
Revised: the last equation following because (cid:80) i n ( ch i ) = n ( ν ) .

Original: Then applying the result about the existence of alphabetic codes (Lemma 1) with the attribute a and considering all objects that reach the subtree
Revised: Then applying the result about the existence of alphabetic codes (Lemma 1) with the attribute a and considering all objects that reach the subtree

Original: M ∗ ch i as a single object, we can get a threshold decision tree A with one leaf for each ch
Revised: M ∗ ch i as a single object, we can get a threshold decision tree A with one leaf for each ch

Original: i such that: 1)
Revised: i such that: 1)

Original: The leaf corresponding to ch i is at height (in A )
Revised: The leaf corresponding to ch i is at height (in A )

Original: d i , and; 2)
Revised: d i , and; 2)

Original: Every object that in M ∗ reaches node ch i , in A reaches the leaf corresponding to ch i .
Revised: Every object that in M ∗ reaches node ch i , in A reaches the leaf corresponding to ch i .

Original: It will be convenient to call A a code tree.
Revised: It will be convenient to call A a code tree.

Original: Then, we proceed like in the proof of the ﬁrst part of Lemma 2 (see also Fig.
Revised: Then, we proceed like in the proof of the ﬁrst part of Lemma 2 (see also Fig.

Original: 3): replace the node ν of M ∗ by the tree A (with the root of the subtree M ∗ ch
Revised: 3): replace the node ν of M ∗ by the tree A (with the root of the subtree M ∗ ch

Original: i identiﬁed with the leaf of A corresponding to ch i ).
Revised: i identiﬁed with the leaf of A corresponding to ch i ).

Original: Perform this replacement for every node of M ∗ , and let T be the resulting threshold tree.
Revised: Perform this replacement for every node of M ∗ , and let T be the resulting threshold tree.

Original: We refer to the nodes of T that are the roots of the code trees A used in the process as the original nodes.
Revised: We refer to the nodes of T that are the roots of the code trees A used in the process as the original nodes.

Original: The motivation for this terminology is that these nodes correspond to the nodes in the original tree M ∗ .
Revised: The motivation for this terminology is that these nodes correspond to the nodes in the original tree M ∗ .

Original: Observe that if v is an original node in T and u its corresponding node in M ∗ , then n T ( v ) =
Revised: Observe that if v is an original node in T and u its corresponding node in M ∗ , then n T ( v ) =

Original: n M ∗ ( u ) .
Revised: n M ∗ ( u ) .

Original: We ﬁrst claim that expl wc ( T ) = expl wc ( M ∗ ) (and hence, by deﬁnition of M ∗ , also equals depth ∗ wc ).
Revised: We ﬁrst claim that expl wc ( T ) = expl wc ( M ∗ ) (and hence, by deﬁnition of M ∗ , also equals depth ∗ wc ).

Original: Let (cid:96) T be a leaf of T and (cid:96) M ∗ be the corresponding leaf of M ∗ .
Revised: Let (cid:96) T be a leaf of T and (cid:96) M ∗ be the corresponding leaf of M ∗ .

Original: By construction, the set of attributes tested on the path reaching (cid:96) T is the same as the set of attributes tested on the path reaching (cid:96) M ∗ , since the tests associated to the nodes ν on the latter path have been expanded in T to a path in the code tree used to replace ν and such a code tree uses only tests on the same attribute tested at ν.
Revised: By construction, the set of attributes tested on the path reaching (cid:96) T is the same as the set of attributes tested on the path reaching (cid:96) M ∗ , since the tests associated to the nodes ν on the latter path have been expanded in T to a path in the code tree used to replace ν and such a code tree uses only tests on the same attribute tested at ν.

Original: Therefore, we have that the explanation size of (cid:96) T is the same as that of (cid:96) M ∗ .
Revised: Therefore, we have that the explanation size of (cid:96) T is the same as that of (cid:96) M ∗ .

Original: Considering the leaves with largest explanation sizes in these trees, we get expl wc ( T ) =
Revised: Considering the leaves with largest explanation sizes in these trees, we get expl wc ( T ) =

Original: expl wc ( M ∗ ) = expl ∗ wc as we wanted.
Revised: expl wc ( M ∗ ) = expl ∗ wc as we wanted.

Original: This gives the desired equality (3).
Revised: This gives the desired equality (3).

Original: Now we claim that depth wc ( T ) ≤ 2 depth wc ( M ∗ )
Revised: Now we claim that depth wc ( T ) ≤ 2 depth wc ( M ∗ )

Original: + log n .
Revised: + log n .

Original: For that, consider any root-to-leaf path
Revised: For that, consider any root-to-leaf path

Original: P in T , and let ν 1 , . .
Revised: P in T , and let ν 1 , . .

Original: , ν p be the original nodes in this path.
Revised: , ν p be the original nodes in this path.

Original: Since both the root and leaves of T are original nodes, we see that ν 

Revised: Since both the root and leaves of T are original nodes, we see that ν

Original: and ν p are the ﬁrst and last nodes of the path P , respectively.
Revised: and ν p are the ﬁrst and last nodes of the path P , respectively.

Original: Letting d ( u, v ) denote the distance (in number of edges) between two nodes u, v in T , we see that the length (number of internal nodes, or equivalently, edges) of P can be written as
Revised: Letting d ( u, v ) denote the distance (in number of edges) between two nodes u, v in T , we see that the length (number of internal nodes, or equivalently, edges) of P can be written as

Original: [Equation]
Revised: [Equation]

Original: Moreover, letting A be the alphabetic code tree used in the construction of T that is rooted at ν
Revised: Moreover, letting A be the alphabetic code tree used in the construction of T that is rooted at ν

Original: i , we have that the subpath between the original nodes ν
Revised: i , we have that the subpath between the original nodes ν

Original: i and ν i +1 coincides with the path of A between its root and its leaf corresponding to ν i +1 .
Revised: i and ν i +1 coincides with the path of A between its root and its leaf corresponding to ν i +1 .

Original: Then, by the construction of A we have
Revised: Then, by the construction of A we have

Original: [Equation]
Revised: [Equation]

Original: Replacing this bound in (6) gives
Revised: Replacing this bound in (6) gives

Original: [Equation]
Revised: [Equation]

Original: where the last inequality follows because all of the original nodes ν 1 , .
Revised: where the last inequality follows because all of the original nodes ν 1 , .

Original: . . , ν p − 1 are (or correspond to) internal nodes in a root-to-leaf path in M ∗ , and so depth wc ( M ∗ ) ≥ p − 1 .
Revised: . . , ν p − 1 are (or correspond to) internal nodes in a root-to-leaf path in M ∗ , and so depth wc ( M ∗ ) ≥ p − 1 .

Original: Since the bound (8) holds for every root-to-leaf path P of T , we obtain that depth wc ( T ) ≤ 2 depth wc ( M ∗ )
Revised: Since the bound (8) holds for every root-to-leaf path P of T , we obtain that depth wc ( T ) ≤ 2 depth wc ( M ∗ )

Original: + log n , proving the claim.
Revised: + log n , proving the claim.

Original: Since by deﬁnition of M ∗ we have depth wc ( M ∗ ) = expl ∗ wc , which is at most depth ∗ wc , this proves the desired inequality (4).
Revised: Since by deﬁnition of M ∗ we have depth wc ( M ∗ ) = expl ∗ wc , which is at most depth ∗ wc , this proves the desired inequality (4).

Original: This concludes the proof of Item 1 of Theorem 1.
Revised: This concludes the proof of Item 1 of Theorem 1.

Original: B.2 Proof of Item 2: Average metrics
Revised: B.2 Proof of Item 2: Average metrics

Original: We now prove Item 2 of Theorem 1, regarding the average explanation size and depth, instead of the worst-case ones, namely we want a threshold tree T satisfying:
Revised: We now prove Item 2 of Theorem 1, regarding the average explanation size and depth, instead of the worst-case ones, namely we want a threshold tree T satisfying:

Original: [Equation]
Revised: [Equation]

Original: where W = (cid:80) o w ( o ) denote the total weight of all objects.
Revised: where W = (cid:80) o w ( o ) denote the total weight of all objects.

Original: The proof is almost identical as that of the previous section, the only difference is that the weights of the items will now be taken into account when deﬁning the d i ’s in the code trees, and we use entropy-based calculations to argue about the average depth of the constructed tree.
Revised: The proof is almost identical as that of the previous section, the only difference is that the weights of the items will now be taken into account when deﬁning the d i ’s in the code trees, and we use entropy-based calculations to argue about the average depth of the constructed tree.

Original: Given a tree T and one of its nodes ν , let w ( ν ) denotes the total weights of the objects reaching node ν
Revised: Given a tree T and one of its nodes ν , let w ( ν ) denotes the total weights of the objects reaching node ν

Original: (we add the subscript w T ( ν ) when we want to emphasize which tree we are referring to).
Revised: (we add the subscript w T ( ν ) when we want to emphasize which tree we are referring to).

Original: Let M ∗ be the multiway tree given by Lemma 3, which satisﬁes the guarantees depth avg ( M ∗ ) =
Revised: Let M ∗ be the multiway tree given by Lemma 3, which satisﬁes the guarantees depth avg ( M ∗ ) =

Original: expl avg ( M ∗ ) = expl ∗ avg for the average metrics.
Revised: expl avg ( M ∗ ) = expl ∗ avg for the average metrics.

Original: We ﬁrst note that one can compute the average cost of a tree by summing up the weights of its internal nodes, for example
Revised: We ﬁrst note that one can compute the average cost of a tree by summing up the weights of its internal nodes, for example

Original: [Equation]
Revised: [Equation]

Original: where again we use int ( · ) to denote the set of internal nodes of a tree.
Revised: where again we use int ( · ) to denote the set of internal nodes of a tree.

Original: We now show how to transform M ∗ into a threshold decision tree T of small average explanation size and depth.
Revised: We now show how to transform M ∗ into a threshold decision tree T of small average explanation size and depth.

Original: For an internal node ν of M ∗ , which tests an attribute a ν and has as children the nodes ν 1 , ν 2 , . . .
Revised: For an internal node ν of M ∗ , which tests an attribute a ν and has as children the nodes ν 1 , ν 2 , . . .

Original: , let p ν be the probability distribution (cid:16) w ( ν 1 ) w ( ν ) , w ( ν 2 )
Revised: , let p ν be the probability distribution (cid:16) w ( ν 1 ) w ( ν ) , w ( ν 2 )

Original: w ( ν ) , . .
Revised: w ( ν ) , . .

Original: , (cid:17) induced by the partition at ν .
Revised: , (cid:17) induced by the partition at ν .

Original: Then consider the code tree T ν given by Lemma 1 with the attribute a and considering all objects that reach the subtree M ∗
Revised: Then consider the code tree T ν given by Lemma 1 with the attribute a and considering all objects that reach the subtree M ∗

Original: ν i as a single object, but such that the leaf in T ν corresponding to ν
Revised: ν i as a single object, but such that the leaf in T ν corresponding to ν

Original: i is now at depth (in T ν )
Revised: i is now at depth (in T ν )

Original: d i := (cid:100) log w ( ν ) w ( ν i ) (cid:101) + 1 (as in (5), we can see that such d i ’s satisfy the requirement of the lemma).
Revised: d i := (cid:100) log w ( ν ) w ( ν i ) (cid:101) + 1 (as in (5), we can see that such d i ’s satisfy the requirement of the lemma).

Original: Then replace the node ν of M ∗ by the tree T ν (with the root of the subtree M ∗
Revised: Then replace the node ν of M ∗ by the tree T ν (with the root of the subtree M ∗

Original: ν i identiﬁed with the leaf of T ν corresponding to ν i ).
Revised: ν i identiﬁed with the leaf of T ν corresponding to ν i ).

Original: Perform this replacement for every node of M ∗ , and let T be the resulting threshold tree.
Revised: Perform this replacement for every node of M ∗ , and let T be the resulting threshold tree.

Original: Again we refer to the nodes of T that are the roots of the code trees used in the process as the original nodes, and observe that if v is an original node in T and u its corresponding node in M ∗ , then w T ( v ) =
Revised: Again we refer to the nodes of T that are the roots of the code trees used in the process as the original nodes, and observe that if v is an original node in T and u its corresponding node in M ∗ , then w T ( v ) =

Original: w M ∗ ( u ) .
Revised: w M ∗ ( u ) .

Original: Just as before, the set of attributes tested on the path in T from its root to one of its leaves (cid:96) T is the same as the set of attributes tested on the path in M ∗ from its root to the leaf (cid:96) M ∗ that corresponds to (cid:96) T .
Revised: Just as before, the set of attributes tested on the path in T from its root to one of its leaves (cid:96) T is the same as the set of attributes tested on the path in M ∗ from its root to the leaf (cid:96) M ∗ that corresponds to (cid:96) T .

Original: This implies that expl avg ( T ) =
Revised: This implies that expl avg ( T ) =

Original: expl avg ( M ∗ ) , which by deﬁnition of M ∗ also equals expl ∗ avg .
Revised: expl avg ( M ∗ ) , which by deﬁnition of M ∗ also equals expl ∗ avg .

Original: This proves inequality (9).
Revised: This proves inequality (9).

Original: To prove inequality (10), let leaf ( M ∗ )
Revised: To prove inequality (10), let leaf ( M ∗ )

Original: = { (cid:96) 1 , . .
Revised: = { (cid:96) 1 , . .

Original: , (cid:96) m } be the set of leaves of the multiway tree
Revised: , (cid:96) m } be the set of leaves of the multiway tree

Original: M ∗ .
Revised: M ∗ .

Original: Let p L :=
Revised: Let p L :=

Original: ( w ( (cid:96) 1 ) W , . . .
Revised: ( w ( (cid:96) 1 ) W , . . .

Original: , w ( (cid:96) m ) W ) be the probability distribution induced on these leaves.
Revised: , w ( (cid:96) m ) W ) be the probability distribution induced on these leaves.

Original: Also, let p ( ν ) :
Revised: Also, let p ( ν ) :

Original: = w ( ν ) W be the sum of the probabilities of the leaves in the subtree rooted at ν.
Revised: = w ( ν ) W be the sum of the probabilities of the leaves in the subtree rooted at ν.

Original: Given a distribution p, we use H ( p ) :
Revised: Given a distribution p, we use H ( p ) :

Original: = (cid:80)
Revised: = (cid:80)

Original: i
Revised: i

Original: p
Revised: p

Original: i log 1 p i to denote its Shannon entropy.
Revised: i log 1 p i to denote its Shannon entropy.

Original: The following fact records a well-known property that allows to compute the Shannon entropy of the leaf distribution as the weighted average of the entropy of the node distributions (see, e.g., [25, Chapter 3]).
Revised: The following fact records a well-known property that allows to compute the Shannon entropy of the leaf distribution as the weighted average of the entropy of the node distributions (see, e.g., [25, Chapter 3]).

Original: Fact 1.
Revised: Fact 1.

Original: It holds that H ( p L ) =
Revised: It holds that H ( p L ) =

Original: [Equation]
Revised: [Equation]

Original: Now, we can compute the average depth of the tree T as follows: Since in T, each node ν of M ∗ is replaced by the code tree T ν , the corresponding contribution to the average depth of T is given by the average depth of T ν , namely
Revised: Now, we can compute the average depth of the tree T as follows: Since in T, each node ν of M ∗ is replaced by the code tree T ν , the corresponding contribution to the average depth of T is given by the average depth of T ν , namely

Original: [Equation]
Revised: [Equation]

Original: We can use this in the computation of the average depth of the tree T , decomposing it into the contributions of the code trees that have been used to replace the nodes of M ∗ :
Revised: We can use this in the computation of the average depth of the tree T , decomposing it into the contributions of the code trees that have been used to replace the nodes of M ∗ :

Original: [Equation]
Revised: [Equation]

Original: where the ﬁrst line follows from (11); inequality (14) follows from using (12) to upper bound depth avg ( T ν ) and then multiplying and dividing the ﬁrst summation by w ( ν ) ; equation (15) follows again from (11); (16) follows from Fact 1; ﬁnally, inequality (17) follows from the standard entropy rank bound (i.e., the entropy of a distribution over m objects is at most log m ).
Revised: where the ﬁrst line follows from (11); inequality (14) follows from using (12) to upper bound depth avg ( T ν ) and then multiplying and dividing the ﬁrst summation by w ( ν ) ; equation (15) follows again from (11); (16) follows from Fact 1; ﬁnally, inequality (17) follows from the standard entropy rank bound (i.e., the entropy of a distribution over m objects is at most log m ).

Original: This proves inequality (10), and concludes the proof of Item 2 of Theorem 1.
Revised: This proves inequality (10), and concludes the proof of Item 2 of Theorem 1.

Original: C Proof of Theorem
Revised: C Proof of Theorem

Original: Construction of the hard instance.
Revised: Construction of the hard instance.

Original: We will construct a deterministic instance for the lower bound as follows.
Revised: We will construct a deterministic instance for the lower bound as follows.

Original: There are n objects, and their classes are from the set C
Revised: There are n objects, and their classes are from the set C

Original: = { 0 , 1 , . .
Revised: = { 0 , 1 , . .

Original: . , c − 1 } and “alternate”: object 1 has class 0, object 2 class 1, object 3 class 2, ..., object c class c − 1 ; then this repeats, object c + 1 has class 0, etc.
Revised: . , c − 1 } and “alternate”: object 1 has class 0, object 2 class 1, object 3 class 2, ..., object c class c − 1 ; then this repeats, object c + 1 has class 0, etc.

Original: We consider throughout that n is at least cst · c for a sufﬁciently large constant cst .
Revised: We consider throughout that n is at least cst · c for a sufﬁciently large constant cst .

Original: The ﬁrst attribute in the instance, named A , has n values and gives the identity of the objects (i.e., object i has A -value equal to i ).
Revised: The ﬁrst attribute in the instance, named A , has n values and gives the identity of the objects (i.e., object i has A -value equal to i ).

Original: The deﬁnition of the other attributes is more intricate and we ﬁrst give their intuition and motivation.
Revised: The deﬁnition of the other attributes is more intricate and we ﬁrst give their intuition and motivation.

Original: First, there is a parameter t ∈ ( 2 cn , 12 ) (these bounds are important so that we can ignore the ﬂoor in (cid:98) t nc (cid:99) ).
Revised: First, there is a parameter t ∈ ( 2 cn , 12 ) (these bounds are important so that we can ignore the ﬂoor in (cid:98) t nc (cid:99) ).

Original: There are several binary attributes each of which discriminates a speciﬁc (1 − t ) -fraction of the objects of one of the classes (cid:54) = 

Revised: There are several binary attributes each of which discriminates a speciﬁc (1 − t ) -fraction of the objects of one of the classes (cid:54) =

Original: from all the other objects (a more detailed description is given below).
Revised: from all the other objects (a more detailed description is given below).

Original: The idea is that the top levels of a decision tree of optimal height (which will only use tests on binary attributes) will have the following effect:
Revised: The idea is that the top levels of a decision tree of optimal height (which will only use tests on binary attributes) will have the following effect:

Original: All of objects of class 0 will traverse the leftmost path 2.
Revised: All of objects of class 0 will traverse the leftmost path 2.

Original: Only t nc of the objects of each class (cid:54) = 0 will follow this left path 3.
Revised: Only t nc of the objects of each class (cid:54) = 0 will follow this left path 3.

Original: All the other (1 − t ) nc objects of each class (cid:54)
Revised: All the other (1 − t ) nc objects of each class (cid:54)

Original: = 0 are classiﬁed correctly by the top levels of the tree.
Revised: = 0 are classiﬁed correctly by the top levels of the tree.

Original: Thus, the top of the tree “peels off” (namely, it completes the classiﬁcation of) a (1 − t ) fraction of the objects of each class (cid:54) = 0 , and what is left to be “solved” is on the leftmost path.
Revised: Thus, the top of the tree “peels off” (namely, it completes the classiﬁcation of) a (1 − t ) fraction of the objects of each class (cid:54) = 0 , and what is left to be “solved” is on the leftmost path.

Original: Then, starting from the last node on the leftmost path of these top levels the tree again will use a similar subtree to peel off another (1 − t ) fraction of the objects of each class (cid:54) = 0 , etc.
Revised: Then, starting from the last node on the leftmost path of these top levels the tree again will use a similar subtree to peel off another (1 − t ) fraction of the objects of each class (cid:54) = 0 , etc.

Original: The key to the lower bound is that for any decision tree not employing all these tests on the distinct binary attributes (in the attempt of reducing the number of the tests on binary attributes, and equivalently, the explanation size), the objects (of class (cid:54) = 0 ) that such not-performed tests would have correctly classiﬁed, will end up on the leftmost path together with the object of class 0.
Revised: The key to the lower bound is that for any decision tree not employing all these tests on the distinct binary attributes (in the attempt of reducing the number of the tests on binary attributes, and equivalently, the explanation size), the objects (of class (cid:54) = 0 ) that such not-performed tests would have correctly classiﬁed, will end up on the leftmost path together with the object of class 0.

Original: But then tests on the A attribute will need to be used to separate these objects and classify them correctly.
Revised: But then tests on the A attribute will need to be used to separate these objects and classify them correctly.

Original: Crucially, the objects separated by one of these binary attribute tests will be “uniformly spread out” over [ n ] , so they will “alternate” with the objects of class 0; this means that you need a lot of threshold tests on the attribute A to separate these remaining 0-class and non-0-class objects, paying a lot in terms height.
Revised: Crucially, the objects separated by one of these binary attribute tests will be “uniformly spread out” over [ n ] , so they will “alternate” with the objects of class 0; this means that you need a lot of threshold tests on the attribute A to separate these remaining 0-class and non-0-class objects, paying a lot in terms height.

Original: To make this more precise, let us deﬁne a round of a decision tree as the sequence of levels that are meant to achieve the result described in the three-item list above.
Revised: To make this more precise, let us deﬁne a round of a decision tree as the sequence of levels that are meant to achieve the result described in the three-item list above.

Original: The sets of objects S i ⊆
Revised: The sets of objects S i ⊆

Original: [ n ] that will be peeled off at round i need to have the following “spread out” property guaranteed by the next lemma, whose proof is deferred to Appendix C.1).
Revised: [ n ] that will be peeled off at round i need to have the following “spread out” property guaranteed by the next lemma, whose proof is deferred to Appendix C.1).

Original: Lemma 4.
Revised: Lemma 4.

Original: There are sets of objects S 1 , . . .
Revised: There are sets of objects S 1 , . . .

Original: , S w (with w = log( n/c ) log(1 /t ) )
Revised: , S w (with w = log( n/c ) log(1 /t ) )

Original: such that:
Revised: such that:

Original: • These sets partition the set of elements of class (cid:54) =
Revised: • These sets partition the set of elements of class (cid:54) =

Original: • (Spread out)
Revised: • (Spread out)

Original: For each i , class χ (cid:54) = 0 , and interval U ∈ [ n ] , the set S
Revised: For each i , class χ (cid:54) = 0 , and interval U ∈ [ n ] , the set S

Original: i ∩ U has at least
Revised: i ∩ U has at least

Original: [Equation]
Revised: [Equation]

Original: objects of class χ .
Revised: objects of class χ .

Original: Given these “peeling-off” sets, we can ﬁnally conclude the deﬁnition of the remaining attributes in the instance: in addition to attribute A , there is a binary attribute ( i, j ) for each i ∈ [log( n/c ) / log(1 /t )] and j ∈ [log c ] such that the test ( i, j )
Revised: Given these “peeling-off” sets, we can ﬁnally conclude the deﬁnition of the remaining attributes in the instance: in addition to attribute A , there is a binary attribute ( i, j ) for each i ∈ [log( n/c ) / log(1 /t )] and j ∈ [log c ] such that the test ( i, j )

Original: • sends to the right all the objects in S
Revised: • sends to the right all the objects in S

Original: i whose class have j -th bit equal to• sends to the left all other objects (in particular all those outside of S i ).
Revised: i whose class have j -th bit equal to• sends to the left all other objects (in particular all those outside of S i ).

Original: This concludes the description of the instance.
Revised: This concludes the description of the instance.

Original: Analysis.
Revised: Analysis.

Original: Given an integer x , we use bits ( x ) to denote the binary expansion of x (as a vector).
Revised: Given an integer x , we use bits ( x ) to denote the binary expansion of x (as a vector).

Original: Given a (partial) decision tree, when all objects that reach a given leaf have the same class we say that the tree classiﬁes them correctly (since this can be naturally obtained by assigning their class to the leaf).
Revised: Given a (partial) decision tree, when all objects that reach a given leaf have the same class we say that the tree classiﬁes them correctly (since this can be naturally obtained by assigning their class to the leaf).

Original: We start by showing that this instance has a tree of “short” height.
Revised: We start by showing that this instance has a tree of “short” height.

Original: Lemma 5.
Revised: Lemma 5.

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: We construct such a short tree that satisﬁes the bound, hence a fortiori it must also hold for the optimum.
Revised: We construct such a short tree that satisﬁes the bound, hence a fortiori it must also hold for the optimum.

Original: Again, we use the term round to refer to a group of consecutive levels, where for each level there is a speciﬁc test which is performed in all nodes of that level.
Revised: Again, we use the term round to refer to a group of consecutive levels, where for each level there is a speciﬁc test which is performed in all nodes of that level.

Original: (Round 1) Build a complete binary tree of depth log c , where at each node on level j the test (1 , j ) is performed.
Revised: (Round 1) Build a complete binary tree of depth log c , where at each node on level j the test (1 , j ) is performed.

Original: We claim that these tests peel off the set S 1 , and classify all of these objects correctly.
Revised: We claim that these tests peel off the set S 1 , and classify all of these objects correctly.

Original: To see this, observe that these tests give a tree with c leaves, each associated with a vector y ∈ { 0 , 1 } log c corresponding to the outcome of these tests (where 0 means “go left” and 1 means “go right”).
Revised: To see this, observe that these tests give a tree with c leaves, each associated with a vector y ∈ { 0 , 1 } log c corresponding to the outcome of these tests (where 0 means “go left” and 1 means “go right”).

Original: The main observation is that every object of S 1 of class χ ∈ { 1 , . . .
Revised: The main observation is that every object of S 1 of class χ ∈ { 1 , . . .

Original: , (log c ) − 1 } will traverse to the leaf corresponding to vector y =
Revised: , (log c ) − 1 } will traverse to the leaf corresponding to vector y =

Original: bits ( χ ) .
Revised: bits ( χ ) .

Original: In addition, all objects outside of S 1 will end up in the left-most leaf (i.e. the one with vector y
Revised: In addition, all objects outside of S 1 will end up in the left-most leaf (i.e. the one with vector y

Original: =
Revised: =

Original: With this we can conclude that:
Revised: With this we can conclude that:

Original: All leaves except the left-most one have only objects with the same class (so we do not need to continue splitting these leaves).
Revised: All leaves except the left-most one have only objects with the same class (so we do not need to continue splitting these leaves).

Original: These leaves contain all elements in S 1 (recall that Sdoes not have any element of class 0).
Revised: These leaves contain all elements in S 1 (recall that Sdoes not have any element of class 0).

Original: The leftmost leaf has exactly all elements outside of
Revised: The leftmost leaf has exactly all elements outside of

Original: S 1 .
Revised: S 1 .

Original: (Round 2)
Revised: (Round 2)

Original: We need to further split this leftmost leaf.
Revised: We need to further split this leftmost leaf.

Original: For that, build a new complete binary tree of depth log c , where at each node on level j test (2 , j ) is performed.
Revised: For that, build a new complete binary tree of depth log c , where at each node on level j test (2 , j ) is performed.

Original: Again by the same argument we get another tree with c leaves, where all objects in S 2 will be peeled out and only the elements in [ n ] − ( S 1 ∪ S 2 ) are remaining on the leftmost leaf.
Revised: Again by the same argument we get another tree with c leaves, where all objects in S 2 will be peeled out and only the elements in [ n ] − ( S 1 ∪ S 2 ) are remaining on the leftmost leaf.

Original: Keep repeating these rounds—each round i = 1 , . .
Revised: Keep repeating these rounds—each round i = 1 , . .

Original: w, corresponding to executing tests { ( i, j ) } j =1 ,..., log c —until you ﬁnish peeling off all sets S i .
Revised: w, corresponding to executing tests { ( i, j ) } j =1 ,..., log c —until you ﬁnish peeling off all sets S i .

Original: At this point the leftmost path has only objects [ n ] − ( S 1 ∪ S 2 ∪ . . .
Revised: At this point the leftmost path has only objects [ n ] − ( S 1 ∪ S 2 ∪ . . .

Original: ∪ S w ) , which are all objects of class 0, so they are correctly classiﬁed by this leaf.
Revised: ∪ S w ) , which are all objects of class 0, so they are correctly classiﬁed by this leaf.

Original: Also, by construction all the remaining objects S 1 ∪ S 2 ∪ . . .
Revised: Also, by construction all the remaining objects S 1 ∪ S 2 ∪ . . .

Original: ∪ S w are also classiﬁed correctly in the rest of the tree.
Revised: ∪ S w are also classiﬁed correctly in the rest of the tree.

Original: So we have obtained a valid tree of total height w · log c = log( n/c )log c log(1 /t ) , as desired.
Revised: So we have obtained a valid tree of total height w · log c = log( n/c )log c log(1 /t ) , as desired.

Original: Now we give a lower bound on the depth of any tree that solves the problem.
Revised: Now we give a lower bound on the depth of any tree that solves the problem.

Original: Lemma 6. Consider a tree that solves the instance described above.
Revised: Lemma 6. Consider a tree that solves the instance described above.

Original: Then there is a root-to-leaf path
Revised: Then there is a root-to-leaf path

Original: P such that
Revised: P such that

Original: [Equation]
Revised: [Equation]

Original: where K A and K B are the number of A tests and binary tests (respectively) done on this path.
Revised: where K A and K B are the number of A tests and binary tests (respectively) done on this path.

Original: Proof.
Revised: Proof.

Original: We construct the path P as follows:
Revised: We construct the path P as follows:

Original: Start at the root of the tree; if the current node ν corresponds to a test on a binary attribute, go to the left; if it corresponds to a threshold test on the attribute A , go to the side where most examples would go if we ignore the tests on the binary attributes performed before node ν
Revised: Start at the root of the tree; if the current node ν corresponds to a test on a binary attribute, go to the left; if it corresponds to a threshold test on the attribute A , go to the side where most examples would go if we ignore the tests on the binary attributes performed before node ν

Original: (but we do take into account the previous tests on attribute A ).
Revised: (but we do take into account the previous tests on attribute A ).

Original: We claim that unless (18) holds, at least one object of class 0 and one object of another class (cid:54) =would reach the end of the path P , contradicting that the tree correctly classiﬁes each object.
Revised: We claim that unless (18) holds, at least one object of class 0 and one object of another class (cid:54) =would reach the end of the path P , contradicting that the tree correctly classiﬁes each object.

Original: To see this claim, ﬁrst consider executing all the binary tests of P and let us see which objects survive (i.e., the result of each test on P make them continue to the next node on P ).
Revised: To see this claim, ﬁrst consider executing all the binary tests of P and let us see which objects survive (i.e., the result of each test on P make them continue to the next node on P ).

Original: First, all objects of class 0 survive these tests (since P only turns left on binary tests).
Revised: First, all objects of class 0 survive these tests (since P only turns left on binary tests).

Original: Moreover, there are log( n/c )log c log1 /t
Revised: Moreover, there are log( n/c )log c log1 /t

Original: binary tests ( i, j ) and P is only doing K B of them.
Revised: binary tests ( i, j ) and P is only doing K B of them.

Original: Thus, log( n/c ) log c log(1 /t
Revised: Thus, log( n/c ) log c log(1 /t

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Original: tests ( i, j ) are not done.
Revised: tests ( i, j ) are not done.

Original: Since there are log c of these j ’s, an averaging argument shows that there is one j where at least
Revised: Since there are log c of these j ’s, an averaging argument shows that there is one j where at least

Original: [Equation]
Revised: [Equation]

Original: tests { ( i, j ) } i are not done; let us denote it with j ∗ .
Revised: tests { ( i, j ) } i are not done; let us denote it with j ∗ .

Original: Since the i ’s range from 1 to log( n/c ) log(1 /t ) , this implies that there is one test ( i, j ∗ ) that is not performed for some i ≤ K B log c .
Revised: Since the i ’s range from 1 to log( n/c ) log(1 /t ) , this implies that there is one test ( i, j ∗ ) that is not performed for some i ≤ K B log c .

Original: Denote this test by ( i ∗ , j ∗ ) .
Revised: Denote this test by ( i ∗ , j ∗ ) .

Original: Look at the objects in the set S i ∗ with class χ where bits ( χ ) =
Revised: Look at the objects in the set S i ∗ with class χ where bits ( χ ) =

Original: . , 0) (with 1 in the j ∗ -th position).
Revised: . , 0) (with 1 in the j ∗ -th position).

Original: Call this set S χi ∗ .
Revised: Call this set S χi ∗ .

Original: Since all binary tests other than ( i ∗ , j ∗ ) send the objects S χi ∗ to the left, again we see that all of these objects survive the binary tests of P .
Revised: Since all binary tests other than ( i ∗ , j ∗ ) send the objects S χi ∗ to the left, again we see that all of these objects survive the binary tests of P .

Original: In summary: if we just perform the binary tests of P , all nc objects of class 0 survive (call them S 0 ) as well as all objects of the set S χi ∗ .
Revised: In summary: if we just perform the binary tests of P , all nc objects of class 0 survive (call them S 0 ) as well as all objects of the set S χi ∗ .

Original: Now we execute all the threshold tests on the attribute A that are performed along the path P .
Revised: Now we execute all the threshold tests on the attribute A that are performed along the path P .

Original: The result of these tests is to select/isolate the objects in some interval U of [ n ] .
Revised: The result of these tests is to select/isolate the objects in some interval U of [ n ] .

Original: That is, the objects ( S 0 ∪ S χi ∗ )
Revised: That is, the objects ( S 0 ∪ S χi ∗ )

Original: ∩ U survive all the tests in the path P .
Revised: ∩ U survive all the tests in the path P .

Original: By construction of P , each one of such tests on the attribute A at most halves the size of the interval of objects selected by the previous tests on A , hence U has size at least n 2 KA .
Revised: By construction of P , each one of such tests on the attribute A at most halves the size of the interval of objects selected by the previous tests on A , hence U has size at least n 2 KA .

Original: Thus, since one every k object has class 0, it is immediate to see that
Revised: Thus, since one every k object has class 0, it is immediate to see that

Original: [Equation]
Revised: [Equation]

Original: that is, at least these many objects of class 0 survive.
Revised: that is, at least these many objects of class 0 survive.

Original: Moreover, by Lemma 4 the number of surviving objects of class χ is at least:
Revised: Moreover, by Lemma 4 the number of surviving objects of class χ is at least:

Original: [Equation]
Revised: [Equation]

Original: By assumption that the decision tree correctly classiﬁes each object, we cannot have both classes surviving the path P .
Revised: By assumption that the decision tree correctly classiﬁes each object, we cannot have both classes surviving the path P .

Original: Hence, at least one of the RHSs in (19) or (20) has to be less than 1.
Revised: Hence, at least one of the RHSs in (19) or (20) has to be less than 1.

Original: By observing that the RHS of (20) is not larger than the RHS of (19), it must hold that
Revised: By observing that the RHS of (20) is not larger than the RHS of (19), it must hold that

Original: [Equation]
Revised: [Equation]

Original: so in particular (since n ≥ cst · c with cst a large constant)
Revised: so in particular (since n ≥ cst · c with cst a large constant)

Original: [Equation]
Revised: [Equation]

Original: Again using the fact that n ≥ cst · c for a sufﬁciently large constant, of the this implies the cleaner bound
Revised: Again using the fact that n ≥ cst · c for a sufﬁciently large constant, of the this implies the cleaner bound

Original: [Equation]
Revised: [Equation]

Original: which concludes the proof.
Revised: which concludes the proof.

Original: Now we are ready to prove the main theorem.
Revised: Now we are ready to prove the main theorem.

Original: Proof of Theorem 2.
Revised: Proof of Theorem 2.

Original: Set t such that α = 12 log1 /t log c . i.e., t = 1 c 2 α .
Revised: Set t such that α = 12 log1 /t log c . i.e., t = 1 c 2 α .

Original: Since we assumed α ∈ [ 1 2log c , 12 ( log( n/ 2) log c − 1)] we have that t ∈ [ 2 cn , 12 ] , as required by the above argument.
Revised: Since we assumed α ∈ [ 1 2log c , 12 ( log( n/ 2) log c − 1)] we have that t ∈ [ 2 cn , 12 ] , as required by the above argument.

Original: Then consider the instance I deﬁned above for this value of t .
Revised: Then consider the instance I deﬁned above for this value of t .

Original: Consider any binary tree T solving the instance I that is an α 2 -approx for the depth.
Revised: Consider any binary tree T solving the instance I that is an α 2 -approx for the depth.

Original: It sufﬁces to show that this tree has expl wc ( T ) ≥ 14 α log( n/c )expl ∗ wc , which means expl wc ( T ) ≥ 14 α log( n/c ) (since expl ∗ wc = 1 for this instance, by using only tests on attribute A ).
Revised: It sufﬁces to show that this tree has expl wc ( T ) ≥ 14 α log( n/c )expl ∗ wc , which means expl wc ( T ) ≥ 14 α log( n/c ) (since expl ∗ wc = 1 for this instance, by using only tests on attribute A ).

Original: For that, look at the path of T given by Lemma 6.
Revised: For that, look at the path of T given by Lemma 6.

Original: Since T is an α 2 -approx for the height, we have in particular (using Lemma 5)
Revised: Since T is an α 2 -approx for the height, we have in particular (using Lemma 5)

Original: [Equation]
Revised: [Equation]

Original: But then from Lemma 6 we have
Revised: But then from Lemma 6 we have

Original: [Equation]
Revised: [Equation]

Original: Since each binary test in K B contributes one unit to the explanation size, this proves that the explanation size of T is at least 14 α log( n/c ) as desired.
Revised: Since each binary test in K B contributes one unit to the explanation size, this proves that the explanation size of T is at least 14 α log( n/c ) as desired.

Original: C.1 Proof of Lemma
Revised: C.1 Proof of Lemma

Original: The lemma will be a direct consequence of the following result.
Revised: The lemma will be a direct consequence of the following result.

Original: Lemma 7.
Revised: Lemma 7.

Original: Consider [ m ] and positive scalars m i such that m 1 + . . .
Revised: Consider [ m ] and positive scalars m i such that m 1 + . . .

Original: + m w ≤ m , with w ≤ m .
Revised: + m w ≤ m , with w ≤ m .

Original: Then there are sets V 1 , . .
Revised: Then there are sets V 1 , . .

Original: , V w ⊆
Revised: , V w ⊆

Original: [ m ] such that:
Revised: [ m ] such that:

Original: The V
Revised: The V

Original: i ’s partition [ m ] 2.
Revised: i ’s partition [ m ] 2.

Original: For every interval J ⊆
Revised: For every interval J ⊆

Original: [ m ] and i , we have
Revised: [ m ] and i , we have

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Without loss of generality assume that (cid:80) i m i
Revised: Without loss of generality assume that (cid:80) i m i

Original: = m , by increasing one of the m i ’s if necessary.
Revised: = m , by increasing one of the m i ’s if necessary.

Original: Construct the sets V
Revised: Construct the sets V

Original: i ’s randomly as follows: independently for each j ∈ [ m ] , randomly put j in one of the sets
Revised: i ’s randomly as follows: independently for each j ∈ [ m ] , randomly put j in one of the sets

Original: V i so that Pr( j ∈ V i )
Revised: V i so that Pr( j ∈ V i )

Original: = m i m .
Revised: = m i m .

Original: Then by deﬁnition the V
Revised: Then by deﬁnition the V

Original: i ’s so created partition [ m ] , giving
Revised: i ’s so created partition [ m ] , giving

Original: Item 1 of the lemma.
Revised: Item 1 of the lemma.

Original: For Item 2, take any interval J .
Revised: For Item 2, take any interval J .

Original: First note that
Revised: First note that

Original: [Equation]
Revised: [Equation]

Original: Moreover, using the multiplicative Chernoff bound (see, e.g., [43, Chapter 4]) we have that for a ﬁxed i ,
Revised: Moreover, using the multiplicative Chernoff bound (see, e.g., [43, Chapter 4]) we have that for a ﬁxed i ,

Original: Pr
Revised: Pr

Original: [Equation]
Revised: [Equation]

Original: Moreover, for any δ ∈ (0 , 1) , by considering the cases 18 | J | m · m i ≥ log(1 /δ ) and 18 | J | m · m i ≤ log(1 /δ )
Revised: Moreover, for any δ ∈ (0 , 1) , by considering the cases 18 | J | m · m i ≥ log(1 /δ ) and 18 | J | m · m i ≤ log(1 /δ )

Original: we see that this implies
Revised: we see that this implies

Original: [Equation]
Revised: [Equation]

Original: Taking a union bound over all i ∈
Revised: Taking a union bound over all i ∈

Original: [ w ] (recall w ≤ m ) and over all intervals J (there are less than mof them)
Revised: [ w ] (recall w ≤ m ) and over all intervals J (there are less than mof them)

Original: we get that with probability strictly more than 1 − m 3 δ we have
Revised: we get that with probability strictly more than 1 − m 3 δ we have

Original: [Equation]
Revised: [Equation]

Original: ∀
Revised: ∀

Original: [Equation]
Revised: [Equation]

Original: ∀
Revised: ∀

Original: [Equation]
Revised: [Equation]

Original: J ⊆
Revised: J ⊆

Original: [ m ] .
Revised: [ m ] .

Original: Setting δ = 1 m 3 shows that there is a scenario where
Revised: Setting δ = 1 m 3 shows that there is a scenario where

Original: [Equation]
Revised: [Equation]

Original: ∀ intervals
Revised: ∀ intervals

Original: [Equation]
Revised: [Equation]

Original: ⊆
Revised: ⊆

Original: [ m ] .
Revised: [ m ] .

Original: This proves the lemma.
Revised: This proves the lemma.

Original: Proof of Lemma 4.
Revised: Proof of Lemma 4.

Original: Bucket the objects [ n ] in buckets B 1 , . . .
Revised: Bucket the objects [ n ] in buckets B 1 , . . .

Original: , B n/c , each of size c (with consecutive objects), that is, we have B 1 = { 1 , 2 , . .
Revised: , B n/c , each of size c (with consecutive objects), that is, we have B 1 = { 1 , 2 , . .

Original: , c } , B 2 = { c + 1 , . .
Revised: , c } , B 2 = { c + 1 , . .

Original: . , 2 c } , ...until bucket B n/c (for the sake of simplifying the notation, we assume that n/c is an integer).
Revised: . , 2 c } , ...until bucket B n/c (for the sake of simplifying the notation, we assume that n/c is an integer).

Original: Set w = log( n/c ) log(1 /t ) and apply the previous lemma with parameter m i
Revised: Set w = log( n/c ) log(1 /t ) and apply the previous lemma with parameter m i

Original: =
Revised: =

Original: t
Revised: t

Original: i − 1 (1 − t ) nc (notice (cid:80) i m i ≤ nc ) to get a partition
Revised: i − 1 (1 − t ) nc (notice (cid:80) i m i ≤ nc ) to get a partition

Original: V 1 , V 2 , . .
Revised: V 1 , V 2 , . .

Original: , V w of the set of indices of buckets [ n/c ] .
Revised: , V w of the set of indices of buckets [ n/c ] .

Original: Then set
Revised: Then set

Original: S i
Revised: S i

Original: [Equation]
Revised: [Equation]

Original: = (cid:91)
Revised: = (cid:91)

Original: [Equation]
Revised: [Equation]

Original: where B (cid:54) =0 (cid:96) denotes the set of objects in bucket B (cid:96) of class different from 0.
Revised: where B (cid:54) =0 (cid:96) denotes the set of objects in bucket B (cid:96) of class different from 0.

Original: By construction we satisfy Item 1 of the lemma, namely that the S i ’s partition all the objects of class (cid:54) = 0 .
Revised: By construction we satisfy Item 1 of the lemma, namely that the S i ’s partition all the objects of class (cid:54) = 0 .

Original: For the second item of the lemma, notice that for any interval U ⊆ [ n ] (recall that S χi is the set of objects in S i of class χ ) |
Revised: For the second item of the lemma, notice that for any interval U ⊆ [ n ] (recall that S χi is the set of objects in S i of class χ ) |

Original: [Equation]
Revised: [Equation]

Original: that compose S i and that are fully contained in U, since each bucket has 1 objects of class χ .
Revised: that compose S i and that are fully contained in U, since each bucket has 1 objects of class χ .

Original: Let J ⊆
Revised: Let J ⊆

Original: [ n/c ] be the indices of the buckets fully contained in U ; so the previous bound is
Revised: [ n/c ] be the indices of the buckets fully contained in U ; so the previous bound is

Original: [Equation]
Revised: [Equation]

Original: where the last inequality follows from Lemma 7.
Revised: where the last inequality follows from Lemma 7.

Original: Moreover, we can see that | J | ≥ | U | c − 2 .
Revised: Moreover, we can see that | J | ≥ | U | c − 2 .

Original: Replacing this bound in the previous displayed inequality and again using the fact n ≥ cst · c for a sufﬁciently large cst gives the desired result.
Revised: Replacing this bound in the previous displayed inequality and again using the fact n ≥ cst · c for a sufﬁciently large cst gives the desired result.

Original: D Proof of TheoremD.1 The Bounds on the average case
Revised: D Proof of TheoremD.1 The Bounds on the average case

Original: In this section we show the bounds
Revised: In this section we show the bounds

Original: [Equation]
Revised: [Equation]

Original: for a tree T produced by the algorithm SER-DT.
Revised: for a tree T produced by the algorithm SER-DT.

Original: In Line 2 of SER-DT, the threshold factor 12 is used to select an attribute that reduces the weighted pairwise misclassiﬁcation.
Revised: In Line 2 of SER-DT, the threshold factor 12 is used to select an attribute that reduces the weighted pairwise misclassiﬁcation.

Original: The argument we present here consider a more general case where 12 is substituted by any γ ∈ [ 12 , 1) .
Revised: The argument we present here consider a more general case where 12 is substituted by any γ ∈ [ 12 , 1) .

Original: To simplify the argument, assume w.l.o.g.
Revised: To simplify the argument, assume w.l.o.g.

Original: that (after the preprocessing) the weights are integers and w min = 1 .
Revised: that (after the preprocessing) the weights are integers and w min = 1 .

Original: To prove both parts of 21, it sufﬁces to show that for the constant α := max { 4 , 2(ln 1 γ ) − 1 } .
Revised: To prove both parts of 21, it sufﬁces to show that for the constant α := max { 4 , 2(ln 1 γ ) − 1 } .

Original: we have
Revised: we have

Original: [Equation]
Revised: [Equation]

Original: where M ∗ is a multiway decision tree 4 achieving the optimal average depth, depth ∗ avg , for the instance I.
Revised: where M ∗ is a multiway decision tree 4 achieving the optimal average depth, depth ∗ avg , for the instance I.

Original: Since ln( P ( O ) w ( O )) ≤ ln( n 2 W )
Revised: Since ln( P ( O ) w ( O )) ≤ ln( n 2 W )

Original: =
Revised: =

Original: O (ln( nW )) , and using the equivalence in Lemma 3, this implies
Revised: O (ln( nW )) , and using the equivalence in Lemma 3, this implies

Original: [Equation]
Revised: [Equation]

Original: To prove (23), we are going to use the following lower bound on the optimum average depth of a multiway tree.
Revised: To prove (23), we are going to use the following lower bound on the optimum average depth of a multiway tree.

Original: Lemma 8.
Revised: Lemma 8.

Original: Let ( a ∗ , i ∗ ) be the pair attribute/value maximizing max a,i P ( O ( a, i ))
Revised: Let ( a ∗ , i ∗ ) be the pair attribute/value maximizing max a,i P ( O ( a, i ))

Original: w ( O ( a, i )) .
Revised: w ( O ( a, i )) .

Original: Let
Revised: Let

Original: M ∗ be an optimal multiway tree, i.e., depth avg ( M ∗ ) = depth ∗ avg .
Revised: M ∗ be an optimal multiway tree, i.e., depth avg ( M ∗ ) = depth ∗ avg .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: For a node ν of M ∗ , let O ν
Revised: For a node ν of M ∗ , let O ν

Original: be the set of objects associated with the leaves of the subtree rooted at ν , and let a ν be the attribute tested at node ν.
Revised: be the set of objects associated with the leaves of the subtree rooted at ν , and let a ν be the attribute tested at node ν.

Original: Like in (11) we compute the average cost of the optimal tree M ∗ by summing up the weights of its internal nodes, depth avg ( M ∗ ) =
Revised: Like in (11) we compute the average cost of the optimal tree M ∗ by summing up the weights of its internal nodes, depth avg ( M ∗ ) =

Original: (cid:80) int ( M ∗ )
Revised: (cid:80) int ( M ∗ )

Original: w ( O ν ) , where again we use int ( · ) to denote the set of internal nodes of a tree.
Revised: w ( O ν ) , where again we use int ( · ) to denote the set of internal nodes of a tree.

Original: For a set of objects S ⊆ O and attribute x , deﬁne the quantity
Revised: For a set of objects S ⊆ O and attribute x , deﬁne the quantity

Original: [Equation]
Revised: [Equation]

Original: Notice that by deﬁnition, the attribute x
Revised: Notice that by deﬁnition, the attribute x

Original: = a ∗ is the one that has the largest value ∆ x ( O ) , that is, ∆ a ( O ) ≤ ∆ a ∗ ( O ) for every attribute a .
Revised: = a ∗ is the one that has the largest value ∆ x ( O ) , that is, ∆ a ( O ) ≤ ∆ a ∗ ( O ) for every attribute a .

Original: Hence, we have
Revised: Hence, we have

Original: [Equation]
Revised: [Equation]

Original: To make the numerator more “local”, we would like to replace ∆ a ν ( O ) by ∆ a ν ( O ν ) .
Revised: To make the numerator more “local”, we would like to replace ∆ a ν ( O ) by ∆ a ν ( O ν ) .

Original: The next lemma shows we can do this and still have a valid lower bound for depth avg ( M ∗ ) .
Revised: The next lemma shows we can do this and still have a valid lower bound for depth avg ( M ∗ ) .

Original: Claim 1.
Revised: Claim 1.

Original: For every attribute x , the function ∆ x ( · ) is monotone, namely for any sets of objects
Revised: For every attribute x , the function ∆ x ( · ) is monotone, namely for any sets of objects

Original: V ⊆ U ⊆
Revised: V ⊆ U ⊆

Original: O we have ∆ x ( V ) ≤ ∆ x ( U ) .
Revised: O we have ∆ x ( V ) ≤ ∆ x ( U ) .

Original: Proof of the claim.
Revised: Proof of the claim.

Original: Let ¯
Revised: Let ¯

Original: ı
Revised: ı

Original: =
Revised: =

Original: argmin
Revised: argmin

Original: i
Revised: i

Original: ( P ( U ) w ( U ) − P
Revised: ( P ( U ) w ( U ) − P

Original: ( U ( x, i ))
Revised: ( U ( x, i ))

Original: w ( U ( x, i ))) be the attribute value that yields the deﬁnition of ∆ x ( U ) .
Revised: w ( U ( x, i ))) be the attribute value that yields the deﬁnition of ∆ x ( U ) .

Original: Since ¯ ı is a feasible value in the deﬁnition of ∆ x ( V ) , we have ∆ x ( V ) ≤ P ( V ) w ( V ) − P ( V ( x, ¯ ı ))
Revised: Since ¯ ı is a feasible value in the deﬁnition of ∆ x ( V ) , we have ∆ x ( V ) ≤ P ( V ) w ( V ) − P ( V ( x, ¯ ı ))

Original: w ( V ( x, ¯ ı )) ; so to show ∆ x ( V ) ≤ ∆ x ( U ) it sufﬁces to show
Revised: w ( V ( x, ¯ ı )) ; so to show ∆ x ( V ) ≤ ∆ x ( U ) it sufﬁces to show

Original: P ( V ) w ( V ) − P ( V ( x, ¯ ı ))
Revised: P ( V ) w ( V ) − P ( V ( x, ¯ ı ))

Original: w ( V ( x, ¯ ı )) ≤ P ( U ) w ( U ) − P
Revised: w ( V ( x, ¯ ı )) ≤ P ( U ) w ( U ) − P

Original: ( U ( x, ¯ ı ))
Revised: ( U ( x, ¯ ı ))

Original: w ( U ( x, ¯ ı )) .
Revised: w ( U ( x, ¯ ı )) .

Original: Let S ( x, (cid:54) = i ) :
Revised: Let S ( x, (cid:54) = i ) :

Original: =
Revised: =

Original: S \ S ( x, i ) be the objects in S with value different from i in the attribute x , and notice w ( S )
Revised: S \ S ( x, i ) be the objects in S with value different from i in the attribute x , and notice w ( S )

Original: =
Revised: =

Original: w ( S ( x, i )) + w ( S ( x, (cid:54) = i )) .
Revised: w ( S ( x, i )) + w ( S ( x, (cid:54) = i )) .

Original: Applying this in the last displayed inequality, we see that it is equivalent to
Revised: Applying this in the last displayed inequality, we see that it is equivalent to

Original: [Equation]
Revised: [Equation]

Original: [Table]
Revised: [Table]

Original: [Equation]
Revised: [Equation]

Original: Since the number of pairs P ( S ) and the weight w ( S ) are monotone in S and since V ⊆ U , it is clear that each term in (25) (except top-term) is at most the corresponding term in (26) (except bottom-term).
Revised: Since the number of pairs P ( S ) and the weight w ( S ) are monotone in S and since V ⊆ U , it is clear that each term in (25) (except top-term) is at most the corresponding term in (26) (except bottom-term).

Original: Thus, to prove this inequality, it sufﬁces to argue that top-term ≤ bottom-term.
Revised: Thus, to prove this inequality, it sufﬁces to argue that top-term ≤ bottom-term.

Original: But notice that top-term is the number of pairs of objects o (cid:54)
Revised: But notice that top-term is the number of pairs of objects o (cid:54)

Original: =
Revised: =

Original: o (cid:48) ∈ V that have different classes and where at least one of objects has value (cid:54) =
Revised: o (cid:48) ∈ V that have different classes and where at least one of objects has value (cid:54) =

Original: ¯ ı in attribute x ; bottom-term is the same thing, but over pairs of objects o (cid:54) =
Revised: ¯ ı in attribute x ; bottom-term is the same thing, but over pairs of objects o (cid:54) =

Original: o (cid:48) in U .
Revised: o (cid:48) in U .

Original: Since every pair in the former also belongs to the latter, we see that top-term ≤ bottom-term.
Revised: Since every pair in the former also belongs to the latter, we see that top-term ≤ bottom-term.

Original: This then proves inequality (25)-(26) and concludes the proof of the claim.
Revised: This then proves inequality (25)-(26) and concludes the proof of the claim.

Original: Applying this on (24) we obtain the lower bound
Revised: Applying this on (24) we obtain the lower bound

Original: [Equation]
Revised: [Equation]

Original: We can think of the numerator above as the cost of a decision tree for an instance ˜
Revised: We can think of the numerator above as the cost of a decision tree for an instance ˜

Original: I with the same tests and object set as I but with the additional property that applying the test on an attribute a at a point where there are S objects incurs a cost equal to ∆ a ( S )
Revised: I with the same tests and object set as I but with the additional property that applying the test on an attribute a at a point where there are S objects incurs a cost equal to ∆ a ( S )

Original: In general, given a decision tree ˜
Revised: In general, given a decision tree ˜

Original: M for the instance ˜
Revised: M for the instance ˜

Original: I , its cost is deﬁned as
Revised: I , its cost is deﬁned as

Original: [Equation]
Revised: [Equation]

Original: where ˜ O ν is the set of objects associated with the leaves of the subtree of ˜ M rooted at ν , and ˜ a ν is the attribute tested at the node ν.
Revised: where ˜ O ν is the set of objects associated with the leaves of the subtree of ˜ M rooted at ν , and ˜ a ν is the attribute tested at the node ν.

Original: Letting depth ∗ avg (˜ I ) denote the minimum cost of a multiway decision tree for ˜ I with costs given by ∆ , from (27) we have
Revised: Letting depth ∗ avg (˜ I ) denote the minimum cost of a multiway decision tree for ˜ I with costs given by ∆ , from (27) we have

Original: [Equation]
Revised: [Equation]

Original: The proof of Lemma 8 is then completed by proving the following claim.
Revised: The proof of Lemma 8 is then completed by proving the following claim.

Original: Claim 2.
Revised: Claim 2.

Original: We have depth ∗ avg (˜ I ) ≥ P ( O ) w (
Revised: We have depth ∗ avg (˜ I ) ≥ P ( O ) w (

Original: [Equation]
Revised: [Equation]

Original: Proof of the claim.
Revised: Proof of the claim.

Original: We argue by induction on the number of pairs of distinct classes P
Revised: We argue by induction on the number of pairs of distinct classes P

Original: ( O ) .
Revised: ( O ) .

Original: For the base case P ( O ) = 1 , we have that in an optimal tree the attribute a tested at the root node ρ must separate the two object belonging to the only pair — note that such an attribute must exists since we assume separability of objects belonging to different classes.
Revised: For the base case P ( O ) = 1 , we have that in an optimal tree the attribute a tested at the root node ρ must separate the two object belonging to the only pair — note that such an attribute must exists since we assume separability of objects belonging to different classes.

Original: Hence, it must hold that
Revised: Hence, it must hold that

Original: P ( O ( a, i a ))
Revised: P ( O ( a, i a ))

Original: = 0 , and ∆ a ( O )
Revised: = 0 , and ∆ a ( O )

Original: =
Revised: =

Original: P ( O ) w ( O ) .
Revised: P ( O ) w ( O ) .

Original: Therefore we have
Revised: Therefore we have

Original: [Equation]
Revised: [Equation]

Original: as desired.
Revised: as desired.

Original: Now assume P ( O ) ≥ 2 and that the claim holds for all instances with the number of pairs of distinct classes smaller than P ( O ) .
Revised: Now assume P ( O ) ≥ 2 and that the claim holds for all instances with the number of pairs of distinct classes smaller than P ( O ) .

Original: Let a be the attribute tested at the root of a tree attaining depth ∗ avg (˜ I ) .
Revised: Let a be the attribute tested at the root of a tree attaining depth ∗ avg (˜ I ) .

Original: Also let i a be the value at attribute a that yields ∆ a ( O ) , namely i
Revised: Also let i a be the value at attribute a that yields ∆ a ( O ) , namely i

Original: a = argmin
Revised: a = argmin

Original: i
Revised: i

Original: ( P ( O ) w ( O ) −
Revised: ( P ( O ) w ( O ) −

Original: P ( O ( a, i ))
Revised: P ( O ( a, i ))

Original: w ( O ( a, i ))) .
Revised: w ( O ( a, i ))) .

Original: Because of the optimality of the tree we can assume that P ( O ( a, i a )) <
Revised: Because of the optimality of the tree we can assume that P ( O ( a, i a )) <

Original: P ( O ) , i.e., the test a separates at least one of the pairs.
Revised: P ( O ) , i.e., the test a separates at least one of the pairs.

Original: Case 1.
Revised: Case 1.

Original: P ( O ( a, i a ))
Revised: P ( O ( a, i a ))

Original: = 0 .
Revised: = 0 .

Original: Then as in the base case ∆ a ( O ) =
Revised: Then as in the base case ∆ a ( O ) =

Original: P ( O ) w ( O ) , and again we have depth
Revised: P ( O ) w ( O ) , and again we have depth

Original: [Equation]
Revised: [Equation]

Original: Case 2.
Revised: Case 2.

Original: 1 ≤ P ( S ( a, i a ))
Revised: 1 ≤ P ( S ( a, i a ))

Original: <
Revised: <

Original: P ( S ) .
Revised: P ( S ) .

Original: Then we consider the contribution of the root node and the subtree rooted at the child corresponding to the value
Revised: Then we consider the contribution of the root node and the subtree rooted at the child corresponding to the value

Original: i a .
Revised: i a .

Original: Let ˜
Revised: Let ˜

Original: I
Revised: I

Original: i a be the instance corresponding to the objects in the set O ( a, i a ) , with number of pairs P
Revised: i a be the instance corresponding to the objects in the set O ( a, i a ) , with number of pairs P

Original: ( O ( a, i a )) .
Revised: ( O ( a, i a )) .

Original: We have
Revised: We have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where (30)) follows from using the inductive hypothesis on depth ∗ avg (˜ I i a ) , (31) follows from the monotonicity of the weights w ( O ( a, i a )) ≤ w ( O ) , and (32) follows from monotonicity of the pairs
Revised: where (30)) follows from using the inductive hypothesis on depth ∗ avg (˜ I i a ) , (31) follows from the monotonicity of the weights w ( O ( a, i a )) ≤ w ( O ) , and (32) follows from monotonicity of the pairs

Original: P ( O ( a, i a )) ≤ P ( O ) .
Revised: P ( O ( a, i a )) ≤ P ( O ) .

Original: This concludes the proof of the inductive step, and hence of the claim.
Revised: This concludes the proof of the inductive step, and hence of the claim.

Original: Given this lower bound on the average depth of an optimal multiway tree depth ∗ avg we now show the desired bound (23).
Revised: Given this lower bound on the average depth of an optimal multiway tree depth ∗ avg we now show the desired bound (23).

Original: Skipping trivialities, we can assume n ≥ 2 .
Revised: Skipping trivialities, we can assume n ≥ 2 .

Original: We argue by induction on the number of pairs of distinct classes P
Revised: We argue by induction on the number of pairs of distinct classes P

Original: ( O ) in the instance .
Revised: ( O ) in the instance .

Original: The base case | P ( O ) | = 1 is easily settled by the assumption that for every pair of objects from distinct classes there exists a test splitting them.
Revised: The base case | P ( O ) | = 1 is easily settled by the assumption that for every pair of objects from distinct classes there exists a test splitting them.

Original: So assume by induction that (23) holds for the tree build by the algorithm on every instance I (cid:48) with objects O (cid:48) such that P ( O (cid:48) )
Revised: So assume by induction that (23) holds for the tree build by the algorithm on every instance I (cid:48) with objects O (cid:48) such that P ( O (cid:48) )

Original: < P ( O ) .
Revised: < P ( O ) .

Original: To prove that it still holds for the instance I , we consider whether the algorithm entered the If in Line 2 or not.
Revised: To prove that it still holds for the instance I , we consider whether the algorithm entered the If in Line 2 or not.

Original: Case 1: The algorithm enters the If at Line 2.
Revised: Case 1: The algorithm enters the If at Line 2.

Original: Let ( a, t ) be the split used by the algorithm at the root.
Revised: Let ( a, t ) be the split used by the algorithm at the root.

Original: For i = 1 , 2 , let T i be the subtree of T rooted at the left and right child of the root of T .
Revised: For i = 1 , 2 , let T i be the subtree of T rooted at the left and right child of the root of T .

Original: Recall that T 1 is the tree built on the instance I 1 with object set O 1 :
Revised: Recall that T 1 is the tree built on the instance I 1 with object set O 1 :

Original: = O ( a, ≤ t ) and T 2 is the the tree recursively built on the instance I 2 with object set
Revised: = O ( a, ≤ t ) and T 2 is the the tree recursively built on the instance I 2 with object set

Original: O 2 :
Revised: O 2 :

Original: = O ( a, > t ) .
Revised: = O ( a, > t ) .

Original: By construction we have wpm ( O ( a, t )) ≤ γ · P ( O ) w ( O ) , which means that
Revised: By construction we have wpm ( O ( a, t )) ≤ γ · P ( O ) w ( O ) , which means that

Original: [Equation]
Revised: [Equation]

Original: The average depth of the tree T build by the algorithm can then be decomposed as depth avg ( T ) =
Revised: The average depth of the tree T build by the algorithm can then be decomposed as depth avg ( T ) =

Original: w ( O )+depth avg ( T 1 )+depth avg ( T 2 ) .
Revised: w ( O )+depth avg ( T 1 )+depth avg ( T 2 ) .

Original: Moreover, we also have the following subadditivity property: depth ∗ avg ( I ) ≥ depth ∗ avg ( I 1 )+depth ∗ avg
Revised: Moreover, we also have the following subadditivity property: depth ∗ avg ( I ) ≥ depth ∗ avg ( I 1 )+depth ∗ avg

Original: I 2 ) .
Revised: I 2 ) .

Original: In fact, letting M ∗ (resp.
Revised: In fact, letting M ∗ (resp.

Original: M ∗ 1 , M ∗ 2 ) being a multiway tree achieving depth ∗ avg ( I ) (resp. depth ∗ avg ( I 1 ) , depth ∗ avg ( I 2 ) )
Revised: M ∗ 1 , M ∗ 2 ) being a multiway tree achieving depth ∗ avg ( I ) (resp. depth ∗ avg ( I 1 ) , depth ∗ avg ( I 2 ) )

Original: we have depth ∗ avg ( I )
Revised: we have depth ∗ avg ( I )

Original: = depth avg
Revised: = depth avg

Original: [Equation]
Revised: [Equation]

Original: M
Revised: M

Original: [Equation]
Revised: [Equation]

Original: where we used the notation depth Mavg ( (cid:96) ( o )) to indicate the explanation size of leaf reached by object o in a tree
Revised: where we used the notation depth Mavg ( (cid:96) ( o )) to indicate the explanation size of leaf reached by object o in a tree

Original: M.
Revised: M.

Original: Then we have
Revised: Then we have

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: But depth ∗ avg ( I ) ≥ w ( O ) , and by the inductive hypothesis depth avg ( T i ) depth ∗ avg ( I i ) ≤ α ln( P ( O i ) w ( O i )) , which is at most α ln( γP ( O ) w ( O )) by (33).
Revised: But depth ∗ avg ( I ) ≥ w ( O ) , and by the inductive hypothesis depth avg ( T i ) depth ∗ avg ( I i ) ≤ α ln( P ( O i ) w ( O i )) , which is at most α ln( γP ( O ) w ( O )) by (33).

Original: Plugging these bounds in the RHS of (34) gives depth avg ( T ) depth
Revised: Plugging these bounds in the RHS of (34) gives depth avg ( T ) depth

Original: [Equation]
Revised: [Equation]

Original: where the last inequality follows from the fact α ≥ (ln 1 γ ) − 1 .
Revised: where the last inequality follows from the fact α ≥ (ln 1 γ ) − 1 .

Original: This proves the inductive hypothesis in this case.
Revised: This proves the inductive hypothesis in this case.

Original: Case 2: The algorithm does not enter the If at Line 2.
Revised: Case 2: The algorithm does not enter the If at Line 2.

Original: In this case, the top of the tree T produced by the algorithm uses the splits ( a ∗ , t ∗ − 1) and ( a ∗ , t ∗ ) , and has (up to) three subtrees T 1 , T 2 , T 3 as children of these nodes, where:
Revised: In this case, the top of the tree T produced by the algorithm uses the splits ( a ∗ , t ∗ − 1) and ( a ∗ , t ∗ ) , and has (up to) three subtrees T 1 , T 2 , T 3 as children of these nodes, where:

Original: • T 1 is constructed recursively for the instance I 1 with object set O 1 :
Revised: • T 1 is constructed recursively for the instance I 1 with object set O 1 :

Original: = O ( a ∗ , ≤ t ∗ − 1) • T 2 is constructed recursively for the instance I 2 with object set O 2 :
Revised: = O ( a ∗ , ≤ t ∗ − 1) • T 2 is constructed recursively for the instance I 2 with object set O 2 :

Original: = O ( a ∗ , t ∗ ) • T 3 is constructed recursively for the instance I 3 with object set O 3 :
Revised: = O ( a ∗ , t ∗ ) • T 3 is constructed recursively for the instance I 3 with object set O 3 :

Original: = O ( a ∗ , > t ∗ ) .
Revised: = O ( a ∗ , > t ∗ ) .

Original: (Note that one of the instances I 1 , I 3 may be empty.)
Revised: (Note that one of the instances I 1 , I 3 may be empty.)

Original: The average depth of T can be decomposed based on the contribution of the two splits ( a ∗ , t ∗ − 1) and ( a ∗ , t ∗ ) (which is at most 2 w ( O ) ) and of the subtrees T i ’s, namely
Revised: The average depth of T can be decomposed based on the contribution of the two splits ( a ∗ , t ∗ − 1) and ( a ∗ , t ∗ ) (which is at most 2 w ( O ) ) and of the subtrees T i ’s, namely

Original: [Equation]
Revised: [Equation]

Original: Thus, as in inequality (34) we have
Revised: Thus, as in inequality (34) we have

Original: [Equation]
Revised: [Equation]

Original: where if an instance I i is empty (so the tree T i does not exist) we ignore the corresponding term in the max.
Revised: where if an instance I i is empty (so the tree T i does not exist) we ignore the corresponding term in the max.

Original: By deﬁnition of t ∗ , for i = 1 , 3 the instance
Revised: By deﬁnition of t ∗ , for i = 1 , 3 the instance

Original: I i satisﬁes P
Revised: I i satisﬁes P

Original: ( O i ) w ( O i ) ≤ γP ( O ) w ( O ) , and so applying the induction hypothesis on the instances
Revised: ( O i ) w ( O i ) ≤ γP ( O ) w ( O ) , and so applying the induction hypothesis on the instances

Original: I i we get
Revised: I i we get

Original: [Equation]
Revised: [Equation]

Original: If P ( O 2 ) w ( O 2 ) ≤ γP ( O ) w ( O ) , the max becomes simply α ln( γP ( O ) w ( O )) and again using depth ∗ avg ( I ) ≥ w ( O ) we get depth avg
Revised: If P ( O 2 ) w ( O 2 ) ≤ γP ( O ) w ( O ) , the max becomes simply α ln( γP ( O ) w ( O )) and again using depth ∗ avg ( I ) ≥ w ( O ) we get depth avg

Original: [Equation]
Revised: [Equation]

Original: T
Revised: T

Original: [Equation]
Revised: [Equation]

Original: where the last inequality uses the fact α ≥ 2(ln 1 γ ) − 1 ; so the inductive hypothesis holds in this case.
Revised: where the last inequality uses the fact α ≥ 2(ln 1 γ ) − 1 ; so the inductive hypothesis holds in this case.

Original: Now consider the case where P ( O 2 ) w ( O 2 ) > γP ( O ) w ( O ) , and hence the max becomes α ln( P ( O 2 ) w ( O 2 )) .
Revised: Now consider the case where P ( O 2 ) w ( O 2 ) > γP ( O ) w ( O ) , and hence the max becomes α ln( P ( O 2 ) w ( O 2 )) .

Original: In this case, we use the lower bound on depth ∗ avg ( I ) from Lemma 8 to obtain (let a ∗
Revised: In this case, we use the lower bound on depth ∗ avg ( I ) from Lemma 8 to obtain (let a ∗

Original: and i ∗ be deﬁned as in the lemma)
Revised: and i ∗ be deﬁned as in the lemma)

Original: [Equation]
Revised: [Equation]

Original: where the second inequality follows the fact P ( O ( a ∗ , i ∗ ))
Revised: where the second inequality follows the fact P ( O ( a ∗ , i ∗ ))

Original: w ( O ( a ∗ , i ∗ ))
Revised: w ( O ( a ∗ , i ∗ ))

Original: ≥ P
Revised: ≥ P

Original: O ( a ∗ , t ∗ ) w ( a ∗ , t ∗ ))
Revised: O ( a ∗ , t ∗ ) w ( a ∗ , t ∗ ))

Original: =
Revised: =

Original: P ( O 2 ) w ( O 2 ) (by optimality of i ∗ ), the third inequality follows from the fact 1 − x ≤ − ln x valid for all x > 0 (since by convexity ln x ≥ ln(1) + ln (cid:48) (1)( x − 1) ), and the last inequality follows from the fact α ≥ 4 .
Revised: P ( O 2 ) w ( O 2 ) (by optimality of i ∗ ), the third inequality follows from the fact 1 − x ≤ − ln x valid for all x > 0 (since by convexity ln x ≥ ln(1) + ln (cid:48) (1)( x − 1) ), and the last inequality follows from the fact α ≥ 4 .

Original: This concludes the proof of the inductive step, and hence that inequality (23) holds and as a consequence, we have
Revised: This concludes the proof of the inductive step, and hence that inequality (23) holds and as a consequence, we have

Original: [Equation]
Revised: [Equation]

Original: where W = w ( O ) is the total weight of the objects, w min = min o ∈ S w ( o ) is the smallest weight of an object.
Revised: where W = w ( O ) is the total weight of the objects, w min = min o ∈ S w ( o ) is the smallest weight of an object.

Original: Finally, by the preprocessing step (following the approach from [35]) we have that the approximation term O (cid:16) ln (cid:16) n Ww min (cid:17)(cid:17) ) becomes O (log n ) , i.e., we can remove the dependency on the weights and achieve the desired O (log n ) approximation bound as given in the statement of the theorem.
Revised: Finally, by the preprocessing step (following the approach from [35]) we have that the approximation term O (cid:16) ln (cid:16) n Ww min (cid:17)(cid:17) ) becomes O (log n ) , i.e., we can remove the dependency on the weights and achieve the desired O (log n ) approximation bound as given in the statement of the theorem.

Original: D.2 Proof of Worst Case
Revised: D.2 Proof of Worst Case

Original: We now focus on the proof of the worst case bounds
Revised: We now focus on the proof of the worst case bounds

Original: [Equation]
Revised: [Equation]

Original: for the decision tree T built by SER-DT.
Revised: for the decision tree T built by SER-DT.

Original: Again to simplify the argument, assume w.l.o.g.
Revised: Again to simplify the argument, assume w.l.o.g.

Original: that (after the preprocessing) the weights are integers and w min = 1 .
Revised: that (after the preprocessing) the weights are integers and w min = 1 .

Original: Let P be longest path from the root to a leaf in T .
Revised: Let P be longest path from the root to a leaf in T .

Original: For a node g in P , its weighted pair-wise misclassiﬁcation ( wpm ) is the wpm of the set of objects that reach g .
Revised: For a node g in P , its weighted pair-wise misclassiﬁcation ( wpm ) is the wpm of the set of objects that reach g .

Original: The nodes of P can be split based on whether they were added in Line 3 or Line 8 of one of the recursive calls of SER-DT.
Revised: The nodes of P can be split based on whether they were added in Line 3 or Line 8 of one of the recursive calls of SER-DT.

Original: More speciﬁcally, let G be the set of nodes in P that are associated with the set of objects S ( a ∗ , t ∗ ) at Line 8 of SER-DT.
Revised: More speciﬁcally, let G be the set of nodes in P that are associated with the set of objects S ( a ∗ , t ∗ ) at Line 8 of SER-DT.

Original: We note that the total number of nodes in P is at most
Revised: We note that the total number of nodes in P is at most

Original: [Equation]
Revised: [Equation]

Original: [Equation]
Revised: [Equation]

Original: since in every recursive call where a node in G is added to P , its parent is also added, hence the factor of 2 in the formula; on the other hand, if the recursive call adds a node to P outside of G (Line 3) then it means the condition in Line 2 was satisﬁed, and so the wpm of the node that is added to P is at most half of the wpm of its parent and, thus, only O (log( P ( O ) w ( O )) additions of this type are possible (recall that we are assuming (scaled) integer weights ≥ 1 ).
Revised: since in every recursive call where a node in G is added to P , its parent is also added, hence the factor of 2 in the formula; on the other hand, if the recursive call adds a node to P outside of G (Line 3) then it means the condition in Line 2 was satisﬁed, and so the wpm of the node that is added to P is at most half of the wpm of its parent and, thus, only O (log( P ( O ) w ( O )) additions of this type are possible (recall that we are assuming (scaled) integer weights ≥ 1 ).

Original: Let M ∗ wc be a multiway decision tree achieving the minimum worst-case depth for the instance.
Revised: Let M ∗ wc be a multiway decision tree achieving the minimum worst-case depth for the instance.

Original: We want to relate | G | to depth wc ( M ∗ wc ) .
Revised: We want to relate | G | to depth wc ( M ∗ wc ) .

Original: The following claim lower bounds the latter quantity and is the worst-case version of Lemma 8.
Revised: The following claim lower bounds the latter quantity and is the worst-case version of Lemma 8.

Original: Lemma 9.
Revised: Lemma 9.

Original: Consider a set S ⊆ O of objects of the instance, and let ( a ∗ , i ∗ ) be the pair attribute/value solving the optimization min a max i P ( S ( a, i ))
Revised: Consider a set S ⊆ O of objects of the instance, and let ( a ∗ , i ∗ ) be the pair attribute/value solving the optimization min a max i P ( S ( a, i ))

Original: w ( S ( a, i )) .
Revised: w ( S ( a, i )) .

Original: Then
Revised: Then

Original: [Equation]
Revised: [Equation]

Original: Proof.
Revised: Proof.

Original: Deﬁne the normalized weights w (cid:48)
Revised: Deﬁne the normalized weights w (cid:48)

Original: ( o ) :
Revised: ( o ) :

Original: = w ( o ) w ( S ) , which then form a probability distribution over the objects in S .
Revised: = w ( o ) w ( S ) , which then form a probability distribution over the objects in S .

Original: Let M ∗ avg be a multiway tree for S with minimum average depth with respect to the weights w (cid:48) , i.e. that—over all multiway decision tree M for the instance—minimizes
Revised: Let M ∗ avg be a multiway tree for S with minimum average depth with respect to the weights w (cid:48) , i.e. that—over all multiway decision tree M for the instance—minimizes

Original: [Equation]
Revised: [Equation]

Original: where we included w (cid:48) in depth avg ( M, w (cid:48) ) to avoid confusion.
Revised: where we included w (cid:48) in depth avg ( M, w (cid:48) ) to avoid confusion.

Original: Using the fact that the maximum is always at least the average (w.r.t.
Revised: Using the fact that the maximum is always at least the average (w.r.t.

Original: a probability distribution) and the optimality of M ∗ avg we get
Revised: a probability distribution) and the optimality of M ∗ avg we get

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Moreover, applying Lemma 8 to the instance with objects S and weights w (cid:48)
Revised: Moreover, applying Lemma 8 to the instance with objects S and weights w (cid:48)

Original: we obtain
Revised: we obtain

Original: [Equation]
Revised: [Equation]

Original: where the equality follows from using the fact w (cid:48) ( S )
Revised: where the equality follows from using the fact w (cid:48) ( S )

Original: = 1 to remove one of these terms in the numerator and then multiplying both numerator and denominator by w ( S ) .
Revised: = 1 to remove one of these terms in the numerator and then multiplying both numerator and denominator by w ( S ) .

Original: (Observe that ( a ∗ , t ∗ ) is exactly the one required in this application of Lemma 8 because the solutions to the problems min a max i P ( S ( a, i ))
Revised: (Observe that ( a ∗ , t ∗ ) is exactly the one required in this application of Lemma 8 because the solutions to the problems min a max i P ( S ( a, i ))

Original: w ( S ( a, i )) and min a max i P ( S ( a, i )) w (cid:48)
Revised: w ( S ( a, i )) and min a max i P ( S ( a, i )) w (cid:48)

Original: ( S ( a, i )) are the same, as w (cid:48) is just a scaling of w .)
Revised: ( S ( a, i )) are the same, as w (cid:48) is just a scaling of w .)

Original: Combining (38) and (39) gives the desired result.
Revised: Combining (38) and (39) gives the desired result.

Original: We are now ready to upper bound | G | with respect to the quantity in (39).
Revised: We are now ready to upper bound | G | with respect to the quantity in (39).

Original: For that, let G =
Revised: For that, let G =

Original: ( g 1 , g 2 , . . .
Revised: ( g 1 , g 2 , . . .

Original: , g | G | ) be the nodes in G in order of traversal of P (from root to leaf).
Revised: , g | G | ) be the nodes in G in order of traversal of P (from root to leaf).

Original: For a node g j , consider the call of SER-DT that created this node (in Line 8).
Revised: For a node g j , consider the call of SER-DT that created this node (in Line 8).

Original: Let S j be the set of objects that was received as input by this call, and let ( a ∗ j , t ∗ j ) be the attribute and threshold value used in Line 8 to create node g j .
Revised: Let S j be the set of objects that was received as input by this call, and let ( a ∗ j , t ∗ j ) be the attribute and threshold value used in Line 8 to create node g j .

Original: Thus, the objects of the whole instance that reach node g j (in the tree T ) are precisely
Revised: Thus, the objects of the whole instance that reach node g j (in the tree T ) are precisely

Original: S j ( a ∗ j , t ∗ j ) .
Revised: S j ( a ∗ j , t ∗ j ) .

Original: Applying Lemma 9 to the set of objects S j and reorganizing the terms gives
Revised: Applying Lemma 9 to the set of objects S j and reorganizing the terms gives

Original: [Equation]
Revised: [Equation]

Original: Thus:
Revised: Thus:

Original: [Equation]
Revised: [Equation]

Original: where the last step uses the inequality 1 − 1 x ≤ ln x which is valid for all x ∈ (0 , 1] .
Revised: where the last step uses the inequality 1 − 1 x ≤ ln x which is valid for all x ∈ (0 , 1] .

Original: But by deﬁnition we have S j +1 ⊆
Revised: But by deﬁnition we have S j +1 ⊆

Original: S j ( a j , t j ) , and hence the denominator in the log can be lower bounded as
Revised: S j ( a j , t j ) , and hence the denominator in the log can be lower bounded as

Original: P
Revised: P

Original: [Equation]
Revised: [Equation]

Original: S j ( a ∗ j , i ∗ j ))
Revised: S j ( a ∗ j , i ∗ j ))

Original: w ( S j ( a ∗ j , i ∗ j )) ≥ P ( S j ) w ( S j ) , and hence we obtain the desired upper bound on G :
Revised: w ( S j ( a ∗ j , i ∗ j )) ≥ P ( S j ) w ( S j ) , and hence we obtain the desired upper bound on G :

Original: [Equation]
Revised: [Equation]

Original: Plugging this bound in (37) and using the equivalence between the optimal worst-case explanation size and the optimal worst-case depth for multiway trees from Lemma 2, i.e., expl ∗ wc = depth ∗ wc = depth wc ( M ∗ wc ) , we obtain depth wc ( T ) ≤ 4 expl ∗ wc
Revised: Plugging this bound in (37) and using the equivalence between the optimal worst-case explanation size and the optimal worst-case depth for multiway trees from Lemma 2, i.e., expl ∗ wc = depth ∗ wc = depth wc ( M ∗ wc ) , we obtain depth wc ( T ) ≤ 4 expl ∗ wc

Original: [Equation]
Revised: [Equation]

Original: ln( P ( O ) w ( O ))
Revised: ln( P ( O ) w ( O ))

Original: ≤ O (log n ) · expl ∗ wc , the last inequality holding since P ( O ) ≤ n 2 and, because of the pre-processing step before the algorithm we have w ( O ) ≤ n 4 .
Revised: ≤ O (log n ) · expl ∗ wc , the last inequality holding since P ( O ) ≤ n 2 and, because of the pre-processing step before the algorithm we have w ( O ) ≤ n 4 .

Original: This implies both the bounds in (35)-(36), expl wc ( T ) ≤ O (log n ) · expl ∗ wc and depth wc ( T ) ≤ O (log n ) · depth ∗ wc , thus completing the proof of the lemma.
Revised: This implies both the bounds in (35)-(36), expl wc ( T ) ≤ O (log n ) · expl ∗ wc and depth wc ( T ) ≤ O (log n ) · depth ∗ wc , thus completing the proof of the lemma.

Original: E Experiments – Additional Details and Statistics
Revised: E Experiments – Additional Details and Statistics

Original: Our experiments were executed with the following settings: Notebook Inspiron 14-7460; Processor: 7th generation Intel Core i5; 16GB RAM (DDR4 at 2400 MHz); Storage type: SSD; GPU NVIDIA
Revised: Our experiments were executed with the following settings: Notebook Inspiron 14-7460; Processor: 7th generation Intel Core i5; 16GB RAM (DDR4 at 2400 MHz); Storage type: SSD; GPU NVIDIA

Original: GeForce 940MX.
Revised: GeForce 940MX.

Original: For our implementation we employed Python 3.8.10; pandas 1.4.2; numpy 1.22.3; sklearn 1.0.2.
Revised: For our implementation we employed Python 3.8.10; pandas 1.4.2; numpy 1.22.3; sklearn 1.0.2.

Original: Our code is available on https://github.com/user-anonymous-researcher/interpretable-dts
Revised: Our code is available on https://github.com/user-anonymous-researcher/interpretable-dts

Original: Our datasets as well as their main characteristics are described in Table
Revised: Our datasets as well as their main characteristics are described in Table

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: To generate the plots from Figure 2, we executed SER-DT with FactorExpl varying in the set { 0 .
Revised: To generate the plots from Figure 2, we executed SER-DT with FactorExpl varying in the set { 0 .

Original: 1 × i | i = 1 , . .
Revised: 1 × i | i = 1 , . .

Original: , 9 } ∪ { 0 . 9 + 0 .
Revised: , 9 } ∪ { 0 . 9 + 0 .

Original: × i | i = 1 , . . .
Revised: × i | i = 1 , . . .

Original: In the following sections, we present some additional metrics and visualizations related to the experiments described in Section 6.
Revised: In the following sections, we present some additional metrics and visualizations related to the experiments described in Section 6.

Original: Moreover, we also present two new experiments: in one of them, we remove the constraint on the depth of the trees while in the other we replace the Gini with entropy as a splitting criterion.
Revised: Moreover, we also present two new experiments: in one of them, we remove the constraint on the depth of the trees while in the other we replace the Gini with entropy as a splitting criterion.

Original: We note that each entry presented in the following tables is given by the the average of 10 runs using different seeds to select the examples in the training and testing set.
Revised: We note that each entry presented in the following tables is given by the the average of 10 runs using different seeds to select the examples in the training and testing set.

Original: E.1 Metrics related to depth and running time
Revised: E.1 Metrics related to depth and running time

Original: Table 3 shows the metrics depth avg and depth wc for the decision trees produced by both CART and
Revised: Table 3 shows the metrics depth avg and depth wc for the decision trees produced by both CART and

Original: SER-DT (experiment from Section 6).
Revised: SER-DT (experiment from Section 6).

Original: The metric depth avg for our algorithm is better than that of
Revised: The metric depth avg for our algorithm is better than that of

Original: CART for 17 datasets and it is worse for only 3.
Revised: CART for 17 datasets and it is worse for only 3.

Original: For depth wc the results are similar, which is somehow expected since we set the maximum allowed depth to 6.
Revised: For depth wc the results are similar, which is somehow expected since we set the maximum allowed depth to 6.

Original: Table 4 shows the average running times in seconds of CART and our algorithm.
Revised: Table 4 shows the average running times in seconds of CART and our algorithm.

Original: The ratio between the fastest and slowest is at most 2 for all datasets.
Revised: The ratio between the fastest and slowest is at most 2 for all datasets.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: E.2 Sensitivity to FactorExpl
Revised: E.2 Sensitivity to FactorExpl

Original: Figure 2 suggests that FactorExpl can be used to provide a trade-of between accuracy and explainability (measured according to expl avg ).
Revised: Figure 2 suggests that FactorExpl can be used to provide a trade-of between accuracy and explainability (measured according to expl avg ).

Original: To provide additional evidence, in Table
Revised: To provide additional evidence, in Table

Original: [Table]
Revised: [Table]

Original: We observe that expl avg has a very predictable behavior: the smaller the FactorExpl the smaller the expl avg (last 3 columns of this table).
Revised: We observe that expl avg has a very predictable behavior: the smaller the FactorExpl the smaller the expl avg (last 3 columns of this table).

Original: In terms of accuracy, for 13 datasets the difference between
Revised: In terms of accuracy, for 13 datasets the difference between

Original: FactorExpl = 0 .
Revised: FactorExpl = 0 .

Original: 99 and FactorExpl = 0 .
Revised: 99 and FactorExpl = 0 .

Original: 9 is smaller than 1% .
Revised: 9 is smaller than 1% .

Original: This number increases to 16 when we consider FactorExpl = 0 . 99 and FactorExpl = 0 .
Revised: This number increases to 16 when we consider FactorExpl = 0 . 99 and FactorExpl = 0 .

Original: We do not recommend using small values for FactorExpl ( < 0 . 9 ) in practice because the loss in terms of accuracy may be severe.
Revised: We do not recommend using small values for FactorExpl ( < 0 . 9 ) in practice because the loss in terms of accuracy may be severe.

Original: In fact, we recommend using SER-DT with FactorExpl ∈ [0 . 95 , 0 . 99] .
Revised: In fact, we recommend using SER-DT with FactorExpl ∈ [0 . 95 , 0 . 99] .

Original: E.3 Removing the maximum depth constraint
Revised: E.3 Removing the maximum depth constraint

Original: In our experiments, in order to prevent (very) large trees, we set the maximum allowed depth to 6.
Revised: In our experiments, in order to prevent (very) large trees, we set the maximum allowed depth to 6.

Original: Table 6 and 7 show the results when this constraint is removed.
Revised: Table 6 and 7 show the results when this constraint is removed.

Original: Some observations are in order:
Revised: Some observations are in order:

Original: • SER-DT builds trees that are much more shallower than those built by CART (Columnsand 7 of Table 7).
Revised: • SER-DT builds trees that are much more shallower than those built by CART (Columnsand 7 of Table 7).

Original: This was not possible to observe when the maximum depth was set to 6; • The gains of SER-DT over CART for all explainability metrics become larger when the depth constraint is removed.
Revised: This was not possible to observe when the maximum depth was set to 6; • The gains of SER-DT over CART for all explainability metrics become larger when the depth constraint is removed.

Original: E.4 Entropy as a split criterion
Revised: E.4 Entropy as a split criterion

Original: In our experiments on Section 6 we compared SER-DT against CART .
Revised: In our experiments on Section 6 we compared SER-DT against CART .

Original: We also implemented a variation of our method that employs the Shannon Entropy, rather than Gini , to evaluate the goodness of a split.
Revised: We also implemented a variation of our method that employs the Shannon Entropy, rather than Gini , to evaluate the goodness of a split.

Original: We note that (a normalized version of) the entropy is employed by the widely used C4.
Revised: We note that (a normalized version of) the entropy is employed by the widely used C4.

Original: algorithm [48].
Revised: algorithm [48].

Original: We tested our variation against a variation of CART that uses the entropy rather than
Revised: We tested our variation against a variation of CART that uses the entropy rather than

Original: Gini to evaluate the quality of a split and also to determine whether a leaf shall be expanded or not (stopping rule).
Revised: Gini to evaluate the quality of a split and also to determine whether a leaf shall be expanded or not (stopping rule).

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: E.5 Boxplots for CART and SER-DT
Revised: E.5 Boxplots for CART and SER-DT

Original: Figures 5-8 present boxplots for the test accuracy of CART and SER-DT for the experiments described on Section 6.
Revised: Figures 5-8 present boxplots for the test accuracy of CART and SER-DT for the experiments described on Section 6.

Original: Figures 9-12 present boxplots for the expl avg .
Revised: Figures 9-12 present boxplots for the expl avg .

Original: E.6 Other examples of trees
Revised: E.6 Other examples of trees

Original: We present some other examples of trees produced by SER-DT and CART .
Revised: We present some other examples of trees produced by SER-DT and CART .

Original: These examples provide additional evidence that our algorithm generates trees with better explainability but similar accuracy compared to CART .
Revised: These examples provide additional evidence that our algorithm generates trees with better explainability but similar accuracy compared to CART .

Original: All trees were constructed by setting a maximum allowed depth to 4.
Revised: All trees were constructed by setting a maximum allowed depth to 4.

Original: For SER-DT we set FactorExpl = 0 .
Revised: For SER-DT we set FactorExpl = 0 .

Original: 97 , such as reported on Section 6.
Revised: 97 , such as reported on Section 6.

Original: Our ﬁrst example employs dataset default credit card .
Revised: Our ﬁrst example employs dataset default credit card .

Original: Figures 13 and 14 show the trees produced by CART and SER-DT, respectively.
Revised: Figures 13 and 14 show the trees produced by CART and SER-DT, respectively.

Original: CART has a Test Accuracy of 81 .
Revised: CART has a Test Accuracy of 81 .

Original: 9% , while SER-DT obtains 81 .
Revised: 9% , while SER-DT obtains 81 .

Original: In contrast, visually the SER-DT tree is much simpler than that of CART , and SER-DT achieves expl avg = 1 .
Revised: In contrast, visually the SER-DT tree is much simpler than that of CART , and SER-DT achieves expl avg = 1 .

Original: 19 against expl avg = 2 .
Revised: 19 against expl avg = 2 .

Original: 27 from CART .
Revised: 27 from CART .

Original: Our second example uses dataset online shoppers intention , whose corresponding decision trees are represented in Figures 15 and 16.
Revised: Our second example uses dataset online shoppers intention , whose corresponding decision trees are represented in Figures 15 and 16.

Original: CART generates a tree with a Test Accuracy of 90 . 0% , while SER-DT achieves 89 .
Revised: CART generates a tree with a Test Accuracy of 90 . 0% , while SER-DT achieves 89 .

Original: 1% for this metric.
Revised: 1% for this metric.

Original: In this case, SER-DT also clearly yields a simpler tree, achieving depth avg = 1 .
Revised: In this case, SER-DT also clearly yields a simpler tree, achieving depth avg = 1 .

Original: against depth avg = 2 . 38 from CART .
Revised: against depth avg = 2 . 38 from CART .

Original: The ﬁnal example shows trees for dataset dry bean , in Figures 17 and 18.
Revised: The ﬁnal example shows trees for dataset dry bean , in Figures 17 and 18.

Original: Although at ﬁrst glance there doesn’t seem to be much improvement, SER-DT yields a tree with expl avg of 2 . 60 , in contrast to 3 . 49 for CART , which is a reduction of 25 . 6% .
Revised: Although at ﬁrst glance there doesn’t seem to be much improvement, SER-DT yields a tree with expl avg of 2 . 60 , in contrast to 3 . 49 for CART , which is a reduction of 25 . 6% .

Original: In terms of prediction, SER-DT gives 82 .
Revised: In terms of prediction, SER-DT gives 82 .

Original: 8% of Test
Revised: 8% of Test

Original: Accuracy, while CART gives 82 . 4% .
Revised: Accuracy, while CART gives 82 . 4% .

Original: [Table]
Revised: [Table]

Original: E.7 Comparision between SER-DT and EC
Revised: E.7 Comparision between SER-DT and EC

Original: As noted in Section 5 and Section 6, SER-DT combines the theoretically best possible guarantee with very competitive performance in practice.
Revised: As noted in Section 5 and Section 6, SER-DT combines the theoretically best possible guarantee with very competitive performance in practice.

Original: The results shown in this section give evidence that in general optimal approximation guarantee on the (average) depth does not imply good results in terms of explanation size and accuracy.
Revised: The results shown in this section give evidence that in general optimal approximation guarantee on the (average) depth does not imply good results in terms of explanation size and accuracy.

Original: Table 9 presents the comparison between the EC 2 algorithm and our SER-DT, with respect to accuracy and explanation size.
Revised: Table 9 presents the comparison between the EC 2 algorithm and our SER-DT, with respect to accuracy and explanation size.

Original: EC 2 is a greedy algorithm proposed by [22] which is proved to achieve O (log n ) - approximation guarantee for the minimization of the average depth.
Revised: EC 2 is a greedy algorithm proposed by [22] which is proved to achieve O (log n ) - approximation guarantee for the minimization of the average depth.

Original: Note that this means that it attains the best approximation guarantee that it is possible in polynomial time, under the hypothesis
Revised: Note that this means that it attains the best approximation guarantee that it is possible in polynomial time, under the hypothesis

Original: P (cid:54) = NP.
Revised: P (cid:54) = NP.

Original: In terms of accuracy SER-DT performs signiﬁcantly better than EC 2 on almost all datasets, and, similarly, in terms of the interpretability metrics the results also show a signiﬁcant difference in favour of our algorithm.
Revised: In terms of accuracy SER-DT performs signiﬁcantly better than EC 2 on almost all datasets, and, similarly, in terms of the interpretability metrics the results also show a signiﬁcant difference in favour of our algorithm.

Original: More precisely, on 14 datasets (bold-faced on columns 2 and 3) we observe a difference larger than 1% in terms of accuracies; on 13 of them, our algorithm outperforms ECwhile only on 1, EC 2 is better.
Revised: More precisely, on 14 datasets (bold-faced on columns 2 and 3) we observe a difference larger than 1% in terms of accuracies; on 13 of them, our algorithm outperforms ECwhile only on 1, EC 2 is better.

Original: When considering the average explanation size, our algorithm is better than EC 2 on 14 datasets, and for 8 of them (bold-faced on column 4), it improves on the expl avg of
Revised: When considering the average explanation size, our algorithm is better than EC 2 on 14 datasets, and for 8 of them (bold-faced on column 4), it improves on the expl avg of

Original: EC 2 by at least 25%.
Revised: EC 2 by at least 25%.

Original: Finally, in terms of expl wc , on 15 datasets our algorithm is at least as good as
Revised: Finally, in terms of expl wc , on 15 datasets our algorithm is at least as good as

Original: EC 2 , and on 5 of them the improvement is at least 25% (bold-faced in column 6).
Revised: EC 2 , and on 5 of them the improvement is at least 25% (bold-faced in column 6).

Original: [Table] [Table]
Revised: [Table] [Table]

Original: avila 61.
Revised: avila 61.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: banknote 97.
Revised: banknote 97.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: bankruptcy polish 96.
Revised: bankruptcy polish 96.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: cardiotocography
Revised: cardiotocography

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: collins
Revised: collins

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: default credit card
Revised: default credit card

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: dry bean 90.
Revised: dry bean 90.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: eeg eye state 74.
Revised: eeg eye state 74.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: htru 97.
Revised: htru 97.

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: iris
Revised: iris

Original: [Table] [Table]
Revised: [Table] [Table]

Original: [Table]
Revised: [Table]

Original: letter recognition
Revised: letter recognition

Original: [Table] [Table]
Revised: [Table] [Table]

Original: mice 99.
Revised: mice 99.

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: obs network 91.
Revised: obs network 91.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Table 10 shows the analogous results for the metrics depth avg and depth wc .
Revised: Table 10 shows the analogous results for the metrics depth avg and depth wc .

Original: We bold-faced values in column 4 and 6 that show an improvements by at least 25% over EC 2 .
Revised: We bold-faced values in column 4 and 6 that show an improvements by at least 25% over EC 2 .

Original: We notice that fordatasets our algorithm has better depth avg than EC 2 .
Revised: We notice that fordatasets our algorithm has better depth avg than EC 2 .

Original: Moreover, considering those where SER-DT’s performance is worse, on only one the difference is larger than 25% (bankruptcy polish).
Revised: Moreover, considering those where SER-DT’s performance is worse, on only one the difference is larger than 25% (bankruptcy polish).

Original: The values for the metric depth wc are similar for both algorithms, which is likely due to having set an upper bound of 6 on the maximum depth.
Revised: The values for the metric depth wc are similar for both algorithms, which is likely due to having set an upper bound of 6 on the maximum depth.

Original: Nonetheless, we observe that for 2 datasets the performance of
Revised: Nonetheless, we observe that for 2 datasets the performance of

Original: SER-DT is signiﬁcantly better (audit risk and mice).
Revised: SER-DT is signiﬁcantly better (audit risk and mice).

Original: E.8 Post-pruning experiments
Revised: E.8 Post-pruning experiments

Original: In this section
Revised: In this section

Original: [Table]
Revised: [Table]

Original: we present the results obtained through the application of a post-pruning strategy for trees.
Revised: we present the results obtained through the application of a post-pruning strategy for trees.

Original: The algorithm consisted of checking if merging two leaves would not decrease the accuracy in a validation set formed by half of the original test set.
Revised: The algorithm consisted of checking if merging two leaves would not decrease the accuracy in a validation set formed by half of the original test set.

Original: If so, these leaves are joined and the algorithm proceeds recursively to the root.
Revised: If so, these leaves are joined and the algorithm proceeds recursively to the root.

Original: Table 11 presents the results obtained before and after post-pruning for SER-DT, limiting the maximum depth to 6 and FactorExpl = 0 .
Revised: Table 11 presents the results obtained before and after post-pruning for SER-DT, limiting the maximum depth to 6 and FactorExpl = 0 .

Original: We observed that post-pruning did not have a signiﬁcant effect in terms of accuracy, leading to differences of not more than 0 .
Revised: We observed that post-pruning did not have a signiﬁcant effect in terms of accuracy, leading to differences of not more than 0 .

Original: 5% for all datasets.
Revised: 5% for all datasets.

Original: In terms of explanation size, post-pruning produces smaller expl avg for all datasets (as expected, since we are joining nodes) and for 4 the gain was more than 20% (bold-faced in table).
Revised: In terms of explanation size, post-pruning produces smaller expl avg for all datasets (as expected, since we are joining nodes) and for 4 the gain was more than 20% (bold-faced in table).

Original: Similarly, the pruning algorithm led to no worse values of expl wc for all datasets
Revised: Similarly, the pruning algorithm led to no worse values of expl wc for all datasets

Original: [Table]
Revised: [Table]

Original: and for 3 of these, the difference was greater than 20%
Revised: and for 3 of these, the difference was greater than 20%

Original: [Table] [Table]
Revised: [Table] [Table]

Original: Table 12 is similar to the previous table but shows metrics depth avg and depth wc .
Revised: Table 12 is similar to the previous table but shows metrics depth avg and depth wc .

Original: For all datasets, there is no worsening in these metrics and for 6 of these there is an improvement of more than 20% in depth avg , while for 1 dataset there is an improvement of more than 20% in depth wc .
Revised: For all datasets, there is no worsening in these metrics and for 6 of these there is an improvement of more than 20% in depth avg , while for 1 dataset there is an improvement of more than 20% in depth wc .

Original: Tables 13 and 14 show the results for CART and SER-DT after applying post-pruning.
Revised: Tables 13 and 14 show the results for CART and SER-DT after applying post-pruning.

Original: We observe similar accuracy values for both algorithms.
Revised: We observe similar accuracy values for both algorithms.

Original: On 7 of the datasets (bold-faced in columns 2 and 3) we observe a difference of more than 1% in accuracies; on 3 of them, SER-DT outperforms CART while on the remaining 4, CART is better.
Revised: On 7 of the datasets (bold-faced in columns 2 and 3) we observe a difference of more than 1% in accuracies; on 3 of them, SER-DT outperforms CART while on the remaining 4, CART is better.

Original: In terms of expl avg and expl wc , our algorithm is clearly better.
Revised: In terms of expl avg and expl wc , our algorithm is clearly better.

Original: Only for 2 datasets CART had better expl avg than SER-DT and for 7 datasets our algorithm outperforms CART by more than 20% .
Revised: Only for 2 datasets CART had better expl avg than SER-DT and for 7 datasets our algorithm outperforms CART by more than 20% .

Original: Regarding expl wc our algorithm was worst for only 3 datasets, and for 4 the improvement was more than 20% compared to CART .
Revised: Regarding expl wc our algorithm was worst for only 3 datasets, and for 4 the improvement was more than 20% compared to CART .

Original: For depth-related metrics, we noted more balanced values between the two algorithms when comparing to Table 3 (without post-pruning).
Revised: For depth-related metrics, we noted more balanced values between the two algorithms when comparing to Table 3 (without post-pruning).

Original: As shown in Table 14, considering depth avg our algorithm is better than CART for 12, worst for 6 and equal for 2 datasets.
Revised: As shown in Table 14, considering depth avg our algorithm is better than CART for 12, worst for 6 and equal for 2 datasets.