Title: Feature Learning for Interpretable, Performant Decision Trees
Venue: NeurIPS 2023
Accepted: yes
Keywords: explainability

interpretability

decision tree

feature learning
Abstract: Decision trees are regarded for high interpretability arising from their hierarchical partitioning structure built on simple decision rules. However, in practice, this is not realized because axis-aligned partitioning of realistic data results in deep trees, and because ensemble methods are used to mitigate overfitting. Even then, model complexity and performance remain sensitive to transformation of the input, and extensive expert crafting of features from the raw data is common. We propose the first system to alternate sparse feature learning with differentiable decision tree construction to produce small, interpretable trees with good performance. We benchmark against conventional tree-based models and demonstrate several notions of interpretation of a model and its predictions.

Feature Learning for Interpretable, Performant Decision Trees

Abstract

Decision trees are regarded for high interpretability arising from their hierarchical partitioning structure built on simple decision rules. However, in practice, this is not realized because axis-aligned partitioning of realistic data results in deep trees, and because ensemble methods are used to mitigate overfitting. Even then, model complexity and performance remain sensitive to transformation of the input, and extensive expert crafting of features from the raw data is common. We propose the first system to alternate sparse feature learning with differentiable decision tree construction to produce small, interpretable trees with good performance. It benchmarks favorably against conventional tree-based models and demonstrates several notions of interpretability of a model and its predictions.

1 Introduction

In recent years, AI has made its way into increasingly critical applications, leading to the popularization of the topic of Trustworthy AI ; AI should not only perform well in its primary task, but exhibit properties that earn trust, such as robustness, provable adherence to safety specifications, and interpretability to humans. For example, predictors that assist medical diagnosis are more useful and more likely to be trusted if their predictions are easily understood by medical personnel.

Interpretability is highly subjective and application-dependent by nature, and there is no universal standard of what makes a system highly interpretable. However, within machine learning (ML), there are some generally agreed-upon rules of thumb; for example, smaller models are considered more interpretable than structurally similar but larger ones, and rule-based representations such as decision trees are considered more interpretable than 'black box' models such as neural networks.

Decision trees in particular are often used as the quintessential example of interpretable ML; they can be interpreted holistically (also called global interpretation ) as a hierarchical partitioning of the input based on simple decision rules, and individual predictions can be interpreted (also called called local interpretation ) as a series of such rules on a path from the tree's root to a leaf. Conventionally, these decision rules have the form x i < t for some feature i of input vector x and decision threshold t , thus representing axis-aligned hyperplanes in the input space, but many other decision rules are possible. For example, rules of the form ∑ i a i x i < b for some a , b are called oblique splits, and the trees using them oblique decision trees (ODTs).

In practice, however, there are barriers to the practical application of decision trees as interpretable models. First, due to various factors such as greedy growth, axis-aligned decision rules, and the lack of any objective to promote robust, large-margin predictions, they tend to generalize poorly to yet-unseen data; while this can be somewhat alleviated by pruning, the most effective and popular solution is to use large ensembles with strategies such as bagging as in Random Forests [9], random thresholds as in ExtraTrees [22], or boosting as in XGBoost [11]. In addition, decision trees often grow quite deep and use many decision rules for any given prediction, even for extremely simple data. Together, these make both global and local interpretation much more difficult.

其

X

X

1

X

We posit that these barriers are due, at least in part, to the sensitivity of decision trees to transformations of the input resulting from greedy construction and simple decision rules. Of these, key limitation is the latter; even if we replace greedy construction with a perfect tree learner, simple distributions can nonetheless require an arbitrarily large axis-aligned tree to fit. However, after a transformation of that distribution, even a greedy algorithm can produce a small tree, as shown in Section 3.5. Figure 1 gives a motivating example of a case where trees do poorly as interpretable learners on simple data, but succeed after a transformation. The sensitivity to the feature representation is also related to other limitations of tree-based models, such as their dependence on features hand-crafted using expert knowledge to achieve good performance on some problems, and their overall weakness on other problems, such as image classification.

This motivates the idea of learning transformations of the input features, or equivalently, more expressive decision rules such as oblique splits, to strike a balance between complexity of the rules and complexity of the overall model. This learning problem is not easy; axis-aligned decision rule learning, as in the popular CART algorithm [8], is only practical because the search is restricted to a finite set of candidate rules that is efficiently exhaustively searchable. However, finding even optimal oblique splits, which are among the simplest decision rules, is NP-Hard [26], and even oblique splits may not be considered interpretable unless they are sparse, that is, having few nonzero coefficients.

Nonetheless, methods have been developed to learn such trees, either by heuristically choosing splits in a tree construction process, or by constructing an axis-aligned tree and then optimizing the parameters of the splits (see Section 2 for examples). However, these suffer from either greedy, suboptimal choice of split, or from keeping the original structure of the tree. They are also usually restricted to a certain kind of split, such as oblique splits.

Here we propose a novel, flexible framework that is the first to alternate optimization of feature transformation parameters with complete regrowth of the tree. This is made possible by recent advances in fuzzy decision tree methods. Refitting the tree completely throughout training reshapes the tree and moves thresholds as needed; it also maintains the property that each splitting rule is maximally informative given the available features and data. Together these result in smaller, more interpretable trees for a given performance. Moreover, the flexibility of this framework to support various feature transformations provides a foundation for future work focusing on interpretable prediction with other data modalities, such as images and time series.

2 Related Work

Decision trees are one of the oldest techniques in machine learning still widely in use today. It has long been known that constructing optimal binary decision trees is NP-Complete [29], so most algorithms use a greedy approach to 'grow' a tree from root to leaf by successively partitioning the data by choosing the split that maximally improves the purity of labels in each partition. The partitions are usually axis-aligned, that is, involving only one feature of the input, to make the search practical. One of the most popular decision tree algorithms today is Classification And Regression Trees (CART) [8]. Unconstrained decision trees can always fit separable data perfectly, but are prone to overfitting; to improve generalization, growth can be constrained, but more often, trees are used in

randomized ensembles such as random forests [9] and ExtraTrees [22], or in boosted ensembles such as AdaBoost [19] and XGboost [11].

There is extensive work using decision trees extracted from more complex models either to entirely replace the model, or as an interpretable proxy. Examples include [12, 37, 43, 3, 35, 36, 25, 7].

Other work attempts to improve the interpretability of decision trees themselves by introducing new decision rules. These rules are more complex than the axis-aligned threshold rules, but compensate by improving performance of single-tree models while reducing their size. For example, [6] uses a branch-and-bound algorithm to search for bivariate splits. [16, 17, 23] use parallel coordinates, which incorporate information from multiple data features while being easy to visualize. [13] uses bilevel optimization to find nonlinear splits for growing trees.

The most common augmented decision rule, however, is linear splits, that is, splitting using arbitrary hyperplanes rather than axis-aligned hyperplanes. These are often called oblique splits, and the trees that use them oblique decision trees (ODTs). Algorithms for growing ODTs are first proposed along with CART [8]. [34] later discusses the difficulty of finding optimal linear splits due to local minima and proposes a heuristic using randomization and deterministic hill climbing.

Another approach is to grow an axis-aligned tree, fix the structure, then treat it as a parametric model in order to learn coefficients for linear splits. This can be made possible using fuzzy splits, wherein a decision is a weighted combination of both subtrees rather than wholly one or the other, to make the tree differentiable. These have gone by names such as fuzzy, soft, neural, or differentiable decision trees; we use fuzzy decision trees (FDTs) as an umbrella term. In the case of ODTs, a splitting function such as sigmoid is used to map a linear combination of feature values to the weight of each subtree. [42] is the first to train globally optimized trees in this way. Later, [32] proposes ensembles of such trees, both as standalone models and as the final layer of a neural architecture, [33] presents various improvements such as balance and sparsity regularizations and dynamic learning rate adjustments, and [40] applies them to reinforcement learning.

Tree Alternating Optimization (TAO) is a recent and successful ODT learning algorithm that initializes similarly, but unlike FDTs, uses a linear discriminator at each node, such as logistic regression or linear SVM, then alternates optimization over each depth level of the tree. It can be used with a sparsity penalty to produce sparse ODTs, that is, where many or most of the coefficients are 0, making linear splits more interpretable. While it is shown that this alternating optimization never causes the loss to increase, there are no convergence or approximation guarantees. It was initially proposed for classification [10], then extended in various ways, including regression ensembles [45]; boosted ensembles [21]; clustering [20]; semi-supervised learning [46]; interpretable image classification [41]; interpretable natural language processing [28]; and as a tool for understanding parts of neural networks [27].

Our approach is different from any existing work in that we consider a broader scope of feature types, but more importantly, in that we continually and completely refit the decision tree throughout the learning process; this requires an efficient differentiable decision tree. For this, we rely on Kernel Density Decision Trees (KDDTs) [24], a recently proposed FDT that represents uncertainty through kernels rather than splitting functions and learns axis-aligned fuzzy splits using a generalization of CART. The kernel representation also has the benefit of avoiding redundant splitting that may occur in other FDT formalisms because repeating similar splits can make the division less 'fuzzy' and therefore reduce loss by inflating the tree size. See the paper for background on axis-aligned FDTs. Our approach is also unique in that, for each output of the feature transformation, we weight the regularization by how much the feature is used for decisions in the training data, resulting in a more balanced enforcement of sparsity throughout the tree.

3 Methods

3.1 Alternating optimization trees and features

Given training data with features X ∈ R n × p and regression targets or one-hot class label vectors Y ∈ R n × q , we aim to learn parameters θ ∈ R d of feature transformation f θ : R p → R p ′ and a decision tree T : R p ′ → R q to minimize a loss function L ( θ, T, X , Y ) for the composed predictive

model T ◦ f θ . The tree is typically subject to constraints such as a max depth, minimum sample weight per node, or cost-complexity pruning.

Viewed within this framework, the prevailing approach in prior work is equivalent to the following process: (1) initialize θ such that f θ is identity; (2) with θ fixed, optimize T using a greedy tree growth strategy such as CART; (3) with T fixed, optimize θ using gradient-based optimization. Some methods use a growing and/or pruning heuristic during (3) so that the structure of T may change.

Instead, we propose an approach inspired by alternating optimization, a common strategy to pragmatically solve difficult optimization problems, whereby we alternate fitting a tree as in (2) with one or more gradient-based updates to θ as in (3). In this case, alternating allows us to use different optimization strategies for the feature transform parameters θ and the non-parametric decision tree T . At the end, we can use the fuzzy tree T , or replace its fuzzy splits with conventional crisp splits; the latter improves interpretability when predictions would otherwise take multiple paths, but can result in a small drop in performance.

3.2 Efficient differentiable trees

Conventional decision trees are piecewise-constant, so there is no meaningful loss gradient with respect to θ . Instead, we use kernel density decision trees (KDDTs) [24], a type of fuzzy decision tree that interprets inputs with local uncertainty defined by a kernel function. In essence, they fit using a kernel density estimate computed from X rather than the raw values, and predictions are smoothed over the same kernel. This smoothing makes both the predictions and loss differentiable. KDDTs are fitted using an efficient generalization of CART, making them suitable for applications such as this where frequently repeated fitting makes efficiency crucial.

Here we present a simplified formulation of KDDTs to focus on this use case. A KDDT T : R p → R q is defined by underlying decision tree T 0 : R p → R q and product kernel k : R p → R with k ( x ) = ∏ p i =1 F ′ i ( x i ) , and F i the marginal CDF of the kernel along each dimension i . For the KDDT fitting algorithm, each F ′ i must be piecewise-constant, but this restriction does not apply for prediction; for example, one might use a histogram approximation of a Gaussian distribution for fitting and the Gaussian itself for prediction.

Then the KDDT is the output element-wise convolution T ( j ) ≡ T ( j ) 0 ∗ k . Suppose T 0 partitions R p into hyper-interval leaves L ⊆ R p with bounds ℓ L,i , u L,i (which may be -∞ or ∞ ) on each dimension i . Each leaf has a value v L ∈ R q . Then the prediction is

where µ L is the leaf membership function µ L ( x ) = ∏ i ∈ [ p ] F i ( x i -ℓ L,i ) -F i ( x i -u L,i ) and v L ∈ R q is the leaf value v ( j ) L = ∑ i ∈ [ n ] µ L ( X i ) Y i,j ∑ i ∈ [ n ] µ L ( X i ) . The Jacobian is thus

̸

using the same traversal algorithm to compute (1) from [24]. Trees are fitted using an information gain based on an impurity measure, so the same impurity should be used as the loss function for feature learning. For classification, we use the Gini index; the resulting loss function, including the feature transformation f , is L ( θ, T, X , Y ) = 1 n ∑ L w L (1 -v T L v L ) with w L = ∑ i ∈ [ n ] µ L ( f θ ( X i )) the sample weight at leaf L . Then we compute the loss gradient ∇ θ L in terms of µ , ∇ µ , and ∇ θ f , using the sparsity of µ for efficient computation during tree traversal. For regression, we use variance of regression target values as the impurity, which can be expressed as a shifted version of the Gini loss, so the gradient computation is the same.

3.3 Kernel choice

In this context, the kernel is mainly used as a smoother to make the tree differentiable, so the choice of shape is not crucial. Good choices are a uniform (box) or Gaussian kernel. However, the size of

the kernel relative to the scale of the input is important; we adopt the term bandwidth from kernel density estimation to describe kernel scaling. If the bandwidth is too small, a KDDT is very close to a conventional decision tree, and the loss gradient is either zero or highly unstable because it results from very few data near split boundaries; if the bandwidth is too large, however, splits are very soft, the tree grows large, the discriminative power is weak, and computation is slow due to dense membership of data in leaves. To control the bandwidth, we use one of two strategies: (1) design the feature transform so that the output range is limited, for example, to [0 , 1] , and choose an appropriate kernel bandwidth; or (2) use a regularization on the feature transform parameters to automatically scale the feature outputs appropriately relative to the kernel. We generally find the latter approach to be more effective and describe the use of regularizations in detail in Section 3.5.

Another related consideration is the support of the kernel, that is, the size of the range on which it has nonzero density. In general, the larger the support, the more decision paths that data belong to, increasing the cost of fitting, gradient computation, and prediction (if using a kernel during prediction). For example, a Gaussian kernel truncated at 3 standard deviations and another with the same variance but truncated at 6 standard deviations are nearly identical, but the latter is more computationally expensive to use with a KDDT.

3.4 Parameterized feature transforms

In principle, any differentiable parameterized class of feature transforms can be used, ranging from simple linear transforms as in ODTs to expressive classes of functions such as MLPs that have high discriminative power on their own. Generally, more expressive feature transforms result in smaller trees, but are probably less interpretable, depending on the application. The goal then is to choose expressive feature classes that remain interpretable in the desired context when conjoined into a rule list, as in the decision path of a tree.

In this work, we focus primarily on tabular data, to which tree-based models are most successfully applied, and suggest some general feature transformation primitives based on feature engineering techniques typically used with tree-based models. These can be composed, or their outputs concatenated so different kinds of transforms can be used together in the tree. We generally recommend starting with linear and prototype features, then proceeding with whatever improves performance while meeting interpretability requirements for the application.

Before describing the primitives, we note a general consideration for features used with decision trees: decision trees consider only the order of inputs and not the distance between values. While this is not strictly true for fuzzy decision trees such as KDDTs, it is best to avoid adding unnecessary feature complexity by using element-wise monotonic transformations of features, such as bias terms, monotonic activation functions, etc., on the final output of the feature transformation.

Identity. It may be beneficial to include the original, unmodified features along with transformed features so that the simplest possible rules are considered during construction of the tree. This also, in some sense, safeguards against situations where transformed features may actually be less informative than the original features.

Element-wise transformation. As noted above, element-wise monotonic transformations are largely meaningless on their own, but can be useful if composed with other transforms. For example, the composition of element-wise square with linear transformation allows for conic decision rules. Some element-wise mappings worth considering have no parameters, such as x ↦→ exp( x ) and x ↦→ log( x ) ; others have parameters to be learned, such as x ↦→ x a . A notable option is the element-wise shifted power transformation

̸

which is used to alter distribution shape, typically to make values more normally distributed.

Linear transformation. The linear transformation is x ↦→ Ax + b with weights A and bias b . The bias is used only if composing with another primitive. Using only a linear transformation results in an ODT. These can be initialized randomly, as identity, or using a linear dimension reduction strategy such as PCA.

Distance to prototype. Another feature type maps a point to its distance to each of k prototypes p i ; in this sense, a decision rule d ( x , p i ; S i ) < t can be interpreted simply as 'is x sufficiently similar to p i '. We use Mahalanobis distance d ( x , p i ; S i ) = ( x -p i ) T S -1 i ( x -p i ) , where the learnable parameters are prototypes p i and inverse covariance matrices S -1 i . The covariance matrix can be reparameterized in various ways to restrict the measurement of distance 1 , for example, by making it diagonal, or by using the same for all prototypes. It should also be parameterized such that it is positive semidefinite; otherwise hyperbolic decision rules may be learned and the distance interpretation no longer applies. S -1 i can be regularized for sparsity, so that the distance is based on few features. The parameters can be initialized randomly or by using a mixture modeling algorithm.

Fuzzy cluster membership. Clustering partitions data into k clusters with high internal similarity; usually each cluster i is defined by a center c i , and each point belongs to the cluster with the closest center. Conventional clustering is not differentiable, so we instead use fuzzy clustering, a variant that assigns a degree of membership to each cluster, with the membership values summing to 1. Given distance functions d i , i ∈ [ k ] , the membership in cluster i is

with m ∈ (1 , ∞ ) a hyperparameter determining the 'softness' of the cluster assignment, usually just set to 2, and d the Mahalanobis distance as defined previously. In this way, soft clustering can be viewed as a transformation of the 'distance to prototype' features so that the resulting transformed features are interdependent; a decision rule w i ( x ) > t is interpreted as ' x is sufficiently closer to c i than the other centers'. The parameters can be initialized randomly or by using a fuzzy clustering algorithm such as fuzzy c -means [15, 4] with all S i initialized as identity.

3.5 Regularization

For each transformed feature j ∈ [ p ′ ] , the parameters θ j ⊆ θ used to compute the j th feature f ( j ) may be subject to a regularization weighted by the total usage of feature j , that is, the sum of total training sample weights over nodes that split using feature j . This weighting serves to balance the effects of the loss and regularization; a feature used in the decisions for relatively few data will have

a relatively smaller loss gradient, and so should have a proportionally smaller regularization, and vice versa.

One reason to apply regularization is to improve the interpretability of features. The L1 regularization increases the sparsity of parameters, which can reduce the complexity of interpretation. Other regularizations might improve interpretability in an application-specific way. Weighting by feature usage also ensures that often-used features are the most interpretable.

Another reason to apply regularization is to automatically regulate kernel bandwidth. By using regularization that shrinks the output range of the features, such as a L1 or L2 regularization on a linear transformation, we effectively increase the bandwidth; meanwhile, minimizing the loss function shrinks the effective bandwidth because a smaller bandwidth allows fewer data to have split decision paths, resulting in purer leaves. This makes the actual choice of kernel bandwidth relative to regularization strength so that only one or the other must be tuned, automatically adapts effective bandwidth individually for each feature, and overall improves performance compared to using a fixed effective bandwidth.

4 Evaluation

This section contains evaluation and demonstration of interpretable models. Comprehensive results, as well as additional experiment details, are in the supplementary material. Unless otherwise noted, all results are from crisp trees.

4.1 Benchmarks

We compare various configurations of our algorithm against popular tree-based baselines including decision trees, random forests, and ExtraTrees. We report 10-fold cross validation accuracy and average number of splits in the model. Additional metrics, including average decision path length, feature sparsity, and inference time are shown in the supplementary material. The data sets are selected from among the the most viewed tabular classification data sets on the UCI machine learning repository [14] at the time of writing. Categorical attributes are one-hot encoded, and the data is normalized to mean 0 and standard deviation 1. For our models, we show in the main paper results for linear features and distance-to-prototype features with diagonal inverse covariance. Each is regularized with L1 coefficient λ 1 = . 01 to promote sparsity. Our models and the conventional decision trees have cost-complexity pruning α selected by cross-validation. Other hyperparameters are fixed and described in the supplementary material.

On every data set, at least one of our models matches or comes close to the best baseline accuracy while being much smaller. Also note that, since we choose the pruning parameter α by cross-validation, there are many cases where a smaller tree than

-0.46flavanoids

+-0.40 OD280/OD315

+ -0.30 hue

+ 0.18 color intensity

<0.65

/yes

no

-0.71 proline

+ -0.52 alcohol

+ -0.11 ash

<-0.14

/yes

no

type 1

type 2

feature

p1

p2

p3

alcohol

2.03

-1.66 -0.15

0.85

-0.35

1.15

ash

0.06

-1.00 -0.62

alcalinity of ash

-1.16

0.11

0.22

magnesium

0.95

-2.06 -0.08

total phenols

1.27

0.70

-1.46

flavanoids

0.04

2.69

-0.54

nonflav. phenols

-0.86 -0.33

0.29

proanthocynanins

0.74

-0.24

0.57

color intensity

0.70

-1.95

2.51

hue

0.43

1.10

-1.05

OD280/OD315

0.51

0.50

-1.31

proline

1.91

-1.55 -1.06

d(x, p1)<12.65

yes

no

type 1

d(x, p2) < 19.41

/yes

no

type 2

d(x,p3)<15.86

/yes

no

type 3

type 2

the one reported performs similarly. Especially for data sets where the ensembles greatly outperform the basic decision tree, the reduction in size for a performant model by using our method is huge. This carries the additional benefit that inference is much faster for our models, as shown in the supplementary material. Models with linear features are often the best for a given dataset, while prototype features lag slightly behind, suggesting that prototype features are best used to supplement linear features rather than on their own. We also note that these benchmarks all use the same hyperparameters; this shows that good performance does not require great tuning effort, but such effort will probably result in a better model for most applications.

(a) The entire model.

(b) Interpretation of a correct prediction.

(c) Interpretation of an incor- rect prediction.

7

4

9

G

2

3

8

5

1

3

We also separately show results on MNIST and fashion-MNIST; these experiments are described in Section 4.3 and the results are shown in Table 2.

4.2 Example interpretation

We show two examples of interpretation for trees trained on the wine data set. Both are trained with α = 0 . 01 as in the benchmarks and achieve 97.2% accuracy on a 20% test split.

A tree with linear features is shown in Figure 2. It has a stronger sparsity regularizer λ 1 = 0 . 1 compared to the benchmarks. Wines with low flavanoids, protein concentration, and hue are classified as type 3. There is also a smaller relationship between type 3 and high color intensity. Of the remaining wines, those with low proline and alcohol are type 2, and the rest are type 1. Similarly, there is also a smaller relationship between ash content and type 1 that may be important if the decision is not clear based on proline and alcohol.

A tree with prototype features is shown in Figure 3. Here we use simple Euclidean distance for decision rules. Each prototype defines a certain wine profile; for instance, prototype 1 is high in alcohol and proline and close to average in other attributes. Wines similar to prototype 1 are type 1; wines similar to prototype 2 are type 2; wines similar to prototype 3 are type 3; the rest are type 2.

4.3 MNIST features

Next we fit models to MNIST and fashion-MNIST to demonstrate performance on simple image classification. Here we use linear features with some extra

MNIST

fashion-MNIST

constraints to improve interpretability. First, in addition to a L1 regularization to promote sparsity, we use a smoothness regularization which penalizes the average squared difference of each weight with its neighbors, not including diagonal. The idea here is that a weight image with smooth shapes is easier to understand than one with a handful of sparse pixels, while also being more expressive. We also constrain the tree's threshold values to be zero to ease local interpretation. In this way, we can interpret a decision rule by asking 'do the pen strokes better match the blue, or the red'? For example, in the root node in Figure 4a, we see that digits with ink primarily in a small near-vertical stroke in the center are classified as 1, whereas those with more ink in the closely surrounding space are other digits. Complete predictions are shown Figures 4b and 4c.

The performance and size of models for various cost-complexity pruning α are shown in Table 2, and the smallest MNIST models are shown in Figure 4. Larger models are shown in the supplementary material. Each internal node shows the linear feature's weight image, with negative values in blue and positive in red. Each leaf shows the average of the training data belonging to it.

Testing on MNIST allows us to make some comparison of our models to TAO [10], which seems to be the best existing method for learning trees with sparse linear features. Ultimately, while their smallest model achieves 89.81% test accuracy with 16 splits, our smallest achieves 92.19% with just 9 splits. Likewise, their overall best-performing model achieves 94.31%, whereas ours achieves 96.10% without fuzzy splitting or 97.08% with it. The latter is on par with performance of random forests for MNIST, accomplished with a single (albeit large) tree with interpretable features. This shows that our proposed approach can achieve state-of-the-art status for learning sparse oblique decision trees, while also having greater flexibility to choose different features and adapt them to application-specific interpretability needs.

4.4 Information hierarchy

An additional property of our models is that, differently from other ODTs such as TAO, we continually refit the tree using CART, which greedily chooses the most informative splits first. Each subtree's class labels are thus as pure as possible, and classes that are easy to separate are separated first. By contrast, in a tree which is trained globally without this greedy splitting, any particular split may not actually result in improvement in label purity, as long as it is informative to splits further down the tree.

A compelling example is shown in Figure 5, which shows the tree trained on fashion-MNIST with α = 10 -3 . Without exception, the tree forms a hierarchy of visual similarity of fashion items, as described in the caption. This allows us to interpret predictions to some extent without even knowing the features . For example, in Figure 5, if the decision path for an image of a tee shirt ends at the upper leftmost leaf (dress),

instead of the one below it (tee shirt), we can tell by the leaf data alone that the model correctly thinks it is a top without long sleeves, but mistakenly thinks that it is sleeveless instead of short-sleeved. A similar structure of visually cohesive subtrees is observed in our MNIST trees, as in Figure 4, but the trend is less obvious because digits are more visually distinct in general compared to the items that comprise fashion-MNIST. We do not observe this property in other ODTs; for example, in [41], a TAO tree fitted to fashion-MNIST does not exhibit this visual hierarchy property.

5 Discussion

We have proposed a new system for learning small, interpretable decision trees and showed examples under several notions of interpretability, including sparse linear combinations, similarity to prototypes, visual matching, and class-based hierarchies. Compared to ensembles and deep models, our single-

tree models are interpretable because they are small, use sparse features, have a sparsely activated hierarchical representation based on logical rules, and ultimately work by simply partitioning the data. Moreover, compared to other methods for single-tree models, ours are highly flexible to customize feature types to meet interpretability needs and produce smaller models with more meaningful hierarchical structure and better performance. The tradeoff is that the models can take relatively long to train due to repeated fitting of fuzzy trees, and while our results show that good models can be obtained without extensive hyperparameter tuning, the absolute best model for a given scenario certainly may require careful tuning of several hyperparameters.

Future work will incorporate new, more specialized feature types to extend the framework to differently structured data such as images (more challenging than MNIST), time series, and text. We will also work to improve the framework for regression tasks, which are not yet analyzed in this work because the piecewise-constant nature of decision trees means that small trees with simple feature transformations are less than ideally suited for challenging regression tasks.

References

Stefan Aeberhard and M. Forina. Wine. UCI Machine Learning Repository, 1991. DOI: 10.24432/C5PC7J .

E. Alpaydin and Fevzi. Alimoglu. Pen-Based Recognition of Handwritten Digits. UCI Machine Learning Repository, 1998. DOI: https://doi.org/10.24432/C5MG6K.

Osbert Bastani, Carolyn Kim, and Hamsa Bastani. Interpretability via model extraction. arXiv preprint arXiv:1706.09773 , 2017.

James C Bezdek. Pattern recognition with fuzzy objective function algorithms . Springer Science &Business Media, 2013.

Marko Bohanec. Car Evaluation. UCI Machine Learning Repository, 1997. DOI: 10.24432/ C5JP48 .

Ferdinand Bollwein and Stephan Westphal. A branch & bound algorithm to determine optimal bivariate splits for oblique decision tree induction. Applied Intelligence , 51(10):7552-7572, Oct 2021.

Valerio Bonsignori, Riccardo Guidotti, and Anna Monreale. Deriving a single interpretable model by merging tree-based classifiers. In Carlos Soares and Luis Torgo, editors, Discovery Science , pages 347-357, Cham, 2021. Springer International Publishing.

L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees . Wadsworth International Group, 1984.

Leo Breiman. Random forests. Machine learning , 45(1):5-32, 2001.

Miguel A. Carreira-Perpinan and Pooya Tavallali. Alternating optimization of decision trees, with application to learning sparse oblique trees. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems , volume 31. Curran Associates, Inc., 2018.

Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , KDD '16, pages 785-794, New York, NY, USA, 2016. ACM.

Mark Craven and Jude Shavlik. Extracting tree-structured representations of trained networks. In D. Touretzky, M.C. Mozer, and M. Hasselmo, editors, Advances in Neural Information Processing Systems , volume 8. MIT Press, 1995.

Yashesh Dhebar and Kalyanmoy Deb. Interpretable rule discovery through bilevel optimization of split-rules of nonlinear decision trees for classification problems. IEEE Transactions on Cybernetics , 51(11):5573-5584, 2021.

Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.

J. C. Dunn. A fuzzy relative of the isodata process and its use in detecting compact wellseparated clusters. Journal of Cybernetics , 3(3):32-57, 1973.

Vladimir Estivill-Castro, Eugene Gilmore, and René Hexel. Constructing interpretable decision trees using parallel coordinates. In Leszek Rutkowski, Rafał Scherer, Marcin Korytkowski, Witold Pedrycz, Ryszard Tadeusiewicz, and Jacek M. Zurada, editors, Artificial Intelligence and Soft Computing , pages 152-164, Cham, 2020. Springer International Publishing.

Vladimir Estivill-Castro, Eugene Gilmore, and René Hexel. Human-in-the-loop construction of decision tree classifiers with parallel coordinates. In 2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC) , pages 3852-3859, 2020.

R. A. Fisher. Iris. UCI Machine Learning Repository, 1988. DOI: 10.24432/C56C76 .

Yoav Freund and Robert E. Schapire. A desicion-theoretic generalization of on-line learning and an application to boosting. In Paul Vitányi, editor, Computational Learning Theory , pages 23-37, Berlin, Heidelberg, 1995. Springer Berlin Heidelberg.

Magzhan Gabidolla and Miguel Á. Carreira-Perpiñán. Optimal interpretable clustering using oblique decision trees. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , KDD '22, page 400-410, New York, NY, USA, 2022. Association for Computing Machinery.

Magzhan Gabidolla, Arman Zharmagambetov, and Miguel Á. Carreira-Perpiñán. Improved multiclass adaboost using sparse oblique decision trees. In 2022 International Joint Conference on Neural Networks (IJCNN) , pages 1-8, 2022.

Pierre Geurts, Damien Ernst, and Louis Wehenkel. Extremely randomized trees. Machine learning , 63(1):3-42, 2006.

Eugene Gilmore, Vladimir Estivill-Castro, and René Hexel. More interpretable decision trees. In Hugo Sanjurjo González, Iker Pastor López, Pablo García Bringas, Héctor Quintián, and Emilio Corchado, editors, Hybrid Artificial Intelligent Systems , pages 280-292, Cham, 2021. Springer International Publishing.

Jack H. Good, Kyle Miller, and Artur Dubrawski. Kernel density decision trees. In Proceedings of the AAAI Spring Symposium on AI Engineering . Curran Associates, Inc., 2022.

Bogdan Gulowaty and Michał Wo´ zniak. Extracting interpretable decision tree ensemble from random forest. In 2021 International Joint Conference on Neural Networks (IJCNN) , pages 1-8, 2021.

Venkatesan Guruswami and Prasad Raghavendra. Hardness of learning halfspaces with noise. In 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06) , pages 543-552, 2006.

Suryabhan Singh Hada, Miguel Á Carreira-Perpiñán, and Arman Zharmagambetov. Sparse oblique decision trees: A tool to understand and manipulate neural net features. Data Mining and Knowledge Discovery , pages 1-40, 2023.

Suryabhan Singh Hada and Miguel Á. Carreira-Perpiñán. Sparse oblique decision trees: A tool to interpret natural language processing datasets. In 2022 International Joint Conference on Neural Networks (IJCNN) , pages 1-8, 2022.

Laurent Hyafil and Ronald L. Rivest. Constructing optimal binary decision trees is np-complete. Information Processing Letters , 5(1):15-17, 1976.

Andras Janosi, William Steinbrunn, Matthias Pfisterer, and Robert Detrano. Heart Disease. UCI Machine Learning Repository, 1988. DOI: 10.24432/C52P4X .

M. Koklu and I.A. Ozkan. Dry Bean Dataset. UCI Machine Learning Repository, 2020.

Peter Kontschieder, Madalina Fiterau, Antonio Criminisi, and Samuel Rota Bulò. Deep neural decision forests. In 2015 IEEE International Conference on Computer Vision (ICCV) , pages 1467-1475, 2015.

Haoran Luo, Fan Cheng, Heng Yu, and Yuqi Yi. Sdtr: Soft decision tree regressor for tabular data. IEEE Access , 9:55999-56011, 2021.

Sreerama K Murthy, Simon Kasif, and Steven Salzberg. A system for induction of oblique decision trees. Journal of artificial intelligence research , 2:1-32, 1994.

Duy T Nguyen, Kathryn E Kasmarik, and Hussein A Abbass. Towards interpretable anns: An exact transformation to multi-class multivariate decision trees. arXiv preprint arXiv:2003.04675 , 2020.

Omer Sagi and Lior Rokach. Approximating xgboost with an interpretable decision tree. Information Sciences , 572:522-542, 2021.

M. Sato and H. Tsukimoto. Rule extraction from neural networks via decision tree induction. In IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222) , volume 3, pages 1870-1875 vol.3, 2001.

Terry Sejnowski and R. Gorman. Connectionist Bench (Sonar, Mines vs. Rocks). UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5T01Q.

Wing S. Hutton L. Sigillito, V. and K. Baker. Ionosphere. UCI Machine Learning Repository, 1989. DOI: https://doi.org/10.24432/C5W01B.

Andrew Silva, Matthew Gombolay, Taylor Killian, Ivan Jimenez, and Sung-Hyun Son. Optimization methods for interpretable differentiable decision trees applied to reinforcement learning. In Silvia Chiappa and Roberto Calandra, editors, International Conference on Artificial Intelligence and Statistics , volume 108 of Proceedings of Machine Learning Research , pages 1855-1865. PMLR, 26-28 Aug 2020.

Suryabhan Singh Hada and Miguel Á. Carreira-Perpiñán. Interpretable image classification using sparse oblique decision trees. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 2759-2763, 2022.

Alberto Suarez and James F. Lutsko. Globally optimal fuzzy decision trees for classification and regression. IEEE Transactions on Pattern Analysis and Machine Intelligence , 21(12):1297-1311, Dec 1999.

Gilles Vandewiele, Olivier Janssens, Femke Ongenae, Filip De Turck, and Sofie Van Hoecke. Genesim: genetic extraction of a single, interpretable model. arXiv preprint arXiv:1611.05722 , 2016.

William Wolberg, Olvi Mangasarian, Nick Street, and W. Street. Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository, 1995. DOI: 10.24432/C5DW2B .

Arman Zharmagambetov and Miguel Carreira-Perpinan. Smaller, more accurate regression forests using tree alternating optimization. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 11398-11408. PMLR, 13-18 Jul 2020.

Arman Zharmagambetov and Miguel A. Carreira-Perpinan. Semi-supervised learning with decision trees: Graph laplacian tree alternating optimization. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems , volume 35, pages 2392-2405. Curran Associates, Inc., 2022.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a method to learn interpretable decision trees by keeping the tree depth limited without sacrificing the performance. Keeping the tree small is achieved through multiple iterations of feature transformation followed by regrowing the tree. Authors have utilized linear and distance to prototype feature transformation. They apply regularization to improve interpretability through increasing feature sparsity and shrinking the output range of the features which results in purer leaves. \nAuthors report empirical results evaluating their method against decision tree, random forests, and EstraTrees on a set of UCI repositories. Experiments report accuracy , average number of nodes in model, average decision path length, feature sparsity, and inference time (all metrics except accuracy are reported in supplementary material). Results show that in 4 out of 6 data sets, a version of the proposed method outperforms the baselines.\n\nSTRENGTHS:\n- Originality: The work builds on top of existing literature on feature transformation, oblique decision trees, CART, differentiable decision trees, etc. The proposed way of alternating through feature transformation and tree re-growth is fairly novel and the introduced regularizations make sense.\n- Quality: The quality of the paper is overall fair. Motivation is clear, related literature is well covered and experimental results are clearly reported with more details in supplementary materials. Refer to weaknesses for rationale on why overall quality is fair.\n- Clarity: The paper is written clearly for the most part. I was not able to completely follow the discussions on MNIST data sets and it is not clear to me how the proposed method reforms compared to the baseline approaches in terms of accuracy and explainability for MNIST.\n- Significance: The work is generally interesting with potentially significant applications. However, the shortcomings that I've listed in the weaknesses sections cast some doubt on the significance of the proposed method as presented.\n\nWEAKNESSES:\n- Interpretation of interpretability: Authors measure interpretability in terms of the tree size and use feature transformation and regularization to keep the tree size under control. However, one can argue that the feature transformation (e.g. as in application of ODT with non axis-aligned partitioning, or distance to prototype) can adversely impact interpretability. For example, consider figure 2 where a node contains a linear combination of a set of features as the decision criteria which is not very interpretable compared with an axis-aligned simple DT method. To get a better idea, consider a domain where raw features are more tangible and interpretable compared to the wine data set such as income prediction based on age where applying a coefficient to a well understood feature such as age and combining it with other scaled features make the whole tree less interpretable.\n- Limited and inconclusive competitive analysis: Authors have compared their method against DT, RF, and ET. Experimental results are limited because authors have not considered linear methods such as linear regression which are arguably comparable with the proposed methods in terms of interpretability. Reason is that linear methods provide one linear formula consisting of the input features whereas the proposed method (with linear transformation) consists of multiple nodes each with a linear combination of a limited set of features. They are inconclusive since, the performance of the proposed method varies drastically based on the alpha value which is not picked programmatically (based on cross validation) like what authors have done for DT. Lastly, the baseline methods do not help getting a clear idea about the delta in accuracy between non-interpretable high performing methods (e.g. xgboost) and more interpretable alternatives.\n\nQUESTIONS:\n- It would be useful to understand author(s)' perspective on linear transformation of input features on interpretability.\n- Can authors clearly explain the main rationale behind reporting MNIST data sets?\n- How would authors justify that the intermediate value of alpha for the most part has the highest performance, although the alpha values that represent highest complexity are expected to outperform the intermediate alpha value.\n- Can author shed more light into the big delta between their method's performance for various alphas and how this could be addressed practically so that a fixed programmatically selected alpha can be used with high confidence?\n\nLIMITATIONS:\n- Time complexity: As stated by the authors, proposed approach requires longer training time due to repeatedly fitting a tree to the data. It would have been more useful if experiments instead of inference time would have reported the training time.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper the authors study the problem of learning meaningful feature representation for decision trees. They propose to interleave the learning of parametrized feature representation by gradient descent, and the learning of tree structures by standard greedy method, e.g. CART. A kernel density decision tree is used for the former step for differentiability. \n\nThe authors propose a few practices to achieve better performance, which includes a couple of classes of feature transformation, regularization and the choice of KDDT kernel. They empirically show that the proposed method easily outperform similar existing methods, and the learned feature representation are visually meaning when the method is applied to image datasets.\n\nSTRENGTHS:\n1. The paper studies a legit and important research question, that it provides a convincing potential solution to the question of effectively applying decision trees on higher dimensional datasets with proper feature learning. \n2. The proposed method has good uses of modern decision tree techniques. I think the authors have exhausted the headroom in their scope, and there is no obvious theoretical improvement left to make. \n3. The writing quality is good. The paper is clear, concise and easy to follow.\n\nWEAKNESSES:\nOne of my major concerns is the divergence of the proposed method from a traditional decision tree and its implied impact on model performance and interpretability. Kernel based fuzzy decision trees are a compromise to differentiability, which already results in a prediction pattern that is not the interpretable yes-no at each node, adding extra computation and more mental overhead. This plus the learned linear feature combination, about whose stability I have doubt, might push the proposed method more like a NN in the shell of a DT.  These observations might have undermined the significance of the submission.\n\nAnother concern is about novelty, that as mentioned by the authors, the interleaved learning of tree structure and feature representation has been studies. If so the novelty of the paper is the proposed of several practices of better feature representation learning, which might fail to justify as a NeurIPS publication.\n\nQUESTIONS:\n1. Regarding novelty, I wonder what is the key difference between the proposed method and prior work, as the phrasings in Line 128-131 and Line 132-135 sound almost the same.\n2. Both the study on the tabular data and on the image datasets suggest that the linear projection is the most important feature transformation, which is also the most compatible with the sparsity regularization. What exactly do other feature transformations, e.g. distance to prototype and fuzzy cluster membership, bring here? Any theoretical / empirical supporting evidence?\n3. Given the linear projection transformation is done in a fashion very similar to the learning of a NN dense layer, I wonder how stable the proposed method is, in particular\na. How's convergence in practice with respect to the learned linear projections? Is the learning robust / can we always reach the same linear features / interpretation of the resulting tree? \nb. How much does the tree T structure changes during model fitting? The introduction of growing / pruning methods (L131) actually prevent the tree structure from changing too much, and why do we not consider such strategies?\n4. The benchmarking against tree methods is less meaningful given the feature learning perspective of the proposed method. A closer comparison is probably between it and a shallow MLP. \n5. Line 126. T \\circ f_{\\theta}. \n\nI am open to raise my rating should the aforementioned questions be addressed.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a novel approach that learns sparse features and constructs decision trees in a differentiable manner. This approach strikes a balance between the complexity of decision rules and the overall model, resulting in small, interpretable trees. Specifically, the authors apply (1)alternating optimization method for trees and features (2)kernel density decision trees (3)different parameterized feature transforms (4)regularization method.\nThey compare their method against popular tree-based baselines and demonstrate its comparable performance in terms of accuracy while producing smaller models.\n\nSTRENGTHS:\n1. The approach proposed by the paper is novel that combines sparse feature learning with differentiable decision tree construction to produce small, interpretable trees. This approach is different from traditional decision tree models and provides a new perspective on feature learning for interpretable models.\n\n2. The paper provides a comprehensive empirical evaluation of the proposed approach on various datasets. The authors compare their method against popular tree-based baselines, including decision trees, random forests, and ExtraTrees. The results show that the proposed approach outperforms or comes very close to the best baseline's accuracy while being much smaller.\n\n3. The paper is well-written and gives detailed explanations for each technique used.\n\nWEAKNESSES:\n1. Lack of comparison with other models: The paper compares the proposed approach against only a few traditional tree-based models like decision trees, random forests, and ExtraTrees. Comparing with more classic baselines can better illustrate the effectiveness of the approach\n\n\n2. Clarity: The paper covers a variety of techniques (such as KDDT), many of which are improvements built upon previous work. It would be beneficial to clearly highlight the unique contributions made by the authors.\n\nQUESTIONS:\n1.I noticed that in Table 1, when α = 0.1, the accuracy on the dry-bean dataset significantly drops to 0.476/0.354. What could be the reason behind this sharp decline? I am somewhat concerned about the robustness of the method.\n\n2.The models using prototype features consistently fall behind the models employing linear features across all six datasets. In practical terms, what strategies can be employed to complement linear features with nonlinear features?\n\nLIMITATIONS:\nYes，the authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors aim to learn small and interpretable decision trees with good predictive performance. To this end, the authors propose efficient differentiable trees based on the kernel density decision trees. To ensure the small size of the tree and increase the model's representability, the authors propose some parameterized feature transformations. Due to its differential structure, the model is capable of learning the parameters in both the tree and feature transformation simultaneously. The experimental results demonstrate the predictive performance and interpretability of the proposed model.\n\nSTRENGTHS:\nThe paper is well written and easy to follow. The idea of learning a parametrized feature transformation is interesting as it enhances the model's expressiveness while still maintaining its interpretability. The authors provided extensive experimental results which demonstrate the interpretability of the model.\n\nWEAKNESSES:\n1. The main contribution of the paper is the proposed differentiable tree and parameterized transformation. Nevertheless, the differentiable tree is not a new concept. Although the idea of learning a transformation during training is interesting. Similar ideas, such as neural prototypes, have been previously proposed in the computer vision domain. See [1] for an example.\n2. No guideline is provided to recommend how to choose transformations in different tasks for users. \n3. The datasets used to show the prediction performance are relatively small.\n4. Some technique details are missing. For example, how these regularizations are used when learn the tree is unclear to me. \n\nRef: \n[1] Nauta, Meike, Ron Van Bree, and Christin Seifert. \"Neural prototype trees for interpretable fine-grained image recognition.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.\n\nQUESTIONS:\n1. In line 147, “we assume a single marginal F_i=F…”. Where is the assumption used in the following part? In the same line, what is F’?\n2. Is the feature transformation mentioned in line 172 the same as these described in Section 3.4? If so, how to guarantee the output range are limit for identiy and element-wise transformation?\n3.What is $\\alpha$ in Section 4? Is it the same as that in line 207? If so, please provide some discussion why larger $\\alpha$ leads to more sparse tree. If not, please provide details how to control the complexity of the model by tuning $\\alpha$?\n4. Is “ours:linear” in Table 1 the same with ODT? If not, please provide experiments results using ODT.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}