SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Distance to prototype. Another feature type maps a point to its distance to each of  $k$  prototypes  $\mathbf{p}_i$ ; in this sense, a decision rule  $d(\mathbf{x}, \mathbf{p}_i; \mathbf{S}_i) < t$  can be interpreted simply as "is  $\mathbf{x}$  sufficiently similar to  $\mathbf{p}_i$ ". We use Mahalanobis distance  $d(\mathbf{x}, \mathbf{p}_i; \mathbf{S}_i) = (\mathbf{x} - \mathbf{p}_i)^T \mathbf{S}_i^{-1} (\mathbf{x} - \mathbf{p}_i)$ , where the learnable parameters are prototypes  $\mathbf{p}_i$  and inverse covariance matrices  $\mathbf{S}_i^{-1}$ . The covariance matrix can be reparameterized in various ways to restrict the measurement of distance<sup>1</sup>, for example, by making it diagonal, or by using the same for all prototypes. It should also be parameterized such that it is positive semidefinite; otherwise hyperbolic decision rules may be learned and the distance interpretation no longer applies.  $\mathbf{S}_i^{-1}$  can be regularized for sparsity, so that the distance is based on few features. The parameters can be initialized randomly or by using a mixture modeling algorithm.
Fuzzy cluster membership. Clustering partitions data into  $k$  clusters with high internal similarity; usually each cluster  $i$  is defined by a center  $\mathbf{c}_i$ , and each point belongs to the cluster with the closest center. Conventional clustering is not differentiable, so we instead use fuzzy clustering, a variant that assigns a degree of membership to each cluster, with the membership values summing to 1. Given distance functions  $d_i$ ,  $i \in [k]$ , the membership in cluster  $i$  is
$$
w _ {i} (\mathbf {x}) = \frac {1}{\sum_ {j = 1} ^ {k} \left(\frac {d (\mathbf {x} , \mathbf {c} _ {i} ; \mathbf {S} _ {i})}{d (\mathbf {x} , \mathbf {c} _ {j} ; \mathbf {S} _ {j})}\right) ^ {\frac {2}{m - 1}}}
$$
with  $m \in (1, \infty)$  a hyperparameter determining the "softness" of the cluster assignment, usually just set to 2, and  $d$  the Mahalanobis distance as defined previously. In this way, soft clustering can be viewed as a transformation of the "distance to prototype" features so that the resulting transformed features are interdependent; a decision rule  $w_{i}(\mathbf{x}) > t$  is interpreted as "x is sufficiently closer to  $\mathbf{c}_i$  than the other centers". The parameters can be initialized randomly or by using a fuzzy clustering algorithm such as fuzzy  $c$ -means [15, 4] with all  $\mathbf{S}_i$  initialized as identity.
3.5 Regularization
For each transformed feature  $j \in [p']$ , the parameters  $\theta_j \subseteq \theta$  used to compute the  $j$ th feature  $f^{(j)}$  may be subject to a regularization weighted by the total usage of feature  $j$ , that is, the sum of total training sample weights over nodes that split using feature  $j$ . This weighting serves to balance the effects of the loss and regularization; a feature used in the decisions for relatively few data will have
For examples of different covariance types, see https://scikit-learn.org/stable/modules/mixture.html.
a relatively smaller loss gradient, and so should have a proportionally smaller regularization, and vice versa.
One reason to apply regularization is to improve the interpretability of features. The L1 regularization increases the sparsity of parameters, which can reduce the complexity of interpretation. Other regularizations might improve interpretability in an application-specific way. Weighting by feature usage also ensures that often-used features are the most interpretable.
Another reason to apply regularization is to automatically regulate kernel bandwidth. By using regularization that shrinks the output range of the features, such as a L1 or L2 regularization on a linear transformation, we effectively increase the bandwidth; meanwhile, minimizing the loss function shrinks the effective bandwidth because a smaller bandwidth allows fewer data to have split decision paths, resulting in purer leaves. This makes the actual choice of kernel bandwidth relative to regularization strength so that only one or the other must be tuned, automatically adapts effective bandwidth individually for each feature, and overall improves performance compared to using a fixed effective bandwidth.
Figure 2: Tree with sparse linear features on the wine data set.
This section contains evaluation and demonstration of interpretable models. Comprehensive results, as well as additional experiment details, are in the supplementary material. Unless otherwise noted, all results are from crisp trees.
We compare various configurations of our algorithm against popular tree-based baselines including decision trees, random forests, and ExtraTrees. We report 10-fold cross validation accuracy and average number of splits in the model. Additional metrics, including average decision path length, feature sparsity, and inference time are shown in the supplementary material. The data sets are selected from among the the most viewed tabular classification data sets on the UCI machine learning repository [14] at the time of writing. Categorical attributes are one-hot encoded, and the data is normalized to mean 0 and standard deviation 1. For our models, we show in the main paper results for linear features and distance-to-prototype features with diagonal inverse covariance. Each is regularized with L1 coefficient  $\lambda_{1} = .01$  to promote sparsity. Our models and the conventional decision trees have cost-complexity pruning  $\alpha$  selected by cross-validation. Other hyperparameters are fixed and described in the supplementary material.
On every data set, at least one of our models matches or comes close to the best baseline accuracy while being much smaller. Also note that, since we choose the pruning parameter  $\alpha$  by cross-validation, there are many cases where a smaller tree than
the one reported performs similarly. Especially for data sets where the ensembles greatly outperform the basic decision tree, the reduction in size for a performant model by using our method is huge. This carries the additional benefit that inference is much faster for our models, as shown in the supplementary material. Models with linear features are often the best for a given dataset, while prototype features lag slightly behind, suggesting that prototype features are best used to supplement linear features rather than on their own. We also note that these benchmarks all use the same hyperparameters; this shows that good performance does not require great tuning effort, but such effort will probably result in a better model for most applications.
Figure 3: Tree with prototype features on the wine data set.
(a) The entire model.
Figure 4: An interpretable MNIST classifier with  $92.19\%$  test accuracy.
(b) Interpretation of a correct prediction.
(c) Interpretation of an incorrect prediction.
We also separately show results on MNIST and fashion-MNIST; these experiments are described in Section 4.3 and the results are shown in Table 2.
4.2 Example interpretation
We show two examples of interpretation for trees trained on the wine data set. Both are trained with  $\alpha = 0.01$  as in the benchmarks and achieve  $97.2\%$  accuracy on a  $20\%$  test split.
A tree with linear features is shown in Figure 2. It has a stronger sparsity regularizer  $\lambda_{1} = 0.1$  compared to the benchmarks. Wines with low flavanoids, protein concentration, and hue are classified as type 3. There is also a smaller relationship between type 3 and high color intensity. Of the remaining wines, those with low proline and alcohol are type 2, and the rest are type 1. Similarly, there is also a smaller relationship between ash content and type 1 that may be important if the decision is not clear based on proline and alcohol.
A tree with prototype features is shown in Figure 3. Here we use simple Euclidean distance for decision rules. Each prototype defines a certain wine profile; for instance, prototype 1 is high in alcohol and proline and close to average in other attributes. Wines similar to prototype 1 are type 1; wines similar to prototype 2 are type 2; wines similar to prototype 3 are type 3; the rest are type 2.
4.3 MNIST features
Next we fit models to MNIST and fashion-MNIST to demonstrate performance on simple image classification. Here we use linear features with some extra
MNIST
α	fuzzy acc.	crisp acc.	splits	path len.
10-2	.9226	.9219	9	4.33
10-3	.9500	.9419	36	5.91
10-4	.9664	.9610	162	7.68
10-5	.9708	.9602	1659	10.05
fashion-MNIST
α	fuzzy acc.	crisp acc.	splits	path len.
10-2	.8059	.8024	8	3.97
10-3	.8439	.8364	20	4.69
10-4	.8669	.8586	98	6.24
10-5	.8675	.8472	2209	11.55
Table 2: Results for MNIST trees.
constraints to improve interpretability. First, in addition to a L1 regularization to promote sparsity, we use a smoothness regularization which penalizes the average squared difference of each weight with its neighbors, not including diagonal. The idea here is that a weight image with smooth shapes is easier to understand than one with a handful of sparse pixels, while also being more expressive. We also constrain the tree's threshold values to be zero to ease local interpretation. In this way, we can interpret a decision rule by asking "do the pen strokes better match the blue, or the red"? For example, in the root node in Figure 4a, we see that digits with ink primarily in a small near-vertical stroke in the center are classified as 1, whereas those with more ink in the closely surrounding space are other digits. Complete predictions are shown Figures 4b and 4c.
The performance and size of models for various cost-complexity pruning  $\alpha$  are shown in Table 2, and the smallest MNIST models are shown in Figure 4. Larger models are shown in the supplementary material. Each internal node shows the linear feature's weight image, with negative values in blue and positive in red. Each leaf shows the average of the training data belonging to it.
Testing on MNIST allows us to make some comparison of our models to TAO [10], which seems to be the best existing method for learning trees with sparse linear features. Ultimately, while their smallest model achieves  $89.81\%$  test accuracy with 16 splits, our smallest achieves  $92.19\%$  with just 9 splits. Likewise, their overall best-performing model achieves  $94.31\%$ , whereas ours achieves  $96.10\%$  without fuzzy splitting or  $97.08\%$  with it. The latter is on par with performance of random forests for MNIST, accomplished with a single (albeit large) tree with interpretable features. This shows that our proposed approach can achieve state-of-the-art status for learning sparse oblique decision trees, while also having greater flexibility to choose different features and adapt them to application-specific interpretability needs.
4.4 Information hierarchy
An additional property of our models is that, differently from other ODTs such as TAO, we continually refit the tree using CART, which greedily chooses the most informative splits first. Each subtree's class labels are thus as pure as possible, and classes that are easy to separate are separated first. By contrast, in a tree which is trained globally without this greedy splitting, any particular split may not actually result in improvement in label purity, as long as it is informative to splits further down the tree.
A compelling example is shown in Figure 5, which shows the tree trained on fashion-MNIST with  $\alpha = 10^{-3}$ . Without exception, the tree forms a hierarchy of visual similarity of fashion items, as described in the caption. This allows us to interpret predictions to some extent without even knowing the features. For example, in Figure 5, if the decision path for an image of a tee shirt ends at the upper leftmost leaf (dress),
Figure 5: An interpretable fashion-MNIST classifier. Because CART selects the most informative splits first, the tree is a visually intuitive hierarchy. For example, tops, bottoms, shoes, and bags each have a distinct subtree; within tops, there are subtrees for sleeve length, collar shape, etc.
instead of the one below it (tee shirt), we can tell by the leaf data alone that the model correctly thinks it is a top without long sleeves, but mistakenly thinks that it is sleeveless instead of short-sleeved. A similar structure of visually cohesive subtrees is observed in our MNIST trees, as in Figure 4, but the trend is less obvious because digits are more visually distinct in general compared to the items that comprise fashion-MNIST. We do not observe this property in other ODTs; for example, in [41], a TAO tree fitted to fashion-MNIST does not exhibit this visual hierarchy property.
We have proposed a new system for learning small, interpretable decision trees and showed examples under several notions of interpretability, including sparse linear combinations, similarity to prototypes, visual matching, and class-based hierarchies. Compared to ensembles and deep models, our single-
tree models are interpretable because they are small, use sparse features, have a sparsely activated hierarchical representation based on logical rules, and ultimately work by simply partitioning the data. Moreover, compared to other methods for single-tree models, ours are highly flexible to customize feature types to meet interpretability needs and produce smaller models with more meaningful hierarchical structure and better performance. The tradeoff is that the models can take relatively long to train due to repeated fitting of fuzzy trees, and while our results show that good models can be obtained without extensive hyperparameter tuning, the absolute best model for a given scenario certainly may require careful tuning of several hyperparameters.
Future work will incorporate new, more specialized feature types to extend the framework to differently structured data such as images (more challenging than MNIST), time series, and text. We will also work to improve the framework for regression tasks, which are not yet analyzed in this work because the piecewise-constant nature of decision trees means that small trees with simple feature transformations are less than ideally suited for challenging regression tasks.
[1] Stefan Aeberhard and M. Forina. Wine. UCI Machine Learning Repository, 1991. DOI: 10.24432/C5PC7J.
[2] E. Alpaydin and Fevzi. Alimoglu. Pen-Based Recognition of Handwritten Digits. UCI Machine Learning Repository, 1998. DOI: https://doi.org/10.24432/C5MG6K.
[3] Osbert Bastani, Carolyn Kim, and Hamsa Bastani. Interpretability via model extraction. arXiv preprint arXiv:1706.09773, 2017.
[4] James C Bezdek. Pattern recognition with fuzzy objective function algorithms. Springer Science & Business Media, 2013.
[5] Marko Bohanec. Car Evaluation. UCI Machine Learning Repository, 1997. DOI: 10.24432/C5JP48.
[6] Ferdinand Bollwein and Stephan Westphal. A branch & bound algorithm to determine optimal bivariate splits for oblique decision tree induction. Applied Intelligence, 51(10):7552-7572, Oct 2021.
[7] Valerio Bonsignori, Riccardo Guidotti, and Anna Monreale. Deriving a single interpretable model by merging tree-based classifiers. In Carlos Soares and Luis Torgo, editors, Discovery Science, pages 347-357, Cham, 2021. Springer International Publishing.
[8] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth International Group, 1984.
[9] Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
[10] Miguel A. Carreira-Perpinan and Pooya Tavallali. Alternating optimization of decision trees, with application to learning sparse oblique trees. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018.
[11] Tianqi Chen and Carlos Guestrin. XGBoost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16, pages 785-794, New York, NY, USA, 2016. ACM.
[12] Mark Craven and Jude Shavlik. Extracting tree-structured representations of trained networks. In D. Touretzky, M.C. Mozer, and M. Hasselmo, editors, Advances in Neural Information Processing Systems, volume 8. MIT Press, 1995.
[13] Yashesh Dhebar and Kalyanmoy Deb. Interpretable rule discovery through bilevel optimization of split-rules of nonlinear decision trees for classification problems. IEEE Transactions on Cybernetics, 51(11):5573-5584, 2021.
[14] Dheeru Dua and Casey Graff. UCI machine learning repository, 2017.
[33] Haoran Luo, Fan Cheng, Heng Yu, and Yuqi Yi. Sdtr: Soft decision tree regressor for tabular data. IEEE Access, 9:55999-56011, 2021.
[34] Sreerama K Murthy, Simon Kasif, and Steven Salzberg. A system for induction of oblique decision trees. Journal of artificial intelligence research, 2:1-32, 1994.
[35] Duy T Nguyen, Kathryn E Kasmarik, and Hussein A Abbass. Towards interpretable anns: An exact transformation to multi-class multivariate decision trees. arXiv preprint arXiv:2003.04675, 2020.
[36] Omer Sagi and Lior Rokach. Approximating xgboost with an interpretable decision tree. Information Sciences, 572:522-542, 2021.
[37] M. Sato and H. Tsukimoto. Rule extraction from neural networks via decision tree induction. In IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222), volume 3, pages 1870-1875 vol.3, 2001.
[38] Terry Sejnowski and R. Gorman. Connectionist Bench (Sonar, Mines vs. Rocks). UCI Machine Learning Repository. DOI: https://doi.org/10.24432/C5T01Q.
[39] Wing S. Hutton L. Sigillito, V. and K. Baker. Ionosphere. UCI Machine Learning Repository, 1989. DOI: https://doi.org/10.24432/C5W01B.
[40] Andrew Silva, Matthew Gombolay, Taylor Killian, Ivan Jimenez, and Sung-Hyun Son. Optimization methods for interpretable differentiable decision trees applied to reinforcement learning. In Silvia Chiappa and Roberto Calandra, editors, International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pages 1855–1865. PMLR, 26–28 Aug 2020.
[41] Suryabhan Singh Hada and Miguel Á. Carreira-Perpínán. Interpretable image classification using sparse oblique decision trees. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2759-2763, 2022.
[42] Alberto Suarez and James F. Lutsko. Globally optimal fuzzy decision trees for classification and regression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12):1297-1311, Dec 1999.
[43] Gilles Vandewiele, Olivier Janssens, Femke Ongenae, Filip De Turck, and Sofie Van Hoecke. Genesim: genetic extraction of a single, interpretable model. arXiv preprint arXiv:1611.05722, 2016.
[44] William Wolberg, Olvi Mangasarian, Nick Street, and W. Street. Breast Cancer Wisconsin (Diagnostic). UCI Machine Learning Repository, 1995. DOI: 10.24432/C5DW2B.
[45] Arman Zharmagambetov and Miguel Carreira-Perpinan. Smaller, more accurate regression forests using tree alternating optimization. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 11398–11408. PMLR, 13–18 Jul 2020.
[46] Arman Zharmagambetov and Miguel A. Carreira-Perpinan. Semi-supervised learning with decision trees: Graph laplacian tree alternating optimization. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 2392–2405. Curran Associates, Inc., 2022.