Empowering Decision Trees via Shape Function Branching
Authors: Nakul Upadhya
Eldan Cohen
Venue: NeurIPS 2025 poster ()
Keywords: interpretability
trustworthy AI
interpretable machine learning
decision tree
Primary area: general_machine_learning
TL;DR: We propose utilizing flexible shape functions to route samples to different branches in decision trees.

Abstract: Decision trees are prized for their interpretability and strong performance on tabular data. Yet, their reliance on simple axis‑aligned linear splits often forces deep, complex structures to capture non‑linear feature effects, undermining human comprehension of the constructed tree. To address this limitation, we propose a novel generalization of a decision tree, the Shape Generalized Tree (SGT), in which each internal node applies a learnable axis‑aligned shape function to a single feature, enabling rich, non‑linear partitioning in one split. As users can easily visualize each node's shape functions, SGTs are inherently interpretable and provide intuitive, visual explanations of the model's decision mechanisms. To learn SGTs from data, we propose ShapeCART, an efficient induction algorithm for SGTs. We further extend the SGT framework to bivariate shape functions (S$^2$GT) and multi‑way trees (SGT$_K$),  and present Shape$^2$CART and ShapeCART$_K$, extensions to ShapeCART for learning S$^2$GTs and SGT$_K$s, respectively. Experiments on various datasets show that SGTs achieve superior performance with reduced model size compared to traditional axis-aligned linear trees.

Empowering Decision Trees via Shape Function Branching
Nakul Upadhya, Eldan Cohen
Department of Mechanical and Industrial Engineering University of Toronto, Toronto, Canada
nakul upadhyammailllll........... utwronto.... ca, eldan e allaa.ccoen@utoooottoocoo
Abstract
Decision trees prized for their interpretability and strong performance on tabular data. Yet, their reliance on simple axis-aligned linear splits often forces deep, complex structures capture non-linear feature effects, undermining human comprehension of the constructed tree. To address this limitation, we propose novel generalization of decision tree, the Shape Generalized Tree (SGT), in which each internal node applies a learnable axis-aligned shape function to a single feature, enabling rich, non-linear partitioning in split. As users visualize each node's s shape function, SGTss are inherently interpretable and provide intuitive, visual explanations of the model's decision mechanisms. To learn SGTs from data, we propose ShapeCART, an efficient induction algorithm for SGTs. We further extend the SGT framework bivariate shape functions GS* and multi way trees (SGT x)) ), and present Shape and ShapeCART K. extensions ShapeCART for learning S2G and SG respectively. Experiments on various datasets show that SGTs achieve superior performance with reduced model size compared traditional axis-aligned linear trees.
easily
Introduction
Tabular datasets are collections of data organized into rows and columns, containing distinct features that can be continuous, categorical, ordinal and remain among the most widely used dataset types machine learning 2 Decisions trees are one of the most widely used approaches on these tabular datasets (3], largely due to their inherent interpretability [4] and robust performance
B. These characteristics have established them as a favored choice in high-stakes domains such as healthcare 55] 6) finance 7 (). and manufacturing [9 10 where the ability to understand the model predictions often just as important as high predictive accuracy. Structurally, decision trees organize nodes hierarchically, with internal nodes responsible for routing data samples based on defined criteria and terminal (leaf) nodes providing the final predictions. A common type of tree is the binary axis- sxis-aligned linear tree, where each internal node routes data to a left right child node based on a simple threshold comparison involving only a single feature: n. < (send to left feature d is less than equal to threshold 6).
While axis- axis-aligned linear trees are simple and interpretable, the impact of feature values on the target is often non-lineat requiring repeated splits on the same features at different levels of the tree to represent the feature-target relationship adequately (see Figure Tb). However, repeatedly employing single feature across multiple nodes in a decision path can hinder end-user comprehension of the model's learned relationship for that feature [IT] and necessitate larger tree with greater depth and node count to attain sufficient performance. This increase in size directly impacts interpretability. with empirical evidence confirming that tree comprehensibility highly sensitive to the depth of decision paths and the number of leaves 4 Furthermore, deeper trees inherently generate larger,
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

less interpretable local explanations [22 13 Finally, larger trees are also challenging to visualize effectively, often requiring complex, interactive tools gain insights [2 14].
Generalized Additive Models (GAMs) 15 are another class of interpretable models where predictions the result of the sum of individual feature contributions, each defined by potentially non-linear
shape function. Researchers have parameterized these shape fuucciinn using diverse model classes such as splines [15), trees [16 17, and neural networks [T8-20], enabling powerful modeling capabilities. Shape functions offer a key interpretability benefit as their flexibility enables GAMs to closely capture the true feature target dynamics in the data 2I while still allowing practitioners to
visualize each function independently, faithfully conveying the learned relationship [5] 21] GAMs
have also been extended incorporate pairwise feature interactions [17 19 20 [2 by adding
bivariate shape functions, creating Ms. Bivariate shape functions can be visualized via heatmaps, improving modelling capacity as they can capture second-order feature interaction effects.
22
Left
Bight
Left
Right
left
(a) Plus Sign dataset.
(b) Linear Axis-Aligned Tree
(c) SGT
Figure 1: Demonstration of SGT on simple toy dataset. As seen here, the SGT can achieve perfect accuracy with smaller depth and number of nodes compared to the linear axis-aligned tree.
In this work, we introduce the Shape Generalized Tree (SGT), a novel class of decision tree models where each internal node utilizes an axis- axis-aligned shape function to determine the routing of samples to child nodes. As shape functions can be parametrized by variety of rich models, they provide the capability to capture non-linear decision boundaries with respect a chosen feature in each node. The benefit of increasing the capabilities of individual nodes can be seen clearly in Figure Figure Ta visualizes simple two dimensional synthetic dataset. A traditional axis-aligned linear tree requires six splits with a maximum depth of four (Figure lb) to represent decision boundaries this dataset, while the SGT can achieve the same result with only two nodes with maximum depth of two (Figure Tel. To build SGTs, we propose ShapeCART, a novel and efficient SGT induction algorithm modeled after the CART framework [23] that constructs shape functions in each node via internal decision trees. Furthermore, we extend the SGT framework by incorporating bivariate shape functions in each node, resulting in GT, and by enabling the construction of trees with higher branching factors (i.e., multi-way trees), resulting in SGT k- Correspondingly, we extend the ShapeCART algorithm to accommodate these extensions, resulting in Shape" and ShapeCART
Overall our contributions are as follows: (1) We introduce Shape Generalized Trees (SGTs), new family of decision trees where internal node splits governed by interpretable axis-aligned shape functions that can capture non-linear feature effects; (2) We introduce ShapeCART, novel and efficient top-down induction algorithm for constructing SGTs; (3) We extend the SGT Framework to accommodate multi-way trees (SGTR) and bivariate shape functions in each node (S² GT) and modify ShapeCART accordingly; (4) We demonstrate across range of datasets that SGTs achieve superior performance with more compact trees compared to traditional linear tres.
2 Background
2.1 Decision Trees
Consider a supervised learning setting where we are given a dataset consisting of N data points X Jx where each x,, R is D-dimensional feature vector, and their corresponding

---

labels Y' = [yn I Within a decision tree framework, let D S X' x ) represent the subset of training data that reaches a specific internal node. The core mechanism of decision tree involves recursively partitioning the data 'D at each internal node based on a defined splitting criterion. Among the most common type of decision trees is the binary axis-aligned linear tree:
Definition 1 (Binary Axis-Aligned Linear Tree). In a binary axis-aligned linear tree, internal nodes partition the data D into two subsets, DD and 'Dr, based on threshold O applied to a single feature encoded by a one-hot vector w:
D = {xxx 3n) E D w X S O; W E (0, 1w/lo 1,0 D'=
(I)
common paradigm for learning binary-axis aligned trees is the Top-Down Induction of Decision Trees (TDIDT) strategy, greedy procedure that recursively partitions data to minimize predefined impurity measure, used in algorithms like the widely adopted Classification and Regression Tree (CART) 233. The appeal of these approaches lies in their empirical performance and efficiency, allowing trees to be trained quickly and scale well. Extensions like SEROT 13 incorporate inductive biases encourage path-level sparsity and improve interpretability. Other methods refine trees after initial training, such as Tree Alternating Optimization (TAO) (24) which iteratively updates nodes to boost predictive performance and prune the tree, and Hierarchical Shrinkage [25) which modifies the empirical target distributions in each leaf based on the parents of each leaf. Alternatively, global optimization methods such as DL8.5 26]. GOSDT 227, and OMT 28 aim to learn optimal trees but typically scale poorly on large datasets [26] 27 and often require pre-discretization of continuous features 28). Recently, there has been work on developing non- gonegyy near-optimal tree induction methods. Some notable approaches include DPDT [29], which formulates tree construction as an MDP, and SPLIT 30, which reduces the runtime of GOSDT (7 via lookahead methods.
Another important type of decision trees aims to enhance node expressivity by utilizing oblique splits, which allow internal nodes to split on linear combinations of multiple features:
Definition 2 (Binary M-variate Oblique Tree). In Binary M-variate oblique tree, nodes use linear combination of at most M features to partition the data:
D {(xn Un) E D w Xn S 0; WER? R. 1/Wlo S M, E R}
(2)
However, oblique trees are difficult to interpret on most datasets due to the high dimensionality of each cut, making it difficult for users reason about model decisions. Some notable approaches for learning unconstrained oblique trees (M- = D) include utilizing the TAO (24) framework and soft decision trees, which use a continuous relaxation of the binary oblique cut to learn a tree using gradient descent. Of particular relevance to our work are pairwise oblique models (M 2) such as BiCART 31,, BiTAO (31) and BiDT [3), which restrict each split to consider linear combinations between exactly two input features. This restriction allows the decision boundary in each node to be visualizable, unlike higher-order (M > 2) oblique trees, and have shown promise in constructing accurate and compact trees.
2.2 Interpretability
Model interpretability, revealing the internal mechanisms driving predictions, is crucial for fostering trust and enabling effective deployment, especially in high-stakes domains. Interpretability is typically understood across two primary scopes: global interpretability, concerning the model's overall logic and the relationships it has learned across the entire dataset, and local interpretability, explaining the model's prediction for a specific instance and the effect of small input perturbations (33). Approaches to achieving interpretability fall into two main categories: intrinsic model-based interpretability, where the model itself is structured to allow for direct understanding and reasoning about its predictions, and post-hoc interpretability, which uses external methods after training to approximate explanations for models lacking this inherent transparency [12 34 Intrinsic modell-aaeed interpretability. which allows for understanding at both local and global scopes, is facilitated by specific structural properties of the model, including: Modularity (or composability) where different parts of the model can be Simulability, where users can reason about and faithfully trace the model's decision process; [3 34 and Sparsity, where the complexity of the model is constrained,
interpreted independently [3 35
through both reducing the number of features used and constraining the number of distinct model
components user must analyze [34 35).
3

---

Decision trees are considered modular and simulable models, and provide mooeel-asee interpretability when their sparsity is enforced through limiting the total number of features used
T3
34 and
restricting the size of the tree, measured by node count and maximum depth [ 12 discussion on the interpretability of trees in comparison to other models can be found Appendix A
13
36 Further
3 Shape Generalized Tree
While effective in many scenarios, linear splitting criterion may result in feature being used multiple times given decision path, leading to larger trees where the learned relation between the feature and the target is difficult for users to understand. To address this drawback, we propose novel generalization of linear trees, the Shape Generalized Tree (SGT), that can represent non-linear decisions within a single node. More formally, we define binary axis-aligned SGT as follows:
Definition (Binary Axis-Aligned Shape Generalized Tree (SGT)). In a Shape Generalized Tree, internal nodes partition the data based on the output of flexible shape function applied to a single selected feature:
D' ('=i(x Bn)) fel xn)
0.0: f E {0, [0,1)":||/lo
(3)
Here, w is a one-hot feature selection vector where d = arg max () is the selected feature, and se dom((a)) RR is a flexible shape function parameterized by O f can be drawn from richer function classes, enabling the capture of more intricate, non-linear decision boundaries. As f operates only on one feature, the shape function is easily visualized and interpreted.
Additionally, we propose extending this generalization to accommodate bivariate shape functions in each node, leading the Binary Bivariate Shape Generalized Tree (S² model:
Definition 4 (Binary Bivariate Shape Generalized Tree GT)) In an S²GT, internal nodes partition the data based on a shape function that utilizes at most two features:
D = {(xx: Vn) E 'D (w( Xn, W Xn θ) S 0.0;
W1, W2 f0, 1) || llo, l| llo 1}
Here, W1, W2 are one-hot feature selection vectors where d1 = arg max wax d2 = arg max W2 are the selected features, and fe dom(Xá,) x dom(Xáa) R is shape function that maps the pair of features to an output branch. The Binary Bivariate Oblique Cut (Definition) 2 can be seen as a special case where (21, 22) = O + O2 Oo.
One significant advantage of employing shape functions is that they can be extended to accommodate multi-way branching (i.c. routing data into K > 2 child nodes simultancously). This facilitates the creation of the K-Wan -Wy Axis- Aligned Shape Generalized Tree (SGT k) model:
Definition (Multi- Way Axis-Alignes Aligned Shape Generalized Tree (SGT In an SGT K, internal nodes partition the data into up to K distinct subsets based on the output of a vector-valued shape function applied single feature:
D* = =[(xxx Yn.) E D arg max
= (0,1), [[/lo 1}
k;
(5)
kE(1....K)
Here, the shape function f(K) dom(Xa) R,K = arg max w maps the selected feature value to a K K-dimensional vector, and the argmax operation determines the branch assignment.
Although our model naturally extends to any K, we limit our exploration of SGT to K 3 to maintain sparsity and, consequently, a high degree of interpretability. By integrating the bivariate splitting and K- K-way branching, we obtain S2GT k trees, which support multi-way splits based on non-linear interactions between two features (formal definition in Appendix B
Expressiveness Guarantees for SGTs We formally establish that SGTs are more expressive than binary exis-aligned linear trees by showing that: (1) SGTs are at least as expressive as binary axis- aligned linear trees with similar number of decision nodes; (2) SGTs can be strictly more expressive than axis-aligned linear trees with similar number of decision nodes. More formally:
Theorem 1. Every function that can be represented by a binary axis-aligned linear tree (Definttion 1. can be represented by an SGT (Definition) 3) with the same number of decision nodes.

---

Algorithm 1: Selecting Shape Function (Node Level Outer Problem)
Input :Features X, Target y. Branching Factor K, Pairwise Candidate Limit P. Coordinate Descent
Iterations R. Pairwise Reg. 7, Branching Reg. A, Impurity H(-), Weighted Impurity Z()
Output of branching partitions D Best Shape Function ()), Best Weighted Impurity L
L' - oc; (-) t None; D + None
// Initialize
for < 1 to D de
// Iterate through all features Ld. fa(:). Da <- FitShapeFunction/22 V, H()), K, A, R,d)
if La L" then
L + La; + fa(:;:D' Da
end
P 0 then
for (d1, d2) E PairviseCommiinttions//D do
1/ Filter Interactions (Section 4.2)
Diserven (D'nD' Da, D
dd)) mimmra))
end Ap t- Best values for (d),d2) E Ap do ((d))d)) fid1)))(-), Dididt + FiShapcFunction(A2, y, H(-)), K, X, R, (21, // Add Pairwise Penalty
// Iterate through candidate pairs of features
((()) Lidi +?
if d2) < L' then L' + La: f*(+) + ()d + ((d1.)2)
end
return D' ,','"
Theorem 2. For every B E N, there exists a function for which a binary axis-aligned tree requires at least B additional decision nodes to represent, compared an SGT.
To prove Theorem we demonstrate that axis-aligned linear trees are special case of SGTs. To prove Theorem we construct a family of neediiensional labeling functions with periodic boundaries and show that SGT can represent any function in this family using fixed number of nodes, while an axis-aligned linear tree requires a number of nodes that grows linearly with the number of intervals. Complete proofs for both theorems can be found in Appendix C
4 ShapeCART
To learn SGTs from data, we propose ShapeCART, novel and efficient tree-induction algorithm inspired by the classic Classification and Regression Tree (CART) framework [23] Furthermore, we extend the core ShapeCART algorithm to learn SGT and S2GTs, creating ShapeCART, and Shape?CARR CART respectively. For simplicity, we present our methods in the classification setting, where each label yn. E ) takes values in finite class set C, but provide detailed description of the regression variant Appendix (Pseudocode in Appendix K). decomposing the global optimization into greedy, recursive node-level learning tasks similar CART. At each node, we aim to learn feature selection parameters w and shape-function parameters θ that minimize the weighted impurity of the empirical class distributions the resulting child nodes. We pose this node-level problem as a bi-level optimization: (1) An inner problem to learn the optimal chape-function parameters for each candidate feature and (2) outer problem to select the shape function that yields the lowest impurity:
[ ShapeCART constructs a decision tree via the TDIDT paradigm
min we (0.1) PI wil
Σ
wa min [ ((D), D51)]
(6)
d=l
s.t. Dj ((x,, vn. E D fa(zn,a) 0.0)
Here, fa(:) (-) and L(-) is the weighted impurity of the resulting branches, defined for any superadditive impurity measure H (e.g. Gini 23 or entropy as:
L(D) Σ (D)H(IT(D)): ITD). DED
Σ
1 c) Vc C
[
(I..WW)ED
5

---

To solve this problem, we construct shape functions for each feature and select the one with the lowest weighted impurity (node level pseudocode in Algorithm D Additionally, this formulation naturally extends to implementing SGT' (Equation 5 by adopting a multi-way shape function in lieu of the binary one, creating ShapeCART
Extending to CART lation to consider both individual feature shape functions and bivariate shape functions. The resulting bi-level optimization problem is as follows:
CART To extend ShapeCART to Chape we extend the above formu-
min w.W
Σ
wa min 0a
[c ((D), DjH)] +
(8)
Σ
W(d
min
F (DDannds) D Pean + +// +l/lo/oo
(2d).d2)
(d) E2)
s.t. Dj = {(xn:3n) Un) E E [a( 5.0)
D( = ((xxn Un) E [(d
( (Fn, ( 1 0.0}
||wlo + [
1), W (0,1)
Here, is one-hot univariate feature selection vector, W is one-hot bivariate interaction selection matrix, is a set representing all pairs of features, and y is a tuneable regularization hyperparameter added penalize selection of bivariate shape function encourage choosing axis-aligned splits unless the bivariate function leads to a significantly lower loss.
4.1 Learning Shape Functions
Recall that for a given feature or pair of features, we would like to learn a shape function Sa(.) (or dz)) for bivariate trees) that assigns samples to branch (e.g., left/right or I,..., K in the multi-way case) such that we minimize the impurity of the empirical class distributions of the resultant branches. To learn this shape function efficiently, we propose two-stage approximation strategy:
1. We first learn function T(-) : - (1....,L that maps samples L mutually exclusive bins based on the value of the d-th feature (or (di, d2) for bivariate shape functions).
2. We then compute an assignment of bins to branches (e.g., left/right) that minimizes the impurity over the induced partitioning of the data.
4.1.1 Binning the samples
We choose represent the binning function T(-) via an internal decision tree. For univariate shape functions, we use CART to train a binary axis-aligned linear tree only on the relevant feature and the target construct L mutually exclusive bins that approximately minimize the chosen impurity criteria H. For bivariate shape functions, we instead train a binary bivariate oblique tree using BíCART (31] the pair of features under consideration and the target. The use of CART and BiCART affords us control over model complexity; by constraining parameters such as the number of leaves and the minimum of samples in each Icaf tree depth, one can ensure that the resulting piecewise-constanl shape function comprises a bounded number of meaningful segments. Each bin / = 1,.... L stores its empirical class distribution тe E R| and weight W, E R, representing the number of samples that fall into the bin. Note that while utilize CART, this step be performed with variety of approaches, and we provide results using an alternative tree induction algorithm (DPDT (29] in Appendix[]
4.1.2 Mapping Bins to Branches
Given the bins from the first stage, our goal is to assign each bin to an output branch (e-g. 1, r) such that we minimize the weighted impurity of the resultant branches. Formally, let a E {1,r}² denote

---

the branch assignment vector. We aim to solve the following discrete optimization problem:
min
(IH + W. -((II,)
(10)
a
L WR Σ 1(a = k)We 11t E 1(ar= = =)
s.t.
(11)
WR
E=1
To solve this optimization problem in a scalable manner, we adopt a coordinate descent procedure.. At each step, we determine the assignment of a single bin while keeping all other assignments fixed. This reduces the optimization problem to iteratively evaluating the impurity resulting from assigning l to each possible branch and selecting the assignment that minimizes the objective. The process is repeated for iterations, with the order of leaf updates randomly shuffled in each pass to mitigate ordering bias (Pscudocode in Appendix K). Coordinate descent guarantees monotonic objective decrease and finite convergence given the discrete search space [38], with time complexity scaling linearly in the number of leaves (Appendix D). However, its sensitivity to initialization suboptimal local minima (38]. To obtain a strong initialization, we choose the better of the following two strategies for setting (1) cluster assignments obtained by applying Weighted K-Means to the empirical distributions of the bins we, weighted by W6; or (2) bin assignments based on whether each bin falls to the left or right of the root node in the inner CART tree.
lead to
Based on the procedure detailed above, we can derive a lower-bound on the information gain at each deci- sion node with respect to the informa- tion gain at each node in CART:
Algorithm 2: Fit Shape Function
Input :X', V, R, K, X, H(-), or (21,d2), Output: Li=,, fa(-)
if was provided then
Lemma 1. Lett be an individual node in T, let IG(t) denote the informa- tion gain obtained by an axis-aligned (threshold) split on the samples in from CART, and let IG (t) denote the information gain obtained by shape function split from ShapeCART on the sumples in L. We show that IG,(1) 2 IG(I).
+ FiC
else
T + FiBiCART Xaz.
end
(W1..... WL), (Tiy..., + ExtractBinStass(TT
[(0)) (0) f- FarmacRoonAssigneen((((
for l: < 2 to K do
(0) WKM L L, + [Wi...,,Wi.
L (0) < L k. WKW then
Given this lemma, ShapeCART inher- its the bound on information gain for any terminal node of the tree estab- lished for CART by Klusowski and Tian [39]. Proof of Lemma can be found in Appendix C.2
(0) <- a(0). root
else
a -- a(0),KMM
end
(a", Lx) + CoordDescent (mm [ m.];
[Wi.... WL]: k:; (:);;;H(+)): I7)
end
k* + arg min (Li + X(k {D) (D......, D* t- RettieweBracchAsiinmnenng a*
Extending to Multi- Branching ShapeCART can be adapted to the multi-way setting, ShapeCARTx. by increasing the number clusters in the Weighted K-Means initialization and considering more options for a; in each step of the coordinate descent procedure. To regular- ize ShapeCART x and prevent the creation of branches that do not provide a significant impurity decrease, we run the Weighted K-Means initialization and Coordinate Descent procedure for each (,...., K} and add a penalty term A(k - 2) to the weighted impurity of the assignments for that k. where À is hyperparameter. We then select the & with the minimum penalized impurity.
f(-) < aT()
return Las, (().(D)..... D²'}
In summary, the shape function for given feature fa(-) is represented jointly by an internal tree Ta(-) which maps samples to bins, and a lookup vector a which maps bins to branches (Pseudocode in Algorithm 2). The time complexity of constructing shape function O (NC log + K²T.C) where C' is the number of classes (see Appendix D for detailed analysis).
7

---

4.2 Limiting Bivariate Shape Function Construction
Recall that to solve the optimization problem presented in Equations 8 we need to construct all univariate and bivariate shape functions to select the one that minimizes the weighted impurity of the resultant branches. A naive approach of constructing a shape function for each pair of features will require constructing C (D2) shape functions, incurring significant computational cost. Instead, we propose a heuristic that utilizes the computation of univariate shape functions to identify a few promising pairs for which shape function will be constructed. We first construct univariate shape functions fi..... fD and denote the set of branches resultant from applying each feature's shape functions the input data as Da = {D), Dj) Vd = L.... D (or Da {D) d..... ., Dí } in the multi-way case). Our goal is to identify pairs of features (d1, d2) that have the potential to have an impurity that is significantly lower than the best impurity of the individual features. We achieve this by calculating the weighted impurity of the sets induced by the Cartesian product of the branching sets of a pair of features and subtracting this value from the best impurity obtained from the univariate shape functions in the pair:
((d),d)) = ((D))) - L((D) n (D') E Dd, X Ddg3)
(12)
We retain the P pairs with the highest 8 values and denote this set as Ap, which we use in lieu of the full set of pairs (A) in Equation g With our heuristic, we only require O((N + CK2) operations to construct the intersection of sets followed by the construction of P' shape functions (O(P N + K LC') vs. ODD N + K LC)). Assuming P < D2, this heuristic can significantly reduce runtime (evaluated in Appendix G.2.3). The choice of P is guided by the user's computational preferences; higher P increases runtime return for increased performance.
4.3 Post-Processing
TDIDT approaches are efficient and allow trees to be trained quickly. However, one downside of TDIDT approaches is their greedy behavior, potentially resulting in globally-suboptimal trees [4 (2)] To overcome this, we globally refine the tree constructed via ShapeCART using Tree
Alternating Optimization (TAO) (24 (1). This post-processing procedure refits the shape function at each internal node of the constructed tree using the predictions of the subtrees rooted at the node's children, while also pruning nodes that do not significantly reduce error. While the original TAO algorithm is restricted to binary trees, we extend it to support K-way branching. Full implementation details and our modifications to TAO are provided in Appendix F
5 Experimental Evaluation
In this section, we evaluate the performance of trees induced by ShapeCART (SGT-C), ShapeCART, (SGT3-C), Shape"CART (S=GT-C) and Shape"CART, GS²GT3-C) against various benchmark ap- proaches on a range of 26 real-world classification datasets (details in Appendix EE. We also present results on the TAO refined ShapeCART models denoted as SGT-T. 5G T. S GT-T. and -T. We benchmark our uxis-aligned approaches against CART [23), SERDT [I3], HSTree (25), Axis- Aligned TAO (AxTAO) [24]. DPDT [29]. and SPLIT (30). We evaluate our bivariate approaches against the newly proposed BiCART and BiTAO algorithms BD. CART, SERDT, HSTree, and BiCART are greedy TDIDT induction approaches, while AxTAO, DPDT, SPLIT and BiTAO are non-greedy induction approaches. Implementation details for all models evaluated can be found in Appendix(F [
Hyperparameter Search and Evaluation In our experiments, we report results for all approaches across reasonable tree depths (2-6), following Souza et al. [13] and for the overall best model. Each dataset is split into three folds using a 70/30 train/validation-ees split, with the non-training data further divided in the same ratio. Hyperparameters are optimized via Bayesian search with Optuna 40 using 50 trials per model and depth to ensure faimess across differing search spaces. Each configuration is scored by mean validation accuracy, and we retain the best per depth and overall. Training time is limited to 15 minutes for univariate and 30 minutes for bivariate models. All runs are executed on GCP N2 instances (8 vCPUs, 32 GB RAM). Hyperparameter search spaces for all evaluated approaches and hyperparameter importance analyses for all ShapeCART variants appear in Appendix F.T We present average test accuracy results across datasets below, and additional evaluations and ablations are presented in Appendix G
8

---

electracAty
eye ey0-11vve on15
0ye $1010
10.00
77.5%
75.0%
15.0%
72.58
0..0%
$0.00
18.0%
45.0%
0A-T
SEROT
H5Tree
507.
507,c
877 17.55
00.00
75.0%
15.0%
70.0%
The
10.0%
15.0%
56.0%
75.0%
D6277
Dape7
1x0
SPLIT
DPST
501T
501y-7
Figure 2: Test accuracy per depth for axis-aligned approaches on eye-movements, elceiissellliiiii electricity, and eye-state. Top: IDDDDe-Approcchss. Bottom: Non-Greedy Approaches.
Table I: Average Test Accuracy (%) across the datasets tested. Top: TDIDT approaches, Bottom:
Non-Greedy Approaches. Best approach per block bolded and second best is italicized.
Axis-Aligned Approaches
(b) Bivariate Approaches
Model
2
Best
Model
5
Best
CART
83.9
81.6
82.3
84.0
85.1
85.2
BiCART
87.3
87.6
87.9
88.7
89.6
89.6
SERDT
83.7
80.9
81.7
83.7
85.1
85.1
SCGT-C
89.3
89.1
89.5
90.5
91.3
91.4
HSTree
83.9
81.6
82.4
84.0
85.1
85.1
S
90.0
90.2
91.1
91.8
91.5
91.7
SGT-C
84.5
82.5
83.7
84.9
86.2
86.3
BiTAO
87.9
88.1
88
89.3
90.1
90.0
C
85.7
84.6
85.9
87.3
87.8
87.8
S
89.7
90.0
90.2
91.2
91.7
91.6
AxTAO
83.9
82.1
82.8
84.5
85.5
85.4
STGT,-T
90.2
90.6
91.2
91.7
92.4
91.9
DPDT
84.9
83.4
83.8
85,2
86.2
86.2
SPLIT
81.7
76.6
77.9
79.5
80.2
80.3
SGT-T 85.0
84.6
86.8
86.8
SG 86.4
86.2
88.0
88.0
Results on Axis-Aligned Trees Table Ia reports the average test accuracy of axis-aligned ap- proaches across datasets. We observe that SGT-C consistently outperforms baseline TDIDT methods every evaluated depth. Moreover, SGT3-C achieves superior performance both overall and at each depth, highlighting the benefit of higher branching factors. It is important to note that despite the increase in the number of nodes, ternary trees have the same size of local explanation as
binary trees of the same depth [3]. Among non non- greedy methods, we observe that the TAO-refined SGTs attain the highest performance across all axis-aligned approaches. At the dataset level (Figure 2). SGT-C often matches or exceeds the maximum-depth accuracy of other TDIDT methods with substantially shallower trees. A similar, though less pronounced, pattern observed for SGT-T when compared to deeper non- non-greedy methods.
Results on Bivariate Trees our evaluation bivariate approaches, we exclude Covertype. Eucalyptus, and Higgs as BiCART and BiTAO require more than the 32GB of available memory on these datasets. When looking the average performance of the bivariate models on the remaining datasets (Table lb), we observe that S²GT-C and S²GTT consistently outperform BiCART and BiTAO, respectively, across all depths as well as overall. Like in the univariate case, we observe that higher branching factor helps improve expressivity as both S²GT,-C and S²GT outperform S*GT-C and S*GT- respectively. We also observe that bivariate shape function branching can significantly compress needed tree size in many cases. More specifically, we observe that on the eye-movementa electricity datasets (See Figure G.4 in Appendix), GTGT with depth of two achieves similar performance to both BiTAO and BiCART with depths of six.

---

Extied ylas
Correct Klaei
Figure 3:
Visualization a SGT induced via ShapeCART (max depth 3) trained on the eye-movements dataset. Purple indicates regions where samples will be directed to the left child; Orange directed to the right.
Visualization To illustrate the interpretability of SGTs, visualize trained ShapeCART model on the eye-movements dataset [41] in Figure 3 We observe that multiple nodes benefit from non- linear shape functions. For example, the decisions represented in Nodes 3, 9, and 10 would each require at least three additional splits in a linear axis-aligned tree. Additional examples, including visualized ShaPe"CART, ShapeCARTs, and ShaPe" trees, are provided in Appendix H
6
Conclusion
This paper introduces the Shape Generalized Tree (SGT), novel generalization of traditional linear decision trees that empowers each node with shape-function branching. Shape Generalized Trees retain the inherent interpretability that makes decision trees a leading approach as the shape functions learned each node can be easily visualized, allowing users understand SGT's internal decision mechanisms. We also introduce ShapeCART, an efficient top-down induction algorithm to construct these SGT models. Additionally, we extend our framework to support bivariate SGTs GT) and multi-way SGTs (SGTx) x), ) and present the corresponding ShapeCART variants (Shape"CART and ShapeCART, k). Overall, we show that SGTs outperform linear trees on various datasets across various model sizes.
Limitations As tree-based models, Shape Generalized Trees (SGTss inherit intrinsic constraints of traditional decision trees, most notably that they are designed primarily for tabular data. Furthermore, interpreting the shape functions at each node can be more challenging than understanding axis- aligned linear splits, potentially increasing the cognitive effort required for analysis. Nevertheless, SGTs preserve essential interpretability properties, including simulability and modularity, while their enhanced expressiveness enables the construction of smaller trees without compromising predictive performance, thereby improving the sparsity of local explanations. Despite these advantages, further work is needed to systematically evaluate the interpretability of SGT's, and human- human-subject subject studies, similar those conducted for GAMs [42 present a valuable direction for future research.
Broader Impact SGTs constitute an inherently interpretable class of models, and our promising results inducing them via ShapeCART may inspire other future inherently interpretable approaches, thereby enhancing transparency and accountability in machine learning systems. Exploring alternative shape function representations representations, such as neural networks used in GAMs 20), and developing clustering SGTs 45] are promising directions for future work. Furthermore, our work opens interesting directions for future work further theoretically characterize SGT's and ShapeCART, including extending theoretical properties of CART such as consistency rate the impact of sparsity 47), and tighter empirical risk bounds 339
[39 46
Acknowledgments and Disclosure of Funding
This research was supported by grant number DSI-CGY3RIP1 IP18 from the Data Sciences Institute the University of Toronto. The authors also gratefully acknowledge funding from the Natural Sciences and Engincering Research Council of Canada (NSERC).
10

---

References
[1] Duncan McElfresh, Sujay Khandagale, Jonathan Valverde, Vishak Prasad C, Ganesh Ramakr- ishnan, Micah Goldblum, and Colin White. When do neural nets outperform boosted trees tabular data? Advances in Neural Information Processing Systems, 36:76336-7600 76336, 2023.
[2] Ravid Shwartz-Ziv and Amitai Armon Tabular data: Deep learning not all you need. Information Fusion, 81:84-90, 2022.
[3] Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data? Advances in neural information processing systems, 35:507-520, 2022.
[4] Mitja Luštrek, Matjaž Gams, Sanda Martinčić-Ipšié, al. What makes classification trees comprehensible? Expert Systems with Applications, 62:333-346 2016.
[5] Gregor Stiglic, Primoz Kocbek, Nino Fijacko, Marinka Zitnik, Katrien Verbert, and Leona Cilar. Interpretability machine learning-bace prediction models in healthcare. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(5):e1379, 2020.
[6] K. Srinivas, G. Raghavendra Rao, and A. Govardhan. Analysis of coronary heart disease and prediction of heart attack in coal mining regions using data mining techniques. 20/0 5th International Conference Computer Science & Education, pages 1344 1349, 2010. doi: 10.1100//CCEE.2010055931
[7] Jozef Zurada. Could decision trees improve the classification accuracy and interpretabilityy of loan granting decisions? In 2010 43rd Hawaii International Conference on System Sciences, pages 1-9. IEEE, 2010.
[8] Pan Tang, Tiantian Tang. and Chennuo Lu. Predicting systemic financial risk with interpretable machine learning. The North American Journal of Economics and Finance, 71:102088, 2024. ISSN 1062-9408. doi: http:://bbooogg///o.lol////mm//p
URL sattps//www
sciencedirect.comm wateer/edrrett.cc//cciiceee////e htt.:///ew...e..oot......//////n
[9] Choi Sungsu, Battulga Lkhagvadorj. Aziz Nasridinov, Kwan-Hee Yoo, et al. A decision tree approach for identifying defective products in the manufacturing process. International Journal of Contents, 13(2):57-65, 2017.
[10) Sadeq Heydarbakian and Mehran Spehri. Interpretable machine learning to improve supply chain resilience, an industry 4.0 recipe. PFACC-PapersOnn.iire (5(10))2834-883) 2022. ISSN 2405- 8963, doi: https://doi. 1ttp//11.11161/11 hfto///..acol..222..... 10. URL http::://////......m //wp:::/wwww... sci ociencedirect comm cen/science/ artaaaleeeea.ea /ttp:///rrr...////////////000010 ing Modelling, Management and Control MIM 2022.
/52405896322021777
10th IFAC Conference on Manufactur-
[11] Federica Di Castro and Enrico Bertini. Surrogate decision tree visualization. In IUI Workshops, 2019.
[12] Christoph Molnar. Interpretable machine learning. Lulu. com, 2020.
[13] Victor Feitosa Souza, Ferdinando Cicalese, Eduardo Laber, and Marco Molinaro. De- cision trees with short explainable rules. In S. Koyejo, S. Mohamed, A. Agar- wal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Informa- tion Processing Systems, volume 35, pages 12365-12379. Curran Associates, Inc., neurips. cc/ httprrrppspppppppp...eeeeea/epee 2022. URL S0067779914ffe0odcceeeeee411eaa3 www.epppppeeereeeeeeeeeeeeeeeeee htt.r//eeeeaappe/22////a
[14] Yan Liu and Gavriel Salvendy. Design and evaluation of visualization support facilitate decision trees classification. International journal of human-computer studies, 65(2):95-110,, 2007.
studies
[15] Trevor Hastie. Generalized additive models. Statistical models S, pages 249-307, 2017.
[16] Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. Interpretml: A unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223, 2019.
11

---

[17] Chun-Hao Chang, Rich Caruana, and Anna Goldenberg. Node- Neural generalized additive model for interpretable deep learning. arXiv preprint 2ttt:220060016133 2021.
[18] Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey E Hinton. Neural additive models: Interpretable machine learning with neural nets. Advances in neural information processing systems, 34:4699-4711, 2021.
[19] Filip Radenovic, Abhimanyu Dubey, and Dhruy Mahajan. Neural basis models for interpretabil- ity. Advances in Neural Information Processing Systems, 35:8414-844 2022.
[20] Nakul Upadhya and Eldan Cohen. Neurcam: Interpretable neural clustering via additive models. In ECAI, 2024.
[21] Chun-Hao Chang, Sarah Tan, Ben Lengerich, Anna Goldenberg, and Rich Caruana. How interpretable and trustworthy are gams? In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining, pages 105, 2021.
[22] Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 623 023-631, 2013.
[23] Leo Breiman. Classification and regression trees. Routledge, 2017.
[24] Miguel A Carreira-Perpinaae and Pooya Tavallali. Alternating optimization of decision trees, with application to learning sparse oblique trees. Advances in neural information processing systems, 31, 2018.
[25] Abhineet Agarwal, Yan Shuo Tan, Omer Ronen, Chandan Singh, and Bin Yu. Hicrarchical shrinkage: Improving the accuracy and interpretabiliit of tree-based models. In International Conference on Machine Learning, pages 111-135. PMLR, 2022.
[26] Gaël Aglin, Siegfried Nijssen, and Pierre Schaus. Learning optimal decision trees using caching branch-and-boumt search. In Proceedings of the AAAI conference artificial intelligence. volume 34, pages 3146-3153, 2020.
[27] Jimmy Lin. Chudi Zhong, Diane Hu, Cynthia Rudin, and Margo Seltzer. Generalized and scalable optimal sparse decision trees. In International conference machine learning, pages 6150-6160. PMLR, 2020.
[28] Shivaram Subramanian and Wei Sun. Scalable optimal multiway-split decision trees with constraints. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 9891-9899 2023.
[29] Hector Kohler, Riad Akrour, and Philippe Preux. Breiman meets bellman: Non-greeds decision trees with mdps. In KDD 2025-The 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2025.
[30] Varun Babbar, Hayden McTavish, Cynthia Rudin, and Margo Seltzer. Near optimal decision trees in a split second. arXiv preprint 250.::::250...888. 2025.
[31] Rasul Kairgeldin and Miguel Á Carreira-Perpiñánn Bivariate decision trees: Smaller, inter- pretable, more accurate. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 1336- 1366,117,,50 2024.
[32] Ferdinand Bollwein and Stephan Westphal. A branch & bound algorithm to determine optimal bivariate splits for oblique decision tree induction. Applied Intelligence, 51(10)7552--572. 2021.
[33] Dylan Slack, Sorelle A Friedler, Carlos Scheidegger, and Chitradeep Dutta Roy. Assessing the local interpretability of machine learning models. arXiv preprint arXiv: 1902. 03501, 2019.
[34] W James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences,
2019.
12

---

[35] Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv: (etii::772.08000 2017.
[36] Dimitris Bertsimas and Jack Dunn. Optimal classification trees. Machine Learning, 106:
1039-1082, 2017.
[37] Ross Quinlan. Induction of decision trees. Machine learning, 1:81-106, 1986.
[38] Stephen J Wright. Coordinate descent algorithms. Mathematical programming. 151(1)33-34, 2015.
[39] Jason M Klusowski and Peter M Tian. Large scale prediction with decision trees. Journal of the American Statistical Association, 1 1195555):255533 537, 2024.
[40] Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pages 2623-2631, 2019.
[41] Jarkko Salojärvi, Kai Puolamäki, Jaana Simola, Lauri Kovanen, Ilpo Kojo, and Samuel Kaski. Inferring relevance from eye movements: Feature extraction. In Workshop at NIPS, page 45, 2005.
[42] Stefan Hegselmann, Thomas Volkert, Hendrik Ohlenburg, Antje Gottschalk, Martin Dugas, and Christian Ertmer. An evaluation of the doctor-interper iectoo-interpretabblity of generalized additive models with interactions. In Machine Learning for Healthcare Conference, pages 46-79. PMLR, 2020.
[43] Eldan Cohen. Interpretable clustering via soft clustering trees. In International Conference on Integration of Constraint Programming, Artificial Intelligence, and Operations Research, pages 281-298. Springer, 2023.
[44] Nave Frost, Michal Moshkovitz, and Cyrus Rashtchian. Exkme: Expanding explainable k-means clustering. arXiv preprint arXiv:2006.00 202.29000..2399, 2020.
[45] Pouya Shati, Yuliang Song, Eldan Cohen, and Sheila Mellraith. Optimal decision trees for interpretable and constrained clustering. Journal of Artificial Intelligence Research, 84, 2025.
[46] Qin-Cheng Zheng, Shen-Huan Lyu, Shao-Que Qun Zhang, Yuan Jiang, and Zhi-Hua Zhou, On the consistency rate of decision tree learning algorithms. In International Conference Artificial Intelligence and Statistics, pages 7824-7848. PMLR, 2023.
[47] Jason Klusowski. Sparse learning with cart. Advances Neural Information Processing Systems, 33:1 161211111.,,,, 2020.
[48] Trevor Hastie, Robert Tibshirani, Jerome Friedman, et al. The elements of statistical learning, 2009.
[49] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, Prettenhofer, R. Weiss, V. Dubourg. J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2830. 2011.
[50] Habiba Muhammad Sani, Ci Lei, and Daniel Neagu. Computational complexity analysis of decision tree algorithms. In Max Bramer and Miltos Petridis, editors, Artificial Intelligence XXXV, pages 191-197, Cham, 2018. Springer International Publishing. ISBN 978-1-030-04191-
[51] Sebastian Raschka. Stat 451: Machine learning lecture notes. University of Wisconsin 2020.
[52] Greg Hamerly and Jonathan Drake. Accelerating lloyd's algorithm for k-means clustering. Partitional clustering algorithums, pages 41-78. 2015.
[53] Chandan Singh, Keyan Nasseri, Yan Shuo Tan, Tiffany Tang, and Bin Yu. imodels: a python package for fitting interpretable models, 2021. URL hntps::: //doi.
03192
13

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: We describe all main claims in abstract and provide list of contributions in our introduction.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: See Conclusion
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification: See Appendix for time complexity proof.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
We provide detailed instructions on our experimental configuration, allowing others to reproduce our experiments if needed. Additionally, we provide detailed pseudocode on how to implement our algorithm.
Justification:
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We utilize open source datasets in our experiments and provide code to replicate our experiments.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: See Section
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We provide standard deviation across folds when applicable in per-dataset results the supplementary material.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
14

---

Justification: See Section 5 for compute resources used.
9. Code of ethies
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification: We agree and confirm our work follows the NeurIPS code of ethics.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
The goal our work is provide an improved decision tree algorithm that addresses many of the interpretability concerns with binary axis- axis-aligned trees. As discussed in the introduction, increased interpretability is important in high-stakes, sensitive fields, and improving interpretability has positive societal impact.
Justification:
11. Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g.. pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification:
Our approach has minor risk of misuse and only consider open source
datasets.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Original producers of relevant code are cited.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No assets introduced
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: No human subjects used.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No study participants.
16. Declaration of LLM usage
15

---

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs are not core part of our work.
16

---

A Interpretability Discussion
Decision trees are widely regarded as modular and simulable models, offering model-based inter- pretability when their complexity is controlled through constraints on the number of features used and through structural restrictions such as limiting the total node count and maximum depth As decision tree variant, the SGT inherits this modularity, while the restriction that each internal node operates on only one or two features preserves simulability. In particular, each node's shape function can be directly visualized and queried to support transparent predictions. For example, consider the SGT trained on the myemmoeeennts dataset (Figure B). For a sample with 4, the tree yields local explanation of the form: "The model predicted (Relevant, Class 1) because regress Dur < 220, lastSaceLen < 5, and In comparable or superior performance with smaller trees, thereby improving sparsity in the resulting explanations and aligning with the desiderata for interpretability proposed by Murdoch et al. [34. Note that higher-order oblique trees (more than features per node) are not naturally visualizable and considered less interpretable (3I] We similarly excluded SGT' with higher-order shape functions beyond bivariate for the same reason.
11 3 36..
regressDur= = 100. lastSaccLen 0, and nWords
[4, 5]." Moreover, the enhanced expressivity of SGT's enables them achieve
One benefit of decision trees that the SGT inherits is that the local explanation length of an SGT is naturally bounded: a user needs to examine at most one piccewise-constant shape function per decision node along path, bounding the explanation's size to the tree's depth. Consequently, interpretability is preserved only if the maximum depth sufficiently small. For this reason, and following prior work compact decision trees [ 29 30
cap the depth of our models six.
Despite being inspired by the feature specific transformations of GAMs, SGTs and GAMs are not directly comparable. In GAMs, a separate shape function is learned for each feature (and for each class multi-class settings, yielding [C] shape functions per feature). As a result, the size of a local explanation GAM is identical its global explanation, since all shape functions must be considered simultaneously. In contrast, decision trees bound the size of a local explanation by the depth of the tree, and the global explanation by the number of nodes (at most 2 1 for depth d). Due to these reasons, GAMs are not directly comparable to trees and we do not include them in our evaluation.
SGTs and GAMs:
B G Formulation
Definition 6 (Binary Multi-Way Shape Generalized Tree GT² In an S'GT internal nodes partition the data into at most K distinct subsets based on the output of a vector shape function that utilizes at most two features:
D* {(Xxny Vn) E D
()2( Xn, W/ Xn, 0)) k;
arg max ke(l.....
(13)
W1. W2 {0, 1) || /wwllo- =1
Here, WI, W2 are one-hot feature selection vectors where d1 arg max W d2 arg max W2 the selected features, and f2.(K) dom (a, dom(Xe, + RRK is a shape function that maps the selected features to a K -i-dimnnsoonl vector, and the argmax operation determines the branch assignment.
C Proofs
C.1
Expressiveness Guarantees
Theorem 1. Every function that can be represented by a binary axis-aligned linear tree (Definition Zp. can be represented by an SGT (Definition 3) with the same number of decision nodes.
Proof. Given binary axis-aligned linear tree, we construct an SGT with similar structure where the shape function in each decision node takes the form fe(w (w x) = w T x,, - O where w and match the values in the corresponding decision node in the axis- axis-aligned linear tree.
17

---

Definition 7 (w-Barr Labeling Function). Let w E N be a frequency parameter. We define the Bars Labeling function g [0, 1] - {0, 1} as follows:
g(x) - (
if cos(2 - 0,
(14)
otherwise.
The w- -Brrs Labeling Function partitions the unit interval [0, 1] into alternating subintervals according
to the sign of cos cos(22ww)) (2mwx).. Within [0, 11, the cosine function completes w full oscillations, producing
+ 1 zero-crossings and hence + 2 disjoint subintervals of equal width. These subintervals alternate between label and label O.
We show that an SGT can directly utilize nonlinear shape functions to represent all functions in this family in fixed number of nodes, while the number of nodes required by an axis-aligned linear tree scales linearly with the number of rectangles utilized by each function.
Lemma 2. The w- w-Bars labelling function can be represented by an SGT with one internal node for any value of w.
Proof. The w- w-Bars Labelling function can be represented by an SGT in single node by utilizing shape function of the form (2) cos(2 where @ learnable parameter. Recall that samples are assigned the left partition if fe x,) S 0.0. As this is univariate dataset, select the only feature by setting w = [1]. Additionally, we set @ = w, therefore samples satisfying (21) cos(2wwzj) S 0 form the left subset, which is assigned to the positive class. The remaining points form the right subset and are assigned to the negative class. The resulting tree matches the definition of the Bars labelling function.
Lemma 3. A binary axis aligned linear tree requires at least internal nodes represent the Bars labelling function.
Proof. A binary axis exis-aligned decision tree partitions the input space into disjoint intervals, each corresponding to a leaf [23]. Since each subinterval of [0, under is class-cons constant,,,,, and adjacent subintervals differ in label, every subinterval must correspond represents y requires at least w 2 leaves. In a binary tree, the number of internal nodes is always one less than the number of leaves. Therefore, the exact representation of the w-Bars Labeling Function requires at least w internal nodes.
distinct leaf. Thus, a tree that
Theorem For every N, there exists a function for which a binary axis-aligned tree requires at least B additional decision nodes to represent, compared to an SGT.
Proof. For a given w E N, the binary axis-aligned linear tree requires w + 1 decision nodes while the SGT requires only one to perfectly represent the w- w-Bars labelling function. To obtain function in which the binary axis-aligned linear tree requires at least B additional decision nodes to represent, can simply set B.
C.2 Information Gain Guarantees
Lemma 1. Let be an individual node T, let It(t) denote the information gain obtained by an axis-aligned (threshold) split on the samples in t from CART, and let (g) denote the information gain obtained by a shape function split from ShapeCART on the samples in We show that
(t) 2 IG(t)
To construct the shape function f, for each feature j. we begin by fitting a univariate CART tree that partitions the data along Zj. The root node of this tree provides the best axis-aligned split, with associ- ated information gain (t. j). For the initial bin-to- -o--n--branch assignment, we consider two alternatives: (1) the left/right partition induced by the root split, and (2) clustering -based assignment obtained via Weighted K-Means. By selecting the better of these two, we guarantee that the initialization of coordinate descent achieves least (t, Since coordinate descent monotonically improves the objective, the resulting shape function satisfies IG I 1) 2 It. Vj. Because ShapeCART employs line search select the feature shape function, follows that
(t) = max ( 2 maxI((t.j) = Ig(t).
3
m
18

---

D Time Complexity Analysis
In this section, we provide the time-complexity of splitting a node using shape function with respect to the number of samples N, number of classes C, branching factor K, and number of bins (internal tree leaves) L.
Internal Tree Construction To assign samples to L bins, an internal tree is constructed using the CART algorithm [3), trained on single feature and the target variable. We can break down the analysis of tree construction into four components: sorting the samples, tabulating class counts, calculating impurity, and the recursive relation.
1. Sorting: Sorting the samples takes ( log N) time. This only needs to be done once for the whole tree operate only when assigned to children nodes. [8|55)
feature and samples retain their relative ordering
2. Class Count Tabulation Once the samples are sorted, CART calculates a cumulative class count at each split point. This involves a scan of all possible splits (N at most).
3. Impurity Calculation:
Once class counts are tabulated, we need to calculate the impurity for the class counts of each candidate split. Calculating the impurity of a single distribution scales linearly with C, which we repeat N times, resulting in a complexity of ((NC).
4. Recursive Relation: To simplify our runtime analysis, we assume the induced decision tree remains approximately balanced (similar to (5T]), such that its maximum depth is on the order of log I., where denotes the number of leaves. Once a node split, the cumulative class tabulations (Step 2) and impurity evaluations (Step 3) must be recomputed for the resulting child nodes. However, the total number of samples processed across all nodes at any given depth remains constant at N, since each split simply redistributes the same data among child nodes. Recall that Steps 2 and 3 have per-node complexities of O(N) and O(NC), respectively. Consequently, the total cost of class tabulation and impurity computation across all nodes at given depth remains ONN+NO Rs O(NC). Aggregating over all levels of the tree yields total complexity of O log L), which can further simplify to O N) as L S N in all cases.
As result, the total complexity of fitting CART on single feature is O(N log N + NC'log N) R O(NC log N).
Bin-to-Branch Optimization Mapping bins (tree leaves) to branches involves running Weighted K-Means on the empirical distributions of the L bins to discover an initial assignment solution, then running coordinate descent to optimize the the assignments to minimize the weighted impurity of the resultant branches.
For a given branching factor k, we run Weighted K-Means on the empirical class distributions of the L leaves/bins to group the bins into & groups. This is equivalent to running Weighted K-Means on L, C-dimensional samples, which has a time complexity of O(LLTC) where T is the number of iterations the algorithm runs for 44 [2].
Our coordinate descent procedure optimizes the partition assignment for each bin indepen- dently. iterating over all leaves R times and evaluates the impurity for k - 1 values for each Icaf assignment. The cost of testing a new branch assignment for a bin does not scale with the number of branches as we only need to recalculate the impurity and weights of the current and new branch while the impurity of the other branches are frozen and can be treated as one unit. Therefore, the number of impurity evaluations in our coordinate descent is R RLR. The cost of impurity evaluation scales linearly with the number of classes (O(C)) for both Gini impurity and entropy, therefore the time complexity of our coordinate descent procedure is (kLRC).
As repeat K-Means and coordinate descent for == 2,.., K, the time complexity of the bin-to- branch optimization is OK2LTTC + L²LRC). As and R are algorithmic constants, we remove them from consideration for simplicity resulting in time complexity of O(K2
The total time complexity of fitting a univariate shape function can then be obtained by adding up the two steps together, resulting in a complexity of O log N + K2 LC). As we construct shape
19

---

functions for each of the D features, the total time complexity of fitting an internal node is:
O (D(NCllog K²))))
(15)
D.1 Shape
For CARP and its variants, we utilize BiCART to construct the internal tree for each pair of features. This algorithm creates H augmented features for each pair of features, then constructs a tree using this augmented feature and passes them into CART [3. We can analyze the cost of CART tree with H features. The runtime of CART
internal tree construction using BiCART scales linearly with the number of features [1] as each step (sorting, tabulating class counts, and evaluating impurity) needs to be done once for each feature. As such, the complexity of fitting bicart is simply O O(HNC) log N), which we can further simplify back to O log N) as we only consider small, fixed values of H. In the naive approach of constructing shape functions for each pair of features, the time-complexity of splitting a node is O (D2 log N + K²I.))
When using pairwise selection heuristic, the complexity can be broken down into the time to score each pair of features and the time to construct shape functions of the top pairs. To evaluate pairs of features, univariate shape functions, resulting in X2 sets. We then calculate the empirical class distribution of each partition, which can be done efficiently in single scan of the data. We then evaluate the impurity of each of the K partitions' empirical class distributions, resulting in a complexity of O(N CK²) for each pair of features, and (( + D2²)D2) when aggregated across all pairs. We then take the top P pairs of features and fit bivariate shape functions for these terms, resulting in time complexity of:
take the Cartesian product of the branching decisions provided from fitting the
D(NClog N K2LC)
+ +(N (NNCK3)D'+ P PNNClog N + L2)
Univariate Shape Function Construction
Pair
Bivariate Shape Function Construction/
O ((D P)(NClog N K3) + (N D²)
Empirical evaluation of the runtime benefits of limiting pairs is provided in Section G.2.3
E
Dataset Information
Table 2: Dataset Information. Dimensionality refers to the number of columns after one-hot encoding.
Dataset
N
ICI
Dimensionality
D
Cat.
Num./Binary
adult
45222
2
108
13
11
2
avila
20867
12
10
0
10
bank
1372
2
4
4
0
4
bean
13611
7
16
0
16
bidding
6321
2
0
9
covtype
581012
7
52
12
2
10
electricity
45312
2
13
7
eucalyptus
736
5
1487
19
14
5
eye-movements
10936
3
27
27
27
eye-state
14980
2
14
14
14
fault 1941
27
gas-drift
13910
6
128
128
128
higgs
11,000,000
29
29
29
htru
17898
2
8
8
magic
13376
2
10
10
mini-boone
130064
2
50
50
50
mushroom
8124
21
16
occupancy
20560
2
5
5
0
5
page
5473
5
10
0
10
pendigits
10992
10
16
16
0
16
20

---

Table 2 continued from previous page
Dataset
N
ICI
Dimensionality
D
Cat.
Num./Binary
raisin
900
2
7
7
0
7
rice
3810
2
7
room
10129
4
16
16
0
16
segment
2310
7
18
18
0
18
skin
245057
2
3
0
3
wilt 4839
2
5
5
0
5
F Model Implementation Details
All models are implemented in Python. For CART, utilize the implementation found in the Scikit- Learn 49 For Axis- Exis-Aligned TAO and HSTree, we utilize the implementation found the imodels [53] package. For SERDT, we utilize the implementation provided by the original authors [13 found at
com/ cwwccoo///aaeeeeeeeeeeeee user- - anonymous www.sasreeeeeeeerrreeeeeeeeee ressearhherinteepreee /ttiiinttererbbleee ddt For DPDT, we utilize the official implementation provided by the original authors (29] found hhtts:///githtt For SPLITT, we utilize the official implementation pro- vided by the original authors found https: thub.....tt...... thub. com/ http///ruaaaaaaaSPPP
com/KohlerHEETOR. TwwEECCTOR/DDDDreeettiiiie
As noted in Section 2. 1, some optimal methods, including GOSDT [27] and SPLIT [30] (which builds on GOSDT), require rrerbinarization of continuous features. Similar the setting from the original work [30), we set sinarize-Truu when running SPLIT, which applies Threshold Guessing to produce sparse set of binary features. While full binarization over all thresholds may improve performance,
is orders of magnitude more computationally expensive 30
Categorical Variables Since all baseline methods require numerical inputs, we one-hot encode categorical variables when relevant. Unlike the baselines that are limited to split on one (axis-aligned) or two (bivariate) levels at time, we note that ShapeCART and its variants support superset branching on categorical variables owing to the flexibility of their shape functions. This is achieved by passing all relevant columns of the one-hot encoded categorical variable into the internal tree.
BiCART and BiTAO BiTAO models, we do our best replicate these approaches following the methods outlined by Kairgeldin and Carreira-Perpiñaar [31] in their original paper. For each pair of features (d1, d2). BiCART and BiTAO create H pre-computed linear combination features. They do this by creating a small, fixed subset of line orientation W E 2 sampled uniformly in two directions by rotating it around the origin times, within a range of 0-180 degrees. The augmented feature set for d₁, d2 is then computed as X (d) (d,,dd) d2) W where X dd) E NNxx is the matrix containing the relevant pair of features. In BiCART, we utilize this augmented feature along with the original features to induce a tree via the CART algorithm [31. For BíTAO, we utilize TAO to refine a bivariate tree constructed by BiCART, finding the best linear split for nodes in reverse depth order. For
As there are no available pre-existing implementations of the BiCART and
pseudocode of this procedure, we refer reads to Kairgeldin and Carreira-Perpiñse. 31
Figure 3.
To implement ShapeTAOO (and Shape train a ShapeCART (and CARPT tree and refine it using TAO. Unlike Axis-Aligned TAO and BiTAO, our FindBestsSpli function trains decision tree via CART (BiCART for Shape?TAAO the "care" set of samples when refining each node. More formally, we follow the pseudocode provided by Kairgeldin and Carreira-Perpiñce 31] Figure 3 and modify the solving procedure the reduced problem (Kairgeldin and Carreira- Perpiñán (31 Equation 3) from considering linear splits, to instead training a decision tree via CART/BICART. For SAa, we only consider pairs of features that were considered in the node under refinement during the initial induction process. To extend TAO to multi-way branching, we make two key changes:
ShapeTAO
1. In the standard TAO procedure, each sample is assigned to a single "correct" branch, which is then used as its pseudolabel. In the multi-way setting, however, a sample may correspond to multiple valid branches. To accommodate this, we modify the pseudolabeling procedure by duplicating each sample across valid branches and passing the new upsampled dataset to CART.
21

---

2. To evaluate whether updated decision function improves upon the previous one, we adapt the accuracy metric to reflect the multi-way setting. Specifically, a prediction is deemed correct if the decision function assigns the sample to any branch that is among the set of valid branches for that sample.
Source code for for ShapeCART. ShapeTAO, and its variants can be found at
F. Hyperparameter Information
For all models, vary the maximum depth between 2-6. Below are the other parameters for each model. Note that when models share hyperparameters, we utilize the same values across all modesl.
CART: criterion: [gini, entropy), min_ samplessspitt: (2,4.8,16,32), Saa__amples__uue _eaff [1,32], min_ min_impurity. impurityy_eccreaee ddecreaee:: {0.0, le-4, 5e-4, le-3, 5e-3, ), ccp_ alpha: (0.0, le-4, 5e-4, le-3, 5e-3, 0.01)
SERDT: min_samples stop: 12,4,8,16,32), factor: (0.9, 0.91, 0.99), ganma_factor: (0.5, 0.65, 0.8)
HSTree: criterion: {gini, entropy], sin_samples_split 12,32) min sin_samples_leaf [1,32], min_ iwt.iinppt_tdcreeeeee
(0.0, le-4, 5e-4, le-3, 5e-3, 0,01), cop_alpaa: |0.0, le-4, 5e-4, 1e-3, Se-3, 0.01), reg-param (different from TAO):
[le-2. 5c-2, 1e-1, 5e-1, 1.0, 5.0, 50}
AxTAO: criterion: (gini, entropy). min sampslsmmsspplaeeeeeeeeeesssssse min_samples_leat [1,32], reg.parmamaaaammlaaaa
(2,4.8.16,32).
{0.0, 1e4, 5e-4, le-3, (0.0, Ic-4, 5e-4, lc-3, 5c-3, 0.01)
DPDT: criterion: (gini. entropy). min_samples_split
12,4.8,16,32),
min_ Sai__smples__ee
[1,32], min_ impurityy_dcrreese decrreeeee: {0.0, le4, 5e-4, le-3, 5e-3, 0.01). cart_ list: ((8,3), (32.), (4.3.), (16,6.), (16,4), (4,4), (3,3), (6,6))
SPLIT: reg: (le-3, le-2, .1), lookahead_deptha (2...., maximum depth of tree),
gbdt_n_estt (10, 20, 20. 100)
BiCART: criterion: (gini, entropy), sii__saples_see min sin_samples__est leaf: [1,32], min_ www.iiipptt_drrreeeee
12,4,8
(0.0, le-4, 5e-4, le-3, 5e-3, 0.01), ccp_alpaa {0.0, le-4, 5e-4, le-3, 5e-3, 0.01), H: (5,6,7.8)
BiTAO: criterion: (gini, entropy), min_samples_split:
(2,4,
(0.0, le4, 5e-4, le-3, 5e-3, 0.01), reg.pl).aagg (0.0, le-4, 5e-4, le-3, Se-3, 0.01), H: (5,6,7.8)
min_samples_leaft [1,32],
ShapeCART/ ShapeCART:: criterion: (gini, entropy), san__ammpee__pplll 16,32), sin_samples__est leaf: [1.32],
(0.0, le- 4, 5e-4, le-3, 5e-3, 0.01), inner inner_max_ _ax max__eaf__ffellleell Leaf_nodes nodes (aka L): (4.8,12.....64) inner Inner_min_er arrin_sample__eee leaff (1, 1e-4, 5e-4, le-3, 5e-3, le-2), branching-penalty.. (aka A): (0.0, Ic-4, 5e-4, le-3, 5c-3. 0.01]
ion__mmurrtt_ddcrreeeee
TAO / ShapeTAOg: criterion: [gini, entropy], sin_ssmples_sple spliit: 12,4.8, 12,4,8 min hin__saples__ee
(0.0, lc-4, Se- 4, le-3, 5e-3, 0.01), 1rs-param/lambda (0.0. le-4, 5e-4, le-3, 5e-3, 0,01), inner [aner_max____ ____max_____
[1.32]min, innp:laiinimpuuuittyooooo.o
mmax_leaf_ne _odde (aka L): 12.8. ..)......)),,, inner inwer_min_sampaaappen arrin_smplle_eee leaf: (1, le-4, 5e-4, le-3, 5e-3, le-2)
S / Shape>CART, criterion: [gini, entropy), min_ saan_sammle_spiii rin_samples__ee leaff [1,32], rin_impurity_des _ecrease (0.0, le- 4, 5e-4, le-3, 5e-3, 0.01), inner www.maaaaaaaaaaaeeeemmmmmaaeamee len_leef_nodes nodes (aka L): 14.8.12....64) inner timerr_min__e wwlmmmnnnnmm_a satp__saples__eeee leaf: (1, le4,, 4e-4 le-3, 5e-3, le-2), branching.penalty, (aka
leaf:
\), H: (5,6,7,8)
Shape2TAO / ShOp: criterion: (gini, entropy), siinssmplesssppe split: (2,4,8,16,)) 16,32), min_ sin__aaples__ese [1,32]min, in.ipuittiim
(0.0, le-4, Se- 4, lc-3, 5e-3, 0.01), reg_param/lambda_ (0.0, lc-4, 5c-4, le-3, 5e-3, 0.01).
22

---

inner _ar_maa______ maa__aaa_f___aaaaaaaaaaaaaaaaaaa _eoeeendeesaaeeeeeeeeeeeeeeeeeee (aka L): 14.8,12.....0 .64), inner sawerr_ii_ssapls__eee (1, le-4, 5e-4, 1e-3, 5e-3, le-2), H: [5.6,7.8}
Note on H: The choice of H was predicated ensuring computational tractability. While prior work by [3T] examined H values from 30 90, our evaluation revealed that these higher values imposed unmanageable memory burdens, preventing successful execution any dataset. Accordingly, more computationally conservative value for H was selected. Despite this restriction, three datasets (Eucalyptus, Higgs, and Covertype) still were not able on BiCART and BiTAO given our 32GB RAM limit.
E.1.1 Hyperparameter Importance
In this section, analyze the important hyperparameters for approaches. Following [2] and [3], we normalize the test accuracy within each model and dataset and train a Random Forest regressor to predict test Z-scores from the model hyperparameters and provide the feature importance in Table
Table 3: Hyperparameter Importance of ShapeCART and its variants.
ShapeCART
ShapeCART,
ShaRe CARRT
StapeTAO
Stap
Shape"TAO
Shap:"TAOg
max depth
0.24
0.179
0.095
0.359
0.221
0.142
0.085
criterion
0.038
D,135
0.033
0.034
0.035
0.025
0.038
0.035
samples split
0.079
min, samples, Jeaf
0.182
0.178
0,188
0.154
0.179
0.159
min impurity decrease
0.094
0.08
0.094
0.08
0.074
0.079
0.082
0.078
leaf podes
0.169
0.192
0.167
0.19
0.137
0.172
0.154
0.195
inner min samples,
0.088
0,084
0.036
branching pecally
0.095
0.088
0.079
0.082
0.066
0.062
0.069
0.054
pairwise penalty
0.088
23

---

G
Extra results
In this section, we provide results for all approaches across all datasets well results from more empirical evaluations.
G.1 Full Results
electricity
eyee
87.5%
Ncarrss
85.0%
60.0%
77.5%
82.5%
55.0%
72.5%
90.0%
50.055
70.0%
67.5%
S/GTy-C
86.0%
84.0%
75.0%
80.0%
33.0%
72.5%
78.0%
50.055
70.0%
Depth
S'GT-T
STGTy-T
Figure G.4: Test accuracy per depth for Bivariate models on eye- sye-movemnnts, electricity, and ete-state. Top: TDIDT-Approaches. Bottom: Non-Greedy Approaches.
Table 4: Test Accuracy (% + std.) for all axis-aligned approaches separated into TDIDT and Non- Greedy approaches by dashed lines per dataset. Best approach is bolded, second-best is italicized.
dataset
model
2
3
4
6
Best
CART 82.60 + 02.60
0.30
83.20
0.30
83.60
0.30
83.60
+ 0.30
83.60 + 0.30
HSTree 82.60 + 82.60 = 0.30
83.20
= 0.30
83.60
+
0.30
83.70
0.50
83.70 + 0.50
SERDT 82.60 1 82.60 1 0.20
82.60
= 0.20
83.00
0.20
83.10
83.10 1 0.30
SGT-C 82.80 1: 0.20 0.30 84.00
0.30
84.30
84.60
84.60 1 0.60
SGT5-C82.80 0.2084.10
84.10
0.40
84.50
0.60
1.40
1: 84.80
adult
AxTAO 82.60 0.2082.60 = 0.30 83.30
0.30
83.50
= 0.50 83.90
83.90 1.20
DPDT 82.60 + 83.0 + 0,20 83.50
= 0.40
83.50 + 0,30
83.90 0.40
83.90 + 0.40
SPLIT 78.00 + 82.60
+ 0.20
83.50
0.30
84.00
+ 0.30
44.20+0.30
84.20 + 0.30
SGT-T 82.90 + 84.1
0.30
84. 10 0.30
84.40
0.20
84.80
0.60
84.80
+ 0.60
SGT;-T 82.90 84.10
84.50
0.40
84.80
85.00
0.50
85.00
CART
57.60
62.30
0.90
65.20
HSTree
57.60 = 1.20
62.30
- 0.90
65.20
65.20 0.90
SERDT
58.10 = 0.30
61.90
- 0.70
66.60
SGT-C
64.50 2.90
71.70
= 1.80
82.00
32.00
SGT=-C
87.00 1.50
98.00
1: 0.50 99.70
+.10
avila
AxTAO
= 0.60
64.10
67.00
67.00
DPDT
61,00 = 0.60
64.70
1: 1.00
68.70
6.000
SPLIT
52.90 = 0.30
56.50
0.40
58.30
58.30 1.10
SGT-T
69.60 1.40 1.40
77.00
0.60
84.40
84.40 1.70
SGT,-T
87.50
1.80
98.10
0.60
99.60
0.30
99.60 + 0.30
CART 89.10 94.0 0.40
95.80
0.60
96.00
+ 2.00
97.90
0.90
97.90 1 0.90
HSTree 89.10 94.40
94.80
2.00
96.20
1.80
97.90
0.90
96.20
SERDT 88.80 = 93.20 2.60
94.70 = 2.00
96.10 = 2.10
97.30 + 1.30
97.30 1.30
bank

---

SGT-C + 3.20
97.00
0.90
96.70 + 0.00
96.70
0.00
97.0M
0.90
SGT,-C 1.
0.70
96.80
96.80
96.80 + 1.90
96.80
1.90
AxTAO 88.80 1.3095 0.60
96.50
1.40
97.70
1.80
98.80
97.70 1.80
DPDT 88.50 1.5095..00
1.00
97.80
= 1.00
97.90
= 1.20
98,40
98.40 = 0.90 0.90
SPLIT 82.70 0.0090.00
1.40
96.50
0.60
97.20
0.80
97.10
97.20 # 0.80
SGT-T 88,80 + 1.3092.00
4.20 97.80
1.00
97.70
0.80
97.90
0.60
97.70 + 0.80
3GT 91.60 + 1.30 96.40
1.10 97.30 = 1.40
97.30
+ 1.40
97.30
1.40
97.30 + 1.40
CART
79.90 = 0.40
= 0.10
89.60
0.70
90.30
0.50
90.30
HSTree
79.90 = 0.40 85.60 = 0.10
89.60
= 0.60
90.30
+: 0.40
90.30 0.40
SEROT
77.10 + 0.60 83.00
= 0.70
88.20
+ 0.60
90.70 0.50
90.70 + 0.50
SGT-C
85.10 + 1.20
88.80
= 0.40
89.70
+ 0.40
89.80 +
89.80 + 0.10
SGT=-C
87.90 + 0.50
90.40
0.60
90.50
+ 0.20
90.30 90.30 + 0.50
bean
AxTAO
81.90 + 0.40
87.20
= 0.10
90.00
0.50
90.40 + 0.40
90.40 0.40
DPDT
85.10 = 0,30 89.80
= 0.30
90.80 = 0.80
90.90 + 0.20
90.90 - 0.20
SPLIT
55.80 = 0,40 66.80
= 1.80
71.20 = 1.00
72.80 + 1.10
72.80 + 1.10
SGT-T
85.70 = 1.10 88.80
0.60
90.10
= 0.80
90.80
0.70
90.80 0.70
SGT3-T
88.20
0.60
90.40
0.70
90.90
0.40
90.00
0.30
90.90 + 0.40
CART 98.10 98
0.60
98.20
0.50
99.70
0.30
99.60
0.20
99.70 + 0.30
HSTree 98.10 1 0.60
98.20
0.50
99.70
0.30
99.60
0.30
+ 99.70
SERDT 98.10 0.60 10
0.60
99.60
= 0.10
99.70 + 0.30
99.70 + 0.10
99.70 1 0.10
SGT-C 98.10 0.70
0.60
99.70
0.20 99.60 + 0.30
99.60 + 0.10
99.60 0.10
9GT1- 9.10
0.20
99.60
0.20 99.60 = 0.20 99.60 + 0.20
99.60 0.20
bidding
AxTAO 98. 0.60 0.60
98.30
0.40
99.50
+ 0.00 99.50
0.20
99.50 0.00
DPDT 98.10 + 99.50
99.70 = 0.30
99.60
99.80 + 0.10
99.70 +
SPLIT 97.30 + 0.60 98. 10 + 0.60
99.40 = 0.10
99.70
+ 0.30
99. + 0.30
99.70 + 0.30
SGT-T 98.10 + 98.50 0.70
99.40 = 0.20
99.60
+ 0.10
99.60 + 0.20
99.40 + 0.20
SGT 99.10 + 0.10 99.30 0.20 99.40
0.20
99.40
- 0.20
99.40
0.20
99.40 1 0.20
CART
67.70 = 0.00
70.10
0.10
70.20 0.10
71.50
+ 0.00
71.50 + 0.00
HSTree
67.70
0.00 70.10
0.10
10.20
71.50
0.00
71.50 0.00
SERDT
67.40 1 69.00
0.10
69.90
0.30
71.80
0.20
71.80 1 0.20
SGT-C
68.90 1 0,20 70.40
0.10
72.10
0,50
73.70
0.20
73.70 1 0,20
SGT,-C
71.00 0.20 72.60
0.20
75.40
0.20
78.10
0.30
78.10 1 0.30
covtype
AxTAO
67.90
70.10
0.10
70.30
72.00
0.20
72.00
0.20
DPDT
68.60 0.00 70.80
0.10
71.60
0.20
73.20
0.20
73.20
0.20
+ 0.20
SPLIT
63.60 + 0.10
63.60
= 0.10
63.60
0.10 63.60
+ 0.10
63.60
0.10
SGT-T
69.50 + 0,20
70.90
0.50
72.50
+ 0.40
74.40
0.20
74.40
+ 0.20
SG
71.20 1 0.10
73.30
0.30
76.20
1 0.00
79.30
+ 0.40
79.30
1 0.40
CART 75.50 1 0.1075.80
0.20
76.00
0.20
77.60
1 0.20
78.00
T 0.10
78.00 1 0.10
HSTree 75.50 0 0.10
0.20
76.00
0.20
77.70
+ 0.10
78.10
0.10
78.10 0.10
SERDT
0.20
76.20
0.10
78.60
0.80
79.40 0.30
79.40
0.30
SGT-C 75.50 0.40
81.30
0.80
84.40
0.10 84.80
84.80 0.60
(UTT=-C76.5511..3000
0.60
84.00
1.20
86.00
+ 0.40
87.80
+
87,80
1.00
electricity
AxTAO
0.10
75.90
0.30
78.50
+ 0.80
79.90
0.20
79.90 1 0.20 0.20
DPDT
0.20
79.10
0.20
80.00
0.30
80.80
0.50
80.80 0.50 0.50
SPLIT 15.40
77.70
= 0.50
78.40
0.20
78.70
0.30
78.70 0.30
SGT-T 79.50
0.40
83.00 = 0.40
86.00
0.40
87.60
0.50
87.60 0,50
SGT,-T 82.60
0.60
84,90 = 0.60
86.70 = 0.60
88.20 + 0.30
88.20 + 0.30
CART
50,90 +: 4.50
52.50 = 3.80
52.00
+ 5.40
55.20 1 5.00
55,20 1 5.00
HSTree
50.90 1: 4.50
52.50 = 3.80
52.00
5.40
54.10
54.10 5.10
SERDT
46.80 = 2.70
44.60
1.80
47.10
3.70
51.40
51.40 1.80
SGT-C
53.60 3.30
55.90
= 4.00
56.30
4.60
58.80
58.80 + 7.20
SGT3-C
55.00 = 2.70
53.40 = 4.10
55.40
6.80
54.70
53.40 1 4.10
eucalyptus
AxTAO
50.90 + 4.50 51.80
= 5.50
52.00
+ 5.40
54.70
54.70 + 3.80
DPDT
49,80 1 4.90
52.50
3.80
53.80
1 3.20
$5.20
55.20 1 3.00
SPLIT
29.30 1 2.60
36.70
2.10
36.30
2.20
34.20
14.20:14: 4.50
34.20 1 4.50
SGT-T
55.40
57.20
2.00
57.00
55.20
4.80
55.20 4.80
SGT.-T
55.40 = 1.80
50.00
= 2.90
57.00 + 2.80
57.00
+ 2.80
55.40 - 1.80
CART 45,90 + 46.30 + 2.60
51.30
= 0.20
52.40
+ 1.40
53.20
+ 0.70
53.20 + 0.70
HSTree 45.90 + 46.30 + 2.60
51.40 = 0.20
52.40 + 1.40
53.10
0.70
53.10 + 0.70
eyc-movements
25

---

SERDT 43.80 0.300
0.20
49.10
0.40
52.70
+
1.00
53.50
+
0.60
53.50
0.60
SGT-C 48.10 2.10
- 0.70
53.00
54.90
1.20
53.90
0.70
57.40
1.20
57.40 1
0.80
56.00
0.30
57.50
60.90
2.60
60.90
1 2.60
AxTAO +5.90
0.80
51,40
0.30
52.30
= 1.20
52.60
52.60
1.60
DPDT 4: = 0.30
52.40
0.80
54.40
= 1.20
36.70
56.70 1.20
SPLIT + 0.30
50.20
= 0.60
51.90
0.20 52.00 + 0.50
51.90 + 0.20
SGT-T 48.60 2.40
1.50
54.80
= 0.50
56.80
2.80 58.00
1.40
58.00
+ 1.40
SGT 0.90 + 0.30 55,40
0.80
56.90 + 2.50
60.30 + 2.90
60.30
+ 2.90
CART 62.70 + 0.60 = 1.30
69.10 = 1.60
70.90
0.30
73.20 + 1.40
73.20 + 1.40
HSTree 62.70 + 0.60 65.80 = 1.30 69.10 = 1.60
70.90 + 0.30
73.20 + 1.40
73.20 + 1.40
SERDT 62.70 + 66,0066400 = 1.50
69.40 = 0.90
12.30+0.00
75.30
0.50
75.30 + 0.50
SGT-C 64.30 0.90 0.20 70.50
1.20
72.40+..3
74.50
+ 1.10
74.50
1.10
SGT,-C 0.700000
0.70
73.30
1.00
76.80
1.30
77.50
0.50
1 77.50
eyc-state
AxTAO 62.10 1.407.
1.00
69.70 = 1.20
72.80
0.70
74.30
74.30 1.60
DPDT 65.90 0.50
1.90
71.40
1.30
73.40
0.50 75.60
+ 0.90
75.60 0.90
SPLIT 58.50 0.40 0.30
69.00
1.30
71.80
= 1.10
73.60
0.30
73.60 0.30
SGT-T 65.60 + 1.20 70.50
0.60
72.70
0.40
73.60
+ 1.20
75.80
0.90
75.80
0.90
SGT,-T 67.10 + 0.59
+ 0.60
75.80
0.60
75.80
+ 0.90
76.30 + 1.20
76.30 + 1.20
CART
61.50 - 66.30
1.30
67.10
= 1.60 70.30
2.30
70.30 +
HSTree
61.50 : 4.50 66.30
= 1.30
67.60 2.30 70.30 + 2.30
70.30 1 2.30
SERDT
63.60 - 0,90 64.80 = 0.80
67.00 +: 1.20 70.50 + 1.00
70.50 1 1.00
SGT-C
65.30
1.80
68.00 = 0.40
69.70
3.60
69.70 + 3.60
C
65.20 5.10 69.60
2.30
69.70
1.30
69.10
69.10 3.40
fault
AxTAO
61.50 4.50 66.40
= 1.70
68.40
2.00 69.80
1.30
69.80
1.30
DPDT
67.40 + 2.10 66.90
2.20
71.00
+ 2.10 70.90 + 2.80
70.90 + 2,80
SPLIT
47.00 = 2.30 59.50
= 1.00
61.90
2.10
+
67.10 3.50
SGT-T
65.70 = 1.40 68.00
3.30
70.10
3.10
5.20
70.70 1 5.20
-
69.30
1.60 70.40
2.20
69.20
2.50
69.20
3.40
69.20 1 3.40
CART
65.40 0.80 72.30
2.10
81.00
0.10
86.90 1.70
86.90 + 1.70
HSTree
65.40 1 72.30
2.10
81.00
0.10
86.80
1.60
86.80 1 1.60
SERDT
65.50 1 0.30 74.70
= 1.00
81.70 0.10
87.60
1 0.90
87.60 1 0.90
SGT-C
66.30 1: 1.30
76.00
1.00
83.10 1. 2.00
89.00
0.90
89.00 1 0.90
C
81.00 = 1.00
87.90
1.90
93.00 = 1.60
94.70
+ 1.20
94.70 + 1.20
gas-drift
AxTAO
66.90 1.00
76.10
1.80
83.50
2.90
89.50
1.30
89.50 1.30
DPDT
70.20 + 0.80
78.50
= 1.30
86.60
0.80
91.90
0.90
0.90
91.90 0.90
SPLIT
51.20 = 0.90
59.00
= 1.00
64.50 + 1.80
69.00
1.50
69.00 + 1.50
SGT-T
70.90 1 3.30
80.00
1.60
86.40
1 0.70
91.10
+ 1.30
91.10 1 1.30
-GT,-T
83.00 1 0.60
89.20
0.80
92.80 1: 1.60
94.80 T 0.30
94.80 1 0.30
CART 64.70 +: 0.30
65.60
0.60
66.00
0.80
66.30 = 1.20
67.60 + 0.30
67.60 + 0.30
HSTree -: 05.0
0.70
66.00
= 0.80
66.50
1.20
67.10
67.10 0.20
0.20
SERDT
0.60
65.70 = 1.30
66.20 = 1.10 66.80
- 0.40
66.80
+.40 0.40
SGT-C 64.40 1: 6320
0.50
64.10
= 1.70
64.90 +: 1.30 63.70 + 1.80
64.90 + 1.70
0,70
63.30
1.20
63.00
64.30 0.90
higgs
AxTAO 62.60 0.20
0.30
65.00
0.70
67.40
65.40
65.40
DPDT 64.70 1 0.60 66.80
65.50
65.50 0.60
66.40
66.40 1.40
SPLIT 61,90 0.60
0.30
65.50
1.10
66.80
1.50
66.00
66.00+2.000
SGT-T 63,80 0,10
1.00
65.90
1.30
65.00
= 2.10
63.90
1.:9022660
SGT3-T 0...000011005.
0.60
64.70
= 0.90
63.70 = 1.00 64.30
64.30 1.00
CART 97.80 1: 0.10 +: 0.20
97.90
0.10
97.90 1: 0.10 97.80
0.00
97.90 1: 0.10
HSTree 97.80 = 07.0
= 0.10
98.00
0.10
97.90
0.10
97.80
0.00
97.90
0.10
SERDT 97.80 0.10
0.10 98.00
= 0.10
98.00
0.10
98.00
0.10
98.00 +
SGT-C 0.10
0.10 97.80
= 0.20
97.90
0.20 97.60
17.60
97.80 0.10
0.10977.0
0,10 97.80
0.10
97.90
97.70
97,80
97.80 0.10
AxTAO 97.80 0.1097.88
0.10
98.00
0.20
97.90
0.10
97.90
0.10
DPDT 98.00 1 0.10
1
0.10
98.00
0.10
97.90
9..90
98.00
SPLIT 97.80 1 0.10
97.90
0.20
98.00 0.10
98.00 0.10
98.00
0.10
SGT-T 97.70 # 0.10
0.10
97.80
0.10
0.10
9.170
9.10
SGT 97.70 0.10
0.10
97.70
0.10
97.70
97.70
07.70+..11
97.70
CART 73.70 +: 1.20 76.80
0.90
78.90 = 0.70
80.10
+ 0.90
81.30 + 0.90
81.30 + 0.90
26
magic

---

HSTree 73.70 7.20 +
0.90
78.90
0.70
80.10 + 0.90
81.30 +
0.90
81.30 0.90
SERDT 73.70 1.20
78.70
0.80
80.00 = 1.20
81.10
0.80
SGT-C 1.00 = 0.10 79.60
1.10
80.10 1.70
81.00
81.00
(TT=-CC77.9011200707 0.80
79.90
0.50
81.10
81.10 81,40
AxTAO
0.90 79.30
0.80
81.10 1.30
82.10
82.10 1.00
DPDT 76.50 + 78.30 0.90 80.30 0.80
81.10
0.30
82.30 1.00
82.30
1.00
SPLIT 7.70 = 1.10 78.70 = 1.20
80.50
= 0.90
+1.20:00.00
81.20 + 0.80
SGT-T 74.80 + 78.00 + 0.80 80.00 = 0.70
80.30
+ 1.10
81.80
1.30
81.80 + 1.30
KGT5-I78.50- 12007 7.20
0.70 81.10 = 0.60
81.00 - 0.40
81.00
+ 0.50
81.00 1 0.50
CART 83.50 + 86.0 + 0.20 87.70 = 0.10
89.00 + 0.20
89.70 + 0.10
89.70 + 0.10
HSTree 83.50 + 0.20 86.60 + 0.20 87.70
0.10
89.00 + 0.20
89.70
89.70 0.10 0.10
SERDT 83.30 0.10 0.10 88.10
0.10 89.00 + 0.20
89.80
89.80 0.10
SGT-C 84.80 1.20 0.50 88.30
0.30
89.00
0.20
89.70
0.10
89.70 0.10
(TT=-C88.6001
0,50 89.10 0.20
89.70
- 0.10
0.20 89.90
89.90 1 0.20
mini-boone
AxTAO 83.50 = 0.10
88.00 = 0.20
89.20
= 0.30
8
89.70 0.30
DPDT 86.50 0.2088.00000000000
0.20
88.30 0.20
89.20 = 0.20
89.90
89,90 0.10
SPLIT 83.20 0.20
0.20
88.00
0.30
88.60
0.30
86.80
0.20
88.60 0.30
SGT-T 85.80 + 1.60
0.20
88.60
= 0.10
89.40
0.10
89.90 + 0.10
89.90 0.10
SGT3-T 87.10 0,20
0.10
89.20
0.20
89.70 + 0.20
90.10 + 0.20
90.10 + 0.20
CART 95.10 + 9.40 + 0.20
99.20 = 0.20
100.00
1 0.00
100.00 1 0.00
100.00 0.00
HSTree 95.10 1 9,40 0.20 99.20 = 0.20 100.00 1 0.00 100.00
0.00
100.00 0.00
SERDT 95.10 0.4098. 9
0.20 99.20 0.20 99.60
0.10 99.90
0.20
99.90
0.20
SGT-C 98.60 0.30 0
0.10 99.80 = 0.20 99.90 0.10 99. 90 0.10
99.90 + 0.10
9, 0.20 100.00 0.00 100.00 0.00 100.00
0.20
0.00 100.00 0.00
0.00
mushroom
AxTAO 95.10 0.40 + 0.20 99.20 = 0.20 100.00 + 0.00 100.00
0.00 100.00 + 0.00
DPDT 96.90 : 99,00 = 0.30 100.00 = 0.00 100.00 + 0,00 100.00 + 0.00 100,00 = 0.00
SPLIT 88.10 1.10 0.30 98.50 0.10 99.70 = 0.30 99.70
+ 0.30 99.70 + 0.30
SGT-T 98.60 9.30
99.90 = 0.10 100.00 + 100.00
0.00 100.00 + 0.00
TG 0.20 99.60 = 0.20 100.00
0.00
100.00 + 0.00 100.00
0.00
100.00 + 0.00
CART 99.00 1 99,00 0.10
99.00 = 0.10
99.00
0.10 99.00
0.10
99.00 1 0.10
HSTree 99.00 1 09.0099.00
+ 0.10 99.00 = 0.10
99.00 0.10
99.10 1. 0.10 0.10
99.10 1 0.10
SERDT 99.00 1: 0.10 99 1 0.10
99.10
0.10
99.10 1. 0.10
99.10 1: 0.10
SGT-C 99.00 = 0.10 98.
= 0.10 99.00
0.10
99.00
99.20
0.20
99.20 + 0.20 0.20
99.000 = 0.10 98.
0.10 99.00
= 0.10
99.20
0.20
09.10
99.00 + 0.10
occupancy
AxTAO 99.00 + 0.10 99.00 + 0.10 99.00
0.10
99.00
0.10
19.10
09.10+0.10 0.10
DPDT 99.00 + 0.10 99.00
+ 0.10 99.10
= 0.20
99.20
+ 0.10
99.20
0.20
99.20 0,10
SPLIT 99.00 1 99.00
- 0.10 99.00
0.10
99.10
1 0.20 99.10
1 0.20
99.10 1 0.20
0.20
SGT-T 99.00 T 0.10
0.20 99.10
0.20
99.20
1 0.30
99.30
T 0.20
99.30 1 0.20
S 98. 90 1 0.20
99.00 = 0.20
99.10
0.20
99.10
0.10
99.10 1
CART
96.00 = 0.30 96.50
0.50
97.30
0.60
97.20
0.80
97.30 0.60
HSTree
96.00 = 0.30 96.50
= 0.50 97.20
= 0.60
97.30
- 0.50
97.20 0.60
SERDT
96.10 + 0.20 96.60
= 0.60
96.90
+ 0.40 96.90
0.20
96.90 + 0.20
SGT-C
95.20 + 0.30 96.40
0.20
96.30
+ 1.20 96.20
96.30 + 1.20
SGT=-C
95.60 + 0.10
95.70
0.30
96.30
+ 0.30
96.00 + 0.10
96.30 + 0.30
page
AxTAO
96.20 1 96.70
0.60
97.20
0.60
97.10
97.10
DPDT
96.70 = 0.40 97.00
= 0.30
97.20
= 0.50
97.40
+ 0.60
97.40 0.60
SPLIT
95.20 = 0.20 96.10
0.20
96.30 = 1.00
96.20
+ 0.60
16.30+1.00
SGT-T
95.50 = 0.00 96.30
0.70
96.50
= 0.50
96.30
0.90
-GT,-T
96.10 - 0.10 96.50
0.70
96.20
+ 0.40
95.70
1.10
96.50
0.70
CART
77.20 2.10
85.70 + 1.50
90.00
0.60
90.00
HSTree
77.20 = 2.10
85.70 = 1.50
90.00
90.00 0.60
0.60
SERDT
77.60 = 3.90
85.30 = 3.00
39.30
89.30 1.40
SGT-C
79.30 3.90
86.40
+ 0.70
88.30 + 1.20
88.30 1.20
SGT--C
87,30 0.50 90.80
+ 0.60
93.50
93.50 + 0.40
pendigits
AxTAO
78.70 2.40
86.00
1. 1.10 87.50
87.50 0.60
DPDT
82.70 - 0.50
89.00
0.50
93.10
03.10-10.3
93.10 0.30
SPLIT
57,90 = 0.50
66.20
0.60
70.20
70.70 2.20
SGT-T
81.50
3,90
87.50
- 0.90
9
91.50
+.60
-
87.70
0.40
92.50
= 0.50
93.90 0.50
93.90 0.50
27

---

CART 85.90 1: 85.90
2.30
85.20
1.20
85.90 1 2.00
85.20
1.20
85.20 1 1.20
HSTree 85.90 2.30 86.30
3.10
85.90
3.30
65.70
86.10
2.00
85.90
SERDT 85.90 2.30
2.00
86.30
2.80
85.60
= 2.50
85.60
86.30
2.80
SGT-C 85.70 1:070011400 1,4055.00 2.30
84.30
3.90
84.30
2.90
2.99
84.30
2.90
(GT==CC34.002223388888
2.00
84.40
3.10
84.60 +
84.60
85,20
2:00
raisin
AxTAO 8.700 2.70 86.30
2.20
86.10
86.30 1 2.20
DPDT 86.50 1 2.30 10 2.50 86.10
3.10
86.10 3.10
85.90
1.70
85,90
1.70
SPLIT 2.5085.07 2,80 85.70
= 3.10
85,90 +: 2.50
85.40 1: 2.20
85.70 1 3.10
SGT-T 85.00 3.000 3.8085.70
2.20 85.40
2.90
85.40 = 2.90
85.40 + 2,90
85,40 + 2,90
SGT,-T 85.90 1.3085.60
2.90
86.10
3.10
86.10
= 3.10
86.10 3.10
85.60 + 2.90
CART 93.10 0,4093.10
0,40
92.70
0.50
92.50 + 0.30
92.30 + 0.30
92.50 1 0.30
HSTree 93.10 0.40
0.40
93.00
0.60
93.00 + 0.20
92.80
0.70
93.00 + 0.60
SERDT 93.10 0.4093.10
0.40 92.80
0.50
92.80
92.50 0.60
92.80 0.50
SGT-C 93.10 0.4092.70 0.80 92.50
0.90
92.10 0.90
92.50 +.500:0..00
0..00=0.50
SGT; 93.20 0.3093.20 = 0.40 93.20
0.50
93.20 = 0.50
93.20
+ 0.50
93.20 + 0.30
rice
AxTAO 93.10 0,4093.10 0.40 92.90
0.50
92.90
= 0.40
92.80 + 0.30
92.80 0.30
DPDT 93.00 0.20 0.40 92.90
0.50 92.70
0.40 92.70
0.40
93.20 0.40
SPLIT 93.10 0.4093.30
93.20
0.50
93.20
93.20
0.50
93.10 0.40
SGT-T 93.10 1 0,4092.30 + 0.50 92.70
= 0.60
92.60 1: 0.60
92.30 1 1.10
92.70 1 0.60
SGT;-T 93.20 0.50
0.70 92.60
0.20
92.60
92.60 + 0.20
92.60 + 0.20
CART 92.50 = 0.30 = 0.50 98. 70 = 0.30
99.40
- 0.20
99.30
0.20
99.40 0.20
HSTree 92.50 + 0.30 = 0.50 98.70
0.30
99.40 0.10
99.40
0.10
99.40 0.10
SERDT 92.50 + 0.30 90.20 0.60 98.90
0.10
99.30
0.20 99.50
0.20
99.50 0.20
SGT-C 93.00 0.40 38
0.20 99.40
0.10
+ 99.40
99.60
0.00
99.60
9 5. 0,20 99.50
0.20
99.50
- 0.20 99.50
99.50 0,20
AxTAO 92.50 0.30
0.30
99.20
0.00
99.50
= 0.20
99.40
0.20
99.40 0.20
DPDT 93.60 1.109900 0.00
99.30
0.20
99.60
= 0.10
99.40
0.10
99.60 0.10
SPLIT 88.80 0,30
0.30 92.40
0.70
94.00
0.70
94.20
94.20
SGT-T + 0.20
99.60 = 99.50
0.20
0.10
99.50
SGT 98.60 + 0.30 99.30
0,10 99.50 = 0.20
99.20 + 0.30
99.50
0.10
99.50 + 0.20
CART
81.30 = 0.10
88.70 = 1.10
90.90 = 0.80
93.40 + 1.00
93.40 + 1.00
HSTree
81.30 = 0.10 88.70 = 1.10
91.30 = 1.10
93.40 + 1.20
93.40 = 1.20
SERDT
67.00 + 3.20
81.50
0.20
88.70 + 1.20
91.00
1.80
91.00 + 1.80
SGT-C
83.30 + 3.60
88.90
0.80
89.80
2.50
91.50
1.90
91.50
1.90
SGT--C
88.70 + 91.20
0.50
92.40 +
02.90
92.40
1.00
segment
AxTAO
81.70 + 0.70
89.40
= 1.00
92.40 + 0,90
93.30
1. 1.00
93.30 + 1.00
DPDT
87.10 = 0.50
90.50
0.60
92.10
= 0.30
94.20
+ 0.70
94.20 = 0.70
SPLIT
52.30 = 2.50
61.90
3.80
67.20
3.50
67.70 : 2.60
67.70 2.60
SGT-T
84.20 = 89.80 = 0.60
91.60
1.50
90.80
+ 90.80 1.70
SGT3-T
89,30 = 1.20 91.70 = 0.10
93.00 = 0.90
92.90
0.90
93.00 0.90
CART 90.80 0. + 0.10 97.40 = 0.10
98.40 + 0.10
98.90
98.90 0.10
HSTree 90.80 1: 0.10%.60 = 0.10 97.50 = 0.10
98.50
- 0.00
99.00 0.00
99.00 0.00
SERDT 90.80 0.10%..0 : 0.10 97.80 = 0.10
98.60
- 0.10 99.30
99.30 30 0.00
SGT-C 0.20 0.10 98.50
0.00
98.90
-: 0.10 99.20
99.2011..20
SGT;-C 0.10
0.00 99.20
0.10
99.60
0.10 99.80
0.00
99.80 0.00
skin
AxTAO 90.80 0.10
0.10
97.90 = 0.00
98.70
+ 99.60
0.00
99.60 0.00
DPDT 92.50 0 96.60
0.10
98.50 = 0.00
99. 10
+ 0.00
99.
40
0.10
0.10
SPLIT 88.40 + 0.10
0.00
96.50
0.10
98.10
+ 0.10
98.50
0.00
98.50
0.00 0.00
SGT-T 93.20 0. 0.00
99.20
0.10
99.70
- 0.10
99.80
0.10
99,80 0.10
SGT 0.10
99.70
0.10
99.70
99.80
0.00
99.80 1
CART 97.30 0.40
0.20
97.80
0.60
98.20
0.50
98. 10 0.70
98.20 0.50
HSTree 97.30 0.40
0.20
97.70
0.70
98.20 - 0.50
98.40
0.50
98.40 + 0.50
SERDT 97.30 1 0.4097.50
0.20 97.90 - 0.60
98.30
-. 0.40
98.30
1 0.60
98.30 1 0.40
SGT-C 97.10 1.60
0.10 97.90 0.50
97.60 0.20 97.60
0.20
97.90 1 0.50
9 0.5097.50
0.20 97.80
0.50
97.60 1.30
97.60
97.50 1200010.000 0.20
wilt
AxTAO 97.30 0.40 0.20 97.80
0.70
98.10 0.70
9..00
08.10
DPDT 97.20 + 0.40 + 0.30 98.20
0.40
98.20
+ 0.50
98.30
98.30 0.60
SPLIT 95.00 + 0.40 + 0.40 97.70
= 0.20
97.80 0.10
97.80
97.80 0.10
SGT-T 97.30 0.40 0.40 97 40
0.30
97.60
0.50
97.60 0.40
97.60 0.40
28

---

SGT 0.60 97.50 50 0.30 97.70 0.30 97.50 + 0.20 97.50 0.20 97.50 + 0.20
Table 5: Test Accuracy (% + std.) for all bivariate approaches separated into TDIDT and Non-Greedy approaches by dashed lines per dataset. Best approach is bolded, second-best is italicized.
dataset
model
3
Best
BICART 83.40 + 0.30
83.60
0.30
83.80
= 0.40
84.10
+ 0.50
84.10
0.50
84.10 + 0.50
S'GT-C 83.60 + 0.30
83.80 - 0.40
84.40
= 0.50
84.90
85.10 0.60
0.60 0.60
85.10 + 0.60
adult
S 84,30 = 0.20
84.10
0.50
84.60
0.70
85.00
8.21
85,10 = 0.20
BiTAO
83.60 0.40
84.00
0.30
84.20
0.40
84.30
+
84.30 + 0.50
STTT 83.90 + 0.60
84.30
0.40
84.90
: 0.40
85.00
0.30
85.00 0.70
-T 84.10 + 0.40
84.50
0.50
85.00 = 0.40
0.60
85.00
0.40
85.10 + 0.60
BiCART
59.80 + 0.50
62.20
:: 0.60
66.90
0.50
66.90 + 0.50
SCGT-C
80.20 + 2.80
88.70
4.30 95.30
0.90
95.30 0.90
avila
-C
97.30 = 0.90 99.90
+ 0.10 99.90
0.10
99.90 + 0.10
BITAO
60.00 = 0.40
66.00
= 1.20 70.90
1.50
70.90 1.50
GT
83.00 = 0.70
90.90
1.80 95.10
1.10
95.10 - 1.10
GT
97.10 1 0.20 99.90
0.10
99.80
0.00
99.80 1 0.00
BiCART 95.60 + 1.90
97.80 + 2.00
99.40 = 0.80
99.50
0.80
99.30
0.70
99.40 + 0.80
G'GT-C 98.40 0.80
98.40 0.80
98.40 + 0.80
98.40
18.40
98.40
98.40 0.80
bank
SC 98.50 0.60
98.40
0.60
98.40 1 0.60
98.40
00:40
98.50 1 0.60
BITAO 95.40 1.80
98.
= 0.80
99.50 = 0.60
0.60
99.50
99.50 + 0,60
GTGT 97.80 + 1.00
98.90
0.40
98.90 = 0.40
98.90
1.40
98.90 0.40
98.90 0.40
GT 98.90 = 1.00
98.50 + 0.70
98.50 = 0.70
98.50
0..50
98.90 + 1.00
BiCART
89.80 + 0.70
90.40 + 0.30
91.80 + 0.20
91.80 + 0.20
S"GT-C
90.10 0.50
90.60 = 0.40
91.30 91.30
91.30 + 0.50
bean
CC
90.30 0.90
91.10 = 0.30
91.30
90.70
0.30
91.10 + 0.30
BiTAO
90.20 + 0.30
91.00 + 0.40
91.50
91.70
0.40
91.70 + 0.40
S'GT-T
91.00 0.30
90.90 0.30
91.60
1:40
91.80
0.50
91.80 +: 0.50
ST
90.60 + 0.30
90.70 = 0.60
91.20
91.40
0.40
91.20 + 0.70
BiCART 97.90 0.40
99.60
0.20
99.80 + 0.10
99.80
99.80
0.10 0.10
99.80 0.10
S'GT-C
99.50 0.60
99.70
0.20
99.70 + 0.20
99.70
0,.7000000
0.20
0.20
99.70 + 0.20
bidding
S'GT3-C 99.70 : 0.30
99.60
0.10
99.60 = 0.10
99.60
99.60
0.10
99.70 0.30
BiTAO 98.00 = 0.50
99.50
0.20
99.80 - 0.10
09.80
99.80 = 0.10
99.80 0.10
S 99.50 0.10
99.80
0.10
99.80 = 0.10
99.80
99.80 = 0.10
99.80 0.10
GT 99.50
99.50
0.20
99.50 1 99.50
0.20
99.50 1 0.20
99.50 0.20
BiCART 75.80 0.20
76.20
0.10
78.30 1 0.70 78.70
-. 0.30
80.40 1 0.30
80.40 0.30
S*GT-C 80.60 + 0.60
82.00
0.60
83.90 = 0.30 86.50 + 1.20
87.50 1.20
0.50
87.50 + 0.50
electricity
SC 81.80 + 0.20
84.50
0.30
86.60
+ 0.80
87.30
1.00
0.20 88.80
0.20
88.80 0.20
BiTAO 75.80 + 0.20
76.20
+ 0.10
79.50
+ 0.50 80.10
80.70
0.40
80.70 0.40
S 81.20 = 0.50
83.60
- 0.50
85.80
= 0.80 86.80
0.70
88.30
0.80
88.30 + 0.80
S² 82.10 0.30
84.90
0.60
86.30
= 0.30
87.70
- 0.20
88.50
0.50
88.50 0.50
BiCART 45.40 + 1.20
51.60
0.70
52.20
+: 1.10 54.70
1.60
56.30
56.30 0.50
0.50
0.50
S"GT-C 57.80 + 1.40
60.00
2,00
61.70
+: 3.20
62.60
3.30
64.80 + 3.50
64.80 +: 3,50
C 59.60 + 2.40
63.60
1.40
66.20 = 1.90
67.50
63.90 2.40
63.90 + 2.40
eyc-movements
BITAO 48. 10 + 2.00
51.60
0.10
53.20 + 0.40
55.80
56.70 + 0.40
56.70 0.40
S3GT- 58.70 + 0.60
60. 10 + 1.70
63.90 + 1.10
66.10
+ 1,20
66.60 + 1.40
66.60 + 1.40
ST 59.80 + 1.70
62.80
1.10
56.90 : 1.10
63.20
1.20
64.90 + 1.30
62.80 + 1.10
BICART 66.20 + 2.40
69.20
1.10
72.90 + 0.60
74.10
+: 2.70
77.60 + 0.80
77.60 +: 0.80
S'GT-C 71.70 + 1.10
72.80
0.70
74.40
+ 2.10
76.10
2.00
80.20 + 0.30
80.20 0.30
S 73.10 + 1.10
75.90
0.40
79.30
+ 1.10 81.10
+ 0.60
80.60 + 1.90
81.10 + 0.60
eye-state
BITAO 68.80 1.20 70.10
0.70
74.50
1 1.50 77.50
79.00 1.40
79.00 1 1.40
S. 72.40 + 1.60
75.90
1.20
77.20
0.80
79.00
+ 1.70
80.90
1.00
80.90 + 1.00
29

---

ST 74.50 1.20
76.30
0.20
79.30 -
80.50
1.70
81.70
1.20
81.70
1.20
BiCART
61.40
3.00
67.40
= 1.90
6..00
73.40
1.50
73,40
1.50
SGGTCC
66.60 +: 5.80
69.00
5.80
69.20
71.90
71.90
+ 2.40
fault
68.50 2.50
70.30
1.70
70.00
71.60
2.80
BiTAO
62.30
67.90
68.00
1..40
73,40
1.20
SGTT
67.90 1.20
70.40 = 3.60
71.80
+
72.40
2.00
71.80 + 1.50
ST
70.40 3.10
73.40 + 1.90
73.20
+.40
72.30
1.50
73.20 3.40
BiCART
86.70 1.10
95.40
0.40 07.90
98.60
0.20
98.60 + 0.20
G'G
83.60 + 4.00
94.90
- 1.20 96.00
0.20
97.20
0.30
97.20 + 0.30
gas-drift
g
96.20 0.50
97.00
0:40 97.00/90000000
97.10
97.10 + 0.40
BiTAO
91.50 0.10
96.20
0.70 97.20
98.20
0.00
98.20 0.00
S
92.00 1 1.10
94.80
+ 1.20 97.30
0.60
98.00 1 0.10
98.00 1 0.10
ST
96.40
0.50
97.90
= 0.20
97.90 + 0.20
97.90 0.20
97.90 + 0.20
BiCART 97.90 +: 0.10
98.00 +: 0.10
98.10
+ 0.20
98.00
1: 0.20
98.00 + 0.10
98.00 + 0.10
S*GT-C 97.80 1 0.00
97.80
0.10
97.80
1 0.20
97.80
0.30
97.80 1 0.30
97.80 0.20
htru
- 97.80 1.00
98.00
0.00
98.00
0,10 08.00
98.00
0.10
98.00 1 0.00
BITAO 97.90 0.10 98.10
0.10
98.00
98.00
98.00 0.00
98.00 + 0.00
S'GT-T 97.90 + 0.20
98.00
0.20
97.80 + 0.20
98.10
98.00 + 0.10
98.00 0.20
STT33T 97.60 + 0.10
98.00 + 0.10
98.10 + 0.10
98.10 0.00
98.10 + 0.00
98.10 0.00
BICART 77.50 + 1.40
81.10 = 0.30
81.10 = 0.30
02.90
83.00 + 0.60
82.90 + 0.20
S'GT-C 81.10 + 1.00
81.70 0.90
81.20 = 0.80 81.90
82.10
0.10
81.90 0.40
S 81,50 +: 0.80
82.90 0.60
82.90
0.40 83.50
82.70
82.90 + 0.40
magic
BiTAO 77.80 1.30
81.70 0.50
82.00
- 0.40
82.70
0.70 83.30
83.30 1 0.70
S*GT-T 81.40 0.10 82.10
0.20
82.60
+ 0.70
82.80 8.:50
0.60
82.50 0.40
STGT-TT 82.20 0.70
83.00
0.80
83.60 + 1.00
82.80
0.20
+.60
82.80 + 0.20
BiCART 87.20 0.10 88.60 + 0.20
89.50 + 0.30
90.30
0.10
90,70
0.20
90.70 0.20
SCGT-C 87.60 + 0.30
89.50 1: 0.10
90.10 = 0.20
90.50
1.20
9
90.60 +: 0.20
mini-boone
SC 88.80 + 0.30
90.00 - 0.20
90.50 = 0.20
90.90
0.20
90,90 + 0.20
BITAO 87.30 0.10
88.40
+ 0.30
89.70 = 0.40 00.20000.100
90.50
90.50 + 0.20
S'GT-T 88.40 1 0.50
89.70
0.20
90.30 1 0.20
90.80
0.10
01.10
91.10 + 0.10
S'GT,-T 89.00 1 0.30
90.10
0.20
90.60 1 0.30 91.20 1 0.30
91.20
0.10
91.20 0.10
BiCART 98.70 + 0.10 100.00
0.00
100.00 = 0.00 100.00
+ 0.00
100.00
+: 0.00
100.00
+ 0.00
S'GT-C 99.40 1 0.20 100.00 1: 0.00 100.00
0.00 100.00
1 0.00 100.00
0.00
100.00 0.00
mushroom
SCGT3-C 100.00 0.00 100.00 0.00 100.00
100.00
0.00 100.00
0.00 100.00
BITAO 98.70 0.10 100.00 + 0.00 100.00 - 0.00 100.00 = 0.00 100.00
0.00
100.00 = 0.00
S 99.40 = 0.10 100.00 = 0.00 100.00
0.00 100.00 = 0.00 100.00
0.00 100.00
= 0.00
ST 100.00 + 0.00 100.00
0.00 100.00 + 0.00 100.00 + 0.00 100.00
0.00
100.00 +
BiCART 99.00 0.10 99.00
0.10 99.10 = 0.10
99.20
0.10 99.20 = 0.10
99.20 0.10
S 99.10 0.20 99.10
0,10
99.20
= 0,10 99.30 - 0.10 99.30 = 0.10
99.10 + 0.10
SC 99.20 0.20
99.30
+ 0.20
99.30
+: 0.10 99.30
+: 0.20
99.30 + 0.20
99.20
0.20
occupancy
BiTAO 99.00 0.10
99.00
0.10
99.10
0.10 99.10
99.10 0.20
99.10 0.20
S 99.10 1 0.20
99.10
+ 0.20
99.30
: 0.20 99.10
1: 0.20
99.20 1 0.20
99.20 1 0.20
GT 99.20 i 0.20
99.20
0.20
99.10 - 99.20
0.20
99.20 1 0.20
99.20 0.20
BiCART
96.40
0.30
97.20
+ 0.20
96.80
0.50
97.10 + 0.60
97.10 0.60
S'GT-C
96.90 0.10
97.20
= 0.70
96.90
1.20
96.10 1: 0.90
97.20 1 0,70
SCGT3-C
96.20 0.40
96.80
= 0.60
96.80
0.30
96.80
0.40
96.80 + 0.60
page
BiTAO
96.60 0.60
97.40 = 0.20
97.40
96.80
0.80
0.80
96.80 0.80
S
97.00 0.20
96.90
1 0.40
97.10
0.80
96.60
1 0.10
97.00 1 0.20
S
96.50
0.50
96.40 1 0.60
96.40
0.60
96.40
0.60
96.40 0.60
BiCART
87,40 = 0.30
91.90 1.30
94.50 1.10
94.50 + 1.10
SºGT-C
87.20 + 1.10
91.00
0.00000800
93.30
0.10
93.30 0.10
pendigits
S'GT,-C
91.80 1 1.10
05.80
94.20
95.80 0.70
BITAO
88.60 - 0.50
93.80 -
95.30
0.90
0.90
95.30
0.90
ST
89.00 = 0.90
92.10
94.40
0.20
94.40 + 0.20
30

---

TT
94.10 -
94.10
0.30
94.80 0.20
94.80
0.20
BICART
89.40 3.10
85.70
1.70
86.50 1.30
86.50
85.40
85.70
1.70
S9GTCC
85.70
+ 1.70
35.00
85.00
1:00:19000
86.50 + 1.30
85.70 + 1.70
S
85.70 1.70
83.00
1.20
83.00
83.00 1.20
85.70
1.70
raisin
BiTAO
89.30 3.40
88.50
3.10
89.40
89.40
89.30 8.550
88.50 - 3.10
SGGTT
85.70 3.70
86.70
2.40
85.00
= 3.10
85.00
3310
85.00 + 3.10
85.70 + 3.70
STGT3TT
85.90 3.70
85.90
3.70
85.90
3.70
85.90
8 3.70
3.70
85.90 + 3.70
BiCART 93.10 + 0.50
93.10
0.50
92.70
- 0.40
92.80 1.:00:0.000
92.80
0.50
92.80 +: 0.60
SGGT-C 92.90 + 0.90
93.00
0.60
92.00
= 0.60
92.50
91.90
0.60
92.90 0.90
S 93.00 0.50
92.70
0.40
92.50 0.60
92.50
92.50
92.50
0.60
93.00 + 0.50
rice
BiTAO 93.10 + 0.50
92.90
0.50
92.90
0.60
93.00
0.50
93.00
0.50
93.00 0.50
S 92.80 1 0.20
92.30
1.40
93.10
+ 0.60
92.90
0.90
92.70
1 0.20
92.30 + 1.40
ST 92.60 + 1.40
92.50 + 0.10
92.50
= 0.10
0.10
92.50
0.10
92.50 + 0.10
BICART 93.30 + 0.30
98.80
0.20
99.50
1 0.20
99.50
1: 0.20
99.40 1 0.10
99.50 +: 0.20
SCGT-C 94.60
99.40
0.10
99.40
1 0.20
99.40
0.20
99.40
0.20
99.40 0.20
S 99.50 1 0.00
99.50
0.20
99.50
0.20
99.50
0.20
99,50 1 0.20
BiTAO 96.60 0.00
99.50
0.20
99.30
99.00
99.50 0.20 0.20
99.60 + 0.20
S'GT-T 98.20 + 99.50
0.20
99.50
+ 0.00
99.60
0.20
99.50 + 0.20
99.50 0.00
ST 99.70 + 0.10
99.60
0.10
99.50 + 0.30
99.50
:: 0.10
99.50 + 0.10
99.70 + 0.10
BiCART
84.50 + 6.60
93.10
1.20
92.90
2.00 - 2.00
94.00 + 1.10
94.00 + 1.10
S'GT-C
92.90 1.10
93.60 = 0.50
93.50
93.90
93.90 1.40
SCGT
92.60 + 1.80
93.10
1,20
94.50
94.50
1.90
93.10 1.20
segment
BITAO
85.20 + 6.80
92.70
0.90 93.70
-.300
94.70
0.50
94.70 +: 0.50
SCGTT
93.40 2.00
93.40
1.50
94.30
94.40 1
94.30
1.20
ST
94.40 + 1.10
94.90
+ 0.50
94.80
1.30
94,90 + 1,20
94.80 + 1.30
BiCART 98.60 0.10
99.40
0.00
99.70
+ 0.00
99.80 1: 0.00
99.80 1 0.00
99.80
0.00
SGG 99.80 + 0.00
99.90
0.00
99.90
+: 0.00 99.90
0.00
99.90
0.00
99.90 1: 0.00
skin
C 99,90 0.00
99.90
0.00
99.90
0.00
09.90
99.90
0.00
99.90 0.00
BITAO 98.60 0.10
99.40
+ 0.00
99.80
= 0.00
99.80
0.00
99.90
0.00
99.90 + 0.00
S'GT-T 99.80 1 0.00
99.90
0.00
99.90
1 0.00
99.90 1: 0.00
99.90
0.00
99.90
0.00
S'GT,-T 99.90 1 0.00
99.90
0.00
99.90 1 0.00
99.90
0.00
99.90 1 0.00
99.90 0.00
BiCART 98.50 + 0.20
98.50
0.20
98.50 + 0.20
98.50
0.20
98.90
0.50
98.90
0.50
S'GT-C 98.80 1 0.20
98.50
0.10
98.20 1 0.60
98.60 1 0.40
98.60 1 0.40
98.80 1 0.20
wilt
SC 98.30 0.40
98.30
0.20
98.50
0.40
98.40
98.40 1 0.30
98.30 0.20
BITAO 98.50 + 0.20
98.40
0.10
98.50 98.50 = 0.10 98.50
0.20 0.20
98.50 0.20
GT 98.50 + 0.10
98.50
0.10
98.10 = 0.40 0.40
98.10
0.40 0.40
98.50 + 0.10
ST 98.40 + 98.70
0.20
98.70 + 0.20
98.70
0.20
98.70 + 0.20
98.70 0.20
G.2
Additional Empirical Evaluation
G.2.1
Runtime
Table 6: Runtime + std.) for all axis-aligned approaches separated into TDIDT and Non-Greedy approaches by dashed lines per dataset. Best approach is bolded, second-best is italicized.
model
CART
0,036
0.000
0.048 + 0.000
0.059 0.00/
0U71 0.00/
0.083 + 0.000
0.083 0.000
I1STree
0.036
0.000
0,046
1 0.001
0.058
0.001
0.067
0.000
0.077 1. 0.001
0.077
0.001
SERDT
20.560
0.037
30.489 + 0.000
36.904
+ 0.157
44.494 = 0.720
$2,905 1: 0.963
$2.905
+ 0.96
SGT-C
adult
SGT C
0.545 +: 0.005
1.069
0.006
1,503 4: 0.013
2.23 0.029
1.729 +: 0.082
1.729 - 0.082
AxTAO
0.562
DPDT
0.311 = 0.003
3,499 + 0.187
7.731
# 0.032
9.249 + 0.088
9.276 + 0.667
9.276 0.667
SPLIT
1.363 1. 0.516
1,353 1 0.511
7.286
2.613
3.477
0.208
7,823 1: 1.706
7.823 1.706
SGT-T
2.029 1 0.020
2.522
0.879
4,038
1,533
37.314
1.389
41.237 + 23.798
237
23,798
SG
2,806 = 0.123
7.232 + 0.066
22,743
19.981
50,972 - 34.416
49,766 + 28.622
49,766 + 28.622
CART
0,050 +: 0.000
0.061 +: 0.002
(.069 +: 0.000
0.069 0.000
31
avila

---

HSTree
0.054 4 0.002
0,061 + 0.000
0,072 - 0.002
0.072 + 0.002
SERDT
2.734 0.010
3.344 0.021
3,995 0.022
SOTC
1,711 # 0.036
2.343 + 0,007
2,733 0.088
2.733 + 0.888
SGT, C
6.553 0.210
8.060 0.358
7887 0,096
7.877 0.096
0.966
AxTAO
0.970 0.319
1681 0.033
3,08
3,038 1.320
DPDT
1.042 + 0.201
7.466 = 0.303
10,148 0.460
10,148 0.460
104,788
70.139 8.502
SGT-T
6.837 4: 1.515
16.545 =: 1.764
31,646 +: 22,865
31,646 22.865
SGT,-T
27.996 + 7.018
28.331
21.534
21,534
CART
0.002 = 0,000
0.002 + 0.000
0,002 = 0.000
0.003 = 0.000
0.003 + 0.000
0,003 0.000
HSeree
0,002 = 0,000
0,003 + 0,000
SERDT
0,033 1 0,000
0,080 1 0,000
0,068 1 0.000
0,085 1 0.003
0,093 1. 0.002
0.093 1 0.000
SGT-C
0.101 = 0.001
0.138 + 0.004
0.165 + 0.015
0.008
bank
SGTx-C
0.142 = 0.000
0,187 = 0.002
0.217 0.008
0.220 0.013
0.216 0.016
0,217 0.008
AxTAO
0.036 +: 0.017
0,115 1 0.025
0.147 1. 0.022
0.256 0.062
0,259 1: 0.083
0.256 0.002
DPDT
00000 1 0.007
0,041 1: 0.001
0,677 0,027
0.308 0,003
0,128 1: 0.001
0.128 0.001
SPLIT
0,653 # 0.609
0.648 + 0.601
0.757 + 0.538
2.495 + 0.512
1,369 + 0.302
2.495 0.512
SGT-T
0,726 = 0.228
3,262
7.584
3,388
ST
1.107 + 0.339
4,162 1: 4.690
2.019 + 0.634
2.130 0.663
2.101 +: 0.646
2.019 0.634
CART
0,107 1 0.001
0.135 1. 0.001
0,160 0.001
0.186 J. 0.002
0,186 0.002
HSTree
0.093 + 0.002
0.119 + 0.004
0.138 + 0.001
0.161 + 0.003
0.161 0.003
SERDT
0.007
4,996 0.009
7.345 0.007
7.345
SOT-C
1.892 + 0.027
2.420 + 0.071
1.797 + 0.041
2,328 + 0.061
2.328 0.061
bean
SGT3 C
1
AXTAO
1.367 + 0.054
2,008 + 0.048
3.189 0.589
4,472 + 0.106
4,472 = 0.106
DPDT
3,258 + 0.174
5,503 = 0.286
7.565
0.361
5,889 + 0.337
5,889 0.337
SPLIT
38.276 7.431
42,074 18.018
$5,909
14,860
71.596 1: 12.543
71.596 12.543
SGT:T
6,180 1: 0.111
19,83 1 9.656
18:729 + 4.516
43,396 +: 24.977
43.396
24.977
SG
8,404 0.343
14,332 2.636
2.059 43.177
26,883
1.676
43. + 2.059
CART 0.005 + 0.000
0.005 + 0.000
0.006 0.000
0.005 0.000
0.005 + 0.000
0,005
HSTree
0,005 = 0,000
0,005 + 0,000
0,006 + 0.000
0.006
0.000
0.006 + 0.000
0L006 0.000
SERDT
0.190 1: 0.002
0.311 1 0.001
0.334 1 0.007
0.341 1 0.002
0,353 1 0.003
0.353 1 0.003
SGT-C
0.132 1 0,001
0,207 + 0.002
0,208 0.044
0.256 0.004
0.333 0.011
0,333 + 0011
bidding
SGT, C 0.276 1. 0.007
0.479 1. 0.025
0.533 1 0.040
0.534 0.042
0.544 1. 0.043
0.479 1 0.025
AXTAO
0,070 1:
1: 0.064
DPDT 0,020 = 0.001
0.426 + 0.088
0.264 +: 0.001
0,955 1: 0,143
0,243 + 0.012
0,264 0.007
SPLIT
0,636 = 0.107
0.647 + 0.111
1,013 + 0.145
1.077 0.176
1,031 + 0.151
1,081 0.151
SGT-T
0,632 = 0.175
1,212 0.532
1.325 = 0.043
1,666 = 0.494
4,025 + 0.473
1,325 = 0.043
SG
1,000 = 0.009
2,184 + 1.188
2,432 = 1.080
2.393 + 0.967
2.472 + 1.033
2,422 = 1.080
CART
1.008 1. 0.009
1.321 1. 0,011
1.636 1: 0.022
1.977 1: 0.021
1.977 1: 0,021
HSTree
0,913 0.011
1,236 0.103
1.466 0.025
1.723 0.012
1.723 +
SERDT
281,991 = 0.979
360.049 + 1.806
447.986 + 0.312
534.572
534,572 = 1.955
SGT-C
6.135 + 0.034
11.224 0.014
10.618 - 0.0K4
15.770 0.096
15.770 0.096
SGT=-C
8,910 0.052
14.011 0.071
19.099
29.561
29,561 0.400
covtype
AXTAO
30.98 + 0.183
52,45 + 12.625
99.446 = 16.476
170.256 28.126
170.256 + 28.126
DPDT
0.671
1.185
SPLIT
912.950 1: 157.383
901
083 1 169.281
877.478 1: 164.931
881.974 1: 153.045
912.950
:: 157.383
SGT-T
34.679 7.011
56.332
+ 5.875
100.787
11.809
148.474 = 9.684
148.474 9.684
ST
73.772 + 15.480
122.400 1 38.379
397.392
65,282
728.959 +: 118.892
728.959
118.892
CART 0,031 = 00000
0.647 0.000
0,062 0.000
0.077
0.093 + 0.001
0.093
HSTree
0.029 = 00,000
0.043 1: 0.001
0.057 1 0.000
0,070 1 0.001
0.085 + 0.000
0.085 : 0.000
SERDT 2923 + 0.036
4.184 + 0.010
5.262 0.046
6.237 - 0.025
7
7A54 0.093
SGFC
0.396 = 0.002
0,438 + 0.006
0,800 + 0.007
1,356 - 0.011
2,145 0.177
2,145 1 0.177
electricity
SGT, C 0,446 1, 0.014
1.138 0.012
2.301 0.081
4.633 1: 0.381
7.367 0.918
7.767 0.918
AxTAO 0,438 1 0,003
1.488 1 0.M88
2,000 1. 0.007
6.282
2,817
10,311 0.628
10.311 1 0.628
DPDT 1.39 153994 # 0.008
3.158 1 0.016
8.492 1 0.024
13:254 - 0.028
19.481 0.057
19.481 + 0.057
SPLIT
0,458 + 0.115
1:
1: 2.394
SGT-T 1.505 1 0,374
4.662 1. 0,122
11,463 1: 1.274
31.993
5.000
72.724 1: 18.726
72.724 1. 18.726
SG
66604 = 1.547
20. 7.617
30.156 + 8.124
79.497
26.969
92.449 6.446
92.449 + 6446
CART
0.006 1 0.000
0.007 1. 0.000
0.008 1 0.000
0.009 1L 0.000
0,009 0.000
0.011
SERDT
4,464 1. 0.019
5.694 1. 0.082
7.129 1: 0.144
8.441 1. 0.177
8.41 1 0.177
0,319 + 0.048
0.523 +.006
0.046
0,995 0.030
SGT+-C
0,688 + 0.048
2.559 0.275
2.658 0.305
1.765 0.119
2.559 0.275
eucalyptus
AxTAO
4.184 + 0.028
8.124 4.652
7.608 0.738
35.022 + 12.345
33,022 12.345
DPDT
0.099 1 0.066
0.096 0.002
0.952 0.225
3.289 1: 0.128
3.289 128
SPLIT
0.267 + 0.023
3,009 + 0.936
2,925 + 0.568
8.046 + 2.917
8.046 + 2.917
SGT-T
7.649 1.954
7.649
ST
3,240 1: 1.273
2.730 1: 0.543
15.882 9.963
14,908 1: 8.775
3,240
1.273
CART
0.023 1: 0.000
0.034 1. 0.000
0.044 1. 0.000
0,053 0.000
0,062 1. 0.000
0.062 0.000
HSTree 0.021 - 0.000
0.031 + 0.001
0.040 1 0.000
0.051 + 0.002
0.060 + 0.003
0.060 0.003
SERDT 1.427 - 0.002
2.177 0.004
2,844
0.008
4,302 - 0.013
SOT-C
0.464 : 0.009
0,910 1: 0.010
1,540 4 0.020
2.501 1: 0.018
3,879 +: 0.329
3,879 :: 0.329
eye-mavements
32

---

SGT3-C
0.661 + 0.015
2.116
4.876 0 0.325
6.152 0.43
12.470
12.470 0.314
DPDT 0,973 10000000000000000000000000000010
AxTAO
0.227 + 0:004
3.89 0.49
0.018
3.191
0.037
5.777 = 0,023
9.090 0.040
5.727 - 0.943
5.727 0.93
SPLIT
19220-1 # 1.249
7,691 + 2.536
6.656 = 1.058
6.736 0.976
12934 : 1.627
6.736 0.766
SGT-T
2.330 = 0.574
5,937
1.198
12,002 = 3.378
24.338 4,59
54,067 + 25.371
$4.067
25.371
SG
4.173 = 0.033
12,936 + 1.726
16.618 # 5.723
8.677 32,929
62,689 = 18.553
62.689 # 18.553
CART 0,015 = 0,000
0,022 + 0.000
0.028 + 0,000
0.036 + 0.000
0,044 0.00/
0,044 0.007
HSTree
0,014 = 0.000
0.020 + 0.000
0,026 = 0.000
0.033 0.000
0,040 = 0.000
0,400 0.000
SERDT
0.005 = 0.036
1,318 0.029
1615 0.033
2
2.495
SGT-C
0.339 = 0.003
0,505 + 0.002
0.678 + 0.005
1171 0.005
2.087 = 0.100
2.087 2.07
eye-stale
-
0.574 = 0.007
1,633 0.008
2
0.111
6.142
6.142
AXTAO
0,449 : 0,081
1.122 + 0.191
1,808 + 0.049
2,954 1 0,564
5,547 + 1.728
5,547 + 1.728
DPDT 0.773 +. 0.012
1: 0.101
0.005
1,080
3,550 0.032
SPLIT 0,405 = 0.251
2.575 + 0.490
2.502 + 0.443
112.005 = 25,574
44.632 11.088
44,632 + 11,038
SGFT 2.187 + 0.956
4,029 1 0.029
9.571 4: 2.318
26.239 6.816
38,509 1: 12.263
38.509 12,263
SGT3-T T 4998 + 1.079
14,493 4.296
23,130 1: 0.825
1.708 37
84.647 +: 14.679
84.647 14.679
CART
+:
HSTree
0,012 + 0,000
0.016 = 0.000
0,019 = 0.000
0.021 + 0.000
0,021 0.000
SERDT
0.455 + 0.003
0.605 0.004
0.777 0.004
0,958 + 0.006
0.958 0.006
SGTC
0,641 1: 0.032
1,04 1 0.045
1,945 0.153
2.599 1. 0,102
2.599 0.102
fault
SGT--C
1,928 +: 0.047
2.356 + 0.10
4.842 0.093
5.925 + 0.192
5,925 : 0.192
AxTAO
0,282 1: 0.108
0.461 + 0.082
0.624 0.134
1.128 +: 0.190
1.128 + 0.190
DPDT
0.609 1. 0.007
0.332 1 0.006
1,336 1: 0.010
1,217 1. 0.008
1.217 1 0.008
SPLIT
+
8,209 0.269
SGT-T
4,148 = 0.724
7,013 + 1,576
12,704 1.537
64,566 + 1.261
64,566 + 1.261
SGT;-T
5.172 + 0.935
12.787 + 0.413
1.168 10.829
42.420 8.558
42.420 + 8.558
CART
0.891 +: 0.005
1.149 + 0.002
1.387 1 0.013
1,569 +: 0.014
1.569 0.014
HSTree
0,763 1 0.011
0.980
SERDT
29.884 + 0.096
39.047 + 0.190
48.294 = 0.116
57.239 + 0.289
57.239 + 0.289
SGT-
7,714 0.016
14.076 + 0.137
17.205 0.505
20.722 0.314
20.722 + 0.314
gas grift
C
35.316 +
26,835 + 0.653
40.630 1.449
47.566 0.873
47.566 + 0.873
AxTAO
8.720 + 1.644
13.83 + 2.509
25.127 = 6.892
34.319 7.455
34.319 +: 7.415
DPDT
36.866 0.395
40,200 0.063
29.836 0.182
37.349 0.300
37.349 + 0,300
SPLIT
135.933 = 4.754
120.914 + 8.115
371.082
39,808
141.962
6.900
141,962 6,909
SGT-T
1 7.772
SGT
89.856 + 31,211
101.685 + 61. 61.669
92.602 9.560
257.699 + 186.309
257.699
186.309
CART
0.045 + 1.335
0.087 1.134
0.076
2.312
0.112 2.103
0.127 + 1,230
0.127 1.230
HSTre 37.872 1.412
51.436 + 2.121
71.554 + 1.768
87.323 = 1.203
111,07 = 2.392
111087 + 2.392
SERDT
54,929
3.237
+: 2.899
5.187
SGTC
38.987 5.122
65.121 1 5.102
84.432 1: 6.939
103.653 1: 7.729
124.762
8.092
103.653 1: 7.729
higgs
SGT
67.214 4.032
75.952 + 7.323
83.548 7.122
11 8.892
141.062
9.321
187.612
11.321
AXTAO
174.298 = 12.321
296.307
19.010
361.994
+ 21.210
448.253
19,453
587.211
20.227
587.211
+ 20.227
DPDT 103.342 1: 9.101
270.478
18.23
430.524 1 24.434
499.492
1: 22.112
645.241 1. 29.110
645.241
1. 29.
SPLIT 643.232 29.211
764.445
22.667
900.000
0.000
900,000
0.000
900.000 0.000
900.000
0.000
SGT-T 299.308 + 7,472
425,915
+ 12,825
563 + 32.081
743.802 = 24.001
900 = 0.000
900.000
0.000
SGT,-T 389.313
CART 0.026 1: 0.000
0.037 1: 0.001
0,049 0.000
0.064 0.004
0.058 : 0.002
0,09 0.000
HSTree
0.027 1 0.001
0.039 1 0.091
0.050 0,000
0.068 0.007
0.059 1: 0.003
0.068 0.007
SERDT 0.983 + 0.030
1,504 + 0.005
1,900 - 0.006
2.30 1 0.008
2.754 + 0.103
1.930 0.006
SGT-C
0.328 0.002
0.020
0.020
SGT--C C 0.495 : 0.003
0.763 1: 0.018
0.731 + 0.100
0.996 1: 0.035
0,882 1: 0.119
0.731 0.100
AxTAO 0.278 + 0.084
0.842 0.122
1.164
DPDT 0.304 = 0.079
0,054 = 0.001
0,483 = 0.044
0,051 = 0.001
0.055 + 0.901
0,483 0.044
SPLIT
5.767 = 3.391
0.757 + 0.085
2,783 - 0.315
9,423 + 3.014
9,901 + 2.383
9,901 2.383
SGTT
1.116 1 0.019
1,819 1 0.601
5,049 1: 1.393
16.676 1 10.774
17.290 2.296
17.290 1. 2.296
SGT T
1. 1,566 1 0.341
1,352 1 0.293
1.635 + 0.351
586 1 0.319
1.665 + 0.335
1,352 0.293
CART
0,023 = 0.000
0.034 + 0.001
0,044 + 0.000
0,064 + 0.001
0.074 + 0.001
0.074 + 0.001
HSTree
0,023 = 0.001
0.034 + 0.000
0.045 0.000
0.064 0.000
0.075 + 0.001
0.075 0.001
SERDI
0.824 1 0.001
1,239 1 0.006
1.605 1: 0.004
2,078
0.010
2.447 1: 0.007
2.447 0.007
SGT-
0.312 1. 0.000
0.477 1. 0.00
0.689 1 0.003
1.671 1: 0.042
1,876 1: 0.061
1.876 11 0.061
magic
SCT's-C
0.325 + 0.006
0,488
0.135
2710
3.284 0.353
2.710
AXTAO
0.349 + 0.081
0,502 + 0.MI2
1.73 + 0.036
2,349 1: 0.095
2.645 + 0.151
2.645 0.151
DPDT
0,231 1
1.
2.812
11.521
SPL.TT
2,514 = 0.667
0.667 + 0.047
3.353
+ 1.291
240.554
202.408
69.764 +: 29.671
69.764 +: 29.671
SGT-T
1,687 = 0.336
4,590 + 1.273
7.290 - 4.003
18.635 = 7.555
38,378 + 36.280
38.378 36,280
S
3.670
0.536
9,933
1.604
2.513 20.429
27.240 7.119
$2.910 1: 15.146
$2,910 15.146
CART
1.327 1: 0.003
2.377 1: 0.006
3.169
0.001
3.292
3,920
HSTree
1.234 +: 0.010
2,112 1. 0.006
2,923 1. 0.260
2,987 1. 0.008
3.561 +: 0.015
3.561 0.015
SERDT
59.664 0.425
0.362 94,866
121.947 0.338
148,979 + 0.959
146.750 0.547
146.750 0,547
SGT-C
11.023 0.078
12,995
0.129
16,074 1 0.116
20.609 0.135
25.352 0.062
25.352 1. 0.062
SGT
13.670 - 0.121
18.370 3.700-0..99
20.669 + 0.178
25.379 = 1.723
66.695 + 4.619
66.695 + 4.619
mini-boone
AxTAO
3.156 + 0.058
10.769 1.910
19.074 + 2.473
28.574 3.315
36061 5.404
36.061 + 5.404
33

---

DPDT
9.395 - 1,738
148.946 = 1,520
256.252 + 1.234
47,860 = 0,757
453 668 4,310
453.668
4.310
SPLIT
8,502 = 0.022
83,926 + 33.161
188.186
19.342
212.160
16.792
152.526
14,973
212.
+ 16.792
SOFT
26,137
5.566
33,979 + 2,929
53.121 = 7.172
66.677 = 12.044
148.000 + 52.45
148,0W0
+ 52.457
SGT, T 63,035
8.172
60.582 1 45.599
71.228
125.311 14,876
256.178
7.425
256.178 7.425
CART
0,007 1 0,000
0.009 + 0.040
0,010 +: 0.000
0.009 : 0.000
0.009 +: 0.M00
0.009 0.000
HSTree 0,007
+ 0.000
+ 0.000
0.010
0.001
0.000
+ 0.001
SERDT 2.501 + 0.030
3,648 + 0.029
4,413 = 0.083
5.367 0.089
6.145 = 0.225
6.145 0.225
SGTC
0.185 + 0.003
0,331 +: 0,008
0.399
1: 0.026
0.327 0.004
0,330 1: 0.004
0.327 0.000
muchroom
-
0.77 0.003
0.326 0.095
0.392 0,028
0.395 0.003
0,400
0.392 0.028
AXTAO 0,111 + 1.011
1.324 + 0.349
1,711 = 0.004
1.463 = 0.004
3,212
1.463 0.004
DPDT 99210 + 51015
0,725
SPLIT E21T0
T 0,011 1
1,065 1 0.484
1,223 1 0.470
0.865 0.241
0,895 1. 0.210
0.865 0.241
SGT-T
0.567 H 0.002
10.795
1,596 +
2,077
SGT3-T
0,919 = 0.007
1,07 = 0.021
1.402 0.020
1,490 0.012
1,535 = 0.019
1.462 0.020
CART 0,012 = 0.000
0.000
0,025 0.002
11STre 0,012 1 0,001
0.016 1 0.000
0.018 1 0.000
0,019 1 0.001
0,017 1 0.003
0.017 0.003
SERDT 0.546 + 0.011
+ 0,186 0.003
0.055
+ 0,406 = 0.024
SGT-C
0,148 + 0.002
0.176 = 0.004
0.197 0.006
0.405 0.024
SGT,-C
0.216 0.002
0.222 0.036
0.365 + 0.015
0.511 0.033
0.579 0.068
0.365 1 0.015
AxTAO E8I1 1 0.000
0.098 1. 0.MI2
0.111 0.009
1,085 0.068
1.666 1. 0.475
1.666 0.475
DPDT 0,088 : 0.001
1.757 + 0.002
1.722 2 0.300
4.697 + 0.181
5,932 + 0.196
4.697 0.181
SPLIT 0206 =
4.228
4.165
4.192 1.823
4.248 1.824
1.823
SOFT
0.40 # 0.013
5,539 + 4,348
2.112 = 0.388
20.543 + 11.216
27,321 + 16,985
27.321 16.985
SGT3-T
1291 0.249
CART
0.007 + 0.000
0.012 + 0.000
0.012 : 0.000
0.014 0.000
0.012 0.000
HSTree
0.008 0.000
0,013
0.013 0.000
0.015 + 0.000
0.013
SERDT
0.335 + 0.002
0.477 + 0.018
0.589 + 0.017
0.711 = 0.022
0.711 = 0.022
1
0.075
SGT--C
0.817 + 0.063
1,053 = 0.122
1.318 0.105
1.798 0.142
1.318 = 0.105
page
AxTAO
0,399 + 0.0%6
0.347 0.137
0.143
0.976 0.366
0.366
DPDT
0.055 + 0.007
1.071 + 0,017
0,315 0.010
0.423 + 0.015
0,423 0,015
SPLIT
5,425 1 2.155
8.316 1 0.927
5.511 1 1.992
9,737 1: 1.043
5.511 1.992
SGT-T
3,422 1 1.302
4.174 + 0.396
11.024 9.302
24,833 + 23.466
24.833
23,466
S
4.892 1 0.064
9.753 1. 1.842
8.558 11.594
23.490 J. 15:246
1.842 9.753
CART
0.021 + 0.000
0,027 + 0.000
0.033 + 0.000
0,033 0.000
I1STre
0,020 1 0.000
0.025 0.000
0.030 1 0.000
0,030 0.000
SERDT
2.127 0.017
2628 0.032
3,216 +: 0.037
3,216 0.037
SGT-C
2.312 1 0.020
3.443 1 0.053
4,788 1 0.033
4,788 0.033
pendigits
SGTy-C
4.421 0.009
7.37
0.429
0.429
AxTAO
1.653 1. 0,364
3,077 1: 1.128
4.157 1. 0.779
4.157 0.779
DPDT
0.811 0.004
0.021
9.062 + 0.068
SPLIT
28.8 828 + 1.348
35.454
1.105
48.427 1.831
48.427 + 1.831
SGT-T
10.774 + 1.335
18.070 = 0.007
48.994 + 24.878
48.994
24.878
SGTa-T
32.178 5.256
180,997
9.845
20.653
282.237 + 20.653
CART 0.002 :: 0.000
0,M02 + 0.000
HSTree 0.002 = (0110
0,003
= 0.000
0,003 = 0.000
0,003 0.000
0.003 + 0.M00
0.M13 = 0.000
raisin
SGT--C
SERDT 0/038 + 0000
SGT-C
0,137 = 2000
0,15 = 10000
100
0,196
0,241 : 0.026
0,58 1: 0.001
0,196
0,233 0.029
0.275 + 0.037
0.080 1: 0.000
0.129 = 0.003
0.09 1: 0.002
0.249 0.034
0.136 = 0.002
0,317 + 0.087
0,106 + 0.001
0.080 0.000
0.241 0.026
0.034
00000 = S000
AXTAO
0.052
+ 0.038
0.052
DPDT 9,016 = 0,000
0.276 1 0.001
0.542 + 0.048
0.779 1 0.064
1,060 1 0.112
1.000 : 0.112
SPLIT
0,780 = 0.394
0.613 + 0.353
0,741 0.303
0.485 0.144
0.115 0.024
0.741 0.303
SGF-
0.764 = 0.259
1.336 + 0.303
2.336 + 0.817
2,599 0.880
2.654 + 1.199
2,336 0.817
SGT, T 0.650 1 0.260
1,859 0.844
2,006 1 0.631
2.788 0.649
2.106 1 0.676
1,859 0.844
CART
0.005 = 0.000
0.007 + 0.000
0 + 0.000
0010 1 0.000
0.012 + 0.001
0.010 0.00
HSTree
0.005 + 0.000
0.007 + 0.000
0.009 0.000
0.000
0.009 - 0.001
SERDT
0.176 0.021
0,255 + 0.019
0.326 - 0.002
0.414 0.006
0.457 + 0.002
0,326 0.002
SGTC
0.134 + 0.002
0.239 1 0.033
0.337 1 0.029
0.441 1 0.062
0.541 0.04
0.541 0.099
SGT
0.204 0.002
0,311 0.022
0.018
0.332
0.022
0.204 0.022
AxTAO
0,058 1. 0.023
0.065 1. 0.024
0.279 1. 0.033
0.354 0.117
0.527 1: 0.114
0.527 0.114
DPDT
0,121 + 0,021
SPLIT
L3344 1 0.386
0.648 1. 0.177
0.645 1 0.167
0.644 1 0.157
0.655 1 0.148
1,334 1. 0.386
SGT-T
0.550 = 0.006
2.356
1,251
5,808 4.53
7.415 1,940
SG
0.933 = 0.349
6,232 + 5.830
0.849 1,991
2,316 1,014
2.158 + 0.941
1,991 0.849
CART
+: 0.000
0.000
0.001
+ 0.001
11STree
0.007 1. 0.000
0.009 1 0.000
0.011 1. 0.001
0.011 1 0.000
0,811 1 0.001
0.011 1 0.000
SERDT
0.541 + 0.019
+ 0,535 0.004
1,546
SGT-C
0.238 = 0.004
0.704 = 0.027
0.770 0.014
0.773 + 0.012
0,773 = 0.012
SGT,-C
0,478 = 0.017
0.884 - 0.053
0.938 + 0.069
0.967 0.075
0.932 + 0.085
0.938 0.069
10om
AxTAO
0.308 1 0.001
0.506 0.M02
0.871 1. 166
0.751 0.260
0.814 1. 0.272
0.804 0.272
DPDT
0,084 = 0.007
0.738 + 0.089
0.578 - 0.006
1,10 0.044
1,021 + 0.072
2,400 0044
SPLIT 0.327 + 0.032
0.614 0.129
5.658 0.178
7.797
6.261 0.343
03343
SOF-T
2.344 1 0.318
3.408 1: 0.033
9.386 4 5.627
6.525 1 1.961
6.014 + 2.769
6.014 2.769
34

---

SGT3-T
1.92 = 0.101
3,035 = 0.150
3.632 + 0.132
8.988 = 4.116
12.089
7.349
3.632 0.132
CART
0.008 1 0.000
0,010 1. 0.000
0.010
0.000
0.011 1. 0.000
0.011 0.000
HSTree
0.009 + 0.000
0,010 - 0.000
0011 0.000
0.012 + 0.000
0.012 0.000
SERDT
1:
SGT-C
0.410 + 0.015
0.468 + 0.016
0.844 0.083
1,099 + 0.109
1,009 0.109
SGT,-C
+ 0.017
1,271 0.084
1.234 0.038
1,988 = 0.096
1,234
segment
AXTAO
0.198 + 0.094
0.524 + 0.158
0,790 0.207
0.732 = 0.038
0,732 0,038
DPDT
0.329 + 0.030
1.110 = 0.011
0.516 0.012
1.142 = 0.064
1.142 0.064
SPLIT
4.735 1,433
4.778 1.408
4,999 1,451
5.115 + 1.427
5,115 1.277
SGT-T
2,355 + 0.157
4,999 = 1.288
7.528 = 1.169
7,641 = 0.787
7.641 0.787
SGI,-T
CART 0.043 = 0.001
0.059 + 0.000
0.067
0.001
0,081 0.000
0.091 + 0.000
0,091
HSTree 0,049 = 0,001
0,044 + 0.091
0.080 + 0.001
0.09 0,002
0.099 + 0.007
0.099 0.007
SERDT
2930 = 0.044
4,499 + 0.101
5.699 + 0.068
6.022 = 0.101
8.007 + 0.374
8.007 = 0.374
SOT-C
0.399 = 0.009
SGT,
C 0.398 1: 0.014
0.537 0.009
0.717 1 0.026
0.944 1: 0.013
1.117 0.013
0.013
1.117 1117.100.01
skin
AxTAO 3.356 + 0.022
62.218
DPDT
2,691 = 0.023
1,058 + 0.011
13,309 + 0,004
19,033
0,066
20,969 + 0.209
20.969 + 0.209
SPLIT
82.441 # 25.752
77,34 + 23.421
96.253 22,971
269.301
47.638
86.591 8.351
86.591 8.351
SGTT
4.319
1.118
9.555 0.206
31,501 1 0.513
41,956
8.365
84.671
0.898
84.671
1 0.898
SGT,-T
9.801
= 2.874
24.299 + 10.724
83. 188 + 1.171
26.254 78.788
176.502 + 16.665
176.502 + 16.665
CART
0,005 = 0,000
0.007 + 0,000
0 = 0.000
0,009 = 0.001
0.008 + 0.001
0.009 0,001
HSTree
0,066 + 0.000
0.007 +: 0.000
0.009 0.000
0010 0.001
0.010 + 0.001
0.010 0.001
SERDT
0.159 1 0.019
0,212 1. 0.000
0,289
0.002
0.368 0.013
0.394 1. 0.022
0.368 0.013
SGT-C
0.127 + 0.002
0,169 + 0.001
0,252 1 0.009
0.292 + 0.033
0,316 + 0.059
0.252 0.009
SGT+-C
0.183 0.006
0.242 0.021
0.352 0.024
0.251
0.256 0.009
0,242 0.021
ArTAO
0.087 + 0.028
0.128 1: 0.000
0,421 + 0.125
0,584 1 0.083
0.462 1: 0.117
0.584 0.083
DPDT
0.035
SPLIT
1612 = 0.771
0,761 + 0.545
1,685 + 0.718
1,898 0.157
1.684 + 0.100
1.898 0.157
SGT-T
0.894
= 0.390
1,666 + 0.538
2,890 0.713
13.621 - 10.497
4,850 + 2.549
4,850 2,549
STT3-T
2.000 0.609
2.338 + 0.916
8,042 + 6.568
10.688 10.439
10.251 + 10.054
10.688 10.439
Table 7: Runtime (s + std.) for all bivariate approaches separated into TDIDT and Non-Greedy approaches by dashed lines per dataset. Best approach is bolded, second-best italicized.
dataset
model
2
Best
2.611
244.966
S²GTC
0.914 1 0.023
1.466
0.018
1,785 1 0.013
2,138
0.251
2,883 1: 0.237
2,883 0.237
adult
5²GT C
1,018 +: 0.013
2.766 1: 0.069
5,652 1: 0.187
9.798
1: 0.353
9.137 +: 0.156
9.137 0.356
519.138
9.067
1: 676.755
715.373
277.318
676.755
SPGT-T
3,048
0.475
5.817
0.526
10.216 0.996
8.5% 0.906
29.082
19.674
8.596
0.906
SGGT--T
8.479 + 1.166
16,80 + 3,316
28.323 7.263
58.823 7.955
227.868 + 25.558
58.823 + 7.955
BICAKT
3.605 + 0,016
2.272 0.004
2.646 0.010
2.646 0.010
Sere
4.248 + 0.131
5,380 1 0.214
6.402 + 0.171
6.402 0.171
avila
S C
9.139 1: 0.178
10.774 0.165
9.608 1 0.245
9,608 0.245
8.205 0.404
27.271 1: 4.852
S²GTT
16.503 1. 11.068
18,244 2.260
23.580
1. 1.872
23,580 1. 1.872
S'GT3-T
19.400
0.481
4.050 22.246
32.641 + 0.356
32.641 + 0.356
BICART
0.011 1: 0.000
0.015 0.001
0.010 1 0.001
0.011
0.001
0.011 1: 0.001
0,010 0.001
S
0.132 1: 0.001
0.144 +: 0.072
0.149 1 0.009
0,142 1: 6.011
0.141 1 0.009
0.132 1: 0.007
bank
S'GT,-C
0.170 - 0.008
0,212 11220000 0.017
0.209 + 0.021
0.213 0.016 0.016
0,214 0.020
0,170 0.008
0.437
0.539
1,500
SCGFT
0.680 + 0.034
0.546 0.137
0,531
1 0.176
0,515 1: 0.152
0.476 1: 0,159
0,546 0.137
ST
0.635 1 0.277
0.718 1 0.202
0,860
0.271
0,877 1. 0.259
1.106 1. 0.353
0.635 0.277
B.CART
5.241 + 0.034
4,492 + 0.016
5.400 1 0.063
6.030 + 0.085
6.030 0,085
S
4.264 1: 0.059
4.134 1 0.057
3,789 1: 0.114
7,849 1: 0.426
7.849 1: 0.426
S
6.099 1: 0.066
9.546 1 0.254
9.039 1: 0.777
6.448 +: 0.122
9.546 0.254
BYTAO
13.466 1. 0.089
17.550
3.502
21.262 4.176
17.836 1. 5.087
17.836 J. 5.087
S
10,078 +: 0.063
14,549
2.278
35.814 + 17.026
49.716 + 23.694
49,716 23.694
S
25.607 +: 0.510
28.688 + 5.987
45.855 = 0.793
36,229 + 4,994
45.855 + 0.793
BICART
0.122 = 0.000
0.226 + 0.004
0J087 + 0.000
0,088 + 0.000
0.087 + 0.001
0,087 0,000
0,379 0.062
0.062
0.420 + 0,076
0-433 + 0.089
0.444 1 0.084
0.444 + 0.085
0.420 1: 0.076
bidding
S²GTs
0.520
1: 0.007
0,631 1: 0.043
0.643 1 0.040
0.641
0.041
0.634
0.045
0.520 1: 0.007
BYTAO
0.455 1 0.155
2,844
1.296
0,881
0.103
0.988
0.127
0.956
0,122
0,881 0.103
SGTT
3.742 1: 2.479
1.552 1 0.201
I 1886
1 0.069
2.064 1. 0.305
2.151 1. 0.424
1,552 1 0.201
S-GT,-T
3.784 4: 2.417
1.336 + 0.026
1.449
+ 0.033
1.363 1: 0.031
1.847 + 0.070
1.336
0.026
BCCART
1.710 1: 0.013
2.459 1 0.015
4.716 1: 0.011
4,399 1: 0.060
4.468 1: 0.027
4,468
0.027
electricity
35

---

S
1,751 + 0,020
3,068 + 0.041
3,943 = 0.022
6,214 + 0.068
7.709 + 0,056
7.70M + 0,056
S'C
2.07 1.2 0.024
4.0K3 1 0.066
6,865 1 0.109
13.089 0.404
16.725 1 0.356
16.725
BITAO
3,090 = 0.013
5.217
19.134
2517
25.4%
23.716 + 5,873
23.716 5873
S
4,900 0.901
13.187 = 2.958
192 8,55
35.407 5.905
64.266 27.552
64.266
27.552
STTT-
16,278 = 1.499
29,933 + 8.363
38.181
0.606
58.046
15.761
186,041 + 81.833
186.041 + 81.33
BICART
2.343 + 0.006
5.074 0.073
6,529 + 0.029
5.365 0.003
6.213 0.023
6.213 0.023
SCGT-C
1.82 = 0.068
3,410 + 0.018
5,833 = 0.091
9.208 + 0.331
14,078 + 0.496
14.078 0,496
eye-movements
S'
2.933 + 0.024
6.749 + 0.222
14.320 0.381
23.759 3.123
41.232 + 2.209
41.232 +: 2.209
BTTAQ
5.639 1.477
1,853
12.694 1 3,756
25.351
15.615
30,017 1: 7.258
30.017 7.258
S'GTT
5.723 + 0.934
13.424 +: 2.150
20.808 2,432
40.531 + 6.476
70.415
28.127
70.415 28.127
STGT3-T
13.826 1: 1.436
31,005 1: 7.743
30,284
4.121
$5.390
7.432
69.564 1. 6.342
31,005 1: 7.743
BICART
1.575 + 0.018
2,290 0.085
3.017 + 0.008
2.397 0.016
1,431 0.008
1,431 0.008
S'GT-C
1,488 = 0.009
2.650 + 0.
4.042 = 0,019
6,192 + 0.074
5.843 + 0.092
5.843 0.092
eye-stale
S'GT--C
1998 + 0.009
3.895 + 0.009
6.083 = 0.386
7.614 + 0.383
14,603 + 0.709
7.614 + 0.383
BITAO
5.084 0.763
6.086 0,920
15,055 3,777
16.758 3,696
17.236 + 4.397
17.236 4.397
S
4.785 + 1.786
10.767 + 5.110
20.799 3.613
25.682 + 10.657
39,077 + 5.903
39,077 + 5.903
ST
10.356 = 2.611
13.256 1: 0.249
36.587
3.486
84.161 1: 64.111
83.416 + 22.508
83,416 1: 22.508
BiCART
0.671 1: 0.004
0,858 1 0.020
1,056 0.021
2,960 0.041
2,960 0.041
S'rre
1.612 + 0,073
3,789 + 0.147
4.264 1: 0.128
3.640 + 0.253
3.640 :: 0.253
fault
S'GTa-C
2.670 1: 0.381
5.217 1: 0.624
4,977 1: 0.401
15.148 1: 1.188
4,927 0.401
BrTAO
27.657 + 8.662
63.598 + 13.320
37,251 2.027
9.754 3.397
9.754 3.397
SGTT
6.341 + 1.211
11.313 7.714
11,403 3.118
18.722 + 1.558
11,403 + 3.118
STTT-TT
9.007 + 1.816
7,998 + 0,319
17,878 0.855
28.492 + 5.414
17,878 +: 0.855
BiCART
306 141 1.082
549.345 + 1.295
595,498 + 2.316
687,982 + 21.935
687 687.982 + 21.935
S
63.441 + 2,416
85,418 5.980
89.618 1.188
107.478 2.900
107.478 2.900
gas-drift
S
69.479 +: 0.213
177.539 +: 4.145
100.374 + 2.373
110.818 1.290
110.818 +: 1.290
BYTAO
383,562 1: 93.594
1327.892.
366.219
582
67.410
1090,
152 + 121.334
1090.
152 121.334
STTT
202.147 1: 53.643
192.221
1: 68.401
208.110
44.917
186.536 1: 20.301
186.536 1: 20.301
S
212.746 1. 133.915
202.622
15.431
249.498
1. 18.637
253,412 1: 18.672
202.622 1: 15.431
BICART
0.465 + 0.000
0.691 + 0.008
1,050 + 0.065
1,035 +: 0.115
1.175 + 0.210
1.175 : 0.210
S*GT-C
1.375 = 0.019
1.614 0.023
1,867 = 0.009
1,888 +: 0.239
3.212 + 0.266
1,867 0.009
SCGT
1.706 = 0.039
1,496 + 0.091
2.098 = 0.188
2,490 0.329
2.681 0.201
1.496 0.091
BITAO
0.490 = 0.004
4,274 0,386
7,087 = 2.078
3,572 + 0.883
6.305 + 0.846
6,305 0.846
S"OFT
1,359 + 0.254
3,521 + 0.623
4,992 = 0,509
11.568 2.664
12,452 + 3.665
3,521 + 0.623
ST
3.907 + 1,259
4.842
1.006
7.099 1 1.676
9.252 1 1.575
13.567 1: 1.836
9.252 1.575
BICART
0.532 1. 0.003
0.761 1: 0.004
0.986 1 0.004
1,214 1: 0.005
1,399 1: 0.007
1,214 1: 0.005
S'GT-C
1,222 +: 0,006
2.023 + 0.076
2.579 + 0,042
3.417 1: 0.036
5.149 1:49:10 0.293
3.417 0.036
magic
CC
3.632
BITAO
2.312 = 1.436
7,309 + 1.476
5,492 + 2.836
11.394 1.136
12,020 + 6,460
12.020 + 6.460
S'GT-T
3,670 + 0.175
6.565 + 2.845
8,800 + 1.447
19.018 4.394
22,002 + 2,200
19.018 + 4.394
S'G13-T
6.552 = 0.972
11.303 + 3.233
28.370 18.524
64.631 +: 32/421
49.323 + 4,491
64.631 32.421
BICART
205.993 + 1.017
283.306 3.623
421. + 6.973
585.503 + 2.113
685 985.981 1.666
685.981 + 1.666
S*GT-
87.626 0.976
87,751 + 0.520
105.152 + 0.643
135,392 + 4.667
196851
0.727
196.851 0.727
mini-boone
S
77.147 0.087
131.214
184.391 2,667
182.661
4.632
275,842
1.822
275,842 1.822
BITAO
513,002 +: 218.
695.079
+ 137.696 696
1580.557 +: 1.59.885
746.081
90.647
692,239 +: 287
692,239 1: 108.287
S'GTT
255.640 60.646
240.491
1: 36.959
249.069
31.002
364.934
26.289
254,313 1: 11.954
254.313 1: 11.954
5'GT3-T557.680
375.451
48913
659,697
BiCART
9.876 + 0.035
11,360 + 0.076
11.661 0.285
11.900 = 0.104
11.424 + 0.047
11,360 + 0.076
S*GT-C
0.450 = 0.004
0.574 + 0.058
0.574 - 0.060
0.060
0.567 0.054
0.574 0.058
mushroom
G
0.674 0.002
0.008
0.676
0.003
=
0.674
BITAO
23.650 = 0.082
32.016 + 0.068
31,680 + 0.114
31.644 0.217
30,900 + 0.695
32.016 + 0.068
S'GFT
0.903 + 0.051
1.237 0.153
1,57 = 0.165
1.254 + 0.178
1.228 0.168
1.237 0.153
STGT3-T
1.725
0.012
BiCART
0.165 + 0.002
0,236 1: 0,002
0,242 1 0.027
0.396 1: 0.011
0,181 + 0.002
0,181 0.002
S
0,653 + 0.025
0.939 0.061
0.744 + 0.021
0.585 0.052
occupancy
S' C
0.739 1. 0.014
0.680 1. 0.002
0.991 1 0.057
1.741 0.049
1,879
0.047
0.739 0.014
BITAO
0.358 + 0.007
0,592 + 0.279
0.966 - 0.297
2,024 + 1.027
2.756 + 1.003
2.756 + 1.033
S'GTT
2.334 = 0.224
3.354 + 1.055
10.974 + 7.008
8.160 1.116
8.196 1.756
8.196 1.756
S'OIS-T
3.550 0.639
3,924
BICART
0.136 0.001
0.265 + 0.002
0.568 + 0.048
0.624 0.088
0.624 0.088
S
0.647
12199 0.055
1.219
S'GT3-C
1.553 0.106
0.106
0.974 1 0.004
1.120 1: 0.110
1,411 +: 0,129
0.974 0.004
RITAO
2.071 +: 0.330
2,233 + 0.756
2.252 1: 0.853
3.958 +: 2.572
3.958 4: 2.572
S²GT-T
3.085 0.805
8,576 - 8.214
13.182
11.022
6.672
1.158
3,085 0.805
TT
6.055 1.312
2.250 10.564
10.963
1.690
15.098 1 2.583
10.564 1: 2.250
36

---

BCCART
3,703 1 0,029
4,409 1 0.038
4,979 1 0.132
4,979 0.132
S'GT-C
5.447 + 0.137
5.681 0.077
5,999 1: 0.100
5,999 100
pendigits
S"GTa-C
13,424 0.194
11,251 =
16.132 0.061
0.546
BITAO
16.04 1.840
27.429 2.592
39,368 + 6,442
39,36 + 6,442
SPGT-T
21.442 = 11,748
20.195 3.336
41,947 + 12,994
41,947 12.994
STGT,-T
31,893 6.406
163.337 + 7.436
56,912 + 9.267
56,912 + 9.267
BiCART
0.019 + 0.000
0.037 + 0.001
0.041 = 0.001
0.030 + 0.001
0.041 + 0.001
0,037 0.001
S+GT-C
0.180 = 0.001
0.329 + 0.003
0.465 = 0.014
0.520 0.028
0.560 0.063
S
0.269 0.001
0064
0.391 = 0.061
0,381 0.055
BITAO
0,285 1 0.219
0,635 1. 0.211
0,471 1 0,088
0,480 1: 0.092
1,322 1: 0.369
0.635 0.211
S'GT-T
0,509 + 0.154
1,571 + 0.680
0.992 = 0.376
1,216 0.535
1,137 + 0.567
0.509 0.154
STGTa-T
0.727 0.285
0,932
0.477
1.159 0.605
0.49]
0.714
BiCART
0.117 + 0.000
0.168 + 0.001
0,213 + 0.006
0:236 1: 0.004
0.169 + 0.002
0.236 0.004
S
0.317 + 6.006
0.468
0,715 + 0.060
1,82 0.077
0.025
G-G
0.339
0.815 0.074
0.083
0.841
0.002
BITAO
0,391 = 0.001
0.595 + 0.363
1,899 = 0,348
1.103 0.452
0.897 + 0.303
1.103 0.452
S
0,728 = 0.013
1,912 + 0.823
5.103 + 3,741
4,694 1.198
2.002 0.473
1,912 0.823
S'GT3-T
3.737
BICART
0.245 i. 0.002
0.329 0.001
0.334 1 0.007
0.371 1: 0.006
1.095 0.039
0.354 0.007
S"GT-
1 1.284 0.063
S'GTa-C
0.802 0,016
1,259 0.06
1.274 0.061
1,286 0.068
1,259 0.076
BITAO
4.12 + 0.520
2.872 + 0.386
1,356 = 0.090
5.522 1 0.832
3,008 + 1.874
5,522 0.832
S
2.757 = 0.664
2.666 0.581
3.444 = 0,002
1.386
4.337 0.795
3.444 0.002
S-GTs-T
4.492
BICART
+
0,856 + 0.008
S*GT-C
1.555 0.106
2,007
0.044
2.454 0.174
2.454
segment
S'GTs-C
2.722 0.065
3.112 + 0.235
3,526
0.332
3,515 + 0.317
3,512 0.235
BTTAO
3,508 11 1.662
2871 + 1.091
4,466 1.403
5.786 1: 0.956
5,786 0.956
S'GT-T
3,402
4.688 0.903
1.683
GT
6,858
5.228 0.796
6,383 1.162
8.472
6,383 1.162
0.350
+:
S*GT-C
1.529 0.086
1.572 0 0.126
2.164 0.047
2.281 0.059
2.371 0.097
2.371 0.097
skin
SCG
1,244 - 0.173
1.668 + 0.109
1.843 = 0.028
1,975 0.152
1.716 + 0.089
1.716 0.089
BITAO
1.343 = 0.007
2.537 0.034
4,490 = 0.230
4.175 1,884
24.620 4.906
24.620 + 4,906
S-GTT
3,988
4.136
ST
4.443 + 0.053
6.694 0.465
5.812 1.213
39.704 +: 31.111
54.114 41.059
5.812 1,213
0.001
S"GT-C
0,280 +: 0.003
0,366 0.019
0.479 1: 0.028
0.401 0.027
0.396 0.034
0,280 0.003
wilt
STG
0.313 + 0.007
0,480 0.020
0.595 = 0.057
0.672 + 0.038
0,680 + 0.035
0,480 + 0.020
BITAO
0,413 = 0.122
0.376 = 0.109
0.213
0.768
S-GT-T
0.781 0,193
0.197
L.244 0.627
1.501 0.732
1.373 0.615
0.781
STGT3-T
0.825 + 0.293
0.678 0,174
0.940 = 0.358
0.762 0.272
1,056 + 0.434
0.678 + 0.174
G.2.2 Optimization Procedure Ablation
In this section, we examine our bin-to-branch optimization procedure. Here. we provide ablations: removing the coordinate descent procedure and only replying on Weighted K-Means to assign bins to branches (denoted "No CD") and replacing the Weighted K-Means initialization with a random initialization (denoted as "No WKM"). ". For these experiments, we work with subset of our datasets Electricity, Eye-Movemente and Etatet We run hyperparameter tuning for both ablations and choose the model with the lowest validation loss for each depth. We report the average test accuracy for the ablated TDIDT approaches Table g and provide runtimes for these approaches Table 9 In these tables, "All" denotes using the full optimization procedure. In general, we observe that both ablations consistently achieve lower test accuracy than the full models across most depths and tested datasets, especially at depths above two, with minimal sacrifice in runtime.
37

---

Table Test Accuracy (% + std.) results for the optimization ablation analysis. Best approach for each model and dataset is bolded.
Model
2
5
Overall
ShapeCART
All
7..5:1..1
78.5
81.3
84.4=0.1
84.8.10.6
84.8
electricity
No CD
74.9:0222
76.5+10.3
79.6:0.55
83.7=1.2
83.410.2
83,410.2
No WKM
74.9+0.2
76.5+0.3
79.4#0.1
83.2=04
83.340.5
83.340.5
All
48.1::2.1
53-0.7
54.91112
53.9-077
57.41122
57.411.2
eye-movements
No CD
47.1+3.3
52.5+-13
52.4+22.7
47.4-1.6
51+0.5
51+0055
No WKM
47.311.2
51.841
51.520.7
47.2=0.8
50.111..
50.1
All
64.3+0.9
69-
70.5+112
1..4-1.3
74.5+111
74.5+1.1
eye-state
No CD
62.5:0.3
68.6110.9
70.31:
7..9=0.9
73.710.7
73.710.7
No WKM
64.41:1.2
68.5+1
7..5+112
72.4=1.3
74.+1111
7..5+1.1
ShapeCART,
All
0.3
80.610.6
8.:
8610,4
87.811
87.8=1
electricity
No CD
7..5+1.2
78.5+0.3
82.3+0.1
85.7=0.3
88:0.55
88:00.5
No WKM
7310.4
78.1
82:0.4
8..8=0.5
88.310.3
88.310.3
All
47.1
54.410.8
5610.3
57.5-1.6
60.912.6
60.912.6
eye-movements
No CD
46+00.5
54.8+1.4
16.1+1.4
54.4-1.7
56.5+3
56.5-3
No WKM
45.6:10.4
55.31:19
56.222.3
5611.
57.21199
17.241.9
All
66.110.77
70.8111
73.311
76.8=1.3
77.5.1..5
77.510..5
eye-state
No CD
64.6+
69.3+1
73.7+11.
7.14-0.1
76.7+
76.7+00.5
No WKM
65.110.8
70.7+1.1
7311
77.5-133
77.710.2
77.710.2
CART
All
80.611..
82-0.6
83.9
86.5=1.2
87,5.0.5
87.5
electricity
No CD
78.3#0.4
77.4+0.7
79.
84.8=0.5
86.3++0.3
86.3++0.3
No WKM
78.540.6
77.410.2
79.81
84.211
85.710.4
85.710.4
All
57.8114
6042
61.713.2
62.6-3.3
64.813.5
64.813.5
eye-movements
No CD
57.743.22
59-1.5
58.243..1
58.4-2.9
60.5
60.5+005
No WKM
56.412
57.5
58.7+0.7
57.1=0.2
61.111
61.1=1
All
71.7111.1
72.81007
74.422.1
76.112
80,210.3
80.2+0.3
eye-state
No CD
71.5+0.5
7..1
71.9+009
74.5=0.9
80.3+00.4
80.3+00.4
No WKM
71.240.3
71.811.4
71.711.2
73.4=1.8
80.211.2
80.2+1.2
ShRPe
All
81.810.2
84.511.33
86.610.8
87.311
88.811.22
88.8+0.22
electricity
No CD
79.8+0.5
81.8+0.3
85400.6
86+00.3
89--0.3
894-0.3
No WKM
8010.4
81.240.3
85.1
85.8=0.9
89.110.5
89.1
All
59.6+22.4
63.6+1..4
66.2+119
67.513
63.91:2.4
63.9
eye-movements
No CD
55.11:1
58110
58.11.7
57.4=3.4
51.9
51.9111.8
No WKM
56.6+1.11
58+2
60.313
62.7=6.3
55.413.6
55.413.6
All
73.13-..
75.910.4
79.311.1
81.1-0.6
80.6.11.9
81.110.66
eye-state
No CD
74+0.8
75.4+1.8
80,2+0.2
81.5-0.4
80.3+1.1
81,5+0,4
No WKM
73.311.11
75.34:
7913
81.7=6.3
78.813.6
78.843.6
38

---

Table Runtime std.) results for the optimization ablation analysis. Best approach for each model and dataset is bolded.
Model
2
3
4
5
6
Overall
ShapeCART
All
0.396++0.002
0.006
0.88:+0.007
1.356=#0.01
2.145=00.77
2.145=0.177
electricity
No CD
0.283=:00088
0.439=0.017
0.692:40.06
0.883:+00188
1.385=0.097
1.385=0.097
WKM
0.294+0.011
0.481-00009
0.722+0.016
0.809+0.012
1.463++0.043
14463-0.433
All
0.464:1.009
0.91-0.01
1.5420.02
2.501=100118
3.879-0.329
3.879=0.329
eye-movements
0.277:10.011
0.48:+0.018
0.036
1.421:10.066
2.482
2.482-00473
WKM
0.311:00112
0.622=0.022
155
1844:10.039
2.693:1.22
2.693-0.2
All
0.339:+0.003
0.505=0.002
0.678:00.005
1.171=0.005
2.087=0.1
2.087=0.1
eye-state
No CD
0.165=0.011 0.292=0.007
0.457=0.029
0.88=0.094
L.30620.089
1306=00089
No WKM
0.172=:0.006
0.238
0.283=0.008
0.501=10.338
0.839
0.039
ShapeCART;
All
0.646:10011
1.138=0.012
2.301
4.603-10.381
0.767=0.918
7.767=0.918
electricity
Nn CD)
0.318+0.014
0.546 0.014
1.087 0.054
1.992++0.045
3.01 +0.35
3.01 3.015
0.438:0.004
0.018=0.052
1.903-0.068
2.098:0012
2.68
2.688-10.02
All
0.661=0.015
2.216=0.119
4.876=0.325
6.152:10.443
12.47=0.314
12.47=0.314
eye-movements
No CD
0.352:000027
0.656=0.105
1.201++0.03
1.749=10.193
2.762=#0077
2.762=0.078
WKM
0.629:10.07
1.564=0.264
3.67211.31
3811110.331
4.829=0.155
4.829=0.155
All
0.574-00.007
1.063-0.008
2.511+0.133
4.777+0.111
6.192-0.148
6.192-0.148
eye-state
0.192:40.008
0.4840.0044
1071
1,975-10.048
4.265-10132
4.265=0.132
No WKM
0.136=00.007
0.014
0.59:0.029
1.167=0.073
0.86:0.154
1.854
ShaRe"CART
AlI
1.751+00002
3.068-0.041
3.943+0.022
6.214+0.068
0.056
7.709
electricity
No CD
1.585=0.005
2.759=0.052
3.713-10.17
4.962:10.14
0.619-00056
0.619
WKM
1885++0.077
2.36+00.04
3.816=0.123
4.508=0.047
5.876=0.025
5.876=00.25
All
1882-10008
3.4110.018
5.833-20091
9.20810.331
14.078:10.496
14.078:10.496
eye-movements
No CD
1553310.124
2.845=0.055
5.28820.141
7.391110.03
9.512-10.076
9.512-00076
WKM
1698:10.046
0.178
0.849
9.436110.206
9.116=20.577
9.516=0.577
All
1428:00009
2.65:10.068
4.012=2.019
6.192:10.074
5.843=20092
5.843=0.092
eye-state
No CD
1.301+00.0
0.014
0.201
3.397=+.191
0.196=0.163
4.196=0.163
WKM
L04810.022
1.72=00029
0.223
2.6210.08
3.191=0.099
3.191=0.099
Shape Shape"CART,
All
2.077:10024
4.083=0.066
6.865=0.109
13.08911.400
16.725:103366
16.725:0.356
electricity
No CD
2.274=10.035
3.113=0.104
4.46
5.756=00742
8.358=0.733
5.756=0.742
No WKM
2.253-1.035
4.182=0.025
5.97620.013
6,394110.111
8.82410.615
8.624=0.615
All
2.973+-0.024
6.749-0.222
04.32+0.381
13.759+3.123
41.232+-2.009
11.233+2.2000
eye-movementt
No CD
2.533-10.071
0.293=0.277
0.387
9.818-10.516
13.094:1..499
9,818 0.516
WKM
2.344-00071
6.604=0.165
16.5++10009
23.284+0.671
21.673++
21.673+1.004
All
1.998100009 3.895-0.009
6.083-10.86
7.61410.383
14.60310.700
7.614=0.383
eyc-state
No CD
1077=10008
1.994000008
3.07820.041
4.739-10.117
9.667-1.879
4.739-00311
WKM
1.386-10.004
2.247=0.017
2.959:1.05
3.863:10.527
6.192:20.195
6.192220.155
G.2.3 Pairwise Selection Heuristic Ablation
In this section, we examine the benefits of utilizing our pairwise selection heuristic. To do this, do not score pairs of features and instead allow Shape"CART and Shape'CARTa to construct all bivariate shape functions at each node. Like in the previous ablation, work with subset of our datasets and run hyperparameter tuning to choose the model with the highest validation accuracy at each depth. We report the average test accuracy in an training runtime in
We observe that utilizing our heuristic has significant benefits in improving the training runtime for both Shape and Shape?CARd across all datasets and depth. This improvement is very apparent on the eye-movements dataset where Chape"CART and Shape"CART, have an 5x and 6x reduction, respectively, in training runtime at depth 6. When looking at test accuracy. we observe that allowing for all pairs does benefit test accuracy. Interestingly, the accuracy of C benefits from limiting pairs most depths on the eyc-state dataset.
39

---

Table 10: Test aceuracy results (% + std.) for the pairwise heuristic ablation.
Dataset
Model
4
5
6
Overall
Shape"CART CART
Heur.
80.6 0.6
82 i 0.6
83.9
86.5 1.2
87.5
87.5 1
electricity
No Heur.
83.1 + 0.8
85.8 + 0.2
86.3 0.3
8.
88.1+1
Heur.
57.8 1.4
60 =
61.7
01.7332 3.2
62.6
64.8 3.5
64.8
+ 3.5
eye-movements
No Heur.
59.3 1.2
63.6 + 2.5
68.4 + 1.6
75.2
77.8 + 2.7
77.8
+ 2.7
Heur.
71.7 - 1.1
72.8
= 0.7
74.4 2.1
76.1
80.2 1
60.2
eye-state
No Heur.
70.8 4 0.8
73.7 1 0.7
76.1 1
77 1 2,1
81 1111
81 1
Chape"CARTI
Heur.
81.8 0.2
84.5 + 0.3
86.6
0.8
87.3
88.8
88.8
electricity
No Heur.
84.7 0.8
88.9 + 0.2
89 +
89 0.1
48.3+00.1
88.9 0.2
Heur.
2.4
63.6
66.2 1 1.6
1.9
67.5
63.9
43.9
eye- mye-movements
No Heur.
62 + 1.2
69.9 :: 2.5
78.6
79.7
74.9
1 2.7
74.9 2.7
Heur.
+3.1-111
75.9+0.4
79.3 111
81.1 4..1
80.6
+ 1.9
81.1 + 0.6
eye-state
No Heur.
73 0.8
76.3 0.7
7.1
121
76.1
1
78.9 1 2.1
Table 11: Runtime results (s. std.) for the pairwise heuristic ablation.
Dataset
Model
3
4
Overal
Shupe"CART
electricity
Hour.
1.751
3,088 + 0.041
3943 0.022
6.214
7.709
0.056
7.709
0.056
No Heur.
3.989 1 0.037
7.202
0.123
9.121 1 0.142
13.878 = 0.319
16.101 1 0.187
16.101
0.187
Hour.
1.882 4: 0.068
3.41 0.018
38333 + 0.091
9.208
0.331
14,078
: 0.496
14.078
0.496
eye
Heur.
16.716
0.204
75.599
77.686
Heur.
1,428
2.68
4.042 0.019
6.192
0.092
5.843 0.092
eye
No Heur.
8.321
0.127
9.455 1 0.201
11.765 1. 0.312
13.482
0.433
14.123
1 0.336
0.336 14.123
Shape CARTss
clectricity
Heur.
2,077
0.024
4,083 1 0.066
6.865 :: 0.109
13,089
0.404
16.725 1: 0,356
16.725
0.356
Heur
5.084
0.037
11,164 0.123
18.321 0.101
26.528
0.319
42.123 + 0.331
11.164
0.123
Heur.
2,933
0.024
6.749 :: 0.222
14.32 : 0.381
23.759 = 3,123
41.232
1: 2.209
41.232
2.209
moveeentttt
No Hear.
30.616
0.204
56.75 + 0.132
155.597 0.361
201.283
4,085
224.403
4.074
224.403 = 4.074
Hour.
1,998
0.009
3.895 + 0.009
6.083 + 0.386
7,614
0.383
14,603 = 0.709
7.614 + 0,383
eye-state
7.594
0.201
17.761
0.483
21.993
H Visualized Trees
CD CBT
Node
Node 13
Node
Mode
Node
hode
Figure H.5: ShapeCART (Depth 3) trained on Bean
40

---

Noda 4
Rusded 6
hoce 10
Moda 12
Mode 7
hook 0
Rode
Mode: 11 Pefowwn
Redd- Dhlloven
Figure H.6: ShapeCART3 (Depth trained on Eye-Movements
1863
23 6- 28355
Swin1 Tineitlee
2579 22200000000 2000000 2212 220322
Node
Node Relavant (Class 1)
Node 5
Node
Correct (Class 2)
Relavant (Class 1)
Inrelovant (Class
Figure H.7: Chape2CART (Depth 2) Trained on the Eye-Movements dataset
Node
relationship
Node
Node
copital gain
num
Node (Class
Node >50k (Class 1)
Node >50k (Class
Node
<=50k (Class
Figure H.8: ShapeCART (Depth 2) Trained on the Adult dataset
41

---

Node
Node 1
Edible (Class
spore
Node
Node Polsonous (Class
Node Edible (Class 0)
Node Poisonous (Class 1)
Figure H.9: ShapeCART (Depth 3) Trained the Mushroom dataset
Malk Inclowad 15
Figure H.10: ChRT''CART,
CARTs (Depth 2) Trained on EyeeMenesents
I Other Shape Function Candidates
While our approach uses CART to construct the internal tree, our two-stage approach can be applied to any algorithm that can group samples, including other trees. To demonstrate this, present version of the ShapeCART algorithm that constructs the internal tree via DPDT [29 (denoted as
SGT-C (D)). Note that exclude Higgs from this evaluation the DPDT version did finish
most depths. Test accuracy results for this algorithm per dataset can be seen in Table 12 and runtime
results Table [3 Overall, we observe minor accuracy differences between constructing the internal tree with CART and DPDT. However, we observe that the runtime of the DPDT variant is much higher across all depths and datasets.
Regression
To adapt ShapeCART to the regression setting (3n, E IR the superadditive impurity metric from Gini/Entropy (class-based impurity measures), to one adapted for regression setting, such as squared-error or absolute enror. To accommodate this, we must modify the calculation of the empirical distribution a set of samples (Equation (7)) instead calculate the mean value of the target the set of samples.
1,..,..,NN we simply need to modify
42

---

Table 12: Test Accuracy comparison (% = std.) between ShapeCART and ShapeCART with DPDT
internal tree.
Dataset
Model
2
4
6
Overall
SGT-C 82.80 0.20 84.10 + 0.30 84.00 = 0.30 84.30 +
0.40
84.60 = 0.60
84.60 + 0.60
adult
SGT-C (D) 81.30 + 1.9083.90
0.20 + 0.30 84.20 0.60
84.50 = 0.60
84.50 = 0.60
SGT-C
64.50 2.90
1.80
82.00
2.10
82.00 + 2.10
avila
SGT-C (D)
67.30 + 2.80 75.50
2.80
83.80 + 3.30
83.80 + 3.30
SGT-C 88.50
2.30 89.50 + 97.00 96,70
0.00 96.70 = 0.00
97.00 + 0.90
bank
SGT-C (D) 88.00
2.00 89.00 + 2.80 1.80000.00000 2.20992.44
1.60
89.30 4.10
92.40 + 1.60
SGT-C
85.10 + 1.20 0.400
0,40
89.80 = 0.10
89.80 : 0.10
bean
SGT-C (D)
85.20 + 08.70 0.30 90.4000000000000
0.90
90.90 = 0.60
90.90 + 0.60
SGT-C 98.10 0.70 98.40 1 0.60 0.20 0.30
99.60 = 0.10
99.60 + 0.10
bidding
SGT-C (D) 98.10 0.70 98.20 + 012.0099995500
0.20 99.60
0.20
99.60 + 0.20
SGT-C
68.90 + 0.2070 0. 0.50
73.70
0.20
73.70 + 0.20
coviype
SGT-C (D)
68.80 0.7070.50
0.10
73.80
0.20
73.80 + 0.20
SGT-C 75.50
7810 + 0,40 0.00040000000000000400000
84.80
0.60
84.80 + 0.60
electricity
SGT-C (D) 75.60 0.20 79.30 + 0.60 0.:.00000...........
0.60
86.60
0.40
86.60 + 0.40
SGT-C
53.60 1 3.30 4.000005...001110000000000.00003 4.60
58.80
= 7.20
58.80 +:: 7.20
eucalyptus
SGT-C (D)
54.10 1 3.50 4.80
7.20 56.10
1.20
57.20 + 7.20
SGT-C 48.10 2.10 53.00 0.70
0.70
57.40
1.20
57.40 + 1.20
eye-movements
SGT-C (D) 47.30 0.30
53.20 0.40 1.20 1.200 0.10 60.40
0.20
60.40 = 0.20
SGT-C 64.30 0.900
0:2070000
1.30
74.50
1.10
74.50 = 1.10
eye-state
SGT-C (D) 66.60 0.60 70.20 0.50 0.9 0.10
75.30
0.40
75.30 + 0.40
SGT-C
59.70 3.90 1.80 0.40
69.70
3.60
69.70 + 3.60
fault
_
SGT-C (D)
60.20 1 5.70 5.0. 1.70 69.90
2.50
69.90 +:: 2.50
SGT-C
10000
:: 1.30 1.000000000000000000000000111110 2.00
1:000110000000011
89.00 = 0.90
89.00 + 0.90
gas-drift
SGT-C (D)
66.00 + 2.10 1..0 0.90 87.70 + 2.70
87.70 + 2.70
SGT-C 97.80 0.10 0.1097.00
0.20 97.60
0.10
97.80 + 0.10
htru
SGT-C (D) 97.80 00000000000000000000000000000000
0.30
0.10
97.80
0.10
97.80 + 0.10
SGT-C 73.90 1.0 0.
1.70
81.00
0.50
81.00 + 0.50
magic
SGT-C (D) 74.90 1.50 1.00
0.30 80.70
0.70
80.70 + 0.70
SGT-C 03000000000000000000000000000000
0.20 89.70
0.10
89.70 = 0.10
mini-boone
0.30 STTCCODDD.......00.........0000. 0.30 89.60
SGT-C 84.70 0.200 00000000000000000000000000000000
0.30
89.60 + 0.30
SGT-C 98.60 + 01000000000000000000000011001000 0.20
0.10 99.90
0.10
99.90 + 0.10
mushroom
SGT-C (D) 98.90 + 0.30 99. 0.20 0...
0.10 100.00
0.00
100.00 + 0.00
SGT-C 99.00 0.10 98.90 0.10 0.10
0.20
99.20
0.20
99.20 + 0.20
occupancy
SGT-C (D) 99.00 0.10 98.90 0.10 0.20
0.20 99.00
0.20
99.00 = 0.20
SGT-C
95.20
0.30 0.20
1.20
96.20
0.40
96.30 + 1.20
page
SGT-C (D)
95.60 + 0.20 0.20 0.20
96.40
0.20
96.70 + 0.20
pendigits
SGT-C
_
79.30 + 3.90 86.40 0.70
88.30
1.20
88.30 + 1.20
SGT-C (D)
78.10 + 16.70
0.10
90.10 0.30
90.10 +:: 0.30
SGT-C 85.70 + 1.40 85.90 + 2.30 84,30 3 84.30
2.90
84.40 = 2.90
84.30 + 2.90
raisin
SGT-C (D) 85.90 + 1.40 85.60 + 2.20 1.40
2.20
85.70 = 2.20
85.60 + 2.20
SGT-C 93.10 0.40 92.70 + 0.80 0.90
0.90
92.50 = 0.50
92.50 + 0.50
rice
SGT-C (D) 93.10 0.40 92.70 + 0.40 0.90 93.00
0.60
93.00 + 0.60
93.00 + 0.60
SGT-C 93.00
0.40 98.30 + 0.20 0.10
0.00
99.60 0.00
99.60 + 0.00
SGT-C (D) 93.20 0.60 98.20 1 0.30 99.0 0.20 99.50
0.30
99.50 +:: 0.30
SGT-C
83.30 + 3.60 88.90 0.80 89.80 2.50 91.50 = 1.90
91.50 + 1.90
segment
SGT-C (D)
83.30 +: 37.08 1.00 89.60 + 2.50 91.10 0.60
91.10 + 0.60
SGT-C 93.20 0.20 97.70 + 08.5 0.00 98.90 + 0.10 99,20 + 0.20
99,20 + 0.20
SGT-C (D) 93.40 + 0.30 97.60 1 9,40 0.20 0.00 99.20 = 0.30
99.20 + 0.30
SGT-C 97.10 + 0.60 0.60 0.10
0.50
0.20 97.60 + 0.20
97.90
+ 0.50
wilt
SGT-C (D) 97.30 0.40 97.20 0.40
0.50 97.90
0.60 97.90 + 0.60
97.90 + 0.60
43

---

Table 13: Runtime comparison (s. + std.) between SGT-C and SGT-C with DPDT internal tree.
Dataset
Model
3
Overall
adult
SGT-C 0,386 0.009
0.596 + 0.009
0.894
0,016
1.120
0.023
1.44 + 0.033
1.144 = 0.033
SGT-C (D) 12.129 +: 0.425
21.990 +: 0.318
33.145 1: 2.011
42.382 1: 5.490
47.805 = 7.702
47.805 7.702
SGT-C
1,711
0.036
2.343
2.733 0.088
2.733 0.088
avila
SGT-C (D)
79.209 + 1,388
117,866
2.197
147.429 + 3,796
147,429 = 3.796
bank
SGT-C SGT (D)
0.101 1: 0.001
0.138 + 0.004
0.165 = 0.015
0.184 :: 0.008
0.179 + 0.010
0.165 : 0.015
3.073
0.234
0.145
0.202
3,492 0.586
1.334 0.202
SGT-C
1892 + 0.027
2.420
0,071
1.797 0.041
2.328 0.061
2.328 + 0.061
bean
SGT-C (D)
35.693 + 0.138
14.431 = 0.125
22.438 + 0.111
32.694 = 0.614
32.694 = 0.614
hidding
C 0.132
0.001
0.207 1: 0.002
0.208
0.004
0.256
0.004
0.333
0.011
0.333 :: 0.011
SGT-C
0.998
0.116
1.516 + 0.195
2.172 0.249
2.395
0.414
2.660
0.374
2.660 0.374
SGT-C
6.135 + 0,034
11,224 + 0.014
10.618 + 0.064
15.770
0,096
15.770 0.096
covtype
SGT-C (D)
439.259
2.396
227.655
2.720
496. 196 1 4.219
603.667
7.049
603.667 = 7.049
SGI-C 0.396
0.002
0.438 + 0,006
0.880
0.007
1.356
0.011
2.145
0.177
2.145 0.177
electricity
SGT-C (D) 5,202 + 0.177
55.811 + 0.318
89,526 + 0.634
142.068
0.423
197.176-
10.258
197,
10.258
eucalyptus
SGT-C
0.319 + 0,048
0.523
0.063
0.799
0.046
0.995 + 0.030
0.995 + 0.030
SGT (D)
9.549
12.593
1.672
2.201
8.036
2.201
SGT-C 0.464 0.009
0.910 1 0.010
1.540
0,020
2.501
0.018
3.879 +: 0,329
3.879 1: 0.329
eye mye-moeemnnts
SGT-C (D) 39.528 = 0.188
74.187 = 0.853
125.956 +: 2.508
193.079 = 8.774
272.248
22.320
272.248 +: 22,320
SGT-C
0.339
0.003
0.505 + 0.002
0.678
0.005
1.171 0.005
2.087 +: 0.100
2.087 + 0.100
eye-state
SGT-C
7.504
0.050
6.531 + 0.032
10.843
0.071
0.487 17.596
27.433
1.789
27.433 1.789
fault
SGT-
0.641 +: 0.032
1.044
0.045
0.153 15.749
1.945
2,599 +: 0.102
2.599 :: 0.102
SGT-CC (D)
12,921
1,536
22.719 + 2.346
21,990
1,183
21.990 1.183
gas-drift
SGT-C
7.714 + 0,016
14.076 + 0.137
17.205
+ 0.505
20.722
0.314
20.722
0.314
SGT-C (D)
167.784 = 2.558
490.627 1: 2.307
763.837
1: 24.354
552.024
10.415
552.024
+: 10.415
htru
SGTC
0.328
0.002
01.466 + 0.004
0.646
0.020
0.586
0.065
0.561 + 0.020
0.456 + 0.004
SGT-C
9.063 0.021
4,563 + 0.040
6.069
0.382
24.929
4.588
7.213 0.460
7,213 0.460
SGT-C 0,312
0.000
0.477 1 0.003
0.689
0.003
1.071 0.042
1.876 + 0.061
1.876 ::: 0.061
magic
SGT (D) 2.937 0.019
5.079 1: 0.021
50.830 1 0.901
28.735 1: 0.621
1.284 17.214
17.214 1.284
mini-boone
SGT-C
11.023 0.078
12.995
0.129
16.074 + 0.116
20.609 + 0.135
25.352
0.062
25.352 0.062
SGT-C (D) 297. + 1.729
153.416
2.880
236.032
2.896
284.030
2. 2.886
738.861
24.597
738.86
861
24.597
SGTC 0.185 1: 0.003
0.331 1: 0.008
0.399 0.026
0.327 1: 0.004
0.330 1: 0.004
0.327 1: 0.004
mushroom
SGT-C (D) 3.644
0.108
4.357
0.128
0.229
0.045
4.881
0.258
0.258
SGT-C
0.148
0,002
0.186 + 0.003
0.176
0,004
0.197 0.006
0.405 0.024
0.405 :: 0.024
occupancy
SGT-C (D)
1.821 1 0.010
7.952 1 0.093
0.673 7.419
9.229 1 1.337
7.998 + 0.908
7.998 1: 0.908
C SGT-C
0.263 + 0.001
0.437 0.022
0.696 0.075
0.933 + 0.015
0.696 + 0.075
page
6.266 + 0.109
5.173 0.086
13.694 + 0.449
6.238 0.361
5.173 0.086
SGT-C
2.312 0.020
3,443 0.053
4.788 + 0.033
4.788 +:: 0.033
pendigits
SGT (D)
128.053 1: 2.424
203.633 1: 0.778
70.396 0.761
70.396 0.761
SGT-C 0.137
0.002
0.1% + 0.011
0.263
0.0029
0.249
0.034
0.317 0.087
0.249 + 0.034
raisin
SGT-C (D) 1.620 :::00:00 0.017
1.262 + 0.275
2.892 0.513
1.762 0.414
1.832 + 0.348
1.262 = 0.275
SGT-C
0.134 1: 0.002
0.239 1 0.003
0.337
0.029
0.441 1: 0.062
0,541 +: 0.049
0.541 1: 0.049
rice
SGT (D)
0.015
1.987
0.017
0.356
3.663 0.810
3.592 0.679
3.663 0.810
SGT-C
0,238
0.004
0,535 + 0.004
0.704
0,027
0.770 0.014
0.773 0.012
0.773 + 0.012
room
SGT-C (D) 20.538 + 0.232
9.141 1 0.236
19.394 1 0.328
22.729 + 1.789
7.991 + 0.356
7.997 1 0.356
C SGT-C
0.410 + 0.015
0.468
0.016
0.844 0.053
1.009 0.109
1.009 +: 0.109
segment
7.757 + 0.310
20.347 + 0.481
12.982 + 0.307
16.016
0.189
16.016 0.189
skin
SGT-C
0.399 : 0.009
0.520 + 0.010
0.645
0.008
0.776
0.009
0.919 + 0.018
0.919 +: 0.018
SGT-C (D) 9.559 0.388
21.933 1 0.846
29.639 1 0.523
35,859 1 0.545
23.851 2.256
23.851 2.256
wilt
SGT-C
0.127
0.002
0.169
0.001
0.033
0.316 0.059
0.252 + 0.0019
SGT-C (D)
1.772 0.048
1,539 + 0.027
1.1 0.260
2.814 0.336
3.213 + 0.159
3.213 # 0.159
44

---

We provide regression results for ShapeCART, ShapeCARTs, Shape"CART, and Shape"CART;; We benchmark our axis-aligned approaches against CART and our Bivariate approaches against BiCART. Our hyperparameter tuning approach and hyperparameter search in this setting are similar to the classification setting. One key modification is that we only use the MSE criterion (rather than tuning for a splitting criterion like in the classification setting). We report the mean squared error (MSE) of each model on five open source regression datasets Ailerons (OpenML ID: 44137). Black Friday (OpenML ID; 41540), California Housing (OpenML ID: 44025), and Diamonds (OpenML ID: 42225)- broken down by depth and overall best model, chosen by best validation MSE, in Table We scale the target zero mean and one standard deviation for all datasets ensure stability during training. We observe that ShapeCART consistently outperforms CART on all datasets except Ailerons, while ShapeCART, outperforms CART on all datasets. A similar trend can be seen in the Bivariate approaches, where Shape'CART CART outperforms BiCART on all datasets except Ailerons. Interestingly, Shape&CART also outperforms Shape?CARe CART, California Housing and Ailerons, while achieving similar performance on Black-Friday and Diamonds.
14
Table 14: Test MSE Results (MSE + std.) on regression datasets. Approaches are split into axis-
aligned and bivariate by the dotted line. Best approach bolded, second best is italicized.
dataset
Model
2
3
Overall
CART
0.453 0.011
0.368 = 0
0.302 0.001
0.257 0.006
0.235 + 0.004
0.235
0.004
SGT-C
0.452 : 0.012
0.364 : 0.004
0.312 0.003
0.258
0.007
0.238 + 0.005
0.238
0.005
-GTa-C
0.375 +: 0.002
0.321 + 0.004
0.263 +: 0.004
0.23 0.004
0.229 + 0.006
0.229
0.006
ailerons
BICART
0.363 +:
0.28 0.006
0.231 + 0.016
0.008 0.206
0.197 0.008
0.197
0.008
S-GT-C
0.358 +: 0.008
0.296 0,005
0.256 =
0.001
0.231 +: 0.009
0.23/
0.009
S'GT3-C
0.37 0.002
0.311 = 0.005
0.265 + 0.001
0.237 - 0.008
0,236 + 0.003
0.236
0.001
CART
0.728 +: 0.012
0.608 = 0.008
0.567 + 0.018
0.513 0.011
0,502 = 0.009
0.502
0.009
SGT-C
0.548 :: 0.019
0.528 0.002
1:50410000
0.496 + 0.016
0.496 = 0.016
flack-friday
SGTa-C
0.528 +: 0.002
0.507 + 0.088
0.088
0.017
+ 0.022
0.484 0.022
BiCART
0.728 + 0.011
0,606 + 0.012
0.561 0.001
0.508 0.011
0.499 + 0.013
0.499
0.013
S-GT-C
0.546 1: 0.001
0.517 + 0.002
0,491 0.011
0.487 1: 0.013
0.487
0.013
S'GT3-C
0.519 - 0.002
0.502 = 0.003
0.49 = 0.008
0.492
0.485 - 0.021
0.021 0.485
CART
0.015
0.459 + 0.008
0,418 0.006
0.001
0.338 + 0.004
0,338
0.004
SGT-C
0,508
0.43 0,007
0.354 0.002
0.272 0.008
0.272
0. 0.008
-GTa-C
0.426 + 0.007
0.33 0.001
0.271 + 0.002
0.254 + 0.006
0,245 + 0.001
0.245
0.001
califomia
BICART
0,442 0.001
0.345 + 0.003
0.297 0.002
0.267 0.007
0.243 0.009
0.243
0.009
S-GT-C
0.358 +: 0.008
0.272 + 0.001
0.228 0.001
0.224 0.008
0.213 + 0.012
0.213
0.012
SCGT--C
0.415 - 0.002
0.332 =
0,276 0.007
0.243 + 0.008
0.236 + 0.001
0.236
0.00!
CART
0.169 +: 0.001
0.125 + 0.001
0.105 0
0.092 + 0
0,081 0
0.081 +
SGT-C
0.169 0.001
0
0.084 1 0.002
0.062 1 0.001
0.047 : 0.001
0.047 = 0.00!
diumonds
SGT,-C
0.116
0.079 0.001
0.048
0.038
0.003
0.001
BiCART
0.163 0.001
0.113 + 0.002
0.084 + 0.001
0.066 1: 0.001
0.051 + 0.001
0.051 = 0.001
S-GT-C
0.128 +: 0.002
0.064 + 0.001
0.044 4 0.003
0.042 1: 0.001
0.036 0
0.036 +
CC
0.106 + 0.001
0.07 0.001
0.041
0.002
0.034 1 0.002
0.031 + 0.002
0,031 0.002
45

---

K Extra Pseudocode
In this section, we provide pseudocode for the TDIDT induction of a tree (Algorithm and of our coordinate descent procedure (Algorithm
Algorithm 3: ShapeCART (TDIDT Pseudocode)
Input: Features X: Target V, Branching Factor K, Pairwise Candidate Limit P. Coordinate Descent Iterations R, Phirwise Reg, "), Branching Rog. Impurity M(()) Weighted Impurity ((-). Max Depth M.x. Minimum Impurity Improvement Cum. Min.
Samples Leaf Steat, Min. Samples Split Syú
Output: I,
T - (j T 4- O
Initialize Empty Pripcity Queue
E, 4- (()))
(D" ( +- SelectShupefunct X, V. K, Rt A.H)).
addd(( [0: (DD L L." (.), x)))
while 1Q)
CD' L', L (-), Ar" D) Pup(Q)
it L' < Lase then
Add mode la T, assign the label the dominant class continue next node end
if MM
May then
Add node to T. assign the label as the dominant class in D. continue next nade
end
ar DI < Sym then
Add node to asign the label as the dominant class D. continue next node
end
[D] Siny Add node to T. assign the label as the dominant class continue next node end
Add node and corresponding shape function f'( I
foreach D E D do
Mark ) as child of
X3 2
(D) L. (() SelectStapeuunet 23,0, K, 7, 10,20)
Quili (D',L' L. P.. ().M* +1,D"1
end
return T.T
Algorithm 4: Coordinate Descent
Input: [ . TT), [W...., .......... Wz), 76(-). R
Output: a,
for k 4- to K do
W, 1(a, = K) P. Ees Mac (W, mT)
W. Ete
+- 3o Bor to R do foreach in Saandonizeddr-ane I.... do
Ps, P. Ws Nz: Wa. Way Wr
ΣE=: W.H(Ps/W.
Store P. + Pt. W. We =k= 1......,K K
for k + 1 K do
// Iterate through all Branches
W: W: - Wi: Pa - Pr. Werre
LE + L W. W.H(P. (W) W.H(PL/W))
if Lu < L' then
// Commit if weighted impurity improves
Leios
StoreState( Pt, W Pk. Wx)
end
end
(P.. W
Pk. Wx) RetrieveStateO
end
end
46


Reviews:
[{"review_id": "BEuBQK31Tf", "reviewer": "Reviewer_w6vq", "summary": "This paper introduces ​Shape Generalized Trees (SGTs)​, a novel class of interpretable decision trees that replace traditional axis-aligned linear splits with ​learnable shape functions​ applied to individual features in each node. These non-linear functions capture complex feature-target relationships within single nodes, significantly reducing tree depth and size while maintaining model transparency. The authors develop ​ShapeCART, an efficient induction algorithm that constructs univariate SGTs using an interpretable binning-and-assignment approach, alongside extensions for bivariate splits (Shape²CART) and multi-way branching (ShapeCARTₖ). Experiments demonstrate that SGT variants consistently outperform conventional trees (CART, SERDT) and bivariate methods (BiCART) across tabular datasets, achieving ​higher accuracy with more compact architectures. Crucially, SGTs retain explainability through ​visualizable node-level decisions, as illustrated in the eye-movements dataset example where non-linear splits replace multiple linear thresholds. This work bridges the expressivity of generalized additive models with the structural clarity of decision trees.", "questions": "See weaknessses", "limitations": "Nan", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Related Work and Theory**: The paper establishes ​empirical superiority​ of SGTs over CART but omits ​theoretical guarantees​ comparable to recent CART analyses (Klusowski 2020, 2024; Zheng 2023). While ShapeCART's greedy optimization shares CART's structure, its ​non-linear shape functions​ fundamentally alter splitting mechanics. Future work should address: (1) ​Approximation bounds​ for function classes captured by piecewise-constant shape functions; (2) ​Consistency rates​ under the binning-and-assignment paradigm; (3) ​Sparsity guarantees​ given the coordinate descent assignment. Connecting to Klusowski's minimax optimality for additive functions or Zheng's consistency analysis would ground SGTs' advantages in modern learning theory.\n* Klusowski, Jason M., and Peter M. Tian. \"Large scale prediction with decision trees.\" Journal of the American Statistical Association 119.545 (2024): 525-537.\n* Klusowski, Jason. \"Sparse learning with CART.\" Advances in Neural Information Processing Systems 33 (2020): 11612-11622.\n* Zheng,  et al. \"On the consistency rate of decision tree learning algorithms.\" International Conference on Artificial Intelligence and Statistics. PMLR, 2023.\n\n**More Experiments**: \n1) Hyperparameter sensitivity analysis​ lacks visualizations of key parameters' impact (e.g., bin count L, pairwise regularization γ) across tasks—critical given their effect on model compactness; ​\n2) Visualization limitations—while Figure 2 illustrates a univariate SGT node, the paper doesn't showcase ​bivariate/multi-way shape functions​ (promised in Sec. 4) or ​comparative architecture diagrams​ against linear trees (as implied in Fig. 1c); \n3) Baseline contemporaneity gaps—recent advances like NODE-GAM and interpretable oblique trees are absent despite relevance to SGT's \"interpretable non-linearity\" claims. While depth-controlled comparisons are rigorous (Tables 1-2), adding ​parameter sensitivity curves, ​bivariate shape visualizations, and ​modern baseline comparisons​ would better substantiate the novel structure-performance trade-offs central to SGTs' contribution.", "was_revised": true, "final_justification": "Thank you for your thorough response! My primary concerns have been addressed, so I am happy to raise my score.", "rebuttal": "We thank the reviewer for their detailed review and suggestions for improvement. In the following, we will address all the reviewer's suggestions and concerns.\n\n## Theoretical Results\n\nWe agree that incorporating theoretical analysis of SGTs and ShapeCART will enhance the work and ground it in a solid theoretical foundation. We are happy to share that we have obtained new theoretical results on: (1) the expressivity of SGTs compared to binary axis-aligned trees; and (2) extending the empirical risk bound for CART in Klusowski and Tian [1] to the proposed ShapeCART algorithm.\n\n### Expressiveness Guarantees for SGTs\n\nWe formally establish that SGTs are more expressive than binary axis-aligned linear trees by showing that: (1) SGTs are at least as expressive as binary axis-aligned linear trees with a similar number of decision nodes; (2) SGTs can be strictly more expressive than axis-aligned linear trees with a similar number of decision nodes.\n\n**Theorem 1**: Every function that can be represented by a binary axis-aligned linear tree (Def. 1), can be represented by an SGT (Def. 3) with the same number of decision nodes.\n\n_Proof:_ Given a binary axis-aligned linear tree, we construct an SGT with similar structure where the shape function in each decision node takes the form $f_\\Theta(\\mathbf{w}^\\top\\mathbf{x}) = \\mathbf{w}^\\top\\mathbf{x}_n -\\Theta$ where $\\mathbf{w}$ and $\\Theta$ match the values in the corresponding decision node in the axis-aligned linear tree.\n\n**Theorem 2**: $\\forall B\\in\\mathbb{N}$, there exists a function for which a binary axis-aligned tree requires at least $B$ additional decision nodes to represent, compared to an SGT.\n\nTo prove Theorem 2, we construct a family of two-dimensional labeling functions with periodic and rectangular boundaries: \n\n**Definition**Let $\\omega \\in \\mathbb{N}$ be a frequency parameter. We define the Extended-Plus Labeling function $g:[0,1]^2 \\to \\{0,1\\}$ as follows:\n$$g(x_1, x_2) = \n\\begin{cases}\n1 &\\text{if } x_2 \\in [0.25, 0.75]\\;\\;\\text{or} \\;\\; \\cos(2\\pi\\omega x_1)\\leq 0, \\\\ 0 &\\text{otherwise}.\\end{cases}$$\n\nWe show that an SGT can represent any function in this family using a fixed number of nodes, while an axis-aligned linear tree requires a number of nodes that grows linearly with the number of rectangles:\n\n**Lemma 1** The Extended-Plus labelling function can be represented exactly by an SGT with two internal nodes for any value of $\\omega$.\n\n**Lemma 2** A binary axis-aligned linear tree requires $4\\omega +2$ internal nodes to represent the Extended-Plus labelling function.\n\n\n### Empirical Risk Bound of ShapeCART\n\nIn addition, we extend the empirical risk bounds for CART in [1] to ShapeCART. To do so, we introduce a small modification to our coordinate descent initialization by choosing the better of (1) Weighted K-Means assignments or (2) bin assignments based on the left/right split of the root node in the shape function's inner CART. With this change, we can show that the information gain for a given node in ShapeCART is at least as high as the information gain for CART. \n\n**Lemma 3.**  Let $t$ be an individual node in $T$, let $\\mathcal{IG}(t)$ denote the information gain obtained by an axis-aligned (threshold) split on the samples in $t$ from CART, and let $\\mathcal{IG}_f(t)$ denote the information gain obtained by a shape function split from ShapeCART on the samples in $t$. We show that $\\mathcal{IG}_f(t)\\geq\\mathcal{IG}(t)$\n\n_Proof Sketch_: To construct the shape function $f_j$ for each feature $j$, we first fit a univariate CART tree that partitions the data along $x_j$. The root node of this tree yields the best axis-aligned split, with information gain $\\mathcal{IG}(t, j)$. As we select the initial assignment of bins to branches based on the best between: (1) the right/left assignment based on the root node; and (2) a clustering-based assignment from weighted K-means, we are guaranteed to initialize the coord. decent with an assignment that achieves at least $\\mathcal{IG}(t, j)$. Since coord. descent monotonically improves the objective, the final shape function satisfies $\\mathcal{IG}_f(f_j, t)\\geq\\mathcal{IG}(t, j),\\forall j$. Given that ShapeCART utilizes the same line search mechanism to select the feature shape function, we can state $\\mathcal{IG}_f(t) = \\max_{j}\\mathcal{IG}_f(f_j, t)\\geq\\mathcal{IG}(t)=\\max_j \\mathcal{IG}(t,j)$. \n\nGiven Lemma 3, ShapeCART naturally satisfies Lemma 4.1 in [1], and consequently the main result of Theorem 4.2 in [1]. We empirically tested this modification and it has a negligible impact on performance. We will add these results to the revised manuscript.\n\n**Future Work on Theoretical Results:** We agree that our work opens additional interesting directions for future work to further theoretically characterize SGTs and ShapeCART (e.g., developing a GridCART variant for ShapeCART following [9]). Naturally this is beyond the scope of this work, but we will extend our discussion on future work to acknowledge these interesting directions, including the references provided for similar works on CART.\n\n## Hyperparameters Sensitivity Analysis\n\nThanks for your suggestion regarding the HP analysis. Following [2] and [3], we normalize the test accuracy within each model and dataset and train a Random Forest regressor to predict test Z-scores from the model hyperparameters and provide the feature importance for our TDIDT approaches in the table below. We will include this analysis in the final manuscript.\n\n| |ShapeCART|ShapeCART$_3$|Shape$^2$CART|Shape$^2$CART$_3$|\n|---|---|---|---|---|\n|max_depth|0.35|0.24|0.148|0.099|\n|criterion |0.038|0.035|0.035 |0.034 |\n|min_samples_split |0.079|0.082 |0.082|0.085 |\n|min_samples_leaf | 0.182 |0.178|0.193|0.191|\n|min_impurity_decrease |0.094|0.08|0.099|0.074|\n|inner_max_leaf_nodes |0.169 |0.192|0.177|0.199|\n|inner_min_samples_leaf|0.088|0.098|0.098|0.083|\n|branching_penalty|---|0.095|---|0.085|\n|H |---|---|0.069|0.062|\n|pairwise_penalty|---|---|0.099|0.088|\n\nWe observe that `max_depth, inner_max_leaf_nodes` ($L$), and `min_samples_leaf` are among the top-3 most important HPs for all models evaluated. \n\n## Visualization\n\nVisualizations of SGT variants can be found the appendix (SGT$_3$: Figure 2, Appendix G; S$^2$GT: Figure 3, Appendix G). In the revised manuscript, we will provide additional examples and visualized S$^2$GT$_3$ models.\n\n## Contemporary Baselines \n\nThanks for your suggestion to add contemporary baselines. We have extended our evaluation to include three additional baselines. Two non-greedy SOTA models from the recent year: DPDT (KDD 2025) [2] and SPLIT (ICML 2025) [4], as well as HSTree (ICML 2022) [6]. Sparse bivariate oblique trees, as well as their non-greedy variant, are already included in our experiments (BiCART and BiTAO, respectively). Note that higher-order oblique trees ($M>2$) are not naturally visualizable and are considered less interpretable [5]. We similarly excluded SGTs with higher-order shape functions beyond bivariate for the same reason.\n\nConsistent with prior works on trees, we did not include GAMs in our evaluation as they are not naturally comparable to trees. GAMs learn a shape function for each feature ($C$ shape functions per feature in multi-class datasets). Further, the local explanation size in GAMs is identical to its global explanation size (all shape functions). In contrast, in trees the local explanation size is bound by the tree depth and the global explanation size is bound by the number of nodes in the tree (at most $2^\\texttt{depth}$).\n\nNote that based on the other reviews, we have improved the robustness and fairness of our hyperparameter tuning setting. Please see our response to Reviewer Vngp for details on these changes. Additionally, we have added additional datasets at the request of Reviewer 85ma bringing our total dataset count to 26. \n\nAggregated test accuracy results can be seen in the table below. We observe that ShapeCART$_3$ and ShapeCART consistently outperform the other TDIDT approaches, while ShapeTAO and ShapeTAO$_3$ outperform the non-greedy approaches. \n\n|model|2|3|4|5|6|Best|\n|---|---|---|---|---|---|---|\n|CART|83.9%|81.6%|82.3%|84.0%|85.1%|85.2%|\n|SERDT|83.7%|80.9%|81.7%|83.7%|85.1%|85.1%|\n|HSTree|83.9%|81.6%|82.4%|84.0%|85.1%|85.1%|\n|ShapeCART|_84.5%_|_82.5%_|_83.7%_|_84.9%_|_86.2%_|_86.3%_|\n|ShapeCART_3|**85.7%**|**84.6%**|**85.9%**|**87.3%**|**87.8%**|**87.8%**|\n|AxTAO|83.9%|82.1%|82.8%|84.5%|85.5%|85.4%|\n|DPDT|84.9%|83.4%|83.8%|85.2%|86.2%|86.2%|\n|SPLIT|81.7%|76.6%|77.9%|79.5%|80.2%|80.3%|\n|ShapeTAO|_85.0%_|_83.5%_|_84.6%_|_85.9%_|_86.8%_|_86.8%_|\n|ShapeTAO_3|**86.4%**|**85.1%**|**86.2%**|**87.5%**|**88.0%**|**88.0%**|\n\n\\* To accommodate slower non-greedy baselines, we downsample our largest dataset (Higgs) from 11m samples to 1m samples to ensure all experiments are completed in time. The final manuscript will include the results for the full dataset.\n\n## References\n\n[1] Klusowski, J. M., and Tian, P. M. (2024). Large scale prediction with decision trees. J. Am. Stat. Assoc, 119(545), 525-537.\n\n[2] Kohler, H., et. al. Breiman meets Bellman: Non-Greedy Decision Trees with MDPs.  KDD 2025. \n\n[3] Grinsztajn, Léo, et. al. \"Why do tree-based models still outperform deep learning on typical tabular data?.\" NeurIPS 2022. \n\n[4] Babbar, V., et. al. Near-Optimal Decision Trees in a SPLIT Second. ICML 2025\n\n[5] Kairgeldin, R., and Carreira-Perpiñán, M. Á.. Bivariate decision trees: Smaller, interpretable, more accurate. KDD 2024\n\n[6] Agarwal, A. et. al. Hierarchical Shrinkage: Improving the accuracy and interpretability of tree-based models. ICML 2022\n\n[7] Zhang, X. et. al. Axiomatic interpretability for multiclass additive models. KDD 2019\n\n[8] Murdoch, W. J. et. al. (2019). Definitions, methods, and applications in interpretable machine learning. PNAS, 116(44), 22071-22080.\n\n[9] Zheng, et al. \"On the consistency rate of decision tree learning algorithms.\" Int. Conf. on AI & Stats. PMLR, 2023."}, {"review_id": "IxJietC5eU", "reviewer": "Reviewer_Vngp", "summary": "This paper proposes to extend the top-down induction tree learning paradigm from only considering linear axis-aligned splits to both (1) non-linear axis-aligned splits on a single feature by using shape functions, (2) non-linear splits on a pair of two features, and (3) multi-way splits (although in practice limited to three-way splits). In comparison with linear axis-aligned splits, the new splits can compactly capture more patterns, whereas in comparison to oblique splits, these shape functions remain more interpretable.", "questions": "1. Categorical variables are transformed using one-hot encoding. This means the only possible shape function is the standard binary split, right? Are other encodings of categorical variables possible to exploit the value of shape functions?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The proposed method is simple and explained well\n- The paper examines a variety of novel split types (shape functions, shape functions on pairs of features, multi-way splits)\n- The formalism is useful and appropriate\n- The related work contains a reflection on the different types of interpretability and the applicability to trees\n\nWeaknesses:\n- The experiment hyperparameter tuning needs more details. There are several hyperparameters that are tuned, but how? (1) L, Number of bins per shape function, (2) min. samples per bin, (3) gamma, regularization hyperparameter, (4) lambda, penalization of multi-way branching \n\nSmall comments\n- “For bivariate shape functions, we instead train a binary bivariate linear tree using BiCART”. More details is needed. Also, what is a linear tree?\n- Eq. 11 needs a comma between the two constraints for readability\n- In Def. 5, k is not defined (only later). Also k>=2 in the text, but in Eq. 5, k >= 1", "was_revised": true, "final_justification": "The proposed method is simple (in the good sense), useful, and well-explained. Their rebuttal shows that my understanding of their binarization of categorical variables was based on a misunderstanding. My main concern was regards to reproducibility because of lack of details in the hyperparameter tuning of the method. This concern is answered well in the rebuttal and should be expanded on in the updated paper. Therefore, I recommend to accept this paper.", "rebuttal": "We thank the reviewer and are happy to hear that they found our approach well-explained, novel, and useful. We hope to answer any remaining concerns and questions below.\n\n### Categorical Variables:\n\nShapeCART and its variants support superset branching on categorical variables without requiring any ordinal assumptions among the categories, owing to the flexibility of their shape functions. This is achieved by passing all relevant columns of the one-hot encoded categorical variable into the internal tree, as described in Section 5 (lines 275–277).  An example illustrating SGT branching over multiple levels of a categorical feature is provided in Figure 5 of the appendix, which visualizes an SGT trained on the mushroom dataset (a fully categorical dataset). We are happy to clarify any remaining questions regarding this implementation detail. \n\n### Tuning and Evaluation:\n\nThanks for highlighting the need for additional detail regarding hyperparameter tuning. In response, we will expand the description of our tuning protocol in Section 5 and Appendix E.1 based on the description below. In addition, since submission we have improved our hyperparameter tuning procedure to ensure fair comparison across models and datasets by allocating fixed tuning budget per model and per dataset.\n\n* We standardize the evaluation by creating three folds for each dataset using a 70/30 train/test-validation split. The non-training portion is then further divided into validation and test sets using the same 70/30 ratio.\n* For hyperparameter optimization, we perform a Bayesian search with a fixed budget of 50 trials for each model and depth via the Optuna Framework [1] to avoid allocating larger budget to model with more hyperparameters. We believe this adjustment strengthens the methodological rigor of our comparisons. \n* We evaluate each configuration by its average validation accuracy across the folds, and select the hyperparameter combination that results in the best average validation accuracy across all folds per explored depth, as well as the single best overall model. \n* Each model’s training time was limited to 15 minutes for axis-aligned (univariate) approaches and 30 minutes for (bivariate) approaches.\n* All experiments are run on GCP N2 Compute Engines (Intel Cascade Lake) with 8vCPUs and 32 GB Memory \n\n### Hyperparameter Importance\n\nGiven the reviewer's interest in the hyper-parameter tuning, we would also like to share that we have conducted an analysis of hyper-parameter importance for our approaches. Following [2] and [5], we normalize the test accuracy within each model and dataset to zero mean and one std. and train a Random Forest regressor to predict test Z-scores from the model hyperparameters and provide the feature importance for our TDIDT approaches in the table below. We will include this analysis in the final manuscript:\n|  | ShapeCART | ShapeCART$_3$ | Shape$^2$CART | Shape$^2$CART$_3$ |\n|---|---|---|---|---|\n| max_depth | 0.35 | 0.24 | 0.148 | 0.099 |\n| criterion | 0.038 | 0.035 | 0.035 | 0.034 |\n| min_samples_split | 0.079 | 0.082 | 0.082 | 0.085 |\n| min_samples_leaf | 0.182 | 0.178 | 0.193 | 0.191 |\n| min_impurity_decrease | 0.094 | 0.08 | 0.099 | 0.074 |\n| inner_max_leaf_nodes | 0.169 | 0.192 | 0.177 | 0.199 |\n| inner_min_samples_leaf | 0.088 | 0.098 | 0.098 | 0.083 |\n| branching_penalty | --- | 0.095 | --- | 0.085 |\n| H | --- | --- | 0.069 | 0.062 |\n| pairwise_penalty | --- | --- | 0.099 | 0.088 |\n\nWe observe that `max_depth, inner_max_leaf_nodes` (aka $L$), and `min_samples_leaf` are among the top-3 most important HPs for all models evaluated. \n\n### Typos\n\nThank you for catching some of our typos and readability issues. Here are the provided fixes:\n* Definition 5: We modify \"...partition the data into $K \\geq k \\geq 2$ distinct subsets..\" to \"...partition the data into at most $K$ distinct subsets..\"\n* Equation 11: We will add a comma between the constraints. \n* Linear Tree: This should read \"bivariate oblique tree.\" This refers to a type of tree where the splitting function is a linear combination of 2 features. Formal definitions of these tree types are provided in Definitions 2 (Page 3) ($M$=2). BiCART [6] is a newly proposed TDIDT method for training bivariate oblique trees.\n\n### References\n[1] Akiba, T. et. al. Optuna: A next-generation hyperparameter optimization framework. (KDD 2019)\n\n[2] Kohler, H., et. al. Breiman meets Bellman: Non-Greedy Decision Trees with MDPs.  KDD 2025. \n\n[3] Babbar, V., et. al. Near-Optimal Decision Trees in a SPLIT Second. ICML 2025\n\n[4] Agarwal, A. et. al. Hierarchical Shrinkage: Improving the accuracy and interpretability of tree-based models. ICML 2022\n\n[5] Grinsztajn, Léo, et. al. \"Why do tree-based models still outperform deep learning on typical tabular data?.\" NeurIPS 2022. \n\n[6] Kairgeldin, R., and Carreira-Perpiñán, M. Á.. Bivariate decision trees: Smaller, interpretable, more accurate. KDD 2024"}, {"review_id": "x04eUKHiiL", "reviewer": "Reviewer_NeDQ", "summary": "This work introduces SGTs, a novel type of decision trees that supports non-linear branching functions at each internal node. In doing so, these models are able to represent complex decision functions more concisely than traditional linear splitting decision trees. Additionally, algorithms for fitting SGTs (ShapeCART) , a generalization of SGTs that supports multi-way splitting (ShapeCART_k), and a generalization that supports bivariate splitting functions are introduced. These trees are shown to improve accuracy relative to baseline greedy decision tree methods across several datasets.", "questions": "I have no major questions, but one minor copyediting note:\n- Should w_d be unbolded in Equation 6? My understanding is that it would be a scalar.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- This work is well motivated, and identifies a nuanced but clear problem in existing decision trees (inefficiency in representing some kinds of splitting functions).\n- The exposition is very clear, and the figures are helpful. In particular, the visualization technique used for SGTs is very clear.\n- Fitting SGTs is a somewhat difficult multipart optimization problem. This work presents reasonable simplifications to make this a feasible problem to solve; in particular, the approach of first learning a binning function, then learning bin assignments to learn each splitting function is quite elegant.\n- The evaluation is thorough and well described, considering multiple datasets of varying scale.\n\nWeaknesses\n- My primary concerns with this work have to do with the presentation of the experiments. If these can be addressed, I think this paper is a solid accept.\n  - The Z-score presentation of results is quite strange, and hard to interpret. Moreover, I see no reason to think that the aggregated test accuracies are normally distributed or support a CLT — they would be drawn from some kind of mixture distribution. As such, Z-scores are not an applicable tool. Additionally, it does not make sense to aggregate Z-scores over depth bounds. It may be more appropriate to present accuracy numbers from a reasonable subset of datasets in the main paper for comparison. I believe tables 1 and 2 could be merged if accuracy is reported directly, allowing space for 3-4 datasets to be included.\n  - In the presentation of results in the appendix, grouping tables by method makes comparisons quite difficult. I would recommend grouping them by dataset, making head to head comparisons easier.\n  - While greedy methods are the most direct competitors, globally optimal trees could be run for the shallower depth bounds, and should be included as competing methods. It does not necessarily take away from ShapeCART if they happen to find superior performance, since they likely scale less well.", "was_revised": true, "final_justification": "This paper represents a clear, solid improvement on existing decision tree methods. The evaluation includes many benchmark datasets, forming a solid evaluation of the method. My primary concerns have been addressed during the rebuttal, so I am happy to recommend acceptance for this paper.", "rebuttal": "We thank the reviewer and are happy to hear that they found our approach well-motivated and elegant. We are happy to hear that the reviewer thinks our paper is a solid accept if the presentation of the experiments can be improved. In the following, we will address all the reviewer's suggestions and concerns.\n\n### Z-Scores\nThanks for pointing out the limitations associated with presenting Z-scores. Our intention was to enable a fairer comparison across methods by standardizing performance, following the approach in [1]. However, following the reviewer's suggestion, we have revised our presentation to report raw accuracy values, averaged per depth and across datasets, to ensure clarity and transparency in our results. Revised tables can be found in the \"Merging Tables and Organization\" section below. \n\n### Extra Baselines \n\nWe agree that our work could benefit from comparisons to optimal methods. However, due to their lack of scalability on both tree depth and dataset size, we instead opted for two recent SOTA **near-optimal models**: DPDT (KDD 2025) [2], a non-greedy induction method based on MDPs that was found to be within 0.35% of optimal trees on average [2]., SPLIT (ICML 2025) [3], a recent method for training near-optimal trees efficiently via lookahead methods that has been shown to effectively approximate the optimal tree in many cases (see Tables 7-9 in [3]). \n\nIn addition to incorporating additional baselines, we introduce TAO-refined variants of our multi-way branching models (ShapeCART$_3$ and Shape$^2$CART$_3$ ) by extending the original TAO algorithm from the binary to the ternary setting. This extension is achieved by modifying the node-level optimization problem to handle multi-way splits, specifically by upsampling samples that can be routed to multiple branches and appropriately adjusting their weights during the fitting of the internal tree. Additional implementation details will be provided in the appendix of the revised manuscript and are available upon request.\n\nResults for the newly considered approaches can be found in the section below.\n\n### Merging Tables and Organization\nThanks for your suggestion to merge tables and increase the available space for content. We would like to clarify that the values reported in Tables 1 and 2 are not directly comparable, as the evaluation of bivariate approaches excludes three datasets (Covertype, Higgs, and Eucalyptus) due to out-of-memory issues with the baselines. Specifically, both BiCART and BiTAO exceed the 32GB memory threshold when training on these datasets, primarily due to their feature augmentation mechanism. That said, we agree that the current table layout could be improved to conserve space and enhance readability. While we are unable to upload a revised version of the paper as part of the rebuttal, we will reorganize the tables in the revised manuscript by displaying them side-by-side and minimizing unnecessary white space in the final version. For reference, we include below the aggregated test accuracy results across datasets and depths.\n\nNote that based on the other reviews, we have improved the robustness and fairness of our hyperparameter tuning setting. Please see our response to Reviewer Vngp for details on these changes. Additionally, we have added additional datasets at the request of Reviewer 85ma, bringing our total dataset count to 26. \n\nAggregated results for all evaluated approaches can be seen in the table below. We observe that ShapeCART$_3$ and ShapeCART consistently outperform the other TDIDT approaches, while ShapeTAO and ShapeTAO$_3$ outperform the non-greedy approaches.\n\n|model|2|3|4|5|6|Best|\n|---|---|---|---|---|---|---|\n|CART|83.9%|81.6%|82.3%|84.0%|85.1%|85.2%|\n|SERDT|83.7%|80.9%|81.7%|83.7%|85.1%|85.1%|\n|HSTree|83.9%|81.6%|82.4%|84.0%|85.1%|85.1%|\n|ShapeCART|_84.5%_|_82.5%_|_83.7%_|_84.9%_|_86.2%_|_86.3%_|\n|ShapeCART_3|**85.7%**|**84.6%**|**85.9%**|**87.3%**|**87.8%**|**87.8%**|\n|AxTAO|83.9%|82.1%|82.8%|84.5%|85.5%|85.4%|\n|DPDT|84.9%|83.4%|83.8%|85.2%|86.2%|86.2%|\n|SPLIT|81.7%|76.6%|77.9%|79.5%|80.2%|80.3%|\n|ShapeTAO|_85.0%_|_83.5%_|_84.6%_|_85.9%_|_86.8%_|_86.8%_|\n|ShapeTAO_3|**86.4%**|**85.1%**|**86.2%**|**87.5%**|**88.0%**|**88.0%**|\n\n\\* To accommodate slower non-greedy baselines, we downsample our largest dataset (Higgs) from 11m samples to 1m samples to ensure all experiments are completed in time. The final manuscript will include the results for the full dataset.\n\nWe appreciate the suggestion to present dataset-level results for a small subset of datasets across depths in the main paper and agree that this will help illustrate that SGTs can achieve high predictive performance with smaller tree sizes. Below we provide test accuracy values on three datasets `eye-movements, electricity, & eye-state` across all depths:\n\n| Dataset | Model | 2 | 3 | 4 | 5 | 6 | Best |\n|---|---|---|---|---|---|---|---|\n| eye-movements | CART | 45.9% | 46.3% | 51.3% | 52.4% | 53.2% | 53.2% |\n|  | SERDT | 43.8% | 49.2% | 49.1% | 52.7% | 53.5% | 53.5% |\n|  | HSTree | 45.9% | 46.3% | 51.4% | 52.4% | 53.1% | 53.1% |\n|  | ShapeCART | **48.1%** | _53.0%_ | _54.9%_ | _53.9%_ | _57.4%_ | _57.4%_ |\n|  | ShapeCART_3 | _47.1%_ | **54.4%** | **56.0%** | **57.5%** | **60.9%** | **60.9%** |\n|  | AxTAO | 45.9% | 48.6% | 51.4% | 52.3% | 52.6% | 52.6% |\n|  | DPDT | _48.7%_ | 51.2% | 52.4% | 54.4% | 56.7% | 56.7% |\n|  | SPLIT | 44.0% | 48.4% | 50.2% | 51.9% | 52.0% | 51.9% |\n|  | ShapeTAO | 48.6% | _53.1%_ | _54.8%_ | _56.8%_ | _58.0%_ | _58.0%_ |\n|  | ShapeTAO_3 | **51.4%** | **54.8%** | **55.4%** | **56.9%** | **60.3%** | **60.3%** |\n| electricity | CART | _75.5%_ | 75.8% | 76.0% | 77.6% | 78.0% | 78.0% |\n|  | SERDT | _75.5%_ | 75.8% | 76.2% | 78.6% | 79.4% | 79.4% |\n|  | HSTree | _75.5%_ | 75.8% | 76.0% | 77.7% | 78.1% | 78.1% |\n|  | ShapeCART | _75.5%_ | _78.5%_ | _81.3%_ | _84.4%_ | _84.8%_ | _84.8%_ |\n|  | ShapeCART_3 | **76.5%** | **80.6%** | **84.0%** | **86.0%** | **87.8%** | **87.8%** |\n|  | AxTAO | 75.5% | 75.9% | 75.9% | 78.5% | 79.9% | 79.9% |\n|  | DPDT | _76.1%_ | 78.3% | 79.1% | 80.0% | 80.8% | 80.8% |\n|  | SPLIT | 75.4% | 76.2% | 77.7% | 78.4% | 78.7% | 78.7% |\n|  | ShapeTAO | 75.5% | _79.5%_ | _83.0%_ | _86.0%_ | _87.6%_ | _87.6%_ |\n|  | ShapeTAO_3 | **79.3%** | **82.6%** | **84.9%** | **86.7%** | **88.2%** | **88.2%** |\n| eye-state | CART | 62.7% | 65.8% | 69.1% | 70.9% | 73.2% | 73.2% |\n|  | SERDT | 62.7% | 66.4% | 69.4% | 72.3% | 75.3% | 75.3% |\n|  | HSTree | 62.7% | 65.8% | 69.1% | 70.9% | 73.2% | 73.2% |\n|  | ShapeCART | _64.3%_ | _69.0%_ | _70.5%_ | _72.4%_ | _74.5%_ | _74.5%_ |\n|  | ShapeCART_3 | **66.1%** | **70.8%** | **73.3%** | **76.8%** | **77.5%** | **77.5%** |\n|  | AxTAO | 62.1% | 67.0% | 69.7% | 72.8% | 74.3% | 74.3% |\n|  | DPDT | _65.9%_ | 69.1% | 71.4% | 73.4% | 75.6% | 75.6% |\n|  | SPLIT | 58.5% | 66.4% | 69.0% | 71.8% | 73.6% | 73.6% |\n|  | ShapeTAO | 65.6% | _70.5%_ | _72.7%_ | _73.6%_ | _75.8%_ | _75.8%_ |\n|  | ShapeTAO_3 | **67.1%** | **71.9%** | **75.8%** | **75.8%** | **76.3%** | **76.3%** |\n\nWe observe that in many cases, ShapeCART can achieve test accuracy with a shallow tree exceeding that of other TDIDT approaches on the maximum depth. While not as significant, we observe that ShapeTAO can also achieve higher test accuracy than deeper non-greedy approaches. \n\nThank you for the advice regarding the tables in the Appendix. Due to character limitations, we are unable to provide tables for all datasets in this rebuttal, but in the revised manuscript, we will re-organize the results in the appendix similar to the per-dataset comparison table above. \n\n**Equation 6:** Thank you for pointing out the typo in Equation 6. This is a scalar and should be unbolded. \n\n### References\n\n[1] Feuer, B. et. al. Tunetables: Context optimization for scalable prior-data fitted networks.  NeurIPS 2024\n\n[2] Kohler, H., et. al. Breiman meets Bellman: Non-Greedy Decision Trees with MDPs.  KDD 2025. \n\n[3] Babbar, V., et. al. Near-Optimal Decision Trees in a SPLIT Second. ICML 2025\n\n[4] Agarwal, A. et. al. Hierarchical Shrinkage: Improving the accuracy and interpretability of tree-based models. ICML 2022"}, {"review_id": "6LId3bci1t", "reviewer": "Reviewer_85Ma", "summary": "The authors introduce the Shape Generalized Tree, which learns a GAM with 1 or 2 features at each tree node (using a decision tree) rather than an axis-aligned split or a linear split. They learn these GAM splits in a recursive CART-style agorithm, and additionally allow for multi-way splits rather than binary splits. Experiments on 12 tabular datasets show that the resulting models can outperform traditional baselines for small model sizes while preserving a form of interpretability that allows for manual inspection of decision paths.", "questions": "It might be interesting to see more explorations of univariate shape function beyond using CART.", "limitations": "Besides the weaknesses listed above, the paper should include a more thorough discussion of the interpretability limitations introduced by the SGT. The nonlinearity introduced by the tree structure may make the SGT more difficult to interpret than a GAM, even one that has many more features (esp. if it makes the individual component functions for different features less understandable). Moreover, interpreting individual shape functions adds complexity beyond a typical binary axis-aligned split.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "- S1: The authors tackle a well-motivated problem\n- S2: The authors propose an intuitive method and explore its natural variants\n- S3: The paper's writing is clear and the authors choices' for variations of the method seem reasonable\n- W1: The authors are missing comparisons to relevant baselines for compact interpretable models such as GAMs (e.g. the [explainable boosting boosting machine](https://github.com/interpretml/interpret)), optimal trees (e.g. [GOSDT](https://arxiv.org/abs/2006.08690)), regularized trees (e.g. [HSTree](https://arxiv.org/abs/2202.00858)), and tree sums with limited feature interactions per tree.\n- W2: The experimental results should be more thorough. The 12 selected datasets are somewhat arbitrary and could include a much wider set of a more standardized benchmark (e.g. [PMLB](https://github.com/EpistasisLab/pmlb)). For the depths that the authors evalute, all methods seem to underfit the data (atleast the performance continues to increase through depth 6, the highest depth evaluated). This suggests that in a realistic setting, users may require larger depths, especially after cross-validation. Perhaps the authors may want to evaluate on commonly used high-stakes datasets which are known to admit interpretable classification models (e.g. the classification datasets in [this work](https://arxiv.org/pdf/2201.11931))\n- W3: It is also unclear whether models of different depths are comparable when shape functions include more parameters and are more difficult to interpret.", "was_revised": true, "final_justification": "I appreciate the authors' response and have increased my score to 4. I maintain my concerns about comparisons to GAMs -- in realistic scenarios they seem to be much easier to globally interpret and often easier to locally interpret than the method proposed by authors, and I urge the authors to engage more seriously with this in the manuscript.", "rebuttal": "We thank the reviewer and are happy to hear that they found our approach well-motivated and reasonable. In the following, we will address all the reviewer's suggestions and concerns. \n\n### Clarification on GAMs\nWe want to clarify that we are not fitting an GAM in each node. Rather, we are learning a piece-wise constant function with a bounded number of pieces ($L$) that best splits the data into two (or $K$) subsets. This is an important distinction as SGTs overcome many interpretability issues GAMs possess. Specifically,  In GAMs, a shape function is learned for each feature ($C$ shape function per feature in multi-class datasets with $C$ classes). The local explanation size in GAMs is identical to its global explanation size (all shape functions). In contrast, in decision trees the local explanation size is bound by the tree depth and the global explanation size is bound by the number of nodes in the tree (at most $2^\\texttt{depth}$). Due to these reasons, GAMs are not comparable to trees and we do not include them in our evaluation. \n\n### Baselines (W1)\n\nWe agree that our work could benefit from comparisons to optimal methods. However, due to their lack of scalability on both tree depth and dataset size, we instead opted for two recent SOTA **near-optimal models**: DPDT (KDD 2025) [1], a non-greedy induction method based on MDPs that was found to be within 0.35% of optimal trees on average [1], SPLIT (ICML 2025) [2], a recent method for training near-optimal trees efficiently via lookahead methods that has been shown to effectively approximate the optimal tree in many cases (see Tables 7-9 in [2]). Additionally, per the reviewer recommendation, we include HSTrees (ICML 2022) [3]. An updated table of results is provided in the section below.  Note that we do not provide results for FIGS [6] as like GAMs, their local explanation sizes are not comparable. \n\n### Experimental Evaluation (W2)\n\n**Benchmark Datasets:** We would like to emphasize that all datasets used in our study are well-established and widely adopted in the literature. They were chosen to ensure diversity in terms of sample size, number of classes, and the inclusion of both numerical and categorical features. However, to further improve the robustness of our empirical evaluations, we have expanded our evaluation to include the Quant-BnB benchmark [5] also used in DPDT [1], bringing the total number of datasets in our empirical analysis to 26. We note that our evaluation spans more datasets than any of the baselines considered. Average results across all datasets are provided below.\n\n**Concerns on Max Depth:** We appreciate the reviewer’s concern regarding the choice of maximum tree depth. Our decision to cap depth at 6 is driven by interpretability considerations and aligns with prior work on compact decision trees ([2,4] explore depths up to 6; [1] focuses primarily on depths 3–5). Since our goal is to improve predictive performance while preserving model interpretability, we believe that focusing on trees of reasonable depths is appropriate and effectively showcases the strengths of our approach.\n\nNote that based on the other reviews, we have improved the robustness and fairness of our hyperparameter tuning setting. Please see our response to Reviewer Vngp for details on these changes.  \n\n|model|2|3|4|5|6|Best|\n|---|---|---|---|---|---|---|\n|CART|83.9%|81.6%|82.3%|84.0%|85.1%|85.2%|\n|SERDT|83.7%|80.9%|81.7%|83.7%|85.1%|85.1%|\n|HSTree|83.9%|81.6%|82.4%|84.0%|85.1%|85.1%|\n|ShapeCART|_84.5%_|_82.5%_|_83.7%_|_84.9%_|_86.2%_|_86.3%_|\n|ShapeCART_3|**85.7%**|**84.6%**|**85.9%**|**87.3%**|**87.8%**|**87.8%**|\n|AxTAO|83.9%|82.1%|82.8%|84.5%|85.5%|85.4%|\n|DPDT|84.9%|83.4%|83.8%|85.2%|86.2%|86.2%|\n|SPLIT|81.7%|76.6%|77.9%|79.5%|80.2%|80.3%|\n|ShapeTAO|_85.0%_|_83.5%_|_84.6%_|_85.9%_|_86.8%_|_86.8%_|\n|ShapeTAO_3|**86.4%**|**85.1%**|**86.2%**|**87.5%**|**88.0%**|**88.0%**|\n\n\\* To accommodate slower non-greedy baselines, we downsample our largest dataset (Higgs) from 11m samples to 1m samples to ensure all experiments are completed in time. The final manuscript will include the results for the full dataset.\n\nWe observe that ShapeCART$_3$ and ShapeCART consistently outperform the other TDIDT approaches, while ShapeTAO and ShapeTAO$_3$ outperform the non-greedy approaches.  In addition to overall average, we provide results on only the Quant-BnB benchmark below and observe a similar trend. \n\n| model | 2 | 3 | 4 | 5 | 6 | Best |\n|---|---|---|---|---|---|---|\n| CART | 89.1% | 88.1% | 87.8% | 89.1% | 90.0% | 90.0% |\n| SERDT | 89.1% | 87.0% | 87.3% | 88.9% | 90.1% | 90.1% |\n| HSTree | 89.1% | 88.1% | 87.8% | 89.2% | 90.1% | 90.0% |\n| ShapeCART | _89.4%_ | _88.4%_ | _88.8%_ | _89.6%_ | _90.7%_ | _90.7%_ |\n| ShapeCART_3 | **91.1%** | **90.3%** | **90.9%** | **92.1%** | **92.2%** | **92.2%** |\n| AxTAO | 89.1% | 88.5% | 88.3% | 89.7% | 90.3% | 90.3% |\n| DPDT | _89.9%_ | _89.9%_ | 89.2% | 90.0% | 90.7% | 90.7% |\n| SPLIT | 87.1% | 82.6% | 84.0% | 85.5% | 86.2% | 86.3% |\n| ShapeTAO | _89.9%_ | 89.3% | _89.6%_ | _90.5%_ | _91.2%_ | _91.2%_ |\n| ShapeTAO_3 | **91.4%** | **90.7%** | **91.4%** | **92.1%** | **92.1%** | **92.2%** |\n\n### SGT Interpretability (W3)\n\nIn contrast to GAMs, the local interpretability of an SGT is inherently bounded: a user need only examine at most one piecewise-constant shape function per decision node along a path, resulting in a local explanation length no greater than the tree’s maximum depth. While we acknowledge that interpreting the shape functions at each node of an SGT may be more complex than interpreting axis-aligned linear splits, our approach nonetheless preserves key interpretability properties such as simulability and modularity (Section 2.2). For example, take the tree trained on the `eye-movements` dataset (Figure 2). Given a sample with `regressDur` = 100, `lastSaccLen` = 0, and `nWordsInTitle` =4, we can read the tree and extract a local explanation of \"The model predicted (Relevant, Class 1) because `regressDur` < 220 & `lastSaccLen` < 5 & `nWordsInTitle` $\\in [4,5]$.\" Moreover, the enhanced expressiveness of SGTs can allow for smaller trees while retaining performance, thereby improving the sparsity of local explanations, satisfying the desiderata proposed by [7]. \n\n### Other Shape Function Representations (Q)\nWe are happy to hear that the reviewer is interested to see more potential shape function representations. While we use CART to construct the internal tree, our two-stage approach can be applied to any algorithm that can group samples, including other trees. To demonstrate this, we present a version of the ShapeCART algorithm that constructs the internal tree via DPDT [1] (ShapeCART-DPDT). Results for this algorithm on three datasets can be seen below: \n\n| dataset |  | 2 | 3 | 4 | 5 | 6 | Overall |\n|---|---|---|---|---|---|---|---|\n| electricity | ShapeCART-DPDT | **75.6%** | **78.9%** | **82.5%** | 84.2% | **86.8%** | **86.8%** |\n|  | ShapeCART | 75.5% | 78.5% | 81.3% | **84.4%** | 84.8% | 84.8% |\n| eye-movements | ShapeCART-DPDT | 47.3% | 51.9% | 54.8% | **54.5%** | **59.2%** | **59.2%** |\n|  | ShapeCART | **48.1%** | **53.0%** | **54.9%** | 53.9% | 57.4% | 57.4% |\n| eye-state | ShapeCART-DPDT | **66.6%** | **70.1%** | **72.5%** | **73.9%** | **75.3%** | **75.3%** |\n|  | ShapeCART | 64.3% | 69.0% | 70.5% | 72.4% | 74.5% | 74.5% |\n\nWe observe that the DPDT internal tree variant outperforms the CART variant across most depths. This is an interesting result and we will perform a full evaluation of this method to add to the revised manuscript. \n\n### References\n\n[1] Kohler, H. et al. Breiman meets Bellman: Non-Greedy Decision Trees with MDPs.  KDD 2025. \n\n[2] Babbar, V. et al. Near-Optimal Decision Trees in a SPLIT Second. ICML 2025\n\n[3] Agarwal, A. et al. Hierarchical Shrinkage: Improving the accuracy and interpretability of tree-based models. ICML 2022\n\n[4] Souza, V. et al. Decision trees with short explainable rules. NeurIPS 2022.\n\n[5] Mazumder, R. et al. Quant-BnB: A scalable branch-and-bound method for optimal decision trees with continuous features ICML 2022. \n\n[6] Tan, Y. et al.(2025). Fast interpretable greedy-tree sums. PNAS, 122_(7), e2310151122.\n\n[7] Murdoch, W. J. et. al. (2019). Definitions, methods, and applications in interpretable machine learning. PNAS, 116(44), 22071-22080."}]

Author rebuttal:

Meta-review: The paper introduces Shape Generalized Trees, which are regression trees that partition the input space along non-linear decision boundaries, and a CART-like algorithm for training such trees.
Although learning optimal non-linear decision rules is quite difficult, the authors propose a clever simplification, first learning a binning function and then learning bin assignments. 
Using 12 benchmark datasets, the authors demonstrate that the proposed algorithms outperform methods that estimate axis-aligned tres in terms of out-of-sample accuracy.

Initial feedback from reviewers was very positive, with most reviewers requesting relatively minor clarifications and for the authors to perform additional experiments and include more competing methods.
Of course, a revised version of the manuscript **must** include results from the 26 benchmark datasets and all competing methods.

In response to w6vq's remarks about the lack of theoretical results, the authors sketched how several of Klusowski's results about axis-aligned trees can be extended to the more general setting considered here.
These are promising preliminary results and I encourage the authors to develop them further and revise the manuscript to include the precise statements and full proofs.
In their reply to w6vq, the authors promise to include a short discussion about these promising directions and defer theoretical analysis to future work.
Importantly, the authors should **not** include the unchecked theoretical results in the camera-ready manuscript; I am recommending only that that the paper reporting the methodological development and empirical results be accepted.
Decision: Accept (poster)