Title: Interpretable Decision Tree Search as a Markov Decision Process
Venue: NeurIPS 2024
Accepted: no
Keywords: mdp

cart

rl

classification
Abstract: Finding an optimal decision tree for a supervised learning task is a challenging combinatorial problem to solve at scale. It was recently proposed to frame this problem as a Markov Decision Problem (MDP) and use deep reinforcement learning to tackle scaling. Unfortunately, these methods are not competitive with the current branch-and-bound state of the art. Instead, we propose to scale the resolution of such MDPs using an information-theoretic tests generating function that heuristically, and dynamically for every state, limits the set of admissible test actions to a few good candidates. As a solver, we show empirically that our algorithm is at the very least competitive with branch-and-bound alternatives. As a machine learning tool, a key advantage of our approach is to solve for multiple complexity-performance trade-offs at virtually no additional cost. With such a set of solutions, a user can then select the tree that generalizes best and which has the interpretability level that best suits their needs, which no current branch-and-bound method allows.

Abstract

Finding an optimal decision tree for a supervised learning task is a challenging combinatorial problem to solve at scale. It was recently proposed to frame this problem as a Markov Decision Problem (MDP) and use deep reinforcement learning to tackle scaling. Unfortunately, these methods are not competitive with the current branch-and-bound state of the art. Instead, we propose to scale the resolution of such MDPs using an information-theoretic tests generating function that heuristically, and dynamically for every state, limits the set of admissible test actions to a few good candidates. As a solver, we show empirically that our algorithm is at the very least competitive with branch-and-bound alternatives. As a machine learning tool, a key advantage of our approach is to solve for multiple complexity-performance trade-offs at virtually no additional cost. With such a set of solutions, a user can then select the tree that generalizes best and which has the interpretability level that best suits their needs, which no current branch-and-bound method allows.

1 Introduction

Decision trees (DTs) remain the dominant machine learning model in applications where interpretabil16 ity is essential [Costa and Pedreira, 2023]. Thanks to recent advances in hardware, a new class of 17 decision tree learning algorithms returning optimal trees has emerged [Bertsimas and Dunn, 2017, 18 Demirovic et al., 2022, Mazumder et al., 2022]. These algorithms are based on a branch-and-bound 19 solver that minimizes a regularized empirical loss, where the number of nodes is used as a regularizer. 20 These optimization problems have long been known to be NP-Hard [Hyafil and Rivest, 1976] and 21 despite hardware improvements, solvers of such problems do not scale well beyond trees of depth 3 22 when attributes take continuous values [Mazumder et al., 2022]. On the other hand, greedy approaches 23 such as CART [Breiman et al., 1984] are still considered state-of-the-art decision tree algorithms 24 because they scale and offer more advanced mechanisms to control the complexity of the tree. By 25 framing decision tree learning as a sequential decision problem, and by carefully controlling the 26 size of the search space, we achieve in this paper a best of both worlds, solving the combinatorial 27 optimization problem with accuracies close to optimal ones, while improving scaling and offering a 28 better control of the complexity-performance trade-off than any existing optimal algorithm. 29

30

To do so, we formulate the problem of decision tree learning as a Markov Decision Problem (MDP,

31

32

33

[L. Puterman, 1994]) for which the optimal policy builds a decision tree. Actions in such an MDP

include tests comparing an attribute to a threshold (a.k.a. splits). This action space could include all

possible splits or a heuristically chosen subset, yielding a continuum between optimal algorithms and

heuristic approaches. Furthermore, the reward function of the MDP encodes a trade-off between the 34

complexity and the performance of the learned tree. In our work, complexity takes the meaning of 35

36

simulatability [Lipton, 2018], i.e. the average number of splits the tree will perform on the train dataset.

The MDP reward is parameterized by α , trading-off between train accuracy and regularization. One 37

of the main benefits of our formulation is that the biggest share of the computational cost is due to 38

39

the construction of the MDP transition function which is completely independent of

40

find optimal policies for a large choice of values of

α

α

at virtually no additional cost.

Branch-and-Bound (BnB) algorithms similarly optimize a complexity performance trade-off 41 [Demirovic et al., 2022, Mazumder et al., 2022] but require the user to provide the maximum 42 number of test nodes as an input to their algorithm. Providing such a value a priori is difficult 43 since a smaller tree (e.g. with 3 test nodes) might be only marginally worse on a given dataset than 44 a larger tree (e.g. with 15 test nodes) with respect to the training accuracy but might generalize 45 better or be deemed more interpretable a posteriori by the user. As such, it is critical to consider 46 the multi-objective nature of the optimization problem and seek algorithms returning a set of trees 47 that are located on the Pareto front of the complexity-performance trade-off. To the best of our 48 knowledge, this has been so far neglected by BnB approaches. None of the BnB implementations 49 return a set of trees for different regularizer weights unlike greedy algorithms like CART or C4.5 50 that can return trees with different complexity-performance trade-offs using minimal complexity 51 post-pruning [Breiman et al., 1984], making it a more useful machine learning tool in practice. 52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

2 Related Work

2.1 Optimal Decision Trees.

Decision tree learning has been formulated as an optimization problem in which the goal is to construct a tree that correctly fits the data while using a minimal number of splits. In [Bertsimas and Dunn, 2017, Aghaei et al., 2020, Verwer and Zhang, 2019], decision tree learning is formulated as a Mixed Integer Program (MIP). Instead of using a generic MIP solver, [Demirovic et al., 2022, Mazumder et al., 2022] design specialized solvers based on the Branch-and-Bound (BnB) principle. Quant-BnB [Mazumder et al., 2022] is currently the latest work in this line of research for datasets with continuous attributes and is considered state-of-the-art. However, direct optimization is not a convenient approach since finding the optimal tree is known to be NP-Hard [Hyafil and Rivest, 1976]. Despite hardware improvements, Quant-BnB does not scale beyond trees depth of 3. To reduce the search space, optimal decision tree algorithms on binary datasets, such as MurTree, Blossom and Pystreed [Demirovic et al., 2022, Demirovi´ c et al., 2023, van der Linden et al., 2023], employ heuristics to binarize a dataset with continuous attributes during a pre-processing step following for example the Minimum Description Length Principle [Rissanen, 1978]. The tests generating function of our MDP formulation is similar in principle except that it is state-dependent, which, as demonstrated experimentally, greatly improves the performance of our solver.

2.2 Greedy approaches.

Greedy approaches like CART iteratively partition the training dataset by taking the most informative splits in the sense of the Gini index or the entropy gain. CART is only one-step optimal but can scale to very deep trees. This might lead to overfitting and algorithms such as Minimal Complexity Post-Pruning (see Section 3.3 from [Breiman et al., 1984]) iteratively prune the deep tree, returning a set of smaller trees with decreasing complexity and potentially improved generalization. The trees returned by our algorithms provably dominate-in the multi-objective optimization sense-all the above smaller trees in terms of train accuracy vs. average number of tests performed, and we experimentally show that they often generalize better than the trees returned by CART.

2.3 Markov Decision Problem formulations.

In [Topin et al., 2021], a base MDP is extended to an Iterative Bounding MDP (IBMDP) allowing 80 the use of any Deep Reinforcement Learning (DRL) algorithm to learn DT policies solving the 81 base MDP. While more general and scalable, this method is not state-of-the-art for learning DTs for 82 supervised learning tasks. Prior to IBMDPs, [Garlapati et al., 2015] formulated the learning of DTs 83 for classification tasks with ordinal attributes as an MDP. To be able to handle continuous features, 84 [Nunes et al., 2020] used Monte-Carlo tree search [Kocsis and Szepesvári, 2006] in combination 85 with a tests generating function that limits the branching factor of the tree. Our MDP formulation is 86 different as it considers a regularized objective while [Nunes et al., 2020] optimize accuracy on a 87 validation set. Our tests generating function is also different and dramatically improves scaling as 88

, allowing us to

shown in the comparison of Sec. 5.1.1, making our algorithm competitive with BnB solvers, while 89 [Nunes et al., 2020] only compared their algorithm against greedy approaches. A comparison of our 90 method with other MDP approaches is presented in the supplementary material. 91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

2.4 Interpretability of Decision Trees.

The interpretability of a decision tree is usually associated with its complexity, e.g. its depth or its total number of nodes. For trees with 3 to 12 leaves, [Piltaver et al., 2016] observed a strong negative correlation between the number of leaves in a tree and a 'comprehensibility' score given by users. Most of the literature considers the total number of test nodes as its complexity measure, but other definitions of complexity exist. [Lipton, 2018] coined the term simulatability , which is related to the average number of tests performed before taking a decision. This quantity naturally arises in our MDP formulation. We show in a qualitative study that both criteria are often correlated but on some datasets, DPDT returns an unbalanced tree with more test nodes that are only traversed by a few samples.

3 Decision Trees for Supervised Learning

Let us consider a training dataset D = { ( x i , y i ) } i ∈{ 1 ,...,N } , made of (data, label) pairs, ( x i , y i ) ∈ ( X,Y ) , where X ⊆ R p . A decision tree T sequentially applies tests to x i ∈ X before assigning it a value in Y , which we denote T ( x i ) ∈ Y . The tree has two types of nodes: test nodes that apply a test and leaf nodes that assign a value in Y . A test compares the value of an attribute with a given threshold value, x ., 2 ≤ 3 ". In this paper, we focus on binary decision trees, where decision nodes split into a left and a right child with axis aligned splits as in [Breiman et al., 1984]. However, all our results generalize straitghforwardly to tests involving functions of multiple attributes. Furthermore, we look for trees with a maximum depth D , where D is the maximum number of tests a tree can apply to classify a single x i ∈ X . We let T D be the set of all binary decision trees of depth ≤ D . Given a loss ℓ defined on Y × Y we look for trees in T D satisfying

̸

where C : T → R is a function that quantifies the complexity of a tree. It could be the number of nodes as in [Mazumder et al., 2022]. In our work, we are interested in the expected number of tests a tree applies on any arbitrary data x ∈ D . As for ℓ , in a regression problem Y ⊂ R and ℓ ( y i , T ( x i )) can be ( y i -T ( x i )) 2 . For supervised classification problems, Y = { 1 , ..., K } , where K is the number of class labels, and ℓ ( y i , T ( x i )) = 1 { y i = T ( x i ) } . In our work, we focus on supervised classification but the MDP formulation extends naturally to regression.

4 Decision Tree Learning as an MDP

Our approach encodes the decision tree learning problem expressed by Eq. (2) as a finite horizon Markov Decision Problem (MDP) ⟨ S, A, R α , P, D ⟩ . We present this MDP for a supervised classification problem with continuous features, but again, our method extends to regression and to other types of features. The state space of this MDP is made of subsets X of the dataset D as well as a depth value d : S = { ( X,d ) ∈ P ( D ) ×{ 0 , ..., D }} , where P ( D ) is the power set of D . Let F = { f : f ( . ) = 1 { . ≤ x ij } , ∀ i ∈ { 1 , ..., N } , ∀ j ∈ { 1 , ..., p }} be a set of binary functions. We consider only tests that compare attributes to values within the dataset because comparing attributes to other values cannot further reduce the training objective. The action space A of the MDP is then the set of all possible binary tests as well as class assignments: A = F ∪ { 1 , ..., K } . When taking an action a ∈ F , the MDP will transit from state ( X,d ) to either its 'left" state s l = ( X l , d +1) or its 'right" state s r = ( X r , d +1) . In particular the MDP will transit to s l = ( { ( x i , y i ) ∈ X : a ( x i ) = 1 } , d +1) with probability p l = | X l | | X | or to s r = ( X \ X l , d +1) with probability p r = 1 -p l . Furthermore, to enforce a maximum tree depth of D , whenever a state is s = ( ., D ) then only class assignment actions are possible in s . When taking an action in { 1 , ..., K } the MDP will transit to a terminal state

denoted s done that is absorbing and has null rewards. The reward of taking an action a in state s is 134 given by the parameterized mapping R α : S × A → R that enforces a trade-off between the expected 135 number of tests and the classification accuracy. It is defined by: 136

̸

The complexity-performance trade-off is encoded by the value 0 ≤ α ≤ 1 , which is the price to 137 pay to obtain more information by testing a feature. A more detailed study of the trade-off is given 138 in section 6.4. The maximum depth parameter D is a time horizon, i.e. the number of actions it is 139 possible to take in one episode. An algorithm solving such an MDP can always return a deterministic 140 policy [L. Puterman, 1994] of the form: π : S → A that maximizes the expected sum of rewards 141 during an episode: 142

where the expectation is w.r.t. random variables s t +1 ∼ P ( s t , π ( a t )) with initial state s 0 = ( D , 0) .

From deterministic policy to binary DT. One can transform any deterministic policy π of the above MDP into a binary decision tree T with a simple extraction routine E ( π, s ) , where s ∈ S is a state. E is defined recursively in the following manner. If π ( s ) is a class assignment then E ( π, s ) returns a leaf node with class assignment π ( s ) . Otherwise E ( π, s ) returns a binary decision tree that has a test node π ( s ) at its root, and E ( π, s l ) and E ( π, s r ) as, respectively, the left and right sub-trees of the root node. To obtain T from π , we call E ( π, s 0 ) on the initial state s 0 = ( D , 0) .

Equivalence of objectives. When the complexity measure C of L α is the expected number of tests performed by a decision tree, the key property of our MDP formulation is that finding the optimal policy in the MDP is equivalent to finding T ∗ , as given by the following proposition

Proposition 1: Let π be a deterministic policy of the MDP and π ∗ one of its optimal deterministic policies, then J α ( π ) = -L α ( E ( π, s 0 )) and T ∗ = E ( π ∗ , s 0 ) .

The proof is given in the Appendix H.

5 Algorithm

We now present the Dynamic Programming Decision Tree (DPDT) algorithm. The algorithm is made of two essential steps. The first and most computationally expensive step constructs the MDP of Section 4. The second step is to solve it to obtain policies maximizing Eq.(4) for different values of α . Both steps are now detailed.

5.1 Constructing the MDP

An algorithm constructing the MDP of Section 4 essentially computes the set of all possible decision trees of maximum depth D whose decision nodes are in F . This specific MDP is a directed acyclic graph. Each node of this graph corresponds to a state for which one computes the transition and reward functions. To limit memory usage of non-terminal nodes, instead of storing all the samples in ( X,d ) , we only store d and the binary vector of size N , x bin = ( 1 { x i ∈ X } ) i ∈{ 1 ,...,N } . Even then, considering all possible splits in F will not scale. We thus introduce a state-dependent action space A s , much smaller than A and populated by the tests generating function.

5.1.1 Tests generating functions

A tests generating function is any function ϕ of the form ϕ : S → P ( F ) , where P ( F ) is the power set of all possible data splits F . For a state s ∈ S , the state-dependent action space is defined by A s = ϕ ( s ) ∪ { 1 , ..., K } . Because for a given state s we might have that ϕ ( s ) = F , solving the

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

̸

10 1

10 2

10 3

10 4

10 5

10 6

Number of MDP states

0.970

0.975

0.980

0.985

0.990

Exhaustive

TOP 55

TOP 10

TOP 5

CART (TOP 1)

DPDT-2

DPDT-3

MDP with state-dependent actions A s is not guaranteed to yield the minimizing tree in Eq. (2), as 173 optimization is now carried on a subset of T D . In this section, we compare different choices of ϕ on a 174 sufficiently small dataset such that ϕ ( s ) = F , ∀ s ∈ S remains tractable. As a baseline, we use a tests 175 generating function proposed in [Nunes et al., 2020], and compare with our proposed ϕ in terms of 176 quality of the best tree vs. size of the MDP. 177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

Exhaustive function. When ϕ ( s ) = F , ∀ s ∈ S , the MDP contains all possible data splits. In this case, the MDP 'spans' all trees of depth at most D and the solution to Eq. (4) will be the optimal decision tree of Eq. (2). In this case, the number of states in the MDP

D

1

-

∑

d

K

=0

exponentially with the maximum depth of the tree:

D

this limits the learning to very shallow trees (

of a more heuristic choice of number of splits

s

ϕ

∈

S

)

3

The goal is to have a maximal

|

that is orders

B

ϕ

(

s

)

|

of magnitude smaller than that of the exhaustive case

=

Np now in the order of

for deeper trees.

Top B most informative splits. [Nunes et al., 2020] proposed to generate tests with a function that returns for any state s = ( X,d ) the B most informative splits over X in the sense of entropy gain. In practice, we noticed that the returned set of splits lacked diversity

and often consists of splits on the same attribute with minor changes to the threshold value. While this still leads to improvements over greedy methods-as shown in the study presented next-it is at the expense of a much larger MDP, i.e., search space.

Top B most discriminative splits. Instead of returning the most informative splits, we propose at every state s = ( X,d ) , to find the most discriminative splits, i.e. the attribute comparisons with which one can best predict the class of data points in X . This is similar to the minimum description length principle used in [Demirovic et al., 2022] that transforms a dataset with continuous attributes to a binary dataset. However, we perform this transformation dynamically at every state while building the MDP. In practice, this amounts to calling CART with a maximum depth D cart (a hyperparameter of DPDT) on every state s , and using the test nodes of the tree returned by CART as ϕ ( s ) .

While restricting the action space at a given state s to the actions of the tests generating function ϕ ( s ) loses the guarantees of finding T ∗ , we are still guaranteed to find trees better than those of CART:

Proposition 2: Let π ∗ be an optimal deterministic policy of the MDP , where the action space at every state is restricted to the top B most informative or discriminative splits. Let T 0 be the tree learned by CART and { T 1 , . . . , T M } be the set of trees returned by postprocessing pruning on T 0 , then for any α > 0 , L α ( E ( π ∗ , s 0 )) ≤ min 0 ≤ i ≤ M L α ( T i ) .

such that the size of the MDP, which is

D

-

1

∑

d

d

=0

d

would be of the order of

K

(2

B

(2

)

Np

)

which scales

, remains tractable

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

The proof for Prop. 5.1.1 follows from the fact that policies generating the tree returned by CART and all of its sub-trees (which is a superset of the trees returned by the pruning procedure) are representable in the MDP and by virtue of the optimality of π ∗ and the equivalence in Prop. 4, are worse in terms of regularized loss L α than the tree E ( π ∗ , s 0 ) . The consequences of Prop. 5.1.1 are clearly observed experimentally in Fig. 3. While this proposition holds for the latter two test generating functions, in practice, the tests returned by our proposed function are of much higher quality as discussed next.

Comparing tests generating functions. We conduct a small study comparing the exhaustive ϕ (labeled 'Exhaustive') against the ϕ proposed in [Nunes et al., 2020] (labeled 'Top B') and the one used in our algorithm (labeled 'DPDT-K', where K is the maximum depth given to CART), on the Iris dataset. Figure 1 shows that while the latter two ϕ generalize the greedy approach (labeled 'CART'), DPDT scales much more gracefully than when using the ϕ of [Nunes et al., 2020]. With D cart = 4 , DPDT-4 finds the optimal tree in an MDP having several orders of magnitude less states (a few hundreds vs a few millions) than the one built using the exhaustive ϕ . This favorable comparison against exhaustive methods also holds for larger datasets as shown in Sec. 6.2.

The MDP construction of DPDT-K using the tests generating function is explained in Alg. 1. Starting from s 0 , the state containing the whole dataset, CART with a maximum depth of K is called which generates a tree with up to 2 K -1 split nodes. These splits are what constitutes A s 0 , the set of binary tests admissible at s 0 . For every such action, we compute the reward and transition probabilities to a set of new states at depth 1. This process is then iterated for every state at depth 1, calling CART with the same maximum depth of K on each of the states at depth 1, generating a new set of binary tests A s for each of these states s and so on until reaching the maximum depth. Upon termination of Alg. 1, we compute the rewards for labelling actions at every state and we call the dynamic programming routine below to extract the optimal policy.

5.2 Dynamic Programming

Having built the MDP, we backpropagate using dynamic programming the best optimal actions from the terminal states to the initial states. We use Bellman's optimality equation to compute the value of the best actions recursively:

Pareto front. As our reward function is a linear combination of the complexity and performance 242 measures, we can reach any tree 'spanned' by the MDP that lies on the convex hull of the Pareto 243 front of the complexity-performance trade-off. In DPDT, we compute the optimal policy for several 244 choices of α using a vectorial representation of the Q -function that now depends on α : 245

We can then find all policies greedy w.r.t. Q ∗ π ∗ ( s, α ) = argmax a ∈ A Q ∗ ( s, a, α ) . Such policies satisfy

Eq. (4) for any value of α . Given a set of values of α in [0 , 1] , we can compute in a single backward pass Q ∗ ( s, a, α ) and π ∗ ( s, α ) and return a set of trees, optimal for different values of α (see Fig.7 for an illustrative example). In practice, the computational cost is dominated by the construction of the MDP 1 and one can promptly back-propagate the Q -values of over 10 3 values of α .

6 Experiments

In this section we study DPDT from different perspectives. First, in Sec. 6.2, we study DPDT in terms of its performance as a solver for the combinatorial optimization problem of Eq. (2). Here, we focus on smaller problems (maximum depth ≤ 3 ) in which the optimal solution can be computed by Branch-and-Bound (BnB) algorithms. In this first set of experiments, we only report the training accuracy vs. the wall-clock time as done in prior work [Mazumder et al., 2022]. Then we study DPDT

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

for model selection (Sec. 6.3). From the perspective of the end user, a decision tree algorithm may be used for selecting either a tree that generalizes well to unseen data or a tree that is interpretable. We compare classification of unseen data of trees obtained by DPDT to other baselines described below. Then, we plot the train accuracy of trees learned by CART and DPDT as a function of their complexity to observe how a user can choose the complexity-performance trade-off. We use the 16 classification datasets with continuous attributes experimented with in [Mazumder et al., 2022].

When considering other optimal BnB baselines [Demirovic et al., 2022, van der Linden et al., 2023], two problems arise for fair comparison with DPDT in terms of model selection. First, to obtain a set of tree from such baselines, the optmization algorithms need to be ran as many times as trees wanted by the user. For example, one can obtain a set of trees of depth ≤ 5 by running MurTree 2 5 times with different maximum number of test nodes allowed in the learned trees. This could require up to 2 5 times the runtime of a single optimization. Second, MurTree and Pystreed [Demirovic et al., 2022, van der Linden et al., 2023] require binary attributes. Learned trees are not comparable directly with trees trained on continuous attributes because each tree node testing a binary feature actually does at least two tests on the original continuous feature (see Appendix F.1 or Appendix D1 from [Mazumder et al., 2022]). DPDT is coded in Python and the code is available in the supplementary material. All experiments are run on a single core from a Intel i7-8665U CPU . All the links to code used for the baselines are given in the Appenix A

6.1 Baselines

Quant-BnB. [Mazumder et al., 2022] propose a scalable BnB algorithm that returns optimal trees. We emphasize that Quant-BnB is not meant to scale beyond tree depths of 3 (explicitly stated in the Quant-BnB paper) and the authors' implementation of Quant-BnB does not support learning trees of depth > 3.

MurTree, Pystreed . To use [Demirovic et al., 2022, van der Linden et al., 2023] with continuous features datasets, the minimum length description principle is used to obtain bins in a continuous feature domain, then a one hot encoding is applied to binarize the binned dataset. This can result in datasets with more than 500 features. As MurTree and Pystreed memory scales with the square of number of binary attributes, using those algorithms to find trees of depths greater than 3 often results in Out Of Memory (OOM) errors.

Deep Reinforcement Learning. We use Custard [Topin et al., 2021] as a DRL baseline. Custard has two hyperparameters: the DRL algorithm to learn a policy in the IBMDP and a tests generating function that gives p tests per feature. In our experiments, Custard-5 and Custard-3 correspond to DQN agents [Mnih et al., 2015] that can test each dataset attribute against 5 or 3 values respectively.

CART [Breiman et al., 1984] is a greedy algorithm that can build suboptimal trees for any dataset.

6.2 Optimality gap

Because we use a tests generating function that heuristically reduces the search space, a first question we want to investigate is how good is our solver for the combinatorial problem of decision tree search. To do so, we focus on max depth 3 problems for which T ∗ can be computed exactly using Quant-BnB [Mazumder et al., 2022]. As Quant-BnB has a different complexity regularization (number of nodes in the tree) than DPDT (average number of tests per classified data), we set the complexity regularizing term α to 0 to allow direct comparisons. This does not create an artificial learning and on 14 out of 16 datasets, trees with α = 0 generalize best, and second best on the remaining 2. That is because at depth 3 the risk of overfitting is small.

We run DPDT with calls to CART with maximum depth 4 or 5 as a tests generating function (DPDT-4 and DPDT-5 respectively). Quant-BnB is first run without a time limit to obtain optimal decision trees w.r.t. Eq.(2). Quant-BnB is also run a second time with a time limit equal to DPDT-5's runtime (we also added in the supplementary material results for Quant-BnBT +5 and Quant-BnBT +50 that add extra seconds to Quant-BnBT ). CART is run with the maximum depth set to 3 and the information gain based on entropy. All algorithms are run on the same hardware. Custard is run 5 times per dataset because it is a stochastic algorithm. We use stable-baselines3 implementation of DQN [Raffin et al., 2021] with default hyperparameters. A Custard run usually takes 10 minutes. We provide learning curves in Fig. 4. The key result from Table 1 is that DPDT-5 has better train accuracies than the

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

<= -5

[-2.5:0]

[0:2.5]

[2.5:5]

[5:7.5]

[7.5:10]

[10:12.5]

[12.5:15]

>15

0

2

4

6

8

10

12

14

Number of datasets

DPDT-2

DPDT-3

Quant-BnB

MurTree

Pystreed

<=-3

[-2:-2.5]

[-1.5:-2]

[-1:-1.5]

[-0.5:-1]

[0:-0.5]

> 0.5

[-2.5:-3]

DPDT trees average #tests gain over CART trees

0

1

2

3

4

5

6

Number of datasets

DPDT-2

DPDT-3

other non-greedy methods when run in similar runtimes across all classification tasks. Furthermore, the train accuracy gaps between the optimal decision trees obtained from Quant-BnB, in sometimes several hours, and DPDT are usually small (the maximum gap is 1.5% for the bean dataset).

6.3 Selecting the best tree for unseen data

We now investigate whether DPDT is suited for model selection i.e. whether DPDT can identify an accurate decision tree that will generalize well to unseen data for a given classification task. We used the following model selection procedure for each classification task. First, we learn a set of decision trees of depth D ≤ 5 with DPDT-3, DPDT-2, and CART on a training set using different values of α for DPDT or minimal complexity post-pruning for CART. Because Quant-BnB simply cannot compute trees of depth > 3 , we only report the accuracy on unseen data of Quant-BnB trees from Table 1. Because the BnB baselines MurTree and Pystreed are not designed to return a set of trees, we brute force the computation of at most 2 5 trees from each by setting the maximum tree nodes parameter to 0 , ..., 2 5 -1 . Then, for each baseline we evaluate each learned tree (only one tree for Quant-BnB) on a test set and select the tree with highest test accuracy. Fig 2 reports the number of datasets for which each baseline has better generalization performances than CART, and the number of datasets for which DPDT-K returned trees performing less tests on average than CART trees. A table with accuracies of the selected trees on a validation set, the runtime in seconds to obtain the set of trees to select from, and the average number of tests performed on data in Appendix. All BnB baselines required more than 5 minutes to generate a single tree. As such, the runtime for BnB to obtain the whole set of trees is order of magnitudes higher than CART and DPDT. DPDT learns a set of trees of at most depth 5 on the complexity-performance convex-hull in seconds which highlights its ability to scale to non-shallow trees. For that purpose, DPDT built the MDP of possible solution trees of at most depth 5 using CART as a tests generating function, and backpropagated state-action values for 1000 different α .

After applying the above selection procedure, we see on Table 2 that DPDT generalized better than CART on 10 out of 16 datasets while CART outperformed DPDT on only one dataset. When accuracy on test data for CART is already close to 100%, our approach can of course not largely outperform it. However, the benefits of our method have to also be appreciated in terms of gains in average number

of tests. We can see that when CART does not generalize well, our method can have clear gains in 337 generalization (e.g. avila, eeg and fault). Otherwise, when CART is close to 100% accuracy, our 338 method can achieve similar results with less tests. In raisin, rice and room we need two fewer tests 339 which is substantial when tests are expensive, e.g. an MRI scan when testing patients. 340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

6.4 Selecting the most interpretable tree

0

1

2

3

4

5

average tests per sample

0.55

0.60

0.65

0.70

0.75

accuracy

eeg

DPDT-3 Train in 57.038s

DPDT-2 Train in 4.347s

CART-PP Train in 1.25s

0

1

2

3

4

5

average tests per sample

0.4

0.5

0.6

0.7

0.8

accuracy

fault

DPDT-3 Train in 35.185s

DPDT-2 Train in 2.611s

CART-PP Train in 0.913s

In this section, we show how a user can use DPDT to select a tree with complexity preferences. In Figure 3, we plot the trade-offs of trees returned by CART and DPDT. The trade-off is between accuracy and average number of tests. Because this is the trade-off that DPDT optimizes and because the trees of CART are 'spanned' by the MDP created by DPDT, all trees returned by DPDT will dominate in the multi-objective sense trees returned by CART. This is well demonstrated in practice by Figure 3 where the curve of DPDT is always above that of CART. Learned trees and their accuracies as functions of number of nodes and tests are presented in Appendices 5 8 9. Finally, decision tree search being a combinatorial problem, there are always limits to scalability. In Appendix 6 we scale up to a tree depth of 10 by running DPDT-2 up to a depth of 6 then switch to DPTD-1 (i.e. greedy) thereafter. The rationale is that a non-greedy approach is more critical closer to the root.

7 Limitations, Future Work, and Conclusion

Limitations. In our opinion, both the strength and the weakness of DPDT come from the choice of the tests generating function. If the tests generating function generates too much tests in each MDP state, the runtime will grow and there is a risk for out-of-memory errors. This can be alleviated with parallelizing (expanding MDP states on different processes) and caching (only expand unseen MDP states), similar to [Demirovic et al., 2022]. A rule of thumb for running DPDT on personal CPUs is to choose a tests generating function resulting in an MDP with at most 10 6 states.

Future Work. DPDT could scale to bigger datasets by combining Custard [Topin et al., 2021] with tests generating functions and tabular deep learning techniques [Kossen et al., 2021]. The latter is a promising research avenue. The transformer-based architecture from [Kossen et al., 2021] takes a whole train dataset as input and learns representations taking in account relationships between all training samples and all labels. Test actions are then the output of such a neural architecture: the tests generating function is learned .

Conclusion. In this work we solve MDPs whose optimal policies are decision trees optimizing a tradeoff between tree accuracy and complexity. We introduced the Dynamic Programming Decision Tree algorithm that returns several optimal policies for different reward functions. DPDT has reasonable runtimes and is able to scale to trees with depth greater than 3 using information-theoretic tests generating functions. To the best of our knowledge, DPDT is the first scalable decision tree search algorithm that runs fast enough on continuous attributes to be an alternative to CART for model selection of any-depth trees. DPDT is a promising research avenue for new algorithms offering human users a greater control than CART over tree selection in terms of generalization performance and interpretability.

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

395

396

397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

References

Sina Aghaei, Andres Gomez, and Phebe Vayanos. Learning optimal classification trees: Strong max-flow formulations, 2020.

Dimitris Bertsimas and Jack Dunn. Optimal classification trees. Machine Learning , 106:1039-1082, 2017.

Leo Breiman, Jerome Friedman, R.A. Olshen, and Charles J. Stone. Classification And Regression Trees . Taylor and Francis, New York, 1984.

Vinicius G Costa and Carlos E Pedreira. Recent advances in decision trees: An updated survey. Artificial Intelligence Review , 56(5):4765-4800, 2023.

Emir Demirovic, Anna Lukina, Emmanuel Hebrard, Jeffrey Chan, James Bailey, Christopher Leckie, Kotagiri Ramamohanarao, and Peter J. Stuckey. Murtree: Optimal decision trees via dynamic programming and search. Journal of Machine Learning Research , 23(26):1-47, 2022. URL http://jmlr.org/papers/v23/20-520.html .

Emir Demirovi´ c, Emmanuel Hebrard, and Louis Jean. Blossom: an anytime algorithm for computing optimal decision trees. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning , volume 202 of Proceedings of Machine Learning Research , pages 7533-7562. PMLR, 23-29 Jul 2023. URL https://proceedings.mlr.press/v202/ demirovic23a.html .

Abhinav Garlapati, Aditi Raghunathan, Vaishnavh Nagarajan, and Balaraman Ravindran. A reinforcement learning approach to online learning of decision trees, 2015.

Laurent Hyafil and Ronald L. Rivest. Constructing optimal binary decision trees is np-complete. Information Processing Letters , 5(1):15-17, 1976. ISSN 0020-0190. doi: https://doi.org/10.1016/ 0020-0190(76)90095-8. URL https://www.sciencedirect.com/science/article/pii/ 0020019076900958 .

Levente Kocsis and Csaba Szepesvári. Bandit based monte-carlo planning. In European conference on machine learning , pages 282-293. Springer, 2006.

Hecotr Kohler, Riad Akrour, and Philippe Preux. Limits of actor-critic algorithms for decision tree policies learning in ibmdps, 2023.

Jannik Kossen, Neil Band, Clare Lyle, Aidan N Gomez, Thomas Rainforth, and Yarin Gal. Selfattention between datapoints: Going beyond individual input-output pairs in deep learning. Advances in Neural Information Processing Systems , 34:28742-28756, 2021.

Martin L. Puterman, editor. Markov Decision Processes: Discrete Stochastic Dynamic Programming . John Wiley & Sons, Hoboken, 1994.

Zachary C. Lipton. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue , 16(3):31-57, jun 2018. ISSN 1542-7730. doi: 10.1145/3236386.3241340. URL https://doi.org/10.1145/3236386.3241340 .

Rahul Mazumder, Xiang Meng, and Haoyue Wang. Quant-BnB: A scalable branch-and-bound method for optimal decision trees with continuous features. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 15255-15277. PMLR, 17-23 Jul 2022. URL https://proceedings. mlr.press/v162/mazumder22a.html .

Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinforcement learning. nature , 518(7540):529-533, 2015.

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

Cecília Nunes, Mathieu De Craene, Hélène Langet, Oscar Camara, and Anders Jonsson. Learning decision trees through monte carlo tree search: An empirical evaluation. WIREs Data Mining and Knowledge Discovery , 10(3):e1348, 2020. doi: https://doi.org/10.1002/widm.1348. URL https://wires.onlinelibrary.wiley.com/doi/abs/10.1002/widm.1348 .

Rok Piltaver, Mitja Luštrek, Matjaž Gams, and Sanda Martinˇ ci´ c-Ipši´ c. What makes classification trees comprehensible? Expert Systems with Applications , 62:333-346, 2016. ISSN 0957-4174. doi: https://doi.org/10.1016/j.eswa.2016.06.009. URL https://www.sciencedirect.com/ science/article/pii/S0957417416302901 .

Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-baselines3: Reliable reinforcement learning implementations. Journal of Machine Learning Research , 22(268):1-8, 2021. URL http://jmlr.org/papers/v22/20-1364.html .

Rissanen. Modeling by shortest data description. Automatica , 14(5):465-471, 1978. ISSN 00051098. doi: https://doi.org/10.1016/0005-1098(78)90005-5. URL https://www.sciencedirect. com/science/article/pii/0005109878900055 .

John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. CoRR , abs/1707.06347, 2017. URL http://arxiv.org/abs/1707. 06347 .

Olivier Sigaud and Olivier Buffet. Markov decision processes in artificial intelligence . John Wiley & Sons, 2013.

Nicholay Topin, Stephanie Milani, Fei Fang, and Manuela Veloso. Iterative bounding MDPs: Learning interpretable policies via non-interpretable methods. Proceedings of the AAAI Conference on Artificial Intelligence , 35(11):9923-9931, May 2021. doi: 10.1609/aaai.v35i11.17192. URL https://ojs.aaai.org/index.php/AAAI/article/view/17192 .

Jacobus van der Linden, Mathijs de Weerdt, and Emir Demirovi´ c. Necessary and sufficient conditions for optimal decision trees using dynamic programming. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 9173-9212. Curran Associates, Inc., 2023.

Sicco Verwer and Yingqian Zhang. Learning optimal classification trees using a binary linear program formulation. In Proceedings of the AAAI conference on artificial intelligence , volume 33, pages 1625-1632, 2019.

A Code links

Quant-BnB. The Julia code for Quant-BnB is available at https://github.com/ .

mengxianglgal/Quant-BnB CART. We use the scikit-learn Cython implementation of CART available at https:// scikit-learn.org/stable/modules/tree.html#tree-classification with the criterion parameter fixed to 'entropy'. MurTree, Pystreed . Codes are available at https://github.com/MurTree/pymurtree and at https://github.com/AlgTUDelft/pystreed .

B On the failure of deep reinforcement learning.

100

200

300

400

500

600

walltime in sec.

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

cumulative MDP rewards

DQN

PPO

0

200

400

600

800

1000

walltime in sec.

1.0

0.8

0.6

0.4

0.2

0.0

DQN-5 cumulative MDP rewards

avila

bank

bean

bidding

eeg

fault

magic

occupancy

page

rice

room

segment

skin

wilt

C Tree plots 466

7

x

18

≤ 0.047

3

(a) A=0.5, N=3, T=1

7

6

x

12

≤ 0.5

x

14

≤ 0.192

x

11

≤ 0.236

x

17

≤ 0.462

7

x

18

≤ 0.047

2

2

4

7

x

15

≤ 0

(b) A=0.69, N=13, T=2.9

7

7

7

x

5

≤ 0

x

18

≤ 0.047

x

25

≤ 0.73

x

11

≤ 0.236

x

11

≤ 0.249

3

4

7

(c) A=0.53, N=11, T=3.2

7

7

7

x

5

≤ 0

x

18

≤ 0.047

x

25

≤ 0.73

x

11

≤ 0.236

x

11

≤ 0.249

7

6

3

4

x

14

≤ 0.019

(d) A=0.55, N=13, T=3.6

D Schematics DTDP 467

E Detailed res of model selection 468

469

F Comparisons with baselines operating on binary datasets

470

F.1 Why comparisons with baselines that binarize datasets is not fair in our favor?

Algorithms finding optimal DTs for binary datasets such as MurTree [Demirovic et al., 2022] use 471 a binarization method to transform a dataset with continuous attributes to a dataset with binary 472 attributes. However, a DT learned on the binary dataset, whenever it tests the value of a binary 473 attribute, can lead to up to two tests on the respective continuous attribute. Hence, DTs of a given 474 maximum depth on the binary dataset are actually deeper if transformed into DTs on the original 475 dataset with continuous attributes. Despite this, we show in Table 1 of this supplementary material 476

class

class

y

x

≤ 0.2

x

≤ 0.6

y

≤ 0.3

done

done

p = 1

r = -1/3

p = 1

r = -2/3

p = 1/3

r =

𝛼

p = 1/3

r =

𝛼

p = 2/3

r =

𝛼

p = 1/3

r =

𝛼

class

done

p = 1

r = 0

class

done

p = 1

r = -1/2

class

done

class

done

p = 1

r = 0

class

done

p = 1

r = 0

class

done

class

done

class

done

p = 2/3

r =

𝛼

p = 2/3

r =

𝛼

p = 1

r = 0

p = 1

r = -1/2

p = 1

r = -1/2

p = 1

r = -1/2

class

class

y

x

≤ 0.2

x

≤ 0.6

y

≤ 0.3

done

done

x

class

done

Q

𝛼

*(s,a)

= [

0

,

0

]

class

done

class

done

class

done

class

done

class

done

class

done

class

done

[

0,0

]

[

-1/3

,-1/3]

[-2/3,-2/3]

[

-½,-½

]

[-½,-½ ]

[

0,0

]

[0,0]

[

-½ ,-½

]

[-½ ,-½]

[-4/3 , -1/3]

[-1 ,

0

]

[-4/3 , -1/3]

that DPDT typically finds better solutions (in terms of training accuracy) than MurTree + binarization 477 even though the comparison is not fair in our favor since MurTree is considering deeper trees. 478

To illustrate this unbalance with an example, we present a dataset with 3 samples, 2 classes, and 1 479 continuous attribute. After binning the continuous attribute and binarizing the dataset into 3 binary 480 attributes, we compute the optimal depth 1 tree like [Demirovic et al., 2022] or [Verwer and Zhang, 481

2019] would do. To apply this depth 1 tree to the original continuous attribute dataset, the root node 482 " a ∈ [0 . 2 , 0 . 22] " should be decomposed in two decision nodes " a ≤ 0 . 19 " and " a ≤ 0 . 22 " before 483 making a label assignment. So the corresponding tree that can be applied on the continuous attribute 484 is actually of depth 2. 485

486

487

488

489

490

491

492

493

494

495

496

497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

F.2 Experiments

Comparing baselines such as [Verwer and Zhang, 2019] or [Demirovic et al., 2022] to DPDT or Quant-BnB [Mazumder et al., 2022] that operate directly on continuous attributes with the same maximum depth is not fair in favor of the latter algorithms as discussed above. Still, for the sake of curiosity we performed comparisons on datasets of prior works. These can be split into two groups.

1) MurTree: Demirovic et al. [2022] propose an algorithm that retrieves optimal trees for large datasets with binary features using dynamic programming. They also propose a binarization method to retrieve suboptimal shallow trees for large datasets with continuous features. We do not run MurTree but use of the results in Table 6 from Mazumder et al. [2022] (see the 'approx' column) which previously compared Quant-BnB to MurTree.

2) OCT, MFOCT, BinOCT: Bertsimas and Dunn [2017], Aghaei et al. [2020], Verwer and Zhang [2019] propose optimal tree algorithms which formulate the learning problem as a MIP. OCT and MFOCT can produce optimal trees for small datasets with continuous features. BinOCT can also produce optimal trees for small datasets with continuous features after they have been binarized. We make use of the results available at https://github.com/LucasBoTang/Optimal_ Classification_Trees .

Reproducibility: as mentioned above, we did not run the additional baselines but instead used available results. As such runtimes were provided only when available. OCT, MFOCT, BinOCT were run on a single core of an Intel(R) Core(TM) CPU i7-7700HQ @ 2.80GHz . MurTree was run on a single core of a Intel Xeon 2.30GHz . According to online benchmarks the performances of those machines are similar to our Laptopt CPU Intel® Core™ i7-8665U CPU .

G Markov Decision Problem formulations of the Decision Tree Learning Problem

In this section we compare our Markov Decision Problem (MDP) formulation of decision tree learning from Section 4 to that of prior work, namely [Garlapati et al., 2015] and [Topin et al., 2021]. In a nutshell , prior work viewed the task as a deterministic and Partially Observable MDP [Sigaud and Buffet, 2013] and used algorithms such as Q-learning [Garlapati et al., 2015] or deep Q-learning [Topin et al., 2021] to solve them in an online fashion one datum from the dataset at a time. Our approach is different in that it builds a stochastic and fully observable MDP. Our MDP makes it possible to perform two operations that are critical for DPDT: i) being able to call the tests generating function which does not operate online but needs full offline access of the dataset ii) being able to efficiently compute through dynamic programming optimal policies for different complexity-performance trade-offs, which is critical in practice as our improved training accuracy compared to greedy methods would otherwise quickly lead to overfitting. High level differences

Datasets

Accuracy of depth-3 trees

Names

Samples

Features

Classes

Opt.

DPDT-5

MurTree

CART

avila

10430

10

12

58

.

5%

58

.

5

∗

%

58 %

58

.

5

∗

%

53

.

2%

bank

1097

4

2

98

.

3%

98%

98%

97

.

3%

93

.

3%

bean

10888

16

7

87

.

1%

85

.

6%

85%

86

.

9%

77

.

7%

bidding

5056

9

2

99

.

3%

99

.

3

∗

%

99.3%

98

.

1%

98

.

1%

eeg

11984

14

2

70

.

8%

70

.

3%

70%

68

.

8%

66

.

6%

fault

1552

27

7

68

.

2%

68%

65.7%

67

.

3%

55

.

3%

14318

8

2

98

.

1%

98%

98%

97

.

9%

97

.

9%

magic

15216

10

2

83

.

1%

83%

82.7%

81

.

1%

80

.

1%

occupancy

8143

5

2

99

.

4%

99

.

4

∗

%

99.3%

99

.

1%

98

.

9%

page

4378

10

5

97

.

1%

97%

97%

96

.

6%

96

.

4%

720

7

2

89

.

4%

88

.

5%

88.3%

87

.

5%

86

.

9%

rice

3048

7

2

93

.

8%

93

.

7%

93.6%

93

.

4%

93

.

3%

room

8103

16

4

99

.

2%

99

.

2

∗

%

99.2%

99

.

2

∗

%

96

.

8%

segment

1848

18

7

88

.

7%

88

.

2%

88.2%

88

.

1%

57

.

4%

skin

196045

3

2

96

.

9%

96

.

7%

96.7%

96

.

8%

96

.

6%

wilt

4339

5

2

99

.

6%

99

.

5%

99.5%

98

.

7%

99

.

3%

Datasets

Train Accuracy depth-5

Test Accuracy depth-5

Runtime depth-5

Names

Samples

Features

Classes

DPDT-5

OCT

MFOCT

BinOCT

CART

DPDT-5

OCT

MFOCT

BinOCT

CART

DPDT-5

OCT

MFOCT

BinOCT

CART

balance-scale

624

4

3

90

.

9%

91

.

0%

71

.

8%

82

.

6%

67

.

5%

86

.

5%

77

.

1%

74

.

8%

66

.

9%

71

.

3%

61

.

6%

76

.

4%

68

.

34

401

.

71

605

.

51

600

.

1

603

.

95

<

0

.

001

breast-cancer

276

9

2

94

.

2%

94

.

7%

88

.

6%

91

.

1%

75

.

4%

87

.

9%

66

.

4%

67

.

6%

67

.

1%

73

.

8%

62

.

4%

70

.

3%

19

.

09

62

.

36

603

.

39

600

.

25

603

.

67

0

.

001

car-evaluation

1728

6

4

92

.

2%

92

.

2%

70

.

1%

80

.

4%

84

.

0%

87

.

1%

90

.

3%

90

.

3%

69

.

5%

79

.

8%

82

.

3%

87

.

1%

5

.

39

38

.

07

618

.

09

600

.

49

613

.

14

<

0

.

001

hayes-roth

160

9

3

93

.

3%

94

.

2%

82

.

9%

95

.

4%

64

.

6%

76

.

7%

75

.

4%

71

.

2%

77

.

5%

77

.

5%

54

.

2%

69

.

2%

0

.

91

2

.

58

602

.

02

600

.

19

601

.

83

0

.

001

house-votes-84

232

16

2

100

.

0%

100

.

0%

100

.

0%

100

.

0%

100

.

0%

99

.

4%

95

.

4%

95

.

4%

93

.

7%

94

.

3%

96

.

0%

95

.

1%

0

.

44

0

.

65

105

.

72

10

.

74

6

.

6

<

0

.

001

soybean-small

46

50

4

100

.

0%

100

.

0%

100

.

0%

100

.

0%

76

.

8%

100

.

0%

93

.

1%

93

.

1%

94

.

4%

91

.

7%

72

.

2%

93

.

1%

0

.

01

0

.

01

4

.

18

0

.

41

1

.

84

<

0

.

001

spect

266

22

2

93

.

0%

93

.

0%

92

.

5%

93

.

0%

92

.

2%

88

.

5%

73

.

1%

73

.

9%

75

.

6%

74

.

6%

73

.

1%

75

.

1%

6

.

32

16

.

78

604

.

87

600

.

33

605

.

57

0

.

001

tic-tac-toe

958

24

2

90

.

8%

91

.

1%

68

.

5%

76

.

1%

85

.

7%

85

.

8%

82

.

1%

82

.

5%

69

.

6%

73

.

6%

79

.

6%

81

.

0%

107

.

94

626

.

34

615

.

28

600

.

45

621

.

81

0

.

001

between MDPs are summarized in Table 5. For the sake of self-completeness we then detail both 522 MDPs of [Topin et al., 2021] and [Garlapati et al., 2015] which are to be contrasted with our MDP 523 formulation in Section 4. 524

MDP properties

IBMDP [Topin et al., 2021]

[Garlapati et al., 2015]

Ours

Training samples attributes

Any

Categorical

Any

Discounted

Yes

Yes

No

Horizon

Infinite

Finite

Finite

States

Partial information about a single training sample

Partial information about a single training sample

A full dataset in

P

(

D

)

Actions

Tests and label assignments

State dependent tests and label assignments

State dependent tests and label assignments

Transitions

Deterministic

Deterministic

Stochastic

525

526

527

528

529

530

531

532

533

534

535

536

G.1 Iteratvie Bounding MDPs

An IBMDP [Topin et al., 2021] is an episodic, infinite horizon, discounted MDP. IBMDPs can be used for learning decision trees of any base MDP. We discuss here the case where the base MDP is a classification task. In this case, during each episode, an agent has to classify a hidden training sample x i drawn uniformly from a training dataset with continuous attributes. We assume whiteout loss of generality that the training dataset X ⊂ [0 , 1] N × p has continuous attributes in [0 , 1] . On the other hand, the set of labels is Y = { 1 , ..., K } . An IBMDP is defined as follows.

State space: the state space is the hypercube [0 , 1] 3 · p . A IBMDP state has two parts. The continuous attributes of the hidden training sample x i = ( x i 1 , ..., x ip ) to classify, and a lower and upper bound ( L k , U k ) for each of the p attributes. For each attribute x ik , ( L k , U k ) represents the current agent knowledge about its hidden value. Initially, ( L k , U k ) = (0 , 1) for all k , which are iteratively refined by taking tests actions.

Action space: an agent in an IBMDP can either take an assignment action a ∈ Y , or a test action 537 1 { x ik ≤ v · ( U k -L k )+ L k } with k ∈ { 1 , . . . , p } and v ∈ { 1 d +1 , ..., d d +1 } , with d ∈ N a hyperparameter of 538 the IBMDP. 539

540

541

542

543

544

545

546

547

548

549

550

551

552

553

554

555

556

557

558

559

560

561

562

563

564

565

566

567

568

569

570

571

572

573

574

575

576

577

578

579

580

581

582

583

584

585

Transition function: if an agent takes a label assignment action, the IBMDP transits to a terminal state, a new training sample x is drawn at random from X , and the attributes bounds ( L 1 , ..., L p , U 1 , .., U p ) are reset to 0 or 1 . If an agent takes a test action, the attributes bounds are refined. Let x ik be the value of the kth attribute of the hidden training sample x i , and ( L k , U k ) be the current bounds of x ik . If 1 { x ik ≤ v · ( U k -L k )+ L k } is true, then L k is updated to v · ( U k -L k )+ L k , else, it is U k that is updated to v · ( U k -L k ) + L k .

̸

Reward function: the reward for assigning the label y i ∈ Y to the hidden training sample x i is 1 a = y i · r + + 1 a = y i · r -, with r + > 0 and r -< 0 . The reward for taking a test action is α < 0 .

G.2 MDPformulation of [Garlapati et al., 2015]

MDP formulations based on [Garlapati et al., 2015] assume categorical attributes, i.e, the training dataset D is in Z N × p . The MDP is episodic with a discount factor and a finite horizon p +1 . An episode of this MDP consists of costly queries of a training sample's attributes until a label assignment is made.

State space: a state of the above MDP has partial information about a training sample to classify. At every step of the MDP, an agent queries a hidden attribute and updates its knowledge about the training sample by concatenating all revealed attributes.

Acion space: at every step t in the MDP, an agent can either assign a class label in Y = { 1 , ..., K } , or, make a query a t of a hidden attribute of a training sample: A t = ( { 1 , ..., p } \ t -1 ⋃ h =0 a h ) ∪ Y .

Transition function: the current state of the MDP contains values of previously queried attributes. At t = 0 , s = {} . Assuming the hidden training sample to be classified during the current episode is x i = ( x i 1 , ..., x ip ) , then the deterministic transition function is: T ( s, a = x ij ) = s ∪ x ij or T ( s, a ∈ Y ) = s terminal . At the start of a new episode, a new training sample is drawn uniformly from D .

̸

Reward function: at time t , when the hidden training sample to classify is x i , if the an agent takes an assignment action a ∈ D , the reward is 1 a = y i · r + + 1 a = y i · r -, with r + > 0 and r -< 0 . So an agent gets a positive signal for making a correct label assignment and negative signal otherwise. If the agent takes a query action, the reward is a negative value α in order to discourage taking to much queries and control the tree complexity.

H Proof of equivalence of learning objectives

In this section, we prove the equivalence between learning an optimal policy in the MDP of Section 4 and finding the minimizing tree of Eq. (2). We first define C ( T ) , the expected number of tests performed by tree T on dataset D . Here T is induced by policy π , i.e. T = E ( π, s 0 ) . C ( T ) can be defined recursively as C ( T ) = 0 if T is a leaf node, and C ( T ) = 1 + p l C ( T l ) + p r C ( T r ) , where T l = E ( π, s l ) and T r = E ( π, s r ) . In words, when the root of T is a test node, the expected number of tests is one plus the expected number of tests of the left and right sub-trees of the root node.

For the purpose of the proof, we overload the definition of J α and L α , to make explicit the dependency on the dataset and the maximum depth. As such, J α ( π ) becomes J α ( π, D , D ) and L α ( T ) becomes L α ( T, D ) . Let us first show that the relation J α ( π, D , 0) = -L α ( T, D ) is true. If the maximum depth is D = 0 then π ( s 0 ) is necessarily a class assignment, in which case the expected number of tests is zero and the relation is obviously true since the reward is minus the average classification loss. Now assume it is true for any dataset and tree of depth at most D with D ≥ 0 and let us prove that it holds for all trees of depth D +1 . For a tree T of depth D +1 the root is necessarily a test node. Let T l = E ( π, s l ) and T r = E ( π, s r ) be the left and right sub-trees of the root node of T . Since both sub-trees are of depth at most D , the relation holds and we have J α ( π, X l , D ) = L α ( T l , X l ) and J α ( π, X r , D ) = L α ( T r , X r ) , where X l and X r are the datasets of the 'right" and 'left" states to which the MDP transitions-with probabilities p l and p r -upon application of π ( s 0 ) in s 0 , as

described in the MDP formulation. Moreover, from the definition of the policy return we have 586

587

588

589

590

591

592

593

594

595

596

597

598

599

600

601

602

603

604

605

606

I Deeper trees experiments

In this section, we push the limits of DPDT to learn trees of at most depth 10. We run two instances of DPDT. The first one will generate a MDP using a depth dependant tests generating function. DPDT-2... generates a MDP where actions availabe at states corresponding to depth ≤ 5 are given by running CART with a maximum depth of 2, and actions for other states are given by CART with a maximum depth of 1 (the maximum information gain splits given the dataset X in the state ( ( X,d ) )). DPDT-2+1... generates a bigger MDP than DPDT-2... as actions available to states with depths up to 6 are given by CART run with a maximum depth of 2. On Table 6 we observe that deep trees learnt by CART and DPDT perform similarly well on unseen data of different classificiation problems. CART runs way faster than DPDT to compute deep trees. However, DPDT learns more interpretable trees with respect to the average number of tests performed on data which is a very useful feature for real-life applications such as medicine where each additional test before a diagnostic can be very expensive (for example performing an addition MRI scan).

J Additional comparisons with Quant-BnB

In Table 7 we compare DPDT with Quant-BnB on train and test sets of different classification problems. Quant-BnB has a time limit equal to DPDT-5' runtime on each problem. We also run Quant-BnB with bonuses of 5 and 50 seconds to see if the latter can outperform DPDT with just a little more time or if it would require almost twice the time (see Table 7 for DPDT-5' runtimes). We observe that for both train and test accuracies, Quant-BnB-t+50 (DPDT-5 runtime plus 50 seconds bonus) outperforms DPDT most often.

607

K Additional figures for different complexity measures

We show here the complexity-performance trade-offs for all 16 datasets. We show the plot for two 608 complexity measures: average number of tests (what DPDT optimize) and total number of nodes 609 (what the post-process prunning of CART optimizes). On the first measure, the trees that DPDT finds 610 dominate those of CART, which matches the theory. On the second measure, even though we do not 611 optimize for the total number of nodes, we are still able to find better trade-offs w.r.t. this metric than 612 CART for several datasets. 613

K.1 Average number of tests vs accuracy 614

0

1

2

3

4

5

average tests per sample

0.40

0.45

0.50

0.55

0.60

0.65

accuracy

avila

DPDT-3 in 51s

CART-postpruning in 1s

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

average tests per sample

0.6

0.7

0.8

0.9

1.0

accuracy

bank

DPDT-3 in 2s

CART-postpruning in 0s

0

1

2

3

4

5

average tests per sample

0.3

0.4

0.5

0.6

0.7

0.8

0.9

accuracy

bean

DPDT-3 in 88s

CART-postpruning in 8s

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

average tests per sample

0.90

0.92

0.94

0.96

0.98

1.00

accuracy

bidding

DPDT-3 in 2s

CART-postpruning in 0s

0

1

2

3

4

5

average tests per sample

0.55

0.60

0.65

0.70

0.75

accuracy

eeg

DPDT-3 in 57s

CART-postpruning in 1s

0

1

2

3

4

5

average tests per sample

0.4

0.5

0.6

0.7

0.8

accuracy

fault

DPDT-3 in 35s

CART-postpruning in 0s

0

1

2

3

4

5

average tests per sample

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

accuracy

DPDT-3 in 63s

CART-postpruning in 3s

0

1

2

3

4

5

average tests per sample

0.65

0.70

0.75

0.80

0.85

accuracy

magic

DPDT-3 in 98s

CART-postpruning in 5s

0

1

2

3

4

5

average tests per sample

0.80

0.85

0.90

0.95

1.00

accuracy

occupancy

DPDT-3 in 11s

CART-postpruning in 0s

0

1

2

3

4

5

average tests per sample

0.90

0.92

0.94

0.96

0.98

accuracy

page

DPDT-3 in 26s

CART-postpruning in 0s

0

1

2

3

4

average tests per sample

0.60

0.65

0.70

0.75

0.80

0.85

0.90

0.95

accuracy

rice

DPDT-3 in 17s

CART-postpruning in 0s

0

1

2

3

4

5

average tests per sample

0.5

0.6

0.7

0.8

0.9

accuracy

DPDT-3 in 7s

CART-postpruning in 0s

0

1

2

3

4

average tests per sample

0.2

0.4

0.6

0.8

1.0

accuracy

segment

DPDT-3 in 6s

CART-postpruning in 0s

0

1

2

3

4

average tests per sample

0.825

0.850

0.875

0.900

0.925

0.950

0.975

1.000

accuracy

room

DPDT-3 in 19s

CART-postpruning in 0s

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

average tests per sample

0.9825

0.9850

0.9875

0.9900

0.9925

0.9950

0.9975

1.0000

accuracy

wilt

DPDT-3 in 3s

CART-postpruning in 0s

0

1

2

3

4

average tests per sample

0.800

0.825

0.850

0.875

0.900

0.925

0.950

0.975

1.000

accuracy

skin

DPDT-3 in 265s

CART-postpruning in 3s

K.2 Total number of nodes vs accuracy 615

L Codes to reproduce experiments 616

0

10

20

30

40

50

60

nodes

0.40

0.45

0.50

0.55

0.60

0.65

accuracy

avila

DPDT-3 in 51s

CART-postpruning in 1s

0

10

20

30

40

nodes

0.6

0.7

0.8

0.9

1.0

accuracy

bank

DPDT-3 in 2s

CART-postpruning in 0s

0

10

20

30

40

50

60

nodes

0.3

0.4

0.5

0.6

0.7

0.8

0.9

accuracy

bean

DPDT-3 in 88s

CART-postpruning in 8s

0

5

10

15

20

25

30

35

40

nodes

0.90

0.92

0.94

0.96

0.98

1.00

accuracy

bidding

DPDT-3 in 2s

CART-postpruning in 0s

0

10

20

30

40

50

60

nodes

0.55

0.60

0.65

0.70

0.75

accuracy

eeg

DPDT-3 in 57s

CART-postpruning in 1s

0

10

20

30

40

50

60

nodes

0.4

0.5

0.6

0.7

0.8

accuracy

fault

DPDT-3 in 35s

CART-postpruning in 0s

0

10

20

30

40

50

60

nodes

0.91

0.92

0.93

0.94

0.95

0.96

0.97

0.98

accuracy

DPDT-3 in 63s

CART-postpruning in 3s

0

10

20

30

40

50

60

nodes

0.65

0.70

0.75

0.80

0.85

accuracy

magic

DPDT-3 in 98s

CART-postpruning in 5s

0

10

20

30

40

50

nodes

0.80

0.85

0.90

0.95

1.00

accuracy

occupancy

DPDT-3 in 11s

CART-postpruning in 0s

0

10

20

30

40

50

60

nodes

0.90

0.92

0.94

0.96

0.98

accuracy

page

DPDT-3 in 26s

CART-postpruning in 0s

0

10

20

30

40

50

60

nodes

0.60

0.65

0.70

0.75

0.80

0.85

0.90

0.95

accuracy

rice

DPDT-3 in 17s

CART-postpruning in 0s

0

10

20

30

40

50

60

nodes

0.5

0.6

0.7

0.8

0.9

accuracy

DPDT-3 in 7s

CART-postpruning in 0s

0

5

10

15

20

25

30

35

40

nodes

0.2

0.4

0.6

0.8

1.0

accuracy

segment

DPDT-3 in 6s

CART-postpruning in 0s

0

10

20

30

40

nodes

0.825

0.850

0.875

0.900

0.925

0.950

0.975

1.000

accuracy

room

DPDT-3 in 19s

CART-postpruning in 0s

0

10

20

30

40

nodes

0.9825

0.9850

0.9875

0.9900

0.9925

0.9950

0.9975

1.0000

accuracy

wilt

DPDT-3 in 3s

CART-postpruning in 0s

0

10

20

30

40

nodes

0.800

0.825

0.850

0.875

0.900

0.925

0.950

0.975

1.000

accuracy

skin

DPDT-3 in 265s

CART-postpruning in 3s

NeurIPS Paper Checklist 621

622

623

624

625

626

627

628

629

630

631

632

633

634

635

636

637

638

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: All the algorithms and claims mentionned in the intro are studied and presented in detail in the main paper. Please see 1 2.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

639

Answer: [Yes]

640

Justification: Dedicated sections in the experiments and in the conclusion for limitations. Please see 7.

641

642

643

644

645

646

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: Propositions and theorems are proven. Note that is not paper is not a theory paper. Please see H.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: Code and data links are provided. Algorithms are described explicitly. Please see A L.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Anonymized github repo and data links are provided. Please see A L. Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Evertyhing is detailed clearly in the main paper, in the appendix and in the code repos. Please see 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: When processes are stochastic error values are provided in result tables. Please see 1.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: Exact CPU model as well as ram are provided. Please see 6.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: Research is done ethically with na lot of concerns for reproduciblity and validity of the results.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: No societal impact, our work simply proposes a classification/regression tree algorithms like many before. So the ethical and societal concers are inherited from supervised learning ones.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: no risk (see above)

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: Appropriate credits is given when necessary and other code not from the authors are open sourced. Please see A L.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [Yes]

Justification: Code is documented to the best we could. Please see L.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer:[NA]

908

909

910

911

912

913

914

915

916

917

918

919

920

921

922

923

924

925

926

927

928

929

930

931

932

933

934

935

936

Justification: no user studies

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: no user studies

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper formulated the problem of finding an optimal decision tree as Markov Decision Problem and solve the scalability problem using an information-theoretic test generation function. This method provides a trade-off between the train accuracy and tree sizes, the decision tree naturally offers interpretability over ML algorithms.\n\nSTRENGTHS:\nThis paper is well-written and well-organized, it combines RL and decision tree generation, and building MDP before constructing the decision tree.\n\nWEAKNESSES:\n1. The paper lacks sufficient novelty. The approach of constructing a Markov Decision Process (MDP) and using Decision Trees (DT) to generate actions is not new. Specifically, Algorithm 1 appears to still rely on Classification and Regression Trees (CART) for splitting criteria, which diminishes the originality of the proposed method.\n2. The evaluated scenarios in the paper are not clearly articulated. The algorithm has not been tested against well-known benchmarks, unlike other optimal DT algorithms. This makes it difficult to assess the comparative performance and robustness of the proposed approach.\n3. The advantages of using this algorithm instead of CART are not clearly demonstrated. Both algorithms control tree size and depth. However, CART is known to converge faster and offers a simpler implementation. Without clear evidence of the benefits, it is hard to justify the use of the proposed method over established techniques like CART.\n4. The definition of actions generated by the tree is ambiguous. It is not clear whether the actions are discrete or continuous. If the algorithm is designed to build an MDP, it should be tested on general reinforcement learning (RL) tasks to validate its effectiveness and applicability in broader contexts.\n\nQUESTIONS:\nSee weakness section above.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors pose binary decision tree construction within the framework of Markov Decision Processes. They first propose methods for constructing an MDP from a decision tree construction problem, exploring varying test generating functions that trade off the coverage of the search space vs the size of the search space. They then apply Dynamic programming to solve the resulting MDP and show this learnt method can both create binary trees that minimise the loss over a dataset but that it can also be used to add additional losses a user may have over decision trees such as the prior that trees should be small, making them interpretable. They evaluate their proposed method, comparing with other high performing methods such as Quant-BnB, MurTree and a DeepRL method.\n\nSTRENGTHS:\n- Well written paper and easy to understand the method\n- Clearly an important direction of research\n- Thorough experiments with appropriate baselines and good range of datasets to ensure the conclusions generalise to a wide range of datasets\n- Code fully provided, along with implementations of baselines used\n\nWEAKNESSES:\n- Various versions of Reinforcement Learning for binary tree construction have previously been explored. While the implementation in this paper is ultimately different and appears to significantly improve performance, there is limited novelty of the approach. Novelty largely comes down to the test generating functions explored and the addition of extra losses (interpretability) in addition to just the dataset accuracy.\n\n- Small formatting issue\n     - Table 1 is too small\n\nQUESTIONS:\n- Do you have any intuition as to why Deep Reinforcement Learning methods fail in this domain, compared to non Deep RL approaches. And do these reasons present any problems when scaling this method to larger datasets where neural learning would perhaps be introduced.\n\nLIMITATIONS:\n- Limitations are appropriate addressed", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper models the construction of decision trees as a reinforcement learning problem. Currently SOTA algorithms for constructing decision trees have the drawbacks that 1) they take long to compute at depths > 3, and 2) the trees constructed are complex and difficult to interpret. By modelling the problem as an RL task, the authors hope to make the construction of decision trees scale to larger sizes. They present Dynamic Programming Decision Trees which models tree construction as a MDP solved using dynamic programming. They evaluate the accuracy of trees produced by their approach empirically against other commonly used approaches.\n\nSTRENGTHS:\n**Originality:** The approach presented is a novel approach to constructing dynamic trees.\n\n**Significance:** As datasets become larger and interpretability becomes more important, having an approach that scales DT construction to larger trees is needed now. This makes this work rather significant.\n\n**Clarity:** The first half of the paper (up to Sec. 4) was clear. It becomes harder to understand after. Providing an intuitive explanation certain equations would be helpful. E.g., why is probability $p_l = |X_l| / |X|$?\n\n**Quality:** The technique designed is sound and the experiments chosen were the correct ones to demonstrate their claims.\n\nWEAKNESSES:\nThe experimental evaluation is weak. From what I understood, the algorithms being evaluated were run only once and evaluated once. The results do not statistically back up the authors' claims. Multiple runs with statistical significance testing is needed.\n\nThere is no actual analysis on the interpretability of the trees produced, only the complexity of the trees.\n\nQUESTIONS:\nMy main concern is the experimental evaluation.\n\nLIMITATIONS:\nThe authors mention one limitation (test generation) as a problem. It seems that that would make it difficult for DPDT to actually scale to larger trees. Is that not so? Is scalability not a limitation then?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes to use an approach for learning interpretable\ndecision trees using markov decision processes. The results are shown\nto be competitive with branch and bound methods.\n\nSTRENGTHS:\nNone of notice, given the listed weaknesses.\n\nWEAKNESSES:\nThere exists extensive experimental evidence challenging the claims\nabout the interpretability of decision trees, while simultaneously\ndemonstrating the need for decision trees to be explained, since these\ncan otherwise exhibit arbitrary explanation redundancy.\n\nAs a result, and at present, there is no practical justification\nwhatsoever to learn so-called interpretable optimal decision trees.\nIt is absolutely unclear that optimal decision trees will provide any\nadvantage, regarding computed explanations, over decision trees\ninduced with heuristic algorithms.\n\nGiven the above, I cannot recommend acceptance.\n\nSome references on the necessity of explaining decision trees.\n\nXuanxiang Huang, Yacine Izza, Alexey Ignatiev, João Marques-Silva: On\nEfficiently Explaining Graph-Based Classifiers. KR 2021: 356-367\n\nGilles Audemard, Steve Bellart, Louenas Bounia, Frédéric Koriche,\nJean-Marie Lagniez, Pierre Marquis: On the Computational\nIntelligibility of Boolean Classifiers. KR 2021: 74-86\n\nYacine Izza, Alexey Ignatiev, João Marques-Silva: On Tackling\nExplanation Redundancy in Decision Trees. J. Artif. Intell. Res. 75:\n261-321 (2022)\n\nGilles Audemard, Steve Bellart, Louenas Bounia, Frédéric Koriche,\nJean-Marie Lagniez, Pierre Marquis: On the explanatory power of\nBoolean decision trees. Data Knowl. Eng. 142: 102088 (2022)\n\nGilles Audemard, Steve Bellart, Louenas Bounia, Frédéric Koriche,\nJean-Marie Lagniez, Pierre Marquis: On Preferred Abductive\nExplanations for Decision Trees and Random Forests. IJCAI 2022:\n643-650\n\nJoão Marques-Silva, Alexey Ignatiev: No silver bullet: interpretable\nML models must be explained. Frontiers Artif. Intell. 6 (2023)\n\nQUESTIONS:\nNOne.\n\nLIMITATIONS:\nThese were listed above. I believe the paper is solving a non-relevant\nproblem given practical and theoretical evidence regarding the\nnon-interpretability of decision trees, be these optimal or not.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}