ICML 2025

Branches: Efficiently Seeking Optimal Sparse Decision Trees via AO*
Ayman Chaouki 1 Jesse Read 1 Albert Bifet 2 3
Abstract
Decision Tree (DT) Learning is a fundamental
problem in Interpretable Machine Learning, yet it
poses a formidable optimisation challenge. Prac-
tical algorithms have recently emerged, primarily
leveraging Dynamic Programming and Branch &
Bound. However, most of these approaches rely
on a Depth-First-Search strategy, which is ineffi-
cient when searching for DTs at high depths and
requires the definition of a maximum depth hyper-
parameter. Best-First-Search was also employed
by other methods to circumvent these issues. The
downside of this strategy is its higher memory
consumption, as such, it has to be designed in a
fully efficient manner that takes full advantage of
the problem’s structure. We formulate the prob-
lem within an AND/OR graph search framework
and we solve it with a novel AO*-type algorithm
called BRANCHES. We prove both optimality and
complexity guarantees for BRANCHES and we
show that it is more efficient than the state of
the art theoretically and on a variety of experi-
ments. Furthermore, BRANCHES supports non-
binary features unlike the other methods, we show
that this property can further induce larger gains
in computational efficiency.
1. Introduction
Black-box models are ill-suited for contexts where decisions
carry substantial ramifications. In healthcare for example,
misdiagnoses can delay crucial treatments and lead to severe
outcomes. Likewise, in the criminal justice system, black-
box models can obscure biases and result in discriminatory
rulings. Such risks highlight the necessity of adopting inter-
pretable models in sensitive domains.
Decision Trees (DTs) are highly interpretable due to their
1LIX, Ecole Polytechnique, IP Paris 2AI Institute, University
of Waikato 3LTCI, T´el´ecom Paris, IP Paris. Correspondence to:
Ayman Chaouki <chaoukiayman2@gmail.com>.
Proceedings of the 42 nd International Conference on Mach...