ICML 2025

Leveraging Predictive Equivalence in Decision Trees
Hayden McTavish * 1 Zachery Boner * 1 Jon Donnelly * 1 Margo Seltzer 2 Cynthia Rudin 1
Abstract
Decision trees are widely used for interpretable
machine learning due to their clearly structured
reasoning process. However, this structure belies
a challenge we refer to as predictive equivalence:
a given tree’s decision boundary can be repre-
sented by many different decision trees. The pres-
ence of models with identical decision boundaries
but different evaluation processes makes model
selection challenging. The models will have dif-
ferent variable importance and behave differently
in the presence of missing values, but most opti-
mization procedures will arbitrarily choose one
such model to return. We present a boolean logi-
cal representation of decision trees that does not
exhibit predictive equivalence and is faithful to the
underlying decision boundary. We apply our rep-
resentation to several downstream machine learn-
ing tasks. Using our representation, we show that
decision trees are surprisingly robust to test-time
missingness of feature values; we address predic-
tive equivalence’s impact on quantifying variable
importance; and we present an algorithm to opti-
mize the cost of reaching predictions.
1. Introduction
Decision trees are widely used for interpretable machine
learning (Rudin et al., 2022). Their structure of discrete
decisions has long been leveraged for difficult tasks such as
handling missing data (Therneau et al., 1997) and measur-
ing variable importance (Breiman, 1984). Recent advances
in decision tree optimization (Lin et al., 2020; Demirovi´c
*Equal contribution
1Department of Computer Science,
Duke University, Durham, North Carolina, USA 2Department
of Computer Science, University of British Columbia, Van-
couver,
BC, Canada.
Correspondence to:
Hayden Mc-
Tavish <hayden.mctavish@duke.edu>, Zachery Boner <zach-
ery.boner@duke.edu>.
Proceedings of the 42 nd International Conference on Mac...