Prediction models that learn to avoid missing values
Authors: Lena Stempfle
Anton Matsson
Newton Mwai
Fredrik D. Johansson
Venue: ICML 2025 spotlightposter (2025)
Keywords: Missing values
interpretablity
boosting
decision trees
rule-based methods
Primary area: general_machine_learning->supervised_learning
TL;DR: We propose Missingness Avoiding (MA)-class, a framework that adds classifier-specific regularization to trees, ensembles, and sparse linear models, reducing reliance on missing values while preserving accuracy and interpretability.

Abstract: Handling missing values at test time is challenging for machine learning models, especially when aiming for both high accuracy and interpretability. Established approaches often add bias through imputation or excessive model complexity via missingness indicators. Moreover, either method can obscure interpretability, making it harder to understand how the model utilizes the observed variables in predictions. We propose *missingness-avoiding* (MA) machine learning, a general framework for training models to rarely require the values of missing (or imputed) features at test time. We create tailored MA learning algorithms for decision trees, tree ensembles, and sparse linear models by incorporating classifier-specific regularization terms in their learning objectives. The tree-based models leverage contextual missingness by reducing reliance on missing values based on the observed context. Experiments on real-world datasets demonstrate that **MA-DT, MA-LASSO, MA-RF**, and **MA-GBT** effectively reduce the reliance on features with missing values while maintaining predictive performance competitive with their unregularized counterparts. This shows that our framework gives practitioners a powerful tool to maintain interpretability in predictions with test-time missing values.

Prediction Models That Learn to Avoid Missing Values
Lena Stempfle' * Anton Matsson 1*
Newton Mwai
Fredrik D. Johansson
Abstract
Decision Tree
MA Decision Tree
V/ law n 80% p- 0.77
Age > 65
Handling missing values at test time chal- lenging for machine learning models, especially when aiming for both high accuracy and inter- pretability, Established approaches often add bias through imputation or excessive model com- plexity via missingness indicators. Moreover, ei- ther method can obscure interpretability, making it harder understand how the model utilizes the observed variables in predictions. We propose miisinngness-avoiding (MA) machine learning, a general framework for training models to rarely require the values of missing (or imputed) fea- tures at test time. We create tailored MA learning algorithms for decision trees, tree ensembles, and sparse linear models by incorporating classifier- specific regularization terms in their learning ob- jectives. The tree-based models leverage contex- tual missingness by reducing reliance miss- ing values based on the observed context. Exper- iments on real-world datasets demonstrate that MA-DT, MA-LASSO, MA-RF, and MA-GBT effectively reduce the reliance on features with missing values while maintaining predictive per- formance competitive with their unregularized counterparts. This shows that our framework gives practitioners powerful tool to maintain in- terpretability in predictions with test-time miss- ing values.
80% p D
t1 49.4% P 0.09
Age
10.6% 1
S1.2% 1
MMSE score
n- 40.8% 1
Pr[C 0,20
0.995
Pr(Ci) 0.11
Pr(CI)
T/
AUC - 0.74
n= 38% 1
Vi, low
n- 9.8% 1
Pr(CI) = 0.33
Pr(Ci)) 0.81
AUC= 0.73
Figure Two decision trees built to diagnose cognitive impair- ment (CI) in adult patients. The left side illustrates regu- decision tree fit solely for accuracy, while the right shows a missingness-avoidngg (MA) tree, which incorporates regulariza- tion parameter a to reduce reliance on missing values. The regular tree initially splits positive MRI scan outcome, determined by hippocampal volume (V))). However, this feature missing for many patients who did not undergo scan, resulting in high reliance on missing values (indicated by a large p). In contrast, the MA tree achieves a comparable AUC while avoiding reliance on missing data entirely (p = 0). Orange nodes indicate missing values along the decision path, while blue denotes missingness reliance. Note that zero imputation is used, resulting in complete observations in the right branch of the regular tree.
This can introduce bias and undermine the model's inter- pretability, which may be crucial in safety-critical fields like healthcare.
Classically, research on missing values has focused on learning from incomplete data, fitting regression models to imputed values, and applying these complete cases (Ru- bin, 1976). However, when missing values also oceur in deployment (at "test time"), the impute-then-regress strat- egy is needlessly restrictive. When the distribution of observed and missing values remains unchanged, Bayes- optimal predictions can be made without imputation, based on observed values and a missingness mask-indicators for which values are missing. Attempts achieve the same result using high-capacity imputation models can increase bias (Le Morvan et al., 2021), fail to use informative miss- ingness (Rubin, 1976), and reduce the interpretability of predictions.
1. Introduction
Missing values complicate the deployment of prediction models. In applications where humans provide inputs to models and read off their output, such as when using clin- ical risk scores, users must either assign values to un- observed variables or rely on the model to handle this.
Equal contribution 'Department of Computer Science and Engineering, Chalmers University of Technology and University of Gothenburg, SE-41296 Gothenburg, Sweden. Correspondence to: Lena Stempfie <stempfle @chalmers.se>..
Proceedings of the 1nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
Fitting flexible prediction models to trivially imputed in- puts (e.g., zero imputation) and the missingness mask

---

Prediction Models That Learn to Avoid Missing Values
is well justified when accuracy is the primary tar- get. Such models often achieve strong predictive perfor- mance (Van Ness et al., 2023) but may inadvertently rely on complex dependencies between the missingness mask, observed features, and imputed values, making their deci- sion process challenging to interpret (Stempfle et al., 2023; van Smeden et al., 2020).
time. We (1) implement this idea for decision trees (MA-DT), sparse linear models (MA-LASSO), and ensemble methods (MA-RE and MA-GBT), providing prt-predict implementations for seamless integration; (2) analyze when MA learning is feasible without sacrificing predictive performance; and (3) demonstrate that MA models perform comparably to baselines with low reliance on missing values across real-world datasets: confidence intervals for the AUROC of MA-LASSO, MA-DT, and MA MA-RF and their unregularized counterparts overlap in nearly all cases, while the MA models consistently and substantially reduce missingness reliance, in some cases from 100 % to 0 %.
A small class of methods avoids both imputation and pre- diction with indicators by handling missing values natively. For example, XGBoost (Chen & Guestrin, 2016) and other tree-based models can learn "default" decision paths for missing values (Chen & Guestrin, 2016; Pedregosa et al., 2011). Probabilistic models, like Bayesian networks (Pearl, 2014), or variational autoencoders (Kingma & Welling, 2013) can marginalize over missing variables rather than imputing them (Ma & Chen, 2018). Currently, however, native handling of missing values is limited to specialized models.
2. Related Work
As described in the introduction, impute--hen-reegees. pro- cedures, tree-based methods, and missingness indicators are common strategies for handling missing values in supervised learning. While missingness indicators can reduce prediction errors when missingness is informa- tive (Van Ness et al., 2023), they may also introduce spuri- ous correlations (Kyono et al., 2021), feedback loops (van Smeden et al., 2020), and reduced interpretability by effec- tively doubling the feature set. To address these challenges, McTavish et al. (2024) proposed M-GAM, a generalized additive model that learns sparse interactions between fea- tures and the missingness mask. Although this approach improves interpretability, it does not explicitly restrict re- liance on missing values-aa a key objective of our work. Ad- ditionally, neural architectures like NeuMiss (Le Morvan et al., 2020) integrate missing patterns directly into model training, thereby avoiding imputation. However, this comes at the cost of reduced interpretability.
What if we could sidestep imputation, indicators, and spe- cialized architectures by learning to avoid using features with missing values in prediction? If a feature is missing but not useful for prediction for a particular input, a good model should not need access to it. Recently, Stempfle & Johansson (2024) made use of a related idea by training dis- junctive generalized linear rule models (GRLMs) to exploit covariate redundancy in datasets and learn rules whose val- ues can be evaluated even if some variables in the input are missing. The approach promising, but limited to GRLMs and does not extend to ensemble models, which better cap- ture complex, nonlinear interactions.
Figure 1 illustrates scenario where standard decision tree (left) achieves high accuracy (AUC- = 0.74) in predict- ing cognitive impairment (CT) relies on frequently miss- ing features, such as MRI results (hippocampal volume Vi,). that are unavailable for 77 % of patients (indicated by م 0.77). Patients without MRI results most likely showed signs of CI and could be classified well based cognitive test score alone, such as the Mini-Mental State Examination (MMSE) score. The tree on the right achieves comparable predictive performance (AUC = 0.73) with- out ever asking for feature that is missing (0- = 0): MRI results are only used for patients with low MMSE scores. This work proposes a method for learning such trees by strategically minimizing reliance exploiting informative structure in the data. While avoid- ing missing values at test time completely may not be feasi- ble, our approach reduces dependence on missing features while maintaining predictive performance.
Native handling of missing values in tree-based models can be achieved through "Missingness Incorporated in At- tributes" (MIA), which treats missing values as separate category (Twala et al., 2008; Kapelner & Bleich, 2015; Josse et al., 2024). While effective, this approach is typi- cally restricted to a single model class and is incomparable across models. Some studies instead leverage structured missingness. For example, Fletcher Mercaldo & Blume (2020) proposed fitting separate models per missing pat- tern, so-called pattern submodels, but this can lead in- efficient use of data. Stempfle al. (2023) addressed this limitation by introducing sparsity-induce induced parameter shar- ing, which improves generalization and consistency while preserving interpretability. However, these approaches are limited to linear models and do not extend easily to more complex settings.
missing values, and
Branch-exclusive splits trees (BEST), proposed by Beaulac & Rosenthal (2020), allow users to guide the learning al- gorithm during the data partitioning process by specifying
Contributions. We introduce missingness-avoidee (MA) machine learning, reconciling accuracy with min- imal reliance on features with missing values at test
2

---

Prediction Models That Learn to Avoid Missing Values
gating variables- that restrict the use of other vari- ables for splitting outside designated regions of the pre- dictor space. BEST is particularly effective when there
B
C
D
E
A
x
0
1
na
0
1
(A B or
B
0
1
0
3
1
Falsc
True
na
na
is strong prior knowledge about the gettrrteng pro- cess, such as in the ODDC rule setting described in Sec- tion 5.1. However, its performance may suffer when the gating variables have low predictive value, and the use of explicit mask splits can reduce interpretability by causing the tree structure to be dominated by missingness logic.
Sparsity
Redundancy
Context
Figure 2. Missing values can be avoided in several ways. Sparse models (left) be trained not use features that are frequently missing. Disjunctive rule models (middle) can be to include rules that exploit redundancy in the variable set. Trees (right) be fit so that missing values rarely occur on the decision paths.
3. Problem Statement
Let X = [X1,..,XXJ] Xal" E X' be the input to a hypoth- esis E H. used to predict an outcome Y E y. The input consists of d tabular (numerical or categorical) vari- ables, which may be missing in some observations, tak- ing the value na Hence, X (R U {na))² We view X as generated by applying (random) missingness mask M E 13 to a complete input variable X For each variable j E [d].
liance with the definition (formalized here) that h relies on missing values in an observation I there is a feature j such that 1) zy = na and 2) computing h (ax) requires eval- uating zj or its imputed value æry. We use binary indicator function p(h, x) E {0, 1} to indicate that computing h(x)) relies on at least one missing feature in æ,
p(h, .²) max [ax (2. Ax jE(d]
= na]
Xj. M =0 X, x na M,M--1
{
In principle, we could define missingness reliance for an instance by aggregating over features using functions y other than the max operator, Po, x) = ((([[(n(z.))
We assume that the observed variables follow a fixed, un- known distribution p(X, M,,Y Y). which remains the same during training and test time. Algorithms learn from a dataset D- = ((I,, m, y:) Pi=1 of n samples assumed to be drawn i.i.d., (I:; m. yc) ~ p(X,,,Y) where æ; = [ .... xi,a]". T.,]]
1 A I. nal) For example, ip could compute the average over the set, sp()) = 1/ EE and the corre- sponding reliance (h, x) would represent the fraction of missing values in æ relied on by h. In this paper, we restrict ourselves to binary reliance based on max: Does rely on missing values when predicting for x or not?
A common strategy to learn h is to first impute the missing values in X, obtaining an imputed dataset X = ) | with imputed observations x!. The model is then trained on observed or imputed covariates-a strategy known as impute-then-reegeee reggress estimation. If missingness is infor- mative, p(M Y) & p(M), incorporating the missing- ness mask m as binary indicators into the model input can reduce prediction error, improving downstream perfor-
The expected missingness reliance p of hypothesis h in distribution p(X, M, Y) is then defined as p(h) = Ep (p,, The goal of MA learning is to find a suit- able trade-off between expected predictive performance and missingness reliance:
minimize (ALL(Y,(XX)) + ap(h),
(2)
BER
mance (Van Ness et al., 2023; Rubin, 1976).
controlled by tradeoff parameter a 2 0.
Our goal is to learn hypotheses h that are missingness avoiding, that is, they are unlikely to require the value of a missing variable at test time. Let (xx = denote the event that computing h(z)) requires access to the value of æy (imputed observed) and on j) 0 otherwise. For example, computing the prediction of linear model with imputed inputs, ha(a') = o requires access to x² whenever f5 # 0. A decision tree requires access to x feature j appears on the prediction path from root to leaf for the input x. rule model requires access to 2 if the truth values of its rules are contingent on a/.. Figure 2 illus- trates how different models may avoid relying on missing values.
4. Missingness- Prediction Models
Next, we introduce missingness-avviidnng (MA) learning algorithms for decision trees, sparse linear models, and tree ensembles, showing that classifiersspecffic regulariza- tion can reduce reliance on missing features across diverse model classes. Decision trees capture nonlinear interac- tions, sparse linear models offer interpretability, and tree ensembles boost generalization and performance.
4.1. Miisingness-Avviiding Decision Trees
Trees are inherently well suited to avoid missing values as their reliance on input features contextual: only a sub-
Stempfle & Johansson (2024) introduced missingness re-
3

---

Prediction Models That Learn to Avoid Missing Values
set of observations will pass through given node on their decision path, depending on the values of other features. This property naturally extends to the binary decision tree h which can be defined recursively by a root node uo and its two children lo; ro. their children, grandchildren, and so on. Each internal node u E V in the tree is endowed with a decision stub li, : X' + {0, } which partitions X' into a "left" and "right" half. Here, we restrict ourselves to the typical univariate threshold stubs, (x) [ > Tul for feature j. and threshold Tu.
We include a sample-specific feature weight os,j {0. 1 to enable generalization to tree ensembles, discussed in Section 4.3. For a single tree, we set Øi. 1 for all i,j. However, in principle, we could set 0a.j 0 to remove the missingness reliance penalty when feature j is already used in split along the decision path T(æ:), so that further splits on the same feature would not increase the overall missingness reliance. The criterion C may or may not op- erate imputed data use default rules like, for example, XGBoost (Chen & Guestrin, 2016). We refer to trees that minimize this regularized criterion as MA decision trees, or MA-DT for short.
Let L denote the set of leaves of the nodes with no children. Each leaf & E & has a value De E y used for prediction. The decision path (xx) = (uo, ..., u4) for an input a in a tree h is the sequence of nodes traversed from ug to leaf ur E L such that for all O,...
4.2. Missingness-Avoidingg Sparse Linear Models
Our framework extends naturally to sparse linear models. For a linear model h((x) = 0 æ, we define the missingness reliance as
ux+1 ={
bux
Jug S Ta
Tun: Juz > Tu
p(h, = max 11 [001| > 0, 1 [z,- = na] sEld]
We say that feature zj is on the decision path for an input a if there exists a node u E ()( such that ju = j. Fi- nally, we can define the missingness reliance for the tree in x as
with p(h) = (pll((,x)) as stated in Section 3. As is clear from this definition, linear models cannol contextu- ally avoid missing values like trees do. No matter the values of other features, every time a feature j used by the model is missing, it counts toward p(h). Nevertheless, we can re- duce reliance missing values through variable selection by penalizing coefficients 02 of variables with high fre- quency of missingness My. and solving
(h,x)= = max
1 [m.. na)
(3)
(ET,(.))
That is, a tree// h relies on a missing value in input æ if there is a node on the decision path ny made based on a feature j which has a missing value in æ.
T) (z) where the decision is
FITTING TREES WITH SMALL MISSINGNESS RELIANCE
minimize BERd
1 YI
L(ui OT xTx)+ + E( CA++am E(+ (5)
j=1
To fit trees that minimize prediction error and reliance missing values, we propose modifications to greedy split- ting algorithms like C4.5 (Quinlan, 2014) or ID3 (Quin- lan, 1986), as these dominate decision tree implementa- tions. Global optimization approaches, such as MIP formu- lations (Verwer& & Zhang, 2019) and other solvers (Gurobi Optimization. LLC, 2024; ILOG, IBM, 2010; De Moura & Bjørner, 2008), could also be regularized with missingness reliance.
with parameter > 0 and a 2 0 controlling the regular- ization strength. Here, my = n - May is the empirical
missingness rate of feature j. We refer to models that min- imize (5) as MA LASSO..
The standard Lasso (Tibshirani, 1996) encourages sparsi- ties in linear models by applying an I,- T''-penalty propor- tional to the sum of absolute magnitudes of the coefficients, multiplied by a parameter a > 0. In MA MAASOO,, we intro- duce feature-specifer penalty XG = X + amy, where reg- ularization adjusted based on the missingness frequency of each feature j. Features with higher missing propor- tions receive larger penalties, making them more likely to be dropped from the model. We can solve (5) by applying standard Lasso with parameter X to dataset with rescaled features ãigj = X This approach can be extended to generalized linear models and generalized additive models.
Greedy splitting partitions a dataset 'D by sequentially split- ting a leaf & in a tree using feature j and threshold r, cho- sen by minimizing a splitting criterion C(E, j, r), such as Gini impurity or information gain (Hastie et al., 2009). Let S = (i e [m] : e л (2x,)} be the index set of samples in D assigned to leaf in h. We regularize a given criterion C' to reduce reliance on missingness values, splitting leaves l based on feature-threshold pairs (j, 7) that solve the fol- lowing optimization problem, where a 2 controls the regularization strength:
Why Lasso? While L regularization is widely used to induce sparsity in learned models, it is primarily moti- vated as a convex relaxation of L regularization, which explicitly penalizes the cardinality of the selected variable
Ood St
minimize C(e, + sE[d).rER
[ na] (4)
S,

---

Prediction Models That Learn to Avoid Missing Values
set. Here, we use the L penalty due its compatibility with widely used implementations in packages like scikit- learn (Pedregosa et al., 2011). However, (5) may be anal- ogously defined using the L norm and, in future work. it would be interesting to explore solving the resulting prob- lem directly (Liu et al., 2022; Dedieu et al., 2021).
ingness reliance. Consequently, the ensemble can mini- mize prediction error while preventing potential excessive reliance missing values from independently fit trees.
5. Balancing Missingness Reliance and Predictive Performance
4.3. Missingness-Avoiddnng Tree Ensembles
One motivation for MA learning is that many model classes exhibit large Rashomon sets (Semenova et al.. 2022)- collections of near-optimal models with similar predictive performance but differing feature properties. By adding a missingness reliance penalty to the learning objective, we prioritize models within this set that rely less on missing values. Still, important questions remain: When is favor- able trade-off between accuracy and missingness reliance possible? Are there non-trivial problems where both can be achieved without sacrificing the other? Are there prob- lems where the MA learning objective is unsuitable? Next, we give class of data generating processes (DGPs) where there exists a model h that achieves both optimal predictive aceuracy and zero missingness reliance, p(h) = 0. After that, we study cases where the MA approach may fail to improve over standard solutions for missing values.
Tree ensembles, such as random forests and gradient- boosted trees, typically outperform single trees in terms of predictive accuracy (Curth et al., 2024). We can combine several MA trees into an MA tree ensemble, following the same principles used for standard CART ensembles. For ensemble (h1...., ..., h of M estimators, we generalize the definition of missingness reliance for trees applied to an input x as
p(e, x) == max max
I [x.. = na] BEE (m)
(6)
where we recall that TA (x) is the decision path for x in h. Since adding models to an ensemble can only increase the average reliance on a variable, extending the ensemble can only increase the missingness reliance.
We implement both an MA random forest (MA-RF) well as MA gradient boosted trees (MA-GBT). Our implemen- tations are based on the corresponding meta-estimators in the scikit-learn library, with MA trees replacing the default tree estimators for both classification and regression tasks. In MA-RF, , the individual trees are fit independently, allow- ing the tree-building process to be parallelized. The param- eter otj in the regularized node splitting criterion in (4) is set to 1 for all j.
5.1. Can We Achieve Both Zero Missingness Reliance and Minimal Prediction Error?
As an example where the heading's question holds, con- sider a system in which a variable is never missing under certain conditions but may be missing otherwise.
Example 1. Patients listed with a general healthcare provider undergo annual check-ups to assess their overall health. Certain patient data, such as age (21), are always collected, while test results may be missing due to clinical recommendations and practitioner decisions. For example, cognitive tests are consistently administered to individuals over 65 years of age (2) 65), ensuring cognitive test re- sults (I2) are available for all patients in this group. Addi- tionally, patients with a positive test result undergo an MRI scan, which records hippocampal volume (I²) as above or below average. MRI scans may also be ordered for other reasons, such as spine problems or cartilage abnormalities.
MA-GBT is a gradient-boosting classifier specifically de- signed to minimize missingness reliance. At the core, it follows the structure of standard gradient-boosting where the model hm (z) added step m fits the pseudo-residuals of the ensemble 6 (2r) learned up that point using the splitting objective in (4) with regression criterion (mean squared error) for C. See Algorithm 1 in Appendix B for the pseudo algorithm for MA-GBT. Unlike MA-RF, each model in the boosting process builds on the reliance pat- terns established by previous models: if a feature j has al- ready been used by a prior model and was missing for a given observation 2;, subsequent models can continue to rely on that feature without incurring additional penalty for a. We control this by updating the weight oi, in (4) be- fore fitting the next ensemble model,
We refer to such patterns in the data collection process as observed deterministic data collection rules (ODDC rules).
Definition 1. Suppose a distribution p(X, M) over fea- tures X E TRd and a missingness mask M E (0, 1] are given. Let j be a variable, T S [d] \ {j} a subset of other variables, and C Xr be a subset of their domain Xr. An ODDC rule R (T, A,j) is an implication in p that whenever Xy. are observed and take values in A, variable j is observed,
 = o.j - (1- - 1li E Them (x:)]1[ars = nal)
This ensures that subsequent models in the ensemble prefer to include features that have already been used by models from previous iterations, or features that incur low miss-
R: My = 0, Xy E 7p M, = =0,
(7)
5

---

Prediction Models That Learn to Avoid Missing Values
or, equivalently, PM, Mr = 0, Xr E =1
5.2. When Is Low Missingness Reliance Incompatible With Good Predictions?
When fitting prediction model to data, the learned struc- ture can exploit ODDC rules to limit missingness reliance. Example (Continued). Suppose we want to fit a deci- sion tree the healthcare data to predict whether a pa- tient suffers from cognitive impairment. Due to the ODDC structure, we can construct an accurate tree with zero miss- ingness reliance by first splitting on the patient's 's age (x1), followed by cognitive test results (I2) for patients older than 65, as illustrated in Figure 1. For patients with 21 > 65 and 22 = 1, the MRI sean outcome- the most predic- tive variable-is always available and can be used for pre- diction.
The MA framework is designed to exploit structure in the missingness patterns induced by the data-generating pro- cess, as exemplified in the previous section. When there is no structure in M predictable from observed variables X, the use of any feature X, will incur missingness reliance.
Proposition 2. Suppose that each feature is indepen- dently missing completely at random with probability Pj E [0.1].
p(M, YY 5) Pj
Then, for any model h, with on (2e E {0, 13 indicating whether the feature 2 is used in the prediction h(x))
We say that a node u in a tree h satisfies a set of ODDC rules R = {R,...,,R, Rx} } if the feature 2j,, used by node implied to be observed by RR. Specifically, the truth values of the splits of u's ancestors, an(u), must activate (imply the antecedent of) at least one rule R E R that im- plies Mr = 0 according to (7). MA trees prefer node splits that satisfy ODDC rules by design: as + oo, only splits on variables that have no missingness for observations at the current node will be considered. A full tree h satisfies a set of ODDC rules R if all of its internal nodes obey R. Such trees have no reliance on missing values.
p(h) 2 max (aa,aaannXX)l)l (X,j)lps.
A proof is given in Appendix A. words, the missingness reliance is lower-hounded by the frequency by which a fea- ture is used by the model, multiplied by the missingness frequency, no matter the structure of the model. We give two more challenging cases for MA learning below.
Distribution Shift. If the test-time missingness distri- bution differs from the distribution of training data, de- ployment performance may degrade also for MA models, like prediction with missingness indicators or impute-then- regress strategies. For example, MA trees optimize deci- sion paths based on observed missingness patterns, but if these change, data-collection rule may longer hold, and variables that were previously guaranteed to be ob- served may be missing test time. Both features frequently missing during training but available at test time and vice versa can lead to miscalibrated reliance and a suboptimal tradeoff with accuracy.
Proposition 1. Let set of ODDC rules R hold under distribution p(X, M). Any tree h that satisfies R will have missingness reliance phh) = 0 under p.
Finally, we say that an outcome Y satisfies set of ODDC rules R with respect
input X and the missingness mask M if p(Y X) is fully determined by variables im- plied to be observed by R. We can conclude the following result.
Corollary 1. Consider a distribution p(X,, ((X,MYY M,Y). Sup- pose the outcome Y satisfies set of ODDC rules R in p. Then, there exists a prediction model h with minimal risk and zero missingness reliance, p(h) 0, on p.
MNAR and Informative Missingness. In MNAR (miss- ing not at random) settings, where the missingness mask depends on unobserved features, MA trees cannot strate- gically split to avoid missing values as there may be no discernible pattern in the missingness to influence the sec- ond term in (2). We study this effect empirically in Sec- tion 6.3. Moreover, when missingness is informative, i.e., when P(M Y) + P(M), MA learning may underper- form compared to approaches that include missingness in- dicators, as these can become predictive in such cases. In principle, missingness indicators could be used with the MA penalty, but we refrain from exploring this here.
Proposition 1 and Corollary 1 are proven in Appendix A.
Corollary follows by definition: if the outcome can be described by input features known be observed together, model that fits p(Y X, M) perfectly can be optimally predictive and have zero reliance p. Such outcomes are more common in practice than we may first expect, at least approximately. For example, when predicting a medical di- agnosis or mimicking a treatment policy, the ground-truth truth labels are given by decisions made by doctors based on ob- servations they have made in sequence. Consequently, a tree-based model that matches test ordering patterns will use only information likely to be measured for splitting.
6. Experiments
Note that T can be the empty set. Variables that are never missing all satisfy ODDC rules on the form (0,-,3)
We demonstrate the MA framework in a suite of experi- ments, aiming to show that MA models reduce reliance on
6

---

Prediction Models That Learn to Avoid Missing Values
missing values while preserving predictive performance. We compare MA-DT, MA-LASSO, MA-GRT, and MA-RF to standard models as well as to models designed to handle missing values, using different imputation methods.
and unit variance for all models except MINTY and M- GAM for which we apply discretization. Categorical fea- tures are encoded using one-hot encoding. All models are evaluated on the held-out test set with respect to AUROC and أ. Appendix C.3 provides details on the experimental setup, including hyperparameters. The code is available at https:/ / /gi thub. cwmc Saom/Healhhh--ae AI/malearn.
Datasets. We study six datasets with varying degrees of missingness. We sample 10,000 samples from the National Health and Nutrition Examination Survey (NHANES) for hypertension prediction with 42 features (Johnson et al., 2013). LIFE (2,864 samples) predicts whether life ex- pectancy is above or below the median using 18 fea- tures (World Health Organization (WHO), 2021). ADNI (1,337 samples) predicts diagnosis changes in patients with suspected dementia using 39 features (Weiner et al., 2010). Breast Cancer (1,756 samples) includes 16 features for cancer prediction (Razavi et al., 2018). Pharyngitis (676 samples) is used for pharyngitis prediction using 18 fea- tures (Miyagi, 2023). Explainable Machine Learning Chal- lenge dataset, referred to FICO, consists of 10,549 sam- ples and is used for credit risk classification with 23 fea- tures (FICO et al., 2018). Details on dataset preprocessing are in Appendix C.
6.1. MA Models Match Baselines With Minimal Reliance on Missing Values
In Table I, we report AUROC and missingness reliance (ń) for each model in ADNI, FICO, LIFE, and NHANES us- ing zero imputation. Results using MICE imputation and for the other datasets, Pharyngitis and Breast Cancer, are included in Appendix D. Comparing MA MA-LASSO, MA-DT, and MA-RF to their unregularized counterparts, we find that confidence intervals for AUROC overlap in all cases except for the linear models on FICO. MA models demonstrate su- perior performance in avoiding missingness reliance, with average decrease of 66.0 (MA-LASSO vs. LR), 4.4 (MA-DT vs. DT), and 70.2 (MA-RF vs. RF) across datasets in Table 1. MA-DT stands out by consistently having the lowest missingness reliance while maintaining competi- tive AUROC scores compared to DT. MA-GBT performs slightly worse compared to XGBoost terms of AUROC but reduces missingness reliance by 64.4 on average.
Baselines. We compare MA models with baselines rep- resenting a) impute-then-regress learners, b) models using missingness indicators, and c) models that handle miss- ing values natively. For a), we use rregularre lo- gistic regression (LR), a decision tree (DT), and a ran- dom forest (RF). For b), we consider M-GAM (McTavish et al., 2024), a sparse additive modeling approach that in- corporates missingness indicators (ind) and their interac- tions with observed data (int). For c), we include XG- Boost (Chen & Guestrin, 2016), NeuMiss (Le Morvan et al., 2020>aa neural network architecture with a non- linearity based on missingness patterns in the input data- and MINTY (Stempfle & Johansson, 2024), generalized linear rule model fit to minimize reliance on missing val- ues. For models relying imputation, impute numeri- cal features with zero imputation (1)) and MICE (Van Bu- uren, 1999) (Ia)). Categorical features are imputed with the mode.
MINTY is strong baseline, with AUROC and missing- ness reliance comparable to MA-LASSO in most cases. However, its training time ranges from 18 to 292 seconds, whereas MA-LASSO trains in under a second. The M- GAM models exhibit moderate أ. suggesting that their use of the missingness structure leads to increased missingness reliance compared to MA models. NeuMiss achieves high AUROC but also exhibits high missingness reliance due to dense multiplications of input features.
The extreme differences observed in NHANES likely stem from its structured missingness patterns. Specifically, the missingness mask depends on the unobserved variable year, with different features and feature versions mea- sured in different years. However, some measurements can act as substitutes for others. For example, there are several different features representing blood pressure measured us- ing different equipment. These patterns cause many stan- dard models to over-rely on missing features, whereas MA models effectively mitigate this issue. Interestingly. DT tends to avoid redundancy by selecting only the most in- formative features, whereas LR uses all available features, resulting in a very high missingness reliance.
Experimental Setup. We divide each dataset into train- ing and testing subsets using an 80/20 split. For hyperpa- rameter selection, including a, we perform 3-fold cross- validation, evaluating candidate models based on a combi- nation of the area under the receiver operating character- istic curve (AUROC) and empirical missingness reliance (p). Specifically, we select the candidate model with the lowest Ó among those achieving at least 95 percent of the maximum AUROC. This procedure is repeated for 5 dif- ferent splits of the dataset, and we report 95 % confidence intervals based on the bootstrap distribution of AUROC and é. Numerical features are standardized have zero mean
6.2. Performance of MA Models in the Limits of
In addition to selecting a a based on the AUROC- missingness reliance trade-off, we fit models with 0
7

---

Prediction Models That Learn to Avoid Missing Values
Table 1. Test-set AUROC and missingness reliance (p) across four datasets, using zero imputation. The 95 % confidence intervals are based on the bootstrap distribution of the respective metric. For each dataset and MA estimator, a is selected part of the overall model selection strategy, with the model minimizing ý chosen among those achieving at least 95 % of the maximum AUROC.
ADNI
FICO
LIFE
NHANES
ESTIMATOR
AUROCI (1)
(())
AUROC (t)
P())
AUROC (1)
p())
AUROC (1)
P(L)
LR
72.0 24.3)
64.1 0621. 56.0
79.1 79.7)
(74, 75.9)
98.7 (97.7. 99.3)
63.4 (60.0. 66.8)
85. (84 K5.7)
100,0)
DT
72.7 74,4
11.7 (6.3. 18.5)
73.8 74.5)
5.7 (5..6.))
92.2 (90.1. 92.6)
18.3 (15.8. 19.9)
82.3 (81 83.0)
0.1 (0.00.2)
RF
(2.M.727
66.1 (64.3, 68.3)
77.3 78..
65.4
97.8 (95 99.0)
82.8 83.9)
81.6
100.0 (1000, 100.0
M M-GAM (IND)
(66.9,6, 72.11
14.7 32.5)
77.9 (77.6.76.3)
(2.6, 56.2)
98.2 (96.4, 99.2)
49.4 (42.8. 56.1)
$4.0 (63 84.6)
28.2 50.0,
M-GAM (eer)
69.5 (67.0,22))
46.3 (22.5, 61.23
77.6 (77.3.77))
29.1 (28.1. 30.2)
98.1 (96.3, 99.3)
51.7 (45.3, 60.2)
(83.6, 84.5)
(14.8, 48.3)
XGBooST
(1.7,85))))
60.9 (87.6, 63.4)
78.4 (77.9. 79.00
(5..9, 74.9)
98.2 (96. 1, 99.4)
56.1 (52.3, 61.1)
84.8 (84 83.2)
95.7 (87.3, 100
NEUMISS
76.4 78..078.)))
67.0 (6..9, 70.1)
79.0 (78.4. 79.5)
75.4 (74.9, 75.9)
98.6 (97.3, 99:4)
84.1 (83.0. 95.2)
(85 83.7)
100.0 (100.0, 100.0)
MINTY
(62,4,2)))
1.3 2:55
76.6 (75.2, 77.9)
22.8 (5.0. (8.0.43.6) 43.6)
96.2 (94. 98.3)
21.9 (16.8. 27.3)
81.9 (60 K3.8)
0.3 (0.0, 0.5)
MA-LASSO
68.4 (66.1.70.))
75.8 (74.9.76.8
(5.4. 6.0)
97.7 (96.7, 95.63
21.0 (19.1.24.1
(54.0,
0.4 0.9)
MA-DT
74.1 (71.7,77)7
0. 0..0.0.3)
73.8 (75 74.5)
5.7 (5..
89.7 (57 5, 91.80
12.3 (10.4, 15.3
82.4 (81 K3.7)
0.0 (0.0. 0.0)
MA-RF
78.0
3.2 2..7.
76.3 (75.6.
5.8 (5.5.6.))
97.2
24.2 26.5)
84.0 083
0.2
MA-GBT
78.5 (7.54.40.5)
1.6 (0.7.277)
75.6 (7.35.77.3
5.9 (5.5.6.3
95.9 (92 99.99
16.5 (1..9.21.3
83.2 033 64.4)
0.3 (0.0. 1.0)
0.95
0.95
Missnggoss Telaance (A)
0.90
0.3
0.90
AUROC
AUROC
0.85
0.2
0.85
0.50
0.0
0.1
0.1
Max tree depth
0.75
U.0
0.80
10.0
0.1
10.0
0.70
0.00
3
1
5
1-
6
7
0.1
0.2 Missingpes relionne (0)
0.3
Max.
depth
Max. free depth
(a) LIFE: AUROC vs. max. tree depth.
(b) LIFE: I vs. max. tree depth.
(c) LIFE: AUROC vs. p.
Figure 3. Average test performance across cross-validation folds for MA decision trees to LIFE using varying values of a and max_ aex__ephh. A shallow tree of depth 3 captures the complexity of the data effectively. Missingness regularization with a S 1.0 does not significantly reduce predictive performance but helps decrease missingness reliance from approximately 30 % to approximately 10% %.
and a set high enough to ensure ý ~ 0 while maximiz- ing AUROC (denoted as a = x)). Following scikit-learn's 's convention, a tree with max_depth = is allowed one split a meaningful feature available. Consequently, achieving و may be impossible when all features con- tain missing values, preventing full elimination of miss- ingness reliance in tree-based models. We report AUROC and I for each setting and dataset in Table 6 and 7 in Appendix D. Figure 4a and 4b show how AUROC and م change when transitioning from a = co to a = 0 for ADNI. Here, all MA models achieve و as with a large a. While setting a 0 can significantly boost AUROC. it drastically increases missingness reliance, especially for MA-LASSO and MA-RF.
from 0.1 (a = 1) to 0.3 (a 0), demonstrating clear case where missingness regularization can be applied with- out sacrificing performance. Notably, reducing the trec's maximum depth is a less effective way to control missing- ness reliance, as negatively impacts AUROC.
In Figure 6 in Appendix D, we show examples of MA trees with E {0, a" x at their respective optimal depths for LIFE. When a co, the learning algorithm is forced to split on the always-ohserved feature Regi on instead of coult_mortality. which has around 10% miss- ingness and is used when a E {0, a0)a") With a = a", Region appears in the first split in the right branch of the tree, reducing missingness reliance without sacrificing AU- ROC compared to the unregularized tree, which uses the incomplete feature Under five_deaths
We further examine the impact of increasing missingness regularization when fitting MA DD with varying values for the max_depx_aaaa parameter. Figure 3 and Figure 5 in Ap- pendix D illustrate the trends in AUROC and missingness reliance for LIFE. When a MA- DT behaves as stan- dard decision tree without missingness reliance regulariza- tion. For {0, 0.1, 1), a depth of 3-4 captures the data's 's complexity without overfitting. On the other hand, و varies
6.3. Performance of MA Models Across Missingness Mechanisms
To assess MA models under different missingness mech- anisms, we introduce 50% % synthetic missingness to ran- domly selected features in the Breast Cancer dataset, the
8

---

Prediction Models That Learn to Avoid Missing Values
0.73
Missingrss rellanoo (.)
0.6
0.73
Proportion
AUROC
AUROC
10.1
0.4
0.68
0.6
0.2-
Mechaniism
0.2
0
0.65
MAR MNAR
0.63
0.00
0.0 MA-LASSO
MA-LASSO MA-DT
MA-RF
MA-GBT
MA-DT
MA-RF
MA-GBT
0.0
0.2
0.4
0.6
0.5
1.0
Missingross ces reliince (0)
(a) ADNI: AUROC vs. MA estimator.
(b) ADNI: á vs. MA estimator.
(c) Breast Cancer: AUROC vs p.
Figure 4. (a) and (b): Test-set AUROC and missingness reliance (0) for MA estimators in ADNI when transitioning from = a = O. Removing missingness regularization improves predictive performance, but the increase in missingness reliance much more pronounced, especially for MA-LASSSO and MA-RF. (c): Test-set AUROC versus ő for MA-LASSSO in Breast Cancer, where 50 % synthetic missingness is added to an increasing proportion of input features. Missingness not at random (MNAR) is more challenging than missingnes random (MAR), but -ASSSOO demonstrates robust performance for large fractions of missingness.
most complete in our study. We apply MAR using logistic masking model and MNAR for values below the 0.25 and above the 0.75 quantiles. Figure 4c shows how AUROC varies with missingness reliance for different missingness proportions and mechanisms using MA-LASSO. The per- formance of reregulariee logistic regression models is included with circles for reference. When MAR missing- ness is introduced in up to 40% % of the features, it possi- ble fit MA LASSO with 25 % missingness reliance with- sacrificing much predictive performance. Notably, the trends similar for the more challenging MNAR case, although the curves are slightly shifted toward lower AU- ROCs in all cases.
metric p. which measures the model's dependence on miss- ing features.
As future direction for tree-based MA models, an alter- native to minimize reliance on missing values could be to enforce ý = 0 at test time by adopting a fallback strategy- halting the decision process and returning the label of the current node whenever missing value is encountered. Ad- ditionally, pruning strategies and hyperparameter tuning beyond depth constraints could be explored reduce miss- ingness reliance in these models. In our current implemen- tation of MA-DT, we rely greedy splitting algorithms, but in theory, global optimization approaches (Gurobi Op- timization, LLC, 2024; ILOG, IBM, 2010) could also be regularized with missingness reliance, producing accurate decision trees with potentially improved interpretability (Rudin et al., 2022).
7. Discussion
We have proposed missingness-avoiding (MA) machine learning, a framework for reducing the reliance of mod- els on missing values in prediction, applied to sparse lin- ear models (MR-LASSO), decision trees (MA-DT), random forests (MA-RF) and gradient boosting (MA-GBT). Our ex- periments show that MA models effectively learn to gener- ate accurate predictions while minimizing reliance on miss- ing values at test time by applying classifier-specffic regu- larization to avoid using unobserved feature values.
Moreover, further analysis could explore alternative strate- gies for selecting a" beyond our current approach and in- vestigate different definitions of as discussed in Sec- tion 3. Alternative strategies for selecting a* should con- sider the specific application needs. For instance, rather than focusing solely on the acceptable trade-off in predic- tive performance for achieving the lowest possible a, we might prioritize limiting the number of missing features per individual. Relying on average measure across the dataset may not provide sufficient insight. Additionally, domain knowledge can help define an acceptable level of missingness, but quantifying this threshold is often chal- lenging and subjective. Our focus has been on the tradeoff between predictive accuracy and reliance on missing val- ues in prediction tasks that match the training data in dis- tribution. Future work should explore other effects of MA learning, such as robustness to distributional shifts. Lastly, an open question remains: can our method be extended to other model classes?
To apply the MA framework, begin by selecting model class that suits the interpretability needs of the applica- tion. Linear models and decision trees offer more trans- parency, while ensembles often provide better performance with less interpretability. After choosing model type, ad- just the regularization parameter a balance predictive performance, and reliance missing features. A higher a discourages reliance on frequently missing features, while a lower a favors performance. The right value depends on the acceptable level of missingness reliance. Practitioners can evaluate their strategy using the missingness reliance

---

Prediction Models That Learn to Avoid Missing Values
Acknowledgements
Dedieu, A., Hazimeh, H., and Mazumder, R. Learning sparse classifiers:
Continuous and mixed integer opti- mization perspectives. Journal of Machine Learning Re- search, 22(135):1-45 2021.
This work was partially supported by the Wallenberg AI, Autonomous Systems and Software Program (WASP) funded by the Knut and Alice Wallenberg Foundation.
FICO, Google, Imperial College London, MIT, University of Oxford, UC Irvine, and UC Berkeley. Explainable machine learning challenge, 2018.
The computations and data handling were enabled by re- sources provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS), partially funded by the Swedish Research Council through grant agreement no. 2022-06725.
Fletcher Mercaldo, S. and Blume, J. D. Missing data and prediction: The pattern submodel. Biostatistics, 21(2): 236-252, 2020.
Data used in the preparation of this article were ob- tained from the Alzheimer's Disease Neuroimaging Ini- tiative (ADNI) database (https: : /anii. loni. usc. edu/). The ADNI was launched in 2003 as a public- private partnership. led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has been to test whether serial magnetic resonance imaging (MRI), positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cog- nitive impairment (MCT) and early Alzheimer's disease (AD).
Gardner, J., Popovie, Z., and Schmidt, L. Benchmarking distribution shift in tabular data with tableshift. In Ad- vances in Neural Information Processing Systems 36, pp. 53385-53432, 2023.
Gurobi Optimization, LLC. Gurobi Optimizer Reference Manual, 2024.
Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning: Data Mining. Inference, and Pre- diction. Springer, 2 edition, 2009.
ILOG, IBM. User's 's Manual for CPLEX, 2010.
Impact Statement
Jeanselme, V., De-Arteaga, M., Zhang, Z., Barrett, J., and
Two of our proposed model classes, MA-DT and MA-LASSO, prioritize interpretability ensure robust and practitioner-frienlly models. This focus interpretability is crucial for applying machine learning in high-stakes do- mains like healthcare, where handling missing data poses risks of introducing or reinforcing unfaireess (Jeanselme et al., 2022). Through increased transparency, these mod- els aim to support ethical eeciiion-making in complex en- vironments.
Tom, B. Imputation strategies under clinical presence: Impact on algorithmic fairness. In Proceedings of the 2nd Machine Learning for Health symposium, pp. 12- 34. PMLR, 2022.
Johnson, C. L., Paulose-Ram, R., Ogden, C. L.,, Car- roll, M. D.,, Kruszon- Moran, D., Dohrmann, S. M., and Curtin, L. R. National health and nutrition examina- tion survey: analytic guidelines, 1999-2010. Vital and Health Statistics, ((161):1-22) 2013.
References
Josse, J., Chen, M., Prost, N., Varoquaux, G., and Scor- net, E. On the consistency of supervised learning with missing values. Statistical Papers, 65(9):5447-5479 2024.
Beaulac, C. and Rosenthal, J. S. BEST: A decision tree algorithm that handles missing values. Computational Statistics, 35(3):1001-1026, 2020.
Kapelner, and Bleich, J. Prediction with missing data via Bayesian additive regression trees. Canadian Journal of Statistics, 43(2):224-239, 2015.
Chen, T. and Guestrin, C. XGBoost: A scalable tree boost- ing system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 785-794, 2016.
Kingma, D. P. and Welling, M. Auto-encoding variational bayes. arXiv preprint arXiv: /2. 14, 2013.
Curth, A., Jeffares, A., and van der Schaar, M. Why do random forests work? Understanding tree ensembles as self-regularizing adaptive smoothers. arXiv preprint arXiv:2402.01502 2024.
Kyono, T., Zhang, Y., Bellot, A., and van der Schaar, M. Miracle: Causally-aware imputation via learning miss- ing data mechanisms. In Advances in Neural Informa- tion Processing Systems 34, pp. 23806 2010002231..11 2021.
De Moura, L. and Björner, N. Z3: An efficient SMT solver. In Proceedings of the 14th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, pp. 337-340, 2008
Le Morvan, M., Josse, J., Moreau, T., Scornet, E., and Varoquaux, G. NeuMiss networks: Differenniable pro- gramming for supervised learning with missing values.
10

---

Prediction Models That Learn to Avoid Missing Values
In Advances in Neural Information Processing Systems 33, pp. 5980-5990, 2020.
Rubin, D. B. Inference and missing data. Biometrika, 63 (3):881-592, 1976.
Le Morvan, M., Josse, J., Scornet, E., and Varoquaux, G. What's a good imputation to predict with missing val- ues? In Advances in Neural Information Processing Sys- tems 34, pp. 11530-11540, 2021.
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., and Zhong, C. Interpretable machine learning: Funda- mental principles and 10 grand challenges. Statistic Sur- veys, 16: 1-1,-5,,000 2022.
Semenova, L., Rudin, C., and Parr, R. On the existence of simpler machine learning models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pp. 1827-1858, 2022.
Liu, J. Zhong, C., Seltzer, M., and Rudin, C. Fast sparse classification for generalized linear and additive mod- els. Proceedings of the 25th International Conference on Artificial Intelligence and Statistics, volume 151, pp. 9304-9333, PMLR, 2022.
Shadbahr, T., Roberts, M. Stanczuk, J., Gilbey, J., Teare, P., Dittmer, S., Thorpe, M., Torné, R. V., Sala, E., Lió, P., et al. The impact of imputation quality on machine learn- ing classifiers for datasets with missing values. Commu- nications Medicine, 3(1):139, 2023.
Ma, Z. and Chen, G. Bayesian methods for dealing with missing data problems. Journal of the Korean Statistical Society, 47:297-313 -139 2018.
Mayer, I., Sportisse, A., Tierney, N., Vialaneix, N., and Josse, J. R-miss-tastic: A unified platform for missing values methods and workflows. The R Journal, 14(2): 244-266, 2022.
Stempfle, L. and Johansson, F. MINTY: Rule-based mod- els that minimize the need for imputing features with missing values. In Proceedings of the 27th International Conference Artificial Intelligence and Statistics, pp. 964 972. PMLR, 2024.
McTavish, H., Donnelly, J., Seltzer, M., and Rudin, C. In- terpretable generalized additive models for datasets with missing values. In Advances in Neural Information Pro- cessing Systems 37, 2024.
Stempile, L., Panahi, A., and Johansson, F. D. Sharing pattern submodels for prediction with missing values. In Proceedings of the 37th AAAI Conference on Artificial Intelligence, pp. 9882-9890, 2023.
Miyagi, Y. Identifying group a streptococeal pharyngitis in children through clinical variables using machine learn- ing. Cureus, 15(4), 2023.
Tibshirani, R. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267-28) 1996.
Paszke, A., Gross, S., Massa, F, Lerer, A., Bradbury, J. Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., al. PyTorch: imperative style, high- performance deep learning library. In Advances in neu- ral information processing systems 32, 2019.
Tietz, M.. Fan, T. J., Nouri, D., Bossan, B., and skorch Developers. skorch: A scikit-learn compatible neural network library that wraps PyTorch, 2017.
Twala, B. E., Jones, M., and Hand, D. J. Good methods for coping with missing data in decision trees. Pattern Recognition Letters, 29(7)9500-956. 2008
Pearl, J. Probabilistic reasoning in intelligent systems: Networks of plausible inference. Elsevier, 2014.
Van Buuren, S. Flexible multivariate imputation by MICE. Technical report, TNO Prevention and Health, 1999.
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V.,
Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P.,
Weiss, R., Dubourg, V., et al. Scikit-leam: Machine learning in Python. Journal of Machine Learning Re- search, 12:2825-2830, 2011.
Van Ness, M., Bosschieter, T. M., Halpin-Gregoorio, R., and Udell, M. The missing indicator method: From low to high dimensions. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp.
Quinlan, J. R. Induction of decision trees. Machine Learn- ing. 1:81-106, 1986.
2023.
van Smeden, M., Groenwold, H., and Moons, G. A cautionary note on the use of the missing indicator method for handling missing data in prediction research. Journal of Clinical Epidemiology, 125:188, 2020.
Quinlan, R. C4. 5: Programs for machine learning. EI- sevier, 2014.
Razavi, P., Chang, M. T., Xu, G., Bandlamudi, C., Ross, D. S., Vasan, N., Cai, Y., Bielski, C. M., Donoghue, M. T., Jonsson, P., et al. The genomic landscape of endocrine-resisttat advanced breast cancers. Cancer Cell, 34(3):477443 2018.
Verwer, S. and Zhang, Y. Learning optimal classification trees using a binary linear program formulation. Pro- ceedings of the 33rd AAAI conference on artificial intel- ligence, pp. 1625-1633 2019.

---

Prediction Models That Learn to Avoid Missing Values
Weiner, M. W., Aisen, P. S., Jack Jr, C. R., Jagust, W. J., Trojanowski, J. Q., Shaw, I.., Saykin, J., Morris, J. C., Cairns, N., Beckett, L. A., et al. The Alzheimer's dis- ease neuroimaging initiative: Progress report and future plans. Alzheimer's 's & Dementia, 6(3):202-21,, 2010.
World Health Organization (WHO). Global Health Es- timates: Life expectancy and healthy life expectancy, 2021.
12

---

Prediction Models That Learn to Avoid Missing Values
A. Proof of Propositions 2 and Corollary 1
Proposition 1 (Restated). Let a set of ODDC rules hold under a distribution p(X, M). Any tree that satisfies will have missingness reliance p(h) under p.
Proof. We will prove the proposition by induction on the size (number of nodes) n of the decision tree h.
BASE CASE: TREE SIZE 0 (TREE WITH ONLY A RoOT NODE)
Let h be a decision tree with size 0 that obeys the ODDC rules R. The decision tree h consists solely of root node and it is also a leaf node. Since there are no splits, no features are used in computing h(x). Therefore, for any observation X, p(h, x) = O. Consequently, the expected missingness reliance is:
p(h) = E, [o(h, X)] E0 = O.
Thus, the base case holds.
INDUCTION STEP: SIZE d
Assume that for any decision tree h of size d that satisfies the ODDC rules in R. the expected missingness reliance satisfies:
p(h) 0.
Now, consider a decision tree h' of size d + 1, built from tree h of size d by adding single split (i, to leaf node i E C(h). Assume that h' also satisfies R. Since h satisfies the ODDC rules, p(h) O by the induction assumption.
By Equation (3), splitting a leaf E [hh) using feature and threshold (j. T) can only introduce missingness reliance p(h') > if the feature zy is missing for some imput x such that é (x)) (E is on the decision path for æ f).. However, splitting on feature j could lead to 2 na for any such æ, the resulting tree h'' would not satisfy R, and we would have contradiction. Given that h satisfies R. must be that
Vo f TT(x): p(M, = 1 X x) =0
Thus, the new tree h also has zero missigness reliance, p(h') 0. Since the base case holds and the inductive step has been proven, by mathematical induction, by the inductive hypothesis, for trees h of any size n that satisfy the ODDC rules R. the missingness reliance is p(h) = O.
Corollary (Restated). Consider a distribution p(X, M, Y). Suppose the outcome Y satisfies a set of ODDC rules R in Then, there exists a prediction model with minimal risk and zero missingness reliance, p(h) = 0, on p.
Proof. We can prove the corollary constructively by defining a prediction model with minimal risk on p. For any loss function J., there is (Bayes optimal) classifier ht" which minimizes the expected risk over by making point-wise optimal predictions,
Vo h (x) arg min E[L(y, Y) X = =r),
vr(va=)>0
and therefore,
arg min Ex [ [((x) X]] h² h:X
For the squared loss function L(y.v') (y v22 h is the conditional expectation, h h²(x)) E X=
Since Y satisfies R on p by assumption, the function f(x) = p(Y = = æ)) itself has missingness reliance p(f) 0 on p. Since ht can be computed deterministically from p(Y = y X p(h*) = 0. Therefore, ht has both minimal risk and zero missingness reliance on p.
Note on the Hypothesis Class. In Corollary 1, the optimal model h- = E|Y X] is implicitly assumed to belong to the hypothesis class H. While this assumption benign for universal function approximators like decision trees, becomes more consequential for more restrictive model classes, such as linear models, where h² may lie outside H.
13

---

Prediction Models That Learn to Avoid Missing Values
A Note Convergence. If optimal model h" exists, can proposed algorithms recover it given enough data? The answer depends the model class, the optimization procedure, and the data distribution. For instance, if h is sparse logistic model, then solving the optimization problem (5) using the L L'-based regularization is likely to recover it-similar to how standard Lasso behaves.
Proposition 2 (Restated). Assume that each feature is independently missing completely at random with probability Pj.
p(Mj X, M-,) -
Then, for any model h, with as (2r, {0, 1 indicating whether feature vi is used in the prediction ((x)),
p(h) 2 max Max.)oo(Xx))pp
Proof. By definition, have that
p(h) E|max an [X,, =na]].
It follows that
p(h) 2 و (Xx 1]]
= maxxElas (X) 1]
max
J
where the second row follows due independence between Mj and X.
B. Missingness--voidiing Gradient Boosting
In Algorithm 1, we present the algorithm used to fit MA-GBT.
Algorithm Missingness--voidingg Gradient Boosted Trees (MA-GBT)
1: inputs Data set D = {(x, 3- learning rate 7 2 0, regularization parameter a, loss function L
2: inputs Ensemble size M
3: co(x:) arg min. Σ L(y..c) c)
4: Wi E [m], [ Fij
5: for m 1, ................................ M do
6: Compute residuaresiduals:
Vi E [n] Tim = - To ((x)))
de(m;)
=(=))=m--((x)
7:
Fit MA regression tree Itm D. { (0i Fim )mm)) K- using the splitting objective in (4) with regularization o,
8:
sample multipliers (1o,, -= Update ensemble:
in and mean squared error the splitting criterion C'
9:
10:
(x) = mm=)( + (2)m)))
11:
Update reliance weights:
12:
Vi [m].j E [d] oij = oss (1 - 1lj E The (x:)) [ = nal)
end for
C. Experimental Details
In this section, we provide additional details about our experimental setup.
C.1. Datasets
In Table 2, we provide an overview of the datasets used in our experiments. Each dataset is described in more detail below. The Life, Breast Cancer, and Pharyngitis datasets are available under Creative Commons license. The FICO dataset
14

---

Prediction Models That Learn to Avoid Missing Values
subject to its own licensing terms, which can be accessed at: challenge. The ADNI dataset requires agreement to the data use policy available at: ADNI Data Use Agreement. The NHANES dataset is governed by the data release policy accessible here: NHANES Data Release Policy.
For preprocessing the datasets, we followed the structure from Shadbahr et al. (2023). For the synthetic missingness experiment described in Section 6.3, we adopted the approach from Mayer et al. (2022). Additional preprocessing details can be found on our GitHub project page: https: /tt//gttuuuuu wwt Hooo/oeeelhhyaaa/.a..aaaaaeaaall AI/ /I/malearn.
Table 2. Overview of the datasets used in our experiments. Each dataset corresponds to a binary classification task.
Dataset
Samples
Features
Prediction Task
NHANES (Johnson et al., 2013)
69,118
42
Hypertension
LIFE (World Health Organization (WHO), 2021)
2,864
18
Life expectancy above/below dataset median
ADNI (Weiner et al., 2010)
1,337
39
Diagnosis change over two years Hormone receptor status
Breast Cancer (Razavi et al., 2018)
1,756
16
Pharyngitis (Miyagi, 2023)
676
18
Outcome of rapid antigen detection test
FICO (FICO et al., 2018)
10,549
23
Credit repayment
The National Health and Nutrition Examination Survey (NHANES) (Johnson et al., 2013) is a cross-sectional study designed to assess the health and nutritional status of the U.S. population. It combines interviews, physical exam- inations, and laboratory tests to provide a comprchensive dataset for epidemiological research. While NHANES is not inherently structured for a specific machine learning task, it has been widely used in predictive modeling studies (Gard- ner et al., 2023). In this work, NHANES was utilized to develop models for predicting hypertension, where the outcome variable (Y) wa based on clinical blood pressure measurements. The predictor variables (X) included demographic infor- mation such as age, sex, and ethnicity, examination features like BMI and waist circumference. laboratory values including cholesterol levels and blood glucose, as well as irestionnaiirebaseed factors such smoking status and physical activity. Missingness in NHANES arises due to various factors, including participant refusal, ineligibility, and incomplete examina- tions. Participants may choose not to answer survey questions or refuse to undergo certain medical tests. Some components are only administered to specific demographic groups, such as dietary recalls that apply only to adults, resulting in struc- turally missing values. Additionally, missing data may occur due to technical issues or participant dropout, making certain variables unavailable for some individuals. Notably, missing values due to ineligibility are deterministic, meaning they cannot be reliably imputed as they are structurally absent in specific subgroups. For our experiments, we sampled 10,000 observations with equal distribution of positive and negative cases.
NHANES.
Life Expectancy (WHO). The Life Expectancy dataset, sourced from the World Health Organization (WHO) (World Health Organization (WHO), 2021) and available at https: / http://wwwwwt///// kaggle. com hatm///aassss////e hattss/lssaagohhh aggregates country-level health indicators spanning from 2000 to 2015. It in- cludes a diverse range of demographic, economic, and health-related variables that are used to predict life expectancy through both regression and classification tasks. This dataset comprises 2864 records across 179 countries and con- tains indicators such as population growth, fertility rates, and median age as demographic variables, GDP per capita and government healthcare expenditure as economic indicators, and immunization coverage. prevalence of discases, and mortality rates as health system data. In this work, we created a classification task to determine whether coun- try's 's life expectancy is above or below the dataset median. Starting from the complete dataset obtained via the linked Kaggle project, we followed the preprocessing steps described in (Stempfle et al., 2023), introducing 10 % MCAR missingness across features. Observations with missing country information were excluded, as the train-test split was performed with respect to this variable (which was not included in the feature set). Missing values in the Region variable were imputed based on the corresponding country. The final dataset is available in our GitHub repository: https: //ggttttt/buu// com/ Hee//Halllee Heealthy-AI/malle
Alzheimer's Disease Neuroimaging Initiative (ADNI). The Alzheimer's Disease Neuroimaging Initiative (ADNI) (Weiner et al., 2010) is a longitudinal study designed to collect and standardize clinical, neuroimaging, and genetic data to support the early diagnosis and monitoring of Alzheimer's disease progression. The dataset provides detailed pa- tient assessments at multiple time points, making it a valuable resource for predictive modeling. In this work, baseline data from sample of 1,337 patients was used to construct a classification task: predicting whether a patient's diagnosis will
15

---

Prediction Models That Learn to Avoid Missing Values
change two years after the baseline assessment. The selected dataset includes clinical data (e.g.. cognitive test scores and medical history), neuroimaging data from MRT and PET scans (measuring brain atrophy and amyloid-beta accumulation), and genetic information, including APOE genotype, which is strongly associated with Alzheimer's is risk. Missingness in ADNI arises from inconsistent follow-up visits, participant withdrawal, and variability in the availability of imaging or genetic tests. Since not all participants complete every assessment at each time point, missing data can significantly impact longitudinal analyses.
Breast Cancer. The Breast Cancer dataset, introduced by (Razavi et al., 2018). 1). focuses on predicting hormone receptor (HR) status and survival time using molecular and clinical features. In this work, we focused on a classification task aimed distinguishing between HR-positive (HR+) and HR-negative (HR-) tumors, as HR status is a key biomarker guiding treatment peccisions-partiuune
the use of endocrine therapies. This task leverages features such as estrogen receptor (ER) and progesterone receptor (PR) status, HER2 expression, and mutation burden. The dataset contains 1,711 samples and 18 features, with a class distribution of 1,415 HR- HR-positive and 296 He-negative cases. Three variables exhibit missing values: "ER PCT Primary" (185 missing entries), "Fraction Genome Altered" (22), and "Mutation Count" (75), We followed the preprocessing approach described in McTavvsh et al. (2024), including only a single row per individual patient.
FICO. The FICO Explainable Machine Learning Challenge dataset (FICO et al., 2018) contains anonymized credit data from Home Equity Line of Credit (HELOC) applications, with the primary objective of predicting whether an individual will repay a line of credit within two years. The dataset includes 10,459 applicants and 23 financial attributes derived from credit reports, such as credit utilization ratios, payment history, and debt-to-income ratios, making valuable dataset for credit risk assessment. A unique feature of this dataset is its encoding of missing values through three distinct codes: -7 denotes missing information of a specific type, -8 indicates the complete absence of usable information, and -9 signifies cases where a credit bureau report was either not retrieved or not found. These encodings reflect real-world financial uncertainty and require tailored imputation strategies. The presence of structured missing values poses an additional challenge in developing models that are both interpretable and robust in assessing credit risk. In our preprocessing of the data, we replaced all the different missingness codes with na.
Pharyngitis. The Pharyngitis dataset, introduced by Miyagi (2023), consists of 676 patients with 19 measured features and was used to predict the outcome of rapid antigen detection test (RADT) in children with pharyngitis. Specifically, the outcome was the presence of group A streptococcus (GAS) on throat culture. The predictor variables include age, sudden onset of sore throat, maximum body temperature (as reported by the accompanying parent), throat pain, cough, rhinorrhea, conjunctivitis, headache, pharyngeal erythema, tonsillar swelling, tonsillar exudate, palatal petechiae, nausea and/or vomiting, abdominal pain, diarrhea, tender cervical lymph nodes, and the presence of a scarlatiniform rash. The number of missing values ranged from 0 % to around % per predictor variable.
C.2. Hyperparameters
The hyperparameters we considered are shown in Table 3. We performed an exhaustive search over the hyperpa- rameters for all models except MA-RF, MA-GBT, MINTY, and XGboost, for which we randomly sampled 10 dif- ferent sets of hyperparameters. Note, in our implementation of MA-LASSO X, (2immaett a, where 3 [0.001, 0.01, 0.1, 10, 100, 1000].
C.3. Missingness Reliance Metric for Each Estimator
The missingness reliance metric quantifies a model's dependence on missing values during inference, providing an estinator-specific yet comparable measure. We define missingness reliance for all MA models in Section 4. For tree- based baseline models (DT, RF, and XGBoost), this reliance is defined in the same way as for the corresponding MA model. For linear baseline models, such as LR and M-GAM, reliance is determined by the use of missing features with non-zero coefficients, i.e., in the same way as for MA-LAS NeuMiss explicitly incorporates missingness through mask non-linearity and specialized blocks, with reliance assessed by the proportion of samples requiring such mechanisms. In MINTY, reliance is defined the rule level, indicating whether missing attribute influences a given decision.
16

---

Prediction Models That Learn to Avoid Missing Values
Table 3. Hyperparameters and their respective search space for all models.
Model
Hyperparameter
Search space
LR
regularization strength
{0.1, 1.0, 2.0, 10.0)
DT
max depth
{1,2, 3, 4, 6, 7,
depth
(3,4,5,6,1 8,9
RF
samples to split
(0.05,0.10) 0.220.,0..
M-GAM
lambda 0
0.1, 0.05, 0.02, 0.01, 0.005)
max depth
{3. 6,,,,,677,8,9
XGBoost
learning
(0.01, 0.13
number of estimators
[100, 200, 300, 400,
lcarning rate batch size
{0.001, 0.01, 0.13
NeuMiss
{32, 64, 128)
optimizer max rules
["beam"]
{10, 15)
MINTY
lambda O
{1.0 10 1.0 10-5 0.001,0.01.0001111
lambda
{1.0 10-6, 1.0 10-5.0.001, 0.01,
gamma
1.00001,,.01,, 0.0.0,,.....1110000000 100,0
alpha
10, 100, 1000, 10000) {0.001, 1,, 100, 1000)
MA-LASSO
beta
max depth
{1,2,3,4, 4,5, 6, 7,8,9} 0,000,0.011001,111100
MA-DT
alpha
number of estimators
(50}
MA-RE RF
max depth alpha
11,2,3,4,5,6,7 6,
(0.001, 10)
number of estimators
{10}
loss
(olog-loss")
MA-GBT
lcarning rate max depth
0.01
(1,2,3,4,5, 4,5,,,4,,,,,77 6, 0,001,,001,,.111
alpha
C.4. Implementations
For Neumiss we used PyTorch (Paszke et al., 2019) in combination with skorch (Tietz et al., 2017). Model parameters were optimized using the cross-entropy loss and the Adam optimizer with default parameters. Early stopping was applied during training if there was no improvement in performance for 10 consecutive epochs. The logistic regression model, the decision tree classifier and the random forest classifier were implemented using the scikit-learn library (Pedregosa et al., 2011). For XGBoost, en stssstallee e pttph//ttoo/ooeepp hocichooninnee inwex/..e....///// html), For MINTY and M-GAM, we adapted code provided by Stempfle & Johansson (2024) and McTavish et al. (2024). Our code is available at https: httpt/gtttttt//tt////////tt com Hooomm/eeahhyyaaa Awtchay-AAAmmaaaaa AI/malearr
used the XGBoost Python Package (see https:/ /xgboost readthhocss /
NODE SPLITTING CRITERION FOR TREE-BASED MODELS
In our tree-bases models, we performed node splitting based on the Gini criterion, defined as:
1 [y4 =
C(E, Σ p. p.).
p
Σ
Sel
iES
for the general case with classes. In case, 2. For definitions of additional notations, please refer to Section 4.
D. A Additional Results
In this section, we provide additional results from our experiments. Table presents the same results as Table 1, but using MICE imputation instead of zero imputation. Table 5 shows results using both imputation techniques on two additional datasets: Breast Cancer and Pharyngitis. Table and Table 7 report the performance of MA models under different settings
17

---

Prediction Models That Learn to Avoid Missing Values
Table Test-set AUROC and missingness reliance (p) across four datasets using MICE imputation. The % confidence intervals are based on the bootstrap distribution of the respective metric. For each dataset and MA estimator, selected part of the overall model selection strategy, with the model minimizing ý chosen among those achieving at least 95 % of the maximum AUROC.
ADNI
FICO
LIFE
NHANES
ESTIMATOR
AUROC (1)
(())
AUROC (f)
(C.)
AUROC (1)
(()
AUROC (1)
P(L)
LR
62.9
79.1 (78.3.
(744
99.0
54.9 57.11
85,
100.0
DT
69.8 (67.6.222. 72.11
34.6.25.0, 0.6-2250,44. 44.6)
73.7 (4.50
5.7 (5.466.)
91.0 (87.9, 94.4)
15.8 (:2.0. 19.4)
82.4 (81.7, 83.0)
(00,0.0)
RF
65.9 (64.0
77.5 (7..9.8.31
71.0
97.9 (96.4,
83.6 (52.0.
82.3
100.0
M-GAM (DND)
69.7 (66.9. 72.11
14.7 (0.5. 32.5)
77.9 (27.8.78.3)
42.4 (23.6, 56.2)
98.2 (96.4, 99:2)
49.4 (42.8. 56.1)
84.0 (83 64.6)
28.2 (9.5, 50.01
M-GAM
66,5
46.3 (22.5,
(77.3.77..77)
29.1
98.1
51.7
84.1 64.5)
26.4 (14.8.48.3)
XGBooST
79.0. 82.4)
60.9 (57.6, 60.4)
(27.9 79.0)
69.3 74.9)
98.2 (96. 90-4)
56.1 (52.3, 61.1)
84.8 (84 k3.2)
95.7 (87.8, 100
NEUMISS
76.4 (75.0.
67.0 (648, 70.1)
79.0
75.4 (14.9,
98 (97 99.4)
84,1 (53.0, 85.33
85.5
100. (100 100.0
(6.6, 15.88
(.55
76.6 (75.2,
43.6)
(94 98.3)
(16.8, 27.0)
MA-LASSO
67.4 (65.7.6888
0.4 1.0
75.8 (74.9.664
5.7 (5.4
98.0
19.5
84.5 (84.0,
0.4 0.0)
74.0 75.9)
(..0.0.73
73.7 (75.0) 74.5)
6.0)
91.2 (86. 94.7)
10.5 (10.2.
K3.2)
MA-RF
78.0 (76.2,80)) 80.3)
3.0 (0.6. 7.5)
76.0 (75.4. 76.4)
(S... 6.11
97.5 (96. 98.1)
24.6 (30.9, 28.2)
84.0 64.6)
0.2 (0.0, 0.5)
MA-GBT
774.1.79)))
75.7 (7..5.77.3)
95.01
21...
84.1)
0.4 1.3)
0.8
AUROC
0.7
0.0
0.6
0.1
L.
10.0
3
6 Max. tree depth
4
7
8
9
Figure 5, LIFE: AUROC vs. max. tree depth.
of the missingness reliance parameter a. Figure illlssrates the trade-off between AUROC and missingness reliance (ø) when varying the maximum allowed tree depth LIFE, across different a values. Finally, Figure shows example LIFE under different settings.
18

---

Prediction Models That Learn to Avoid Missing Values
Table Test-set AUROC and missingness reliance (p) across two datascts, Breast Cancer and Pharyngitis, using zero (far) and MICE (1x) imputation. The 95 % confidence intervals based the bootstrap distribution of the respective metric. For each dataset and MA estimator, a is selected as part of the overall model selection strategy, with the model minimizing f chosen among those achieving least 95 % of the maximum AUROC.
BREAST CANCER
PHARYNGITIS
ESTIMATOR
AUROC (1)
(4)
AUROC (t)
(4)
LR (To)
75.0 (73.4, 76.6)
29.5 (17.8, 41.3)
(68.1, 72.4)
15.9 (12.8, 20.0)
LR (IM)
75.8 (7..2.77.4.
(39.0. 41.9)
72.4)
15.9 (52.,
DT (lb)
70.6 (2.85.72.88
0..0, 0.0)
65.5 67.8)
(4.7,,
DT (IM)
70.0 (67.5,
(0.00,
63.6 68.1)
74.6, 10.9)
RF RF (Jo)
74.3 (72.1,
(41.8, 43.8)
73.7 (72.1, 75.0)
25.0 (22.5, 27.2)
RF (IAs)
74.5 (72.5,
(41.8, 43.8)
(72.2, 74.7)
25.3 (22.6, 27.8)
M-GAM (IND)
71.7 (69.3, 74.0)
(70.2,
15.6 (12.9, 19.0)
M-GAM (INT)
72.5 (69.4, 75.7)
(0.5,
71.3 (70.2.211.))
13.7 (12.8, 15.0)
XGBOOST
74.7 76.5)
(41.6, 43.6)
43.6)
(69.8,
19.9 (15.9, 24.1)
NEUMISS
75.5 (73.5, 77.4)
42.6 (41.8, 43.8)
(71.2 75.5)
25.7 (22.8, 28.2)
MINTY
70.9 (68.8, 73.9)
32.6 (3.11,
69.7 (68.2, 70.9)
11.8 (9.1, 13.8)
MA-LASSO (lo)
71.6 (69.7,
0.0 0.0)
18.8 (16.6, 20.4)
MA-LASSO (Ix)
(68.5, 74.1)
8.4 (0.0, 25.2)
70.2 (67.7,,7,2
186.6 20.4)
MA-DT (To)
70.6 (68.6,
0.00, 0.0)
(63.2,
(2.4,666
MA-DT (!M)
70.6 (68.6,,,2..
0.000 0.0)
(63.2, 69.6)
4.3 42.4, 6.2)
MA-RF (Io)
72.2 (69.7,7777))
(011,
(69.9,,
14.1 (12.6, 15.7)
MA-RF (IM)
72.2 (69.7,74.7)
(.11,.1
71.4 (69.5,
(12.6, 15.7)
MA-GBT (Io)
70.8 72.11,
0.6 00.0, 1.2)
68.2 (65.8, 71.0)
6.8 8.8)
MA-GBT (Iar)
71.2 (69.3,
0.7 (0.0,
68.2 (65.8, 71.0)
6.8 (4.7,
Table Test-set AUROC and missingness reliance (p) across four datasets using zero imputation. 95 confidence intervals are based the bootstrap distribution of the respective metric. For each dataset and MA estimator, consider three different values for the missingness regularization parameter a, where a = 00 and a a are selected as part of the overall model selection strategy. For select that is large enough result in near-zero missingness reliance (Ò z= 0) while maximizing AUROC. For a", we select the model minimizing و chosen among those achieving at least 95 % of the maximum AUROC. Finally, a effectively means missingness regularization.
ADNI
ESTIMATOR
AUROC (
p(l)
AUROC (1)
p()
AUROC (1)
AUROC (f)
P())
50.9)
ao)
(50.0. 71.3)
(50.0,3..4
(.2.0.00.3)
MA-LASSO o')
68.4 (86.1,
11.8 (0.1. 34.7)
75.8 (749. 16.5)
5.7 (5.4. 60)
97.7 (96.7. 98.01
(1.0. 24.1)
84.5 (84.0. 85.0
0.4 (0.00.9)
MA. MA-LASSO (a
69.0 (6.3..63))22.3)
66.5 (64.) 58.6
76.4 77.4)
(148. 75.91
98.7 (90.7. 99.4)
84.1 (82.0 85.2)
77.5 (76.7. 78.2:
100.0 c1000, 100.0)
(65.3, 15.3)
(0.00,
(349, 16.4)
(5.A. 6.)
(67.1. 76.81
(83.5. 84.31
MA-DT (a )
74.1 (71.7.
(0.,
73.8 (33.0, 14.5)
5.7 (5.4. 5.0)
89.7 (87.5. 91.81
12.3 (10.4. 15.3)
82.1 (81.6. 83.2
0.0 0.00
MA-DT (o
72.7 01..244
11.7 18.8)
(3.8.03.0 14,5)
5.7 15A.
92.2 (9..1. 91.81
18.3 (15.8. 19.9)
82.3 (81.5. 81.00
0.1 (0.0,0.3
(o
(6866 75.8)
(0.3, 1.5)
(3..). 16.7)
15.A 6.0) 5.8 (5.5,
(67.1. 87.91
15.8)
(80.4, 84.21
MA-RF (a = =a')
78.0 (36.2, 50.4)
3.2 (0.3.
76.3 (15.. 16.5)
97.2 (05.7. 98.31
24.2 (22.0. 25.5)
84.0 (83.3. 84.60
0.2 (0.0, (0.0.0.5)
MA-RF (o 0)
78.4 30.5)
$7.5 (52.1, 62.8)
75.7 (4.8. 16.5)
(K6. 15
95.2 (91.6. 96.4)
20.8 23.80
84.2 (81.5.
0.4 2,
(64.7, 72.3)
(5.4
(91.1.
(79.2. 82.7:
MA-GBT (a a²)
78.5 (36., 30.5)
1.6 (0.), 2.7)
(3..5.4..77 77.1
5,9 15.55.5. 6.2)
95.9 (92.1. 98.91
16.5 (11.9. 21.2)
83,2 (82.1. 84.4)
0.3 (0.0.
MA-GRT (a
(713.713.766.
26.6
74.8 (31.8. 36.3)
1.A.
92.6 (90.3,
19.4 (18.8.88...... 19.9)
82.7 (81 9.87
0.0 oo0.con....
19

---

Prediction Models That Learn to Avoid Missing Values
Table 7. Test-set AUROC and missingness reliance (ÿ) across two datasets, Breast Cancer and Pharyngitis, using zero imputation. The 95 % confidence intervals based on the bootstrap distribution of the respective metrie. For each dataset and MA estimator, consider three different values for the missingness regularization parameter a, where oo and = a" are selected as part of the overall model selection strategy. For oo, we select a that large enough to result near-zere missingness reliance (p 0) while maximizing AUROC. For = a* we select the model minimizing ý chosen among those achieving at least 95 % of the maximum AUROC. Finally, a effectively means no missingness regularization.
BREAST CANCER
PHARYNGITIS
ESTIMATOR
AUROC (1)
(
AUROC (t)
MA-LASSO (a = )0)
71.9 (70.3, 74.3)
0.0 (0.0, 0.0)
54.9 (51.5, 58.2)
0.0 (0.0, 0.0)
MA-LASSO (a =
71.6 (69.7, 74.2)
(0.0, 0.0)
70.2 (67.7, 72.1)
18.8 (16.6, 20.4)
MA-LASSO (a 0)
76.0 (74.5, 77.5)
42.6 (41.8, 43.8)
70.4 (68.2, 73.1)
25.7 (22.8, 28.2)
MA-DT (a = 60)
70.6 (68.6, 72.6)
0.0 (0.0, 0.0)
54.6 (50.8, 58.9)
0.0 (0.0, 0.0)
MA-DT a*)
70.6 (68.6 6, 72.6)
0.0 0.0)
66.6 (63.2, 69.4)
4.3 (2.4, 6.2)
MA-DT (a 0)
70.6 (68.6, 72.6)
0.0 (0.0,
65.6 (63.0, 68.1)
7.8 (4.6, 10.9)
MA-RF (a = 00)
69.6 (67.5, 71.8)
0.1 (0.0, 0.3)
60.2 (55.3, 65.0)
1.6 (0.3, 3.1)
MA-RF (a a*)
72.2 (69.7, 74.7)
0.6 (0.1, 1.5)
71.7 (69.9, 74.0)
14.1 (12.6, 15.7)
MA-RF (a = 0)
73.9 (72.7, 75.1)
6.4 (2.6, 10.1)
69.0 (66.0, 71.2)
12.1 (9.6, 14.1)
MA-GBT (a = 00)
69.6 (67.5, 71.6)
0.0 (0.0, 0.0)
57.6 (51.8. 65.2)
1.2 (0.0, 2.6)
MA-GBT (a a')
70.8 (69.1, 72.1)
0.6 (0.0, 1.2)
68.2 (65.8, 71.0)
6.8 (4.7, 8.8)
MA-GBT (a = 0)
71.4 (69.2, 73.3)
1.0 (0.0, 2.1)
68.6 (66.5, 70.5)
9.3 (7.8, 11.2)
20

---

Prediction Models That Learn to Avoid Missing Values
MA-DT with (AUC 0,90, 0.18)
MA-DT with a' (AUC 0.90, 0.12)
Aduls exoriality 167.32 Snapiee: 100,0%
Adult mortally 167.52 32
Samples: 10000%
Pt(LE median)
Pr(LE medianl
Trun
Falso
Trne
Fal
Infant deatlis 28.79
Under deaths 19.70
Adult mocealliy 100.50 Sampless 48, 48.0% Pr(LE > median) 0.50
Region Europeon Union
Sumples 40.0%
Samples 51.0%
Sumples 51.065 Px(LE: > molian) 0.17
PxILE > mollan) 0.50
Pr(LE moliun) 0.17
0.09
0.28
0.00
0,20
(...)
(-)
6--)
(..))
{..)
(-))
(...)
(-)
(a) a
(b) = a
MA-DT with a oa (AUC = 0.67, , 0.00)
Region Africa
Sumples: 100.0%
Pr(LE > median) = 0.53 0
True
False
Samples: 73.4%
Sumples: 26.6%
Pr(LE median) 0.67 A = 0.00
Pr(LE median) 0.12 = 0.00
(c) 00
a², and a fit to LIFE, The nodes are colored hased on the missingness reliance The goal is to predict whether a country's 's life expectancy (LE) is above or below the median life expectancy. (a): MA-DT with 0 behaves hetween 15 and 60 years per 1000 population) and Infant_deatha. (number of infant deaths per 1000 population) (b): MA-DT with selected to balance the trade-off between predictive performance and AUROC. Similar the standard decision tree, MA-DT first splits on adult_mortal ality at the root node, but then splits by the region corresponding to the Europcan Union, reducing the missingness reliance for the overall tree. (c): With large a, the tree is built with missingness reliance, only splitting by the region "Africa". While this tree achieves zero missingness reliance, it performs significantly worse than the others in terms of predictive accuracy.
Figure 6. Example decision trees for
regular decision splitting highly predictive features such
(probability of dying
21


Reviews:
[{"review_id": "E2meZMt7ZE", "reviewer": "Reviewer_dApi", "summary": "The paper proposes a novel framework – “missingness-avoiding” (MA) machine learning – designed to mitigate the impact of missing data during prediction. The core idea is to train models that inherently minimize reliance on features with missing values at test time. This is achieved by incorporating classifier-specific regularization terms into the learning objectives for decision trees, sparse linear models, and ensemble methods.  The authors demonstrate this approach on several datasets, showing improved performance compared to standard imputation techniques while maintaining interpretability", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful feedback and appreciate their recognition of our well-motivated and novel MA framework. We have addressed all suggestions to improve clarity and respond to specific comments below.\n\n**Re: relevance of MA learning**\n\nWe would like to direct the reviewer to Sec. 5, where we outline when MA learning is expected to perform well (Sec. 5.1) and when it may face challenges (Sec. 5.2)—a discussion also highlighted as a strength by Reviewer go58. MA learning is expected to work well when the missingness mask follows a clear structure,  such as when ODDC rules govern the data-generating process. When there is no structure in the missingness patterns related to observed features, such as in an MCAR or a fully MNAR setting, we expect it to be harder for MA learning to achieve low missingness reliance with maintained accuracy. As the reviewer noted, our experiments show that MA learning outperforms baselines across datasets while greatly reducing reliance on missing values. Finally, MA learning may benefit from being combined with missingness indicators when missingness itself is highly informative—the reliance penalty will just make sure to keep such features to a minimum. \n\n**Re: guidelines for practitioners**\n\nTo apply the MA framework in practice, we recommend starting by selecting a model class that aligns with the interpretability requirements of the application. Linear models and decision trees offer greater transparency, while ensembles may yield better performance with reduced interpretability. Once the model type is chosen, the trade-off between predictive performance and reliance on missing features can be adjusted through the regularization parameter $\\alpha$. A higher $\\alpha$ encourages the model to avoid relying on features that are often missing at test time, while a lower $\\alpha$ prioritizes performance. The appropriate choice of $\\alpha$ depends on how much missingness reliance is acceptable in the given context. We describe different strategies in the response to Reviewer eoMs. To evaluate this, practitioners can compute the reliance score ($\\hat{\\rho}$), which quantifies how frequently a model depends on missing features. Importantly, MA learning requires no assumptions about the underlying missingness mechanism (although the achievable accuracy-reliance tradeoff will be affected by its structure), making it a practical and robust approach across a wide range of real-world settings.\n\n**Re: qualitative analysis**\n\nWe agree that qualitative analysis of MA trees is valuable. In Section 6.2, we refer to Figure 6 (appendix), which shows how MA-DT behavior varies with different values of the missingness regularization parameter $\\alpha$ on the Life dataset. For instance, when $\\alpha$ is tuned to balance accuracy and reliance on missing values, MA-DT reduces reliance on missing data by 33% without sacrificing accuracy compared to a standard decision tree. For the camera-ready version, we will consider including these figures in the main text and adding MA tree examples from the ADNI dataset. These illustrate how MA learning favors complete features over those with missingness, enhancing interpretability without compromising accuracy at test time.\n\n**Re: theoretical foundations and stacked models**\n\nThank you for the insightful comment. We agree that strengthening the theoretical foundations is important. As for stacked models, we're unsure how they would address the challenges in our setting. Stacking typically involves training a meta-model on the outputs of several base models (e.g., decision trees, logistic regression). The reviewer suggests that these may be trained using complete data, but this is not generally available without imputation, which is what we try to avoid in the first place. Alternatively, we could train base models on different data subsets based on their missingness pattern, but this poses the same problem—it seems to us that the stacking question is orthogonal to reducing missingness reliance, but could be an alternative to boosting. We’re very open to considering this direction and would greatly appreciate any additional insights the reviewer can share on their suggestion.\n\n**Re: clarification on MA learning**\n\nMany model classes have large Rashomon sets—sets of near-optimal models that have similar predictive performance but differ on other properties. By adding the missingness reliance penalty, we prioritize models within this set that rely less on missing values. This can exploit nonlinear patterns (as with MA trees, where reliance is contextual) or linear ones (as with variable selection in MA-LASSO). We will include this in our Section 5 discussion.\n\n**Re: performance metrics**\n\nWe agree that multiple metrics are important to capture different aspects of performance. Due to space constraints, we focused on AUROC and missingness reliance in the main text and will consider including additional metrics (e.g., F1 score) in the appendix.", "claims_and_evidence": "The central claim – that models can be trained to actively avoid using missing values – is clearly articulated in the abstract and introduction. However, the level of detail regarding when this avoidance is feasible and effective isn’t consistently clear throughout the paper. The evidence supporting this claim is primarily demonstrated through experimental results across multiple datasets. While the quantitative performance improvements are notable (especially with MA-DT), the paper could benefit from a more rigorous discussion of the conditions under which these gains are most pronounced.", "methods_and_evaluation_criteria": "Methods: The authors’ method is well-defined, outlining the specific regularization terms applied to different model types (decision trees, sparse linear models, ensemble methods). The incorporation of classifier-specific regularization is a clever and theoretically sound approach. The use of MA-DT, MA-LASSO, MA-RF, and MA-GBT provides a good range of model implementations for comparison.\n\nEvaluation Criteria: The evaluation primarily focuses on AUROC (Area Under the Receiver Operating Characteristic curve) as a measure of predictive performance and missingness reliance. The use of cross-validation to assess model stability is appropriate. However, the paper could benefit from exploring other relevant metrics beyond AUROC, such as precision, recall, or F1-score, particularly when considering the implications for clinical applications where false positives and negatives have different costs.", "theoretical_claims": "The theoretical underpinning – that models can learn to exploit missingness patterns – is rooted in ideas related to Bayesian inference and the concept of minimizing expected risk. The paper does a reasonable job of connecting this with the idea of avoiding reliance on features with high missingness rates, but it could be strengthened by explicitly stating how the regularization terms are mathematically derived from these principles.  For example, it might be useful to compare this approach to simpler stacked models and use this to situate where MA-learning could be useful.", "experimental_designs_or_analyses": "Experimental Design: The experimental design is commendable, utilizing multiple datasets (NHANES, LIFE, ADNI, Breast Cancer, Pharyngitis) with varying characteristics to demonstrate the generalizability of the MA approach. The inclusion of zero imputation as a baseline comparison is also sensible.\n\nAnalysis: The analysis effectively presents the quantitative results, clearly showing the performance gains achieved by the MA models compared to standard methods. However, the paper could benefit from a more in-depth discussion of why these improvements occur. For example, are the MA models better at capturing non-linear relationships that are obscured when missingness is treated as simple imputation? A qualitative analysis of the decision trees generated by the MA-DT model would also be valuable. The paper could also benefit from a more thorough comparison to stacked models trained on complete data", "relation_to_broader_scientific_literature": "Overall the paper has sufficient literature review", "essential_references_not_discussed": "Most essential references are in the paper", "other_strengths_and_weaknesses": "The paper presents an interesting and potentially valuable concept with promising experimental results. However, the lack of clear guidance on when this approach is most effective, coupled with a somewhat superficial theoretical justification, warrants a \"weak accept\" recommendation. Further work is needed to solidify the theoretical foundations, provide more robust comparisons against alternative methods (particularly stacked models), and offer practical guidelines for practitioners on how to best apply this framework.", "other_comments_or_suggestions": "Please consider expanding on the theoretical framework, especially around when MA-learning is suitable.", "questions_for_authors": "See above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e766b8tWcp", "reviewer": "Reviewer_go58", "summary": "This work introduces a generic framework for encouraging models to avoid accessing missing values through regularization. Specific implementations of this framework for Lasso, greedy decision trees, and tree ensembles are introduced. A thorough discussion of the settings in which missingness can and cannot safely be avoided is included, followed by experiments showing that the level of reliance on missing features can be substantially reduced without sacrificing much predictive performance on several real world datasets.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their thoughtful and valuable feedback, including their appreciation of our methods, empirical effectiveness, and the helpful discussion. We have addressed all suggestions to improve clarity and respond to specific comments below.\n\n**Re: motivation of avoiding missing values (w4)**\n\nMA learning is designed for prediction tasks with missing data where interpretability matters—clinical risk scores are a key example. Standard approaches (imputation, missingness indicators, default rules) often yield models that rely on unavailable inputs, as shown by their high missingness reliance in our experiments, which can reduce interpretability. For example, it may be difficult to justify that a physician or model should guess the value of a missing test (if using imputation) or that a patient’s risk of readmission should go down because the test result is missing (if using indicators or default rules in trees)—especially if the test is irrelevant based on the values of other features, such as a diagnosis. Though such tests may be predictive in other contexts, many learning tasks are underspecified, and models with similar accuracy may differ in missingness reliance. MA learning favors models that use information that is both predictive *and* available. Finally, we argue that MA learning and reasoning about missingness patterns go hand in hand. Increasing the penalty $\\alpha$ encourages trees to split first on commonly observed features (e.g., demographics), and then on contextually available ones (e.g., tests for certain conditions). As discussed in Section 5 on ODDC rules, these observation patterns may be correlated with predictiveness, e.g., when the outcome reflects a diagnosis based on observed information.\n\n**Re: relevant literature (w2)**\n\nWe thank the reviewer for highlighting the work by Beaulac & Rosenthal and have added it to the related work section. As noted, BEST suits ODDC settings with clear knowledge of the data-generating process, but defining gating variables is less straightforward with limited knowledge. In this sense, we view the MA framework as more general. A key difference between our methods is that MA trees do not explicitly split on the missingness mask. On the one hand, BEST can explicitly build trees using mask splits—often at the top—approaching the pattern submodel (PSM) of Mercaldo & Blume. On the other hand, MA trees cannot split based on non-MAR patterns, which may limit their expressiveness in such cases. However, avoiding explicit mask splits improves interpretability by preventing the tree structure from being dominated by missingness logic. Comparing MA-DT to BEST on the dataset from Beaulac and Rosenthal would be interesting, but the dataset does not appear to be publicly available.\n\n**Re: informative missingness (w3)**\n\nWe introduce informative missingness in Section 3 but agree that Appendix C felt somewhat isolated. The reviewer is correct: when missingness is informative, adding missingness indicators is generally preferable to using an MA approach. For example, Beaulac & Rosenthal show that when missingness depends on the label, indicators can be predictive. In such cases, MA trees cannot leverage this information and may underperform, depending on the imputation. We appreciate the feedback and address this in the revised manuscript (Section 5.2).\n\n**Re: convergence (w5)**\n\nThe reviewer raises an important question: if an optimal model $h^*$ exists, can our algorithm recover it with enough data? This depends on the model class, optimization, and data distribution. For example, if $h^*$ is a sparse logistic model, our L1 objective can likely recover it—similar to LASSO. The optimization in Section 4 supports Corollary 1: under ODDC rules, we can learn models with low missingness reliance and strong performance. We will add this discussion to the final paper.\n\n**Re: clarity (w1), notation, and regularization**\n\n- We agree it's important to clarify the relationship between $h^*$ and $\\mathcal{H}$ and now state explicitly that $h^*(x) = \\mathbb{E}[Y \\mid x]$ may lie outside $\\mathcal{H}$. This is benign for universal approximators (e.g., decision trees) but more significant for restricted classes like linear models.\n- We agree that introducing $\\sigma_{i,j}$ in Section 4.1 may be premature; we now assume equal feature contribution initially and introduce $\\sigma_{i,j} \\in {0,1}$ later in Section 4.3, where it supports the generalization to tree ensembles. \n- We also acknowledge that starting with L0 regularization to penalize missing features would have been natural; however, we focused on L1 (LASSO) due to its compatibility with widely used implementations (e.g., scikit-learn, statsmodels). We will clarify this choice, along with a discussion of L0-based alternatives.\n\nLastly, we thank the reviewer for their valuable feedback and excellent suggestions, and for noting inconsistencies and typos, which have been corrected in the revision.", "claims_and_evidence": "Yes. The clear discussion of settings in which the MA framework is expected to fail (Section 5.2) is particularly appreciated, and helps clarify when MA can safely be used.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes; I read all three proofs in Appendix B. The proofs are correct, although I have a minor suggestion for Corollary 1. (w1) The given proof is valid, but does not specify the hypothesis class from which h* is drawn. From the notation used in the paper, it is implied that h* must be a member of the hypothesis class H we are currently considering. This is benign for universal approximators like decision trees, but may be misleading when H is the class of linear models (there may be no linear model h st h(x) = E[Y | x] ). For clarity, I recommend specifying that h* may not be a member of H where appropriate.", "experimental_designs_or_analyses": "I did not check the code, but the description of the experimental setup is clear and thorough, and the described experimental setup makes sense.", "relation_to_broader_scientific_literature": "This work takes a different angle on the well-studied problem of handling missing data. It appropriately considers impute-then-predict, model-specific missingness handling, and missingness as a value baselines, which each have slightly different goals than the MA approach.", "essential_references_not_discussed": "(w2) One notable omission from the set of baselines considered is [1]. Similar to this work, [1] introduces an approach that aims to avoid including missing values in the splits used in a decision tree. This is particularly relevant because [1] aims to split on x_j only in the subspace where x_j is not missing; this is an approach that would perform particularly well in the ODDC rule setting considered in the present work.\n\n[1] Beaulac, Cédric, and Jeffrey S. Rosenthal. \"BEST: A decision tree algorithm that handles missing values.\" Computational Statistics 35.3 (2020): 1001-1026.", "other_strengths_and_weaknesses": "Strengths\n- The paper is quite well written, with clean and clear notation used throughout. \n- The proposed methods are elegant, and empirically shown to be effective.\n- This work provides a clear discussion of settings in which we expect MA regularization to be viable in general (5.1) and in which we don't expect MA regularization to be viable (5.2). This is very helpful in understanding the method, and will support future research.\n\nWeaknesses\n- (w1) See \"Theoretical Claims\"\n- (w2) See \"Essential References Not Discussed\"\n- (w3) An (arguably) less restrictive setting than MNAR in which this strategy may be expected to fail is the case of informative missingness. This setting is discussed in Appendix C, but is not connected with the main paper in any way. I recommend adding some discussion of this point to the end of 5.2, since, in this case, we would expect MA to find inferior accuracy to missingness indicator approaches.\n- (w4) See \"Questions For Authors\" <- This is my primary concern, and I am inclined to increase my score if it can be addressed in a compelling way.\n- (w5) See \"Questions For Authors\"", "other_comments_or_suggestions": "To help respond to this review, I've labelled each point on which a response would be appreciated with (w#). \n\nSome additional suggestions to which I do not expect a response:\n- It is somewhat confusing to introduce sigma_ij in section 4.1 — I would recommend holding off on adding this term until 4.3, and simply defining an updated splitting rule there.\n- 4.2 describes an l1 based regularization, but the target problem seems to suggest an L0 regularization (i.e., we want to guide coefficients to exactly 0). It may be interesting to consider methods that optimize L0 regularized classification directly (e.g., [2, 3]) rather than Lasso. I do not expect this experiment to be done for rebuttal, and it does not effect my rating.\n- Throughout the paper there is discussion using the language of regression (i.e., “impute-then-regress”). However, the experiments focus on classification. Unifying the language would improve readability.\n- Typos\n    - Line 162, right column — “we propose medications” should be “we propose modifications”\n    - Line 216 left — it would be more accurate given equation 5 to say that Lasso uses a parameter lambda > 0, rather than alpha > 0\n    - Line 194 right — “individual trees ar fit” should be “individual trees are fit”\n\n[2] Liu, Jiachang, et al. \"Fast sparse classification for generalized linear and additive models.\" Proceedings of machine learning research 151 (2022): 9304.\n[3] Dedieu, Antoine, Hussein Hazimeh, and Rahul Mazumder. \"Learning sparse classifiers: Continuous and mixed integer optimization perspectives.\" Journal of Machine Learning Research 22.135 (2021): 1-47.", "questions_for_authors": "- (w4) Why should practitioners prefer to avoid accessing missing variables, rather than reasoning based on missingness/handling it natively with, e.g., defaults? This seems like a key question in motivating this work, but is not compellingly answered in the current version of the paper. This is my primary concern, and I am inclined to increase my score if it can be addressed in a compelling way.\n- (w5) Corollary 1 speaks to the existence of a Baye's optimal model with 0 reliance on missing features, but it says nothing about convergence/relevance of the given loss functions to finding this h*. Is it possible to say anything stronger/connect the optimization problems described in Section 4 to this Corollary?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Ez8hikIYB6", "reviewer": "Reviewer_HDgG", "summary": "This manuscript focuses on improving the reliability of machine learning methods when encounter missing value at inference time. A novel framework termed Missingness-Avoiding learning is proposed to reduce the models' reliance of missing values for decision trees, sparse linear models and ensemble methods. Specifically, a classifier-specific regularization technique is proposed to alivate the models' dependency on missing values. Experiments on six datasets are carried out to validate the proposed method.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their insightful feedback and appreciate their recognition of our work addressing a practically important yet underexplored challenge in the healthcare domain—enhancing model reliability under test-time missingness. We address the specific comments below.\n\n**Re: MA learning in neural networks**\n\nStandard feed-forward neural networks typically rely on all input features to a non-zero degree unless the learning objective is regularized to yield sparse weights. Even then, the reliance is generally not contextual but remains the same for every input, regardless of the observed features. In principle, contextual reliance could be achieved by combining the MA penalty with attention modules that attend only to observed features. However, our focus is on tabular data, where neural networks tend to underperform. We instead target model classes like decision trees and linear models, which not only perform well on tabular data but also offer interpretability—an essential requirement in our primary application domain, healthcare. Exploring MA learning with attention modules for tabular representations is an interesting direction for future work. \n\n**Re: dataset sizes**\n\nWe thank the reviewer for their observation and agree that the number of features in our datasets is relatively small (ranging from 16 to 42). However, we include a diverse set of datasets in terms of sample size—from a few hundred (e.g., Pharyngitis: 676, ADNI: 1,337, Breast Cancer 1,756, LIFE 2,864) to over 10,000 samples (e.g., FICO: 10,549, NHANES: 10,000). These dataset sizes are typical in the tabular ML literature and reflect realistic constraints in many application domains.\n\n\n**Re: advanced imputation methods as baselines** \n\nThank you for sharing the two works. We agree that the methods could be compared to more advanced imputation approaches. However, our goal for the baselines was to include representative methods from different strategies for handling missing data: impute-then-predict, model-specific handling of missingness, and treating missingness as an informative signal. Each of these serves a slightly different purpose compared to the MA approach. We chose zero imputation and MICE because they are both widely studied in the literature and commonly used in practice, making them strong reference points for comparison. More advanced imputation methods are likely to yield similar reliance on missing values when combined with similar regressions or classifiers. Thus, the choice of imputation method is secondary in our study. To enable a fair comparison with our method, we also calculate the missingness reliance for the baselines. Since the baseline methods were not designed with the goal of learning to avoid missing values, but rather to provide accurate imputations, they may be incentivized to rely on well-imputed values—potentially leading to a higher missingness reliance metric.", "claims_and_evidence": "First of all, I would like to acknowledge that I am not familiar with the related works. I tried my best to understand this work and do the review. It is highly possible that I miss some importance aspects of this submission.\n\n- The authors claim that the proposed method is a general framework for training models with missing values. However, the proposed MA is only implemented with three types of classic machine learning methods, i.e., decision tree, linear model and ensembles. Is MA applicable for more general models like neural networks?", "methods_and_evaluation_criteria": "- Most of the datasets used in the experiments are small-scale tabular data, which may be insufficient to well support the proposed method.\n\n- The compared baselines may be insufficient. For examople, the proposed MA is only compared with zero-imputation and MICE. Other advanced imputation-based methods such as [1] is missed.\n\n[1] GAIN: Missing Data Imputation using Generative Adversarial Nets, NIPS'18\n\n[2] TabDDPM: Modelling Tabular Data with Diffusion Models, ICML'23", "theoretical_claims": "There is no proofs in this manuscript that need to be checked.", "experimental_designs_or_analyses": "- The experimental designs make sense to me. I have no further comments on this point.", "relation_to_broader_scientific_literature": "None.", "essential_references_not_discussed": "None.", "other_strengths_and_weaknesses": "None.", "other_comments_or_suggestions": "None.", "questions_for_authors": "None.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jx7rzRTXYr", "reviewer": "Reviewer_eoMS", "summary": "The authors introduce missingness-avoiding (MA) machine learning, a framework for altering model training to avoid reliance on missing features. Through experiments on decision trees, lasso, and tree ensembles, they show that MA can reduce reliance on missing-value features with only a minor hit to predictive performance. This may yield improved interpretability of the model.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback and appreciate their recognition of the clear explanations and sensible evaluation criteria. We respond to specific comments below.\n\n**Re: real-world importance of reducing missingness reliance**\n\nThanks for raising this point—we understand its importance in demonstrating the improved interpretability of our MA method. In the last paragraph of Section 6.2, we refer to MA trees fitted to the Life dataset, shown in Figure 6 in the appendix. This example demonstrates how MA-DT behaves under different values of the missingness regularization parameter $\\alpha$. For instance, when $\\alpha$ is tuned to balance accuracy and reliance on missing values, MA-DT reduces reliance on missing data by 33% without sacrificing accuracy compared to a standard decision tree. We will consider including these figures in the main text in a potential camera-ready version. Additionally, we will consider adding examples of MA trees fitted to the ADNI dataset. These examples illustrate how MA learning avoids splitting on features that are likely to be missing, instead favoring more complete features that maintain predictive performance. At test time, such trees are more interpretable since they do not rely on features with missing values.\n\n**Re: different strategies for choosing $\\alpha$**\n\nIn the submitted version of the work, we primarily considered two strategies for selecting $\\alpha$. We chose $\\alpha=\\alpha^{*}$ as part of the overall model selection process, selecting the candidate model with the lowest missingness reliance among those achieving at least 95% of the maximum AUC. If predictive performance is a priority, this percentage could be increased (at the potential cost of increased missingness reliance). Conversely, we chose $\\alpha=\\infty$ by selecting the model with the highest AUC among those achieving near-zero missingness reliance (ensuring that the range of $\\alpha$ included sufficiently high values). A more general approach could involve setting a threshold for missingness reliance, such as 20%, and then maximizing AUC among models meeting this threshold. Another alternative is to maximize $\\mathrm{AUC} – \\gamma\\cdot\\hat{\\rho}$, where $\\gamma$ controls the trade-off between AUC and missingness reliance and needs to be set for the specific problem at hand.\n\nWe thank the reviewer for raising this point and we have added the following discussion to the revised paper: Alternative strategies for selecting $\\alpha^*$ should consider the specific application needs. For instance, rather than focusing solely on the acceptable trade-off in predictive performance for achieving the lowest possible $\\alpha$, we might prioritize limiting the number of missing features per individual. Relying on an average measure across the dataset may not provide sufficient insight. Additionally, domain knowledge can help define an acceptable level of missingness, but quantifying this threshold is often challenging and subjective.", "claims_and_evidence": "Yes, the experiments clearly show that MA estimators can mostly maintain performance while substantially reducing reliance on missing features.", "methods_and_evaluation_criteria": "Yes the proposed methods are clearly explained and the evaluation criteria make sense.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Yes, the experimental designs seem sound.\n\nOne arbitrary choice seems to be the selection of alpha, which is chosen as the candidate model with the lowest ρ among those achieving at least 95 percent of the maximum AUROC. This seems sensible but I am curious if there are other reasonable choices, particularly if any of them can lead to improved test AUROC (regardless of missingness reliance).", "relation_to_broader_scientific_literature": "The results generalize the results of Stempfle & Johansson 2024, which are specific to generalized linear rule models", "essential_references_not_discussed": "None that I know of", "other_strengths_and_weaknesses": "The paper addresses an interesting problem and a clear and well-explained solution. Some of the novelty may be slightly limited in light of Stempfle & Johansson 2024, but the extension to trees seems to be quite useful in practice.", "other_comments_or_suggestions": "N/a", "questions_for_authors": "Can the authors show, even qualitatively, examples where the decrease in missingness reliance translates to real-world importance, e.g. improved interpretability on one of the datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This submission contributes a framework to facilitate machine-learning predictions in the presence of missing values by adding corresponding regularization. It led to interest and discussion with the reviewers, which appreciated the prediction framework and the empricial study. The work could be improved by demonstrating the framework beyond trees.
Decision: Accept (spotlight poster)