Interpretable Generalized Additive Models for Datasets with Missing Values
Authors: Hayden McTavish
Jon Donnelly
Margo Seltzer
Cynthia Rudin
Venue: NeurIPS 2024 poster (2024)
Keywords: Interpretability
Missing Data
Generalized Additive Models
Sparsity
Primary area: interpretability_and_explainability
TL;DR: We introduce an interpretable GAM approach for missing data which improves accuracy under synthetic missingness while globally improving sparsity, all with no significant cost to real-world accuracy or runtime.

Abstract: Many important datasets contain samples that are missing one or more feature values. Maintaining the interpretability of machine learning models in the presence of such missing data is challenging. Singly or multiply imputing missing values complicates the model’s mapping from features to labels. On the other hand, reasoning on indicator variables that represent missingness introduces a potentially large number of additional terms, sacrificing sparsity. We solve these problems with M-GAM, a sparse, generalized, additive modeling approach that incorporates missingness indicators and their interaction terms while maintaining sparsity through $\ell_0$ regularization. We show that M-GAM provides similar or superior accuracy to prior methods while significantly improving sparsity relative to either imputation or naïve inclusion of indicator variables.

Interpretable Generalized Additive Models for Datasets with Missing Values
Hayden McTavish
Jon Donnelly*
Department of Computer Science
Department of Computer Science
Duke University
Duke University
Durham, NC 27705
Durham, NC 27705
bayden. www.wmstdwtiiscueeeee.e... edu
jon donnelly@duke edu
Margo Seltzer
Cynthia Rudin
Department of Computer Science University of British Columbia Vancouver, BC V6T 1Z4 mseltzer@cs.ube ubc..cccsuubcc.
Department of Computer Science
Duke University
Durham, NC 27705 cynthia@cs. duke. edu
Abstract
Many important datasets contain samples that are missing one more feature values. Maintaining the interpretability of machine learning models the pres- ence of such missing data is challenging. Singly multiply imputing missing values complicates the model's mapping from features to labels. On the other hand, reasoning indicator variables that represent missingness introduces potentially large number of additional terms, sacrificing sparsity. We solve these problems with M-GAM,, sparse, generalized, additive modeling approach that incorporates missingness indicators and their interaction terms while maintaining sparsity through to regularization. We show that M-GAM provides similar superior accuracy to prior methods while significantly improving sparsity relative either imputation naive inclusion of indicator variables.
Introduction
Interpretability is essential for wide range of machine learning applications Rudin et al. (2022). Missing data pose a challenge to interpretability, because many simple models (e.g., linear models) are not well-defined when data are missing. This raises the question: how can interpretability be maintained for datasets with missing values?
We introduce an interpretable model class, M-GAM. that extends Generalized Additive Models (GAMs) to handle missing data. GAMs take the form linear combination of univariate component functions, with one function corresponding to each feature; this univariate nature is the core reason for their interpretability (Rudin et al. (202). We introduce two sets of boolean variables for each feature. The first consists of missingness indicators that identify which features have missing values. The second consists of missingness adjustment terms that adjust the shape curves for other features for each missing features in a sample. This maintains
of univariate ability to view GAM M-GAM much shape functions even when modeling interactions with missing data. As such, an simpler to interpret than GAM built on imputed data, since it avoids creating multivariate features as happens when imputing features from multiple others. This is illustrated in Figure]
* These authors contributed equally to this work.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Original Model (All Variables Present)
Score
Score
Spore
0.5
X2
X3
10
Not Univariate
Impute Then Prodict
Score
Score
Score
XI
X2
XI
X3 Missing
1.2
M-GAM (Ours)
Spore
Score
0.5 0
10
Figure I:
A comparison of how GAMs that use underlying imputation (middle row) and M-GAMs (bottom row) behave when feature is missing. Top: When data are missing, the overall output logit for both models is the sum of three univariate shape functions. Middle: When X3 is missing, imputed X3 X1 2X2, producing 3D shape function that is difficult understand. Bottom: M-GAM uses simple adjustments to existing univariate shape curves when X3 is missing (using the green curves instead of the light blue ones), making its reasoning process simple to follow. the data were more than 3 dimensional, we would not be able to visualize the model with imputation, but M-GAM would still be easily visualized.
M-GAMs explicitly encourage sparsity. This reduces overfitting, which has been identified as a concern in prior work using missingness indicators (Van Ness et al. 2023), since realistic data may
produce an overwhelming number of missingness indicators. Unlike prior methods that leverage missingness indicators, we use tu regularization (Liu et al. 2022; Dedieu et al. 2021 Hazimeh &
Mazumder, 22000] Hazimeh et al.) (003), which directly optimizes for sparser. more interpretable models. Our ability to create sparse models allows us to include not only simple missingness indicators but also combinations of missingness indicator variables with M-GAM's missingness Saluureellssttttsss M-GAM fit on real data.
adjustment terms, without overfitting.
With this modeling framework, we make the following contributions: (1) We introduce M-GAM, form of sparse generalized additive model that incorporates missingness directly into its reasoning. (2) We show that M-GAM provides substantial performance benefits relative to impute-then- predictt models when synthetic missing- aataad- (MAR) missingness is added to real datasets, while maintaining performance real world settings with only naturally occurring missingness. (3) We show that M-GAM substantially reduces runtime relative impute-then thee--diiit methods built multiple imputation while producing sparse, interpretable models.
Related Work
Missing data is a well studied problem in statistics. Traditionally, mechanisms by which data can be missing are organized into three categories: missing completely random (MCAR), where missing- ness is independent of the value of all covariates; missing at random (MAR), where missingness in a variable X, is conditionally independent of the value of X2 given all other variables: and missing not at random (MNAR), where missingness may depend on any variable (Little & Rubin 2019).
For supervised learning. there are two common approaches to dealing with missing data: impute- trendpredict, in which standard machine learning model is fit on top of imputed data and used for prediction, and incorporating missing data handling directly in the predictive model.
broad body of work studies imputation, particularly in the MAR setting - for a more thorough review, see Shadbahr et al. (2023). Imputation methods can broadly be sorted into single imputation methods (see Van Buuren 2018, for a review of such methods), where each missing value is
imputed once, and multiple imputation (Rubin 1988
Van Buuren & Oudshoorn, 1999. Schafer &

---

Variable
0.8
Missing Data
Variable 4
Variable 5
Variable
Variable
40
80
75
20
0.8
Missing Variable 8 (No Usable Information)
0.4
40
80
20
0.8
Missing Variable (Not Investigated Not Found)
Not Missingness Dependent
40
80
75
Missingness
Dependent
Figure 2: A generalized additive model (GAM) for the Explainable MI. Challenge data from FICO al. 2018) with missingness incorporated. This model handles missingness interpretably by explicitly providing alternative shape functions when a variable is missing. For example, in this model the shape function for variable 2 is adjusted when variable 3 is missing, and the shape function for variable 3 is removed. This model achieves comparable performance to convoluted black box approaches (such as random forests and/or MICE), but provides global interpretability (the entire model can easily be inspected) and local interpretability (the shape functions applied for given sample can be easily visualized). An expanded version of this figure with variable names can be found in Appendix Figure [6] Shape functions in the right section shared across all missing variable combinations. The type of missingness indicated in parentheses next to the missing variable. Section F visualizes additional M-GAMs.
Graham 2002,
Stekhoven & Bühlmann 2012, Mattei & Frellsen 2019). where many alternative
imputations are provided for each missing value. Multiple imputation is convenient because it integrates uncertainty into its imputations by providing a range of alternatives (Van Buuren 2018).
Recent approaches directly incorporate missing data in the predictive model. Le Morvan et al. (2020)
showed that, even when the target of prediction is linear function, in the presence of missing data, the optimal model need not be linear in the original features. Rather, their optimal model was linear the observed data and interactions between indicators for missing data and the observed data. Van Ness et al. (2023) showed that when missingness contains information about an outcome, linear models that directly include missingness indicators outperform models excluding this information. This inclusion of missingness indicators is especially recommended in the practical setting of predictive modeling, rather than the setting of statistical inference where MCAR, MAR and MNAR concepts are more commonly used (Sperrin et al. 2020).
There are also a wide variety of ad-hoc methods for handling missing data in tree-based models (Kapelner & Bleich 2015] Twala et al.) 2008 [Baaulac & Rosenthal] 2020 Therneau et al. 1997) C00nnCChen
and boosting models (Wang & Feng 2010 Chen & Guestrin (016)) that involve prediction without
explicit imputation. Beaulae & Rosenthal (2020) learn decision trees which avoid splitting on missing data when missingness follows deterministic structure based on other known features. This sidesteps any need to query features when data is missing, but does not generalize to settings with less structured missingness. More generally, tree-based models can learn branch direction for each split to use when the queried feature is unknown. This effectively imputes the response to the query, but keeps the model itself simple. This is the method used in XGBoost (Chen & Guestrin (016)) and SKLeam decision tree classifier (Pedregosa et al., 2011)) Twala et al. (2008) and Kapelner & Bleich (2015) additionally incorporate the option to split on missingness itself, effectively encoding missingness as value. Ding & Simonoff (2010) provides empirical support for incorporating this option to treat missingness as a value. We compare XGBoost and SKleear in our experimental section, finding that M-GAM better balances interpretability and performance,
when allow missingness
3

---

indicator splits like in Twala et al. (2008); Kapelner & Bleich (015)); Ding & Simonoff (2010): Wang & Feng (2010). Most similarly to our own approach, Therneau et al. (1997) and Breiman (0017)
discuss an alternative approach of surrogate splits: when feature that is split on is missing, a set of other splits is used in place of the missing feature to evaluate the split. This practice of adjusting which features are used when one feature is missing bears some similarities to our use of missingness interaction splits that adjust the shape functions for some features when other features are missing, though these surrogate splits do not optimize for sparsity like M-GAM, and underperform more standard multiple imputation approaches Valdiviezo & Van Aelst (2013); [eelders (1999). Further work explores the idea of developing distinct models for under different cases of missing features (Fletcher Mercaldo & Blume, 2020 Stemptle et al. 2023) or developing additive logical models with disjunctions, such that reliance imputed values is low Stempfle & Johansson (2024).
is critical to note that, for a dataset with features, adding indicators for missing data results in 2d features, and adding first order interactions between features and missingness results in d(d-1] + 2d features. As such, without careful regularization, these models that explicitly handle missingness are complex and uninterpretable. This poses a challenge for their application in high stakes domains such as justice and medicine, where there have been calls to enshrine interpretability as a requirement for the use of machine learning methods (US Food and Drug Administration] 2021 [Eropean Commision 2021]. In contrast, M-GAM provides sparse, transparent models that handle missingness indicators and interactions by extending sparse generalized additive models (Liu et al. (2022). M-GAM provides an expressive model class for handling missingness while controlling the exploding number of missingness interaction terms through to regularization.
3 Methodology
We denote a dataset of n samples by D (X.y)= - ( 9:) where x; E (R U [NAJ) is a d-dimensional vector of features, NA denotes missing entry, and y E {0,1) is our target label. We use Zú,j to denote the j-th feature of the i-th sample. We use bold capital letters (X) to denote matrices, bold lowercase letters to denote vectors (x,), capital letters to denote random variables (X), and lowercase letters to represent scalars (Ix,r). denotes noise; any other Greek characters denote model parameters. We encode all binary comparisons to the value NA as O. That is, follow the convention that [NAga] = 0 for any value a E TR. where 1 denotes the indicator function.
Note that, in practice, data are often missing for distinct yet identifiable reasons; for example, measurement for one sample may be missing because it was never taken, while another may be missing because researcher spilled coffee the notes containing the data. As such, we explicitly consider distinct reasons for missing data. For a dataset with e E potential reasons for data to be missing, define the mapping meal: IR U } + {0, 1. l..... c) to map from entry of X to natural number indicating the reason that entry is missing (0 if the entry is not missing).
With notation established, we begin with a motivational proposition. Proposition 3.1 states that even we can perfectly impute missing values, we may find greater predictive power by using missingness itself as a feature rather than by imputing missing values.
Proposition 3.1. Let I (R U NA)á - Rdd be an oracle imputation function that replaces all missing values in a vector with the correct non-missing entry. For a random variable X E Rd, let fi() = (())))()))) be the Bayes' optimal model using perfectly imputed data and fa(X) == 1 [] be the Bayes optimal model using missingness as a value. There exist data generating
processes for X and Y where P(y = fx(X)) < P(Y = f2(X)).
The key insight behind Proposition 3.1 is that, when missingness is dependent the label Y, missingness itself can be a powerful predictor of the label (this setting is called informative missingness in Van Ness et al.) 2023 In particular, we can gain information about our label that is not available in other covariates (e.g., information from E1).
Section A of the appendix provides a proof by construction for Proposition 3.1
may appear to conflict with Theorem 3.1 of Le Morvan et al. (2021), which states that a Bayes optimal model may be produced using impute-then-preett model. This theorem hinges on the idea that, for most imputations, it is still possible to distinguish imputed data entries from non-misse mon-missing entries. This is not the case for perfect imputation, which yields Corollary 3.2
Proposition]] 3.1
with almost any imputation

---

Corollary 3.2. Let R(f.X.y) denote the risk of model f for data X.y. and R" the optimal risk. Let T: (R NA)e + Rd denote the oracle imputation function of Proposition 3.1 Under perfect imputation, it possible for there to be Bayes optimal model built on imputed data. That is,
3(X y) P R( o y, = R*)
Corollary 3.2 states that perfectly imputing missing data can reduce the best possible performance of predictive model. This has substantial implications for how imputation is understood for prediction: perfect imputation imputation is optimized to maintain the downstream performance of impute-then-predicc models, the imputed data loses some of meaning since is no longer our "best guess" of the missing data's value, as we need to deliberately avoid perfect imputation guarantee that maintain performance.
achieved, then impute-then perenntredict models sacrifice expressiveness.
Motivated by Proposition 3.1 and Corollary 3.2 we interested in constructing predictive models that explicitly use missingness as value in their prediction rather than imputing first. More generally, we may also consider using the indicator for each rype of missing data directly in a prediction. Generalized additive models (GAMs) provide a natural choice for such model.
GAM y Rd I consists of bias term 30 R and d shape functions S1 . parameterized by vectors .i,..., ,,,...,,a, Sa. Given a sample Xs, a GAM forms a prediction as:
g(x; Bo + Σ fT())::)))
(1)
j=1
In practice, it is common for each shape function to be linear combination of difffrrnnt thresholds its input variable, i.e., J3 (1 = Ile((tz) 3y. ( (t,,a);; where len(t,) N is the number of thresholds applied variable j. each f IR is threshold value, and each By, IR is a learned weight.
These functions provide a convenient framework for considering missing values. In particular, we can form new shape function h (#t.j 8j, (3 miiiiiii that explicitly handles missing data by introducing additional "missingness indicator" terms, such that our shape functions take the form
(z Bj,, 3 = (
3 1 (mmat((ss.))
m==1
where gmiss E IRC is an additional vector of parameters. Recall that there are e distinct reasons for missingness, with mcat(x.,5) = if æıj is not missing and meat(x;;)) = m if Fi.5 missing for the m-th reason.
may further extend this augmentation to include interaction terms between missingness indicators and standard threshold functions. The "missingness interaction" function between feature j and feature j' takes the form
len(t;)
hg. (1 Tr. ago"))
aj (mmm x Stypeb
m==1
where each km E IR is a learned weight. We thus define missingness-GAM (M-GAM) Imiss as follows:
Definition 3.3. Given parameters a, griss, and B, an M-GAM is defined as
d
d
Jmiss (x;; B, gmiss a:) = Bu +
Σ
h
+ Σ
(2)
1=l
==
where
ken(t,)
Σ
Ogj'' k. [mcal[( and Styrell
and
hy B5, = (2
3 [mca((s,))=m)
m=1
5

---

These augmentation terms are quite powerful. Theorem 3.4 shows that, for any impute- then peeei-p approach using an affine imputer and a GAM predictor, we can construct an M-GAM that recovers the expected classification score over imputations.
Theorem 3.4. Consider any GAM g R + R. parameterized by with shape functions defined as linear combinations over boolean features (either thresholds fş(mi.; P3)= Ek=1, B5.k lus.) Sty,a]
or a feature that was originally boolean). Suppose some observations missing boolean feature b, and that this feature imputed such that the modeled probability of vi. being true, PP 1 1xx;,-b) (where Xi,-b refers to all covariates except b) is an affine function h. Xi.-b + [0, 1]. For any parameterization B of a GAM g. let Elg(x;; B)] == P(zz,b
1|xx,-))g(( (b+), 3) + P(zz,b 0 (x) ;B),, where x (b-) denotes x4 with Fi,b = 1 and x; there exists a model in the model class M-GAM (which does not use imputations), that recovers this score [[g(x;;; [3] for all i.
(b-) denotes x; with L4.. O. Then,
More broadly. Theorem 3.4 suggests that M-GAM is able to express scores comparable to those of any impute-then predict GAM, if the imputation probabilities can be approximated by an additive model. One advantage is that M-GAM can be optimized directly for classification performance rather than first optimizing an imputation step to recover missing values, and then optimizing model on the imputed data. Together, Proposition [3.1 and Theorem 3.4 show that M-GAM is comparable to mmpue--teen-rredicc in a broad range of settings and that M-GAM is strictly better than thepute-then-preiic in some settings. Appendix C contains the proof for Theorem 3.4
3.1 Sparsity
Building a GAM with missingness indicators and interaction terms provides superior expressive power but causes an explosion of the number of covariates the model must consider. The GAM in Equation I consists of Σj len(t,) coefficients, while the M-GAM in Definition 3.3 consists of E [Eree len(ty-)) + Σÿ (len(t,)) + cd + coefficients (or, d (le(tt)) + d when c = 1). This increases the risk of overfitting and may lead to complex. uninterpretable models. The same problem arises when adding similar interaction terms to a linear model, as diagnosed by Van Ness al. (2023), who propose hypothesis testing style framework for variable selection. Notably, this framework does not explicitly encourage sparsity it only discourages overfitting.
Rather than applying pre- rre-processing step for variable selection, we use to regularization, which we can optimize directly alongside accuracy. This encourages the model coefficients to be O, resulting in sparse models despite the potentially large number of input features. We optimize classification performance using the exponential loss, as yields faster convergence rates than logistic loss during optimization (Liu et al., 2022). Thus, our goal is to solve the following optimization problem:
( Σ
n
V..mii (x.iB, gaiis aa) + >n(11/11o + ||,3miss/| llo + l|allo))
min 9,guas aa
(3)
where An is hyperparameter that determines the strength of the lo regularization.
To simplify
( so it can be solved directly, we construct a new set of features X {0, 1302; (((tt))) consisting of the indicator for each threshold, missing value, and interaction term for each feature in the original dataset. For a large coefficient vector R +1 --dd++1 and bias coefficient to R, our optimization problem
becomes:
min
Tcctrl Aollyllo.
4
which is solved using the optimization framework of Liu et al. (2022). This allows us to quickly produce sparse M-GAMs, overcoming the large number of input values.
4 Experiments
We now evaluate the performance, runtime, and sparsity of M-GAM in comparison to other methods. To evaluate M-GAM in realistic setting, we require datasets with some missing entries. We primarily

---

consider four datasets: the Explainable Machine Learning Challenge dataset (FICO et al. 2018 (referred to as FICO). breast cancer dataset introduced by Razavi et al. (2018) (referred as Breast Cancer), the MIMIC-III critical care dataset (Johnson et al. 2016) (referred to as MIMIC), and a dataset concerning the prediction of pharyngitis introduced by Miyagi (2023) (referred as Pharyngitis). FICO contains 10,459 individuals, measuring 23 predictor variables used to predict whether each individual will repay a line of credit within 2 years. FICO contains three distinct encodings for missingness: -7, indicating no information of a given type is available, -8, indicating there was no usable information, and -9, indicating that a credit bureau report was not investigated or not found. Breast Cancer measures 27 features for 1,756 patients, MIMIC measures 49 features for 30,238 patients, and Pharyngitis measures 19 features for 676 patients. We use AUC, rather than accuracy, when evaluating model performance for Breast Cancer and MIMIC because these two datasets are heavily imbalanced. Breast Cancer, MIMIC, and Pharyngitis contain only one missingness encoding. Two additional datasets studied in Section E of the appendix.
Each dataset contains missing entries. Because these are real datasets, we do not know the exact mechanism(s) (i.e., MCAR, MAR, or MNAR) by which data are missing. These datasets allow us to evaluate M-GAM on data with (Section 4.1 and without (Section 4.4) added MAR missingness.
We then study the interpretabllityyacccurrayy tradeoff for M-GAM using sparsity versus accuracy plots (Section (42) and evaluate the runtime of M-GAM (Section 4.3). We use multivariate imputation by chained equations (MICE) (Van Buuren & Oudshoorn 1999), MIWAE(Mattei & Frellsen 2019). and
MissForest Stekhoven MissshvrestSteehoveen & Bühlmann 2012) as multiple imputation baselines.
We compare M-GAM to a variety of standard machine learning models used in an impute-then-prediin framework. We further compare standard machine learning models used with both the missingness augmentation described by Van Ness et al. (2023) and imputation. Section D.3 of the appendix contains full experimental details.
4.1 M-GAM Provides Superior Performance Given Informative MAR Missingness
To demonstrate the added expressive capability of our model relative impute-then, then-predict models, we created versions of each dataset with added synthetic missingness. Missingness is added to an arbitrary column of the data according an MAR mechanism, where missingness is dependent the outcome Y and one other randomly chosen predictor variable. We encode this added missingness as a new value distinct from the value(s) used to indicate missingness in the original dataset. A conditional probability table for this synthetic missingness is provided in Appendix Section D.I
This adjustment falls under the MAR setting where imputation is often suggested. Nevertheless, as shown Figure 3 M-GAM with interactions provides much greater accuracy than imputation, because the missingness depends on the outcome. The gain in performance due to considering interaction terms grows larger with increasing MAR missingness (from left to right).
Note that missingness depending on the outcome of interest is realistic. For example, individuals who are unlikely to have a particular disease are unlikely to receive medical tests related to that disease. One theory about why polls were wrong before the 2016 US presidential election was that non-response bias was associated with less education and distrust in the media, both predictors of votes for Donald Trump (Kurtzleben) 2016).
4.2 M-GAM Achieves High Performance While Maintaining Sparsity
The most important benefit of the missingness handling in M-GAM is that it enables simple, sparse models. As such, we show the tradeoff' between complexity and performance for M-GAM.
Figure 4 demonstrates the sparsity- trade-off for M-GAM relative to a GAM fit using Scikit Learn's logistic regression package (Pedregosa et al. 2011) over binarized features, trained on data from 10 imputations using a multiple imputation method. We also contrast two different levels of missing variable parameterizattin: the full set of indicators and interactions versus using just indicators.
We quantify interpretability using the number of nonzero coefficients selected by M-GAM, since a large number of non-zero coefficients leads to dense, complicated mapping from the input data to prediction. Meanwhile, running impute-then-predic is not interpretable: the method
7

---

Performance under Added Synthetic MAR Missingness
Rate S 0.25
Rate S 0.5
Test Accuracy
FICO
0.8
0.8
0.7
30
125
150"
0.70
50
125
150
Breast Cancer
Test AUC
0.8
0.8
0.7
50 70 80 90
0.7g
50 70 80
90
Nonzero Coefficients
Nonzero Coefficients
M-GAM (Indicators Only)
SMIM
GAM
MIWAE
M-GAM (w/ Interactions)
GAM w/ MVI
MICE
MissForest
Figure 3: Sparsity of M-GAM when synthetic MAR missingness is added to up to 25% (left column) and 50% (right column) of entries in FICO (top row) and Breast Cancer (bottom row). We compare to several alternatives for GAMs with missing data: ensembling 10 GAMs fit on multiple imputation (for MIWAE, MICE, and MissForest), O-value imputation ("GAM"), mean-value imputation ("GAM w/ MVT"), and selective addition of missingness indicators ("SMIM"".. The number of non-zero coefficients for multiple imputation cannot be evaluated because the models depend both the GAM coefficients and the underlying imputation mechanisms, resulting in high dimensional shape functions in Figure [ Error bars report standard error 10 train-test splits.
requires ensembling many different GAMs, and the imputations themselves introduce complicated relationships between the data and the classifications, similar to what illustrated in Figure I
We show that with fewer than 40 total coefficients (including all step functions for all variables), M-GAM achieve accuracy comparable that of GAMs with multiple imputation. On FICO, M-GAM using just 20 non-zero coefficients achieves superior accuracy to a variety of dense, complicated alternatives.
4.3 M-GAM is Faster than Impute-then-Preed
We next turn runtime comparison between the impute-then-predde framework and M-GAM. For impute-then-predife models, we first imputed datasets and recorded the time required to do so. We then fit predictive model on each imputed training dataset and recorded the total time required. We recorded the time required to an M-GAM with missingness indicators and an M-GAM with missingness interactions for comparison. This was repeated for each of ten distinct train-test splits of the original dataset.
Figure 5 shows the runtime of our approach relative to each impute-thene then--predie baseline, as well as decision trees and random forests without imputation. M-GAM consistently produces models least an order of magnitude more quickly than impute-then-predic with any non-trivial imputation. While decision trees without imputation tend to produced faster than M-GAM, they tend to have lower accuracy than M-GAM as discussed in the next section. We repeat this experiment for four distinct subsamples of each dataset (1/4, 1/2, 3/4, and all the data) to study how each method scales the number of samples in Appendix E.3

---

Test Performance vs Sparsity
FICO
Pharyngitis
0.74
Test Accuracy
0.73
0.70
0.72
0.71
20
40
125
150
0.60
20
40
60
Breast Cancer
MIMIC
Test AUC
0.75
0.86
0.73
0.84
0.70
70
80
90
50
100 300
350
# Nonzero Coefficients M-GAM (Indicators Only)
# Nonzero Coefficients
SMIM
GAM MICE
MIWAE
M GAM (w/ Interactions)
GAM w/ MVI
MissFooret
Figure 4: Test performance of three models at various levels of sparsity the unaltered FICO and Breast Cancer datasets, with the same baselines as in Figure 3
Breast Cancer
FICO
MIMIC
Pharyngitis
Time (Seconds)
M-GAM (Ind)
M-GAM (Int)
Mean
MICE
MissForest
MIWAE
M M-GGAM (Ind)
M- M-GAM (Int)
Mean
MICE
MissForest
MIWAE
M M-GAM (Ind)
M-GAM (Int)
Mean
Miss
MICE
MIWAE
M-GAM (Ind)
M-GAM (int)
Mean
MisoForest
MICE
MIWAE
Figure 5: Runtime of different methods on Breast Cancer, FICO, MIMIC, and Pharyngitis. For each imputation method, we report the total time required to impute missing data and fit the best performing impute- then-predin classifier for that dataset and imputation method. M-GAM (Ind) is an M-GAM with indicators and M-GAM (Int) is an M-GAM with indicators and interaction terms. Error bars report standard error of total runtime over 10 train-test splits.
4.4 M-GAM is as Accurate as Impute-then-Predict on Real Data
While M-GAM outperforms imputation on semi-synthetic data, there is risk that this comes at the cost of performance on real data. To evaluate whether this the case, we used several multiple imputation methods to impute 10 distinct datasets for each setting, then fit variety of predictive models on these datasets. We used cross validation to select hyperparameters separately for each imputed training dataset and ensembled the resulting 10 models for each model class produce single predictive model for each model class. We repeated this procedure for ten distinct train-test splits for each dataset considered.
Figure (6 shows the test accuracy each model. find that, on real datasets, no alternative method substantially outperforms M-GAM. This suggests that M-GAM does not harm predictive performance on real datasets, while providing substantial benefits in interpretability (Section 4.2) and superior power under informative missingness (Section 4.1)

---

M RA-GAM (Indicators Only)
treeclaffeee (SMIM)
RandomForast Imputation
M-GAM (w/ Interactions)
Logistic Regression (No SMIM)
RandonforestClassifee (No SMIM)
AdaBoostC fler SMIM)
LoglsticRegnesseen (SMIM)
Fandon (SMIM)
AdaBoostC fier (SMIM)
MLPClessifier (No SMIM)
XGBClessifier (No SMIM)
DecisionTree No Imputation (No SMIM)
ML PClassiifeer (SMIM)
XGBClasstfer (SMIM)
Decision/reeChessffee SMIM)
FICO
Pharyngitis
Breast Cancer
MIMIC
0.74
0.80
3.80
0.90
Test Accuracy
0.72
Test AUC
0.70
0.71
0.65
0.80
0.60
0.55
3.60
0.75
0.69
0.70
0.68
0.45
3.50
Figure 6: mox--ad-whise plots comparing test performance of baseline models to M-GAM on four datasets over ten train splits. All methods except M-GAMs, "DecisionTree No Imputation", and "RandomForest No Imputation" impute ten datasets using MICE then ensemble the models fit on each dataset.
5 Conclusion
We introduced M-GAM, framework for producing accurate, sparse GAMs in the presence of missing data. We demonstrated that M-GAM achieves comparable accuracy impute-ther preepct on real datasets and superior accuracy under informative synthetic missingness. M-GAM produces models substantially more quickly than impute- then- predictt models with multiple imputation, and provides simple, transparent reasoning missing data.
While the ta penalty in M-GAM encourages sparsity, it is limited in that the lo regularization is applied uniformly across all coefficients. Consider the case when we are adding interaction terms to handle missingness. We might encourage the model to rely features is already using to predict y. rather than using new features. It may be more effective regularization - and more interpretable to have a reduced lo penalty for such cases. Future work should investigate applying distinct levels of regularization to observed variables, missingness indicators, and missingness interactions when a variable already included in the model.
An important caveat to models that reason on missing features that missingness can be especially vulnerable distribution shift, particularly in medical domain (Sperrin et al.) 2020 Groenwold 2020). The interpretability enabled by M-GAM crucial in allowing models to be closely monitored and adjusted in the presence of potential distribution shift. Future work could more thoroughly investigate potential distribution shift and ways to adjust model which reasons on missing data.
On the whole, M-GAM quickly produces accurate, interpretable models, providing new degree of transparency to predictions in the presence missing data. The code used for this work is available at htttp://gthuub..o//jdomlll////// GAM
Societal Impacts. M-GAM offers an interpretable way to deploy machine learning high stakes domains like medicine, even when data is missing. Modeling decisions for missing data risk unfaieeessDeansllme
introducing or perpetuating unfairness Jeanselme et al. (002)). We view interpretability as key tool for addressing this.
Acknowledgements
We acknowledge funding from the National Institutes of Health under 5R01-DA054994, the National Science Foundation under grant HRD-2222336 and the Department of Energy under grant DE- SC002135. Additionally, this material is based upon work supported by the National Science Foundation Graduate Research Fellowship under Grant No. DGE 2139754. Finally, we thank Jiachang Liu for his helpful advice.
10

---

References
Beaulac, C. and Rosenthal, J. BEST: A decision tree algorithm that handles missing values.
Computational Statistics, 35(3):1001-1026, 2020.
Breiman, I.. Random forests. Machine learning, 45:5-32, 2001.
Breiman, L. Classification and regression trees. Routledge, 2017.
Chen, T. and Guestrin, C. Xgboost: A Scalable Tree Boosting System. Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. pp. 785-794, 2016.
Dedieu, A., Hazimeh, H., and Mazumder, R. Learning Sparse Classifiers: Continuous and Mixed Integer Optimization Perspectives. Journal of Machine Learning Research, 22(135):1-47, 2021.
Ding, Y. and Simonoff, J. S. An investigation of missing data methods for classification trees applied to binary response data. Journal of Machine Learning Research, 11(1), 2010.
European Commision. Laying Down Harmonised Rules ON ARTIFICIAL INTELLIGENCE (ARTI-
FICIAL INTELLIGENCE ACT) AND AMENDING CERTAIN UNION LEGISLATIVE ACTS. Technical report, European Commision, 2021.
Feelders, Handling missing data in trees: Surrogate splits statistical imputation? European Conference on Principles of Data Mining and Knowledge Discovery, pp. 329-334. Springer, 1999.
FICO, Google, Imperial College London, MIT, University of Oxford, UC Irvine, and UC Berkeley, Explainable Machine Learning Challenge. lhttps:/ccommniis fico. com/ // expplainablee-nachine-eeeeers challenge 2018.
Fletcher Mercaldo, and Blume, J. D. Missing data and prediction: the pattern submodel. Biostatis- fics, 21(2)233-225, 2020.
Freund, Y. and Schapire, R. E. A teeision- generalization of on-line learning and an application to boosting. Journal of computer and system sciences, 55(1):119-139, 1997.
Groenwold, Informative missingness in electronic health record systems: the curse of knowing. Diagnostic and prognostic research, 4(1):8, 2020.
Hazimeh, H. and Mazumder, R. Fast Best Subset Selection: Coordinate Descent and Local
Combinatorial Optimization Algorithms. Operations Research, 68(5) 2020. doi: 00.1287/opre.219.1991
Hazimeh, H., Mazumder, R., and Nonet, T. LOlearn: A scalable package for sparse learning using 10 regularization. Journal of Machine Learning Research, 24(205): 2023.
Janosi, A., Steinbrunn, W. Pfisterer, M., and Detrano, R. Heart Disease. UCI Machine Learning Repository, 1988, DOI: https://doi.or///02444////55
Jeanselme, V., De-Arteaga, M., Zhang, Z., Barrett, J., and Tom, B. Imputation Strategies Under Clinical Presence: Impact on Algorithmic Fairness. In Machine Leurning for Health, pp. 12-34.
PMLR, 2022.
Johnson, A. E., Pollard, J., Shen, L., Lehman, L.-w. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., and Mark, R. G. MIMIC-III, freely accessible critical care database. Scientific data, 3(1):1-9, 2016.
Johnson, A. E. W., Stone, D. J., Celi, L. A., and Pollard, T. The mimic code repository: enabling reproducibility critical care research. Journal of the American Medical Informatics Association, 25(1):32-39, 2018.
Kapelner, A. and Bleich, J. Prediction with Missing Data via Bayesian Additive Regression Trees. Canadian Journal of Statistics, 43(2):224 239, 2015.
Kurtzleben, D. possible reasons the polls got so wrong this year. NPR, 2016. URL. https
7/www. npr. httpp///rgg/////////////00/00000

---

Le Morvan, M., Prost, N., Josse, J., Scornet, E., and Varoquaux, G. Linear predictor on linearly- generated data with missing values: non consistency and solutions. International Conference on Artificial Intelligence and Statistics, pp. 3165-3174. PMLR, 2020.
Le Morvan, M., Josse, J., Scornet, E.,, and Varoquaux, G. What's good imputation to predict with missing values? Advances in Neural Information Processing Systems, 1011111101540 2021.
Little, R. J. and Rubin, D. B. Statistical Analysis with Missing Data, volume 793. John Wiley & Sons, 2019.
Liu, J., Zhong, C., Seltzer, M., and Rudin, C. Fast Sparse Classification for Generalized Linear and Additive Models. Proceedings of machine learning research, 151:9304, 2022.
Mattei, P.-A. and Frellsen, J. MIWAE: Deep Generative Modelling and Imputation of Incomplete Data Sets. In International conference on machine learning, pp. 4413-4423. PMLR, 2019.
Miyagi, Y. Identifying Group A Streptococcal Pharyngitis in Children Through Clinical Variables Using Machine Learning. Cureus, 15(4), 2023.
Pedregosa, F,, Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825- 28300, 2011.
Razavi, P., Chang, M. T., Xu, G., Bandlamudi, C., Ross, D. S., Vasan, N., Cai, Y., Bielski, C. M., Donoghue, M. T., Jonsson, P., et al. The Genomic Landscape of Endocrine-Resiee Advanced Breast Cancers. Cancer cell, 34(3):477-43, 2018.
Rubin, D. B. An Overview of Multiple Imputation. In Proceedings the section of the American statistical association, volume 79, pp. 84. Citeseer, 1988.
the survey research methods
Rubini, L., Soundarapandian, P., and Eswaran, P. Chronic Kidney Disease. UCI Machine Learning Repository, 2015. DOE:
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., and Zhong, C. Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges. Statistic Surveys, 16:1-85, 2022.
Schafer, J. L. and Graham, J. W. Missing data: Our view of the state of the art. Psychological methods, 7(2):147, 2002.
Shadbahr, T., Roberts, M., Stanezuk, J., Gilbey, J., Teare, Dittmer, S., Thorpe, M., Torné, R. V., Sala, E., Lió, P., et al. The impact of imputation quality on machine learning classifiers for datasets with missing values. Communications Medicine, 3(1):139, 2023.
Sperrin, M., Martin, G. P., Sisk, R., and Peek, N. Missing data should be handled differently for prediction than for description or causal explanation. Journal of clinicul epidemiology, 125: 183-187, 2020.
Stekhoven, D. J. and Bühlmann, P. Missforest parameerii missing value imputation for mixed- type data. Bioinformatics, 28(1):112-115 2012.
Stempfle, L. and Johansson, F. Minty: Rule-based models that minimize the need for imputing features with missing values. In International Conference on Artificial Intelligence and Statistics, pp. 964 972. PMLR, 2024.
Stempile, L., Panahi, A., and Johansson, F. D. Sharing pattern submodels for prediction with missing values. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp. 9882-9890, 2023.
Therneau, T. M., Atkinson, E. J., et al. An introduction to recursive partitioning using the rpart routines. Technical report, Technical report Mayo Foundation, 1997.
Twala, F., Jones, M., and Hand, D. Good methods for coping with missing data in decision trees. Pattern Recognition Letters, 2977)99509956, 2008.
12

---

US Food and Drug Administration. Artificial Intelligence/Machine Learning (Al/L)--Basse Software as a Medical Device (SaMD) Action Plan. Technical report, Office of the United Nations High Commissioner for Human Rights, 2021.
Valdiviezo, H. C. and Van Aelst, S. Tree-based prediction incomplete data using imputation or surrogate decisions. Information Sciences, 311:163-188 2015.
Van Buuren, S. Flexible Imputation of Missing Data. CRC press, 2018.
Van Buuren, and Oudshoorn, K. Flexible multivariate imputation by MICE. Leiden: TNO, 1999.
Van Ness, M. Bosschieter, T. M., Method: From Low High Dimensions. Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 5004-5015, 2023.
R., and Udell, M. The Missing Indicator
Wang, C. and Feng, Z. Boosting with missing predictors. Biostatistics, 11(2):95-212,, 2010.
Zhu, C. Q. Tian, M., Semenova, L., Liu, J., Xu, J.. Scarpa, J., and Rudin, C. Fast and interpretable mortality risk scores for critical care patients, 2023.
13

---

X
X
M
Figure 7: The case constructed prove Proposition A.1 Model fi takes as input variables in blue circles, f2 takes as input variables in red circles, and dashed red circles indicate variables involved in generating missingness in X1, denoted by M. Here, 61 and (2 are unmeasured noise. By using M as an input, f2 can infer information about Y after noise from is considered.
A Proof of Proposition 3.1
First, recall Proposition 3.1
Proposition A.1. Let I Rd + R&d be an oracle imputation function that replaces all miss- ing values in vector with the correct non-missing entry. For random variable X R. let fx(X) == (x))0)(()))005) be the Bayes' optimal model using perfectly imputed data and f2(X) == 1 [XYisss0. be the Bayes' optimal model using missingness as a value. There exist data generating processes for X and Y where P(Y
= f(())) < P(Y f2()))
Proof. We prove Proposition 3.1 by construction.
Let X2 Bernoullipp 0.5) and Y |X1 eil where e1 Bernoullipp k1) is unobserved noise. Let M denote missingness in X1, where M = |Y E2l with 62 ~ Bernoulli(p k2) being unobserved noise. Let lyg < K1 0.5.
Consider two oracle models f1 and f2, defined as: fi(X1, := (x))20(())
,x, X1l20.s) f2( ||//|///20.5] = IM Intuitively, fi perfectly imputes X1 when X2 is missing, then predicts using X1 and X2, while fz predicts Y using M and X2.
We first evaluate the expected accuracy of the imputation model, i.e., P(f((X, X2) Y). Using the law of total probability, we have:
(X X2) Y)= Σ ((),, 2221,
P(z))F(2)PP(( 2221,22)
Since X₁ 1 X2
.22
0.25
Σ
((f((11)2)) = 22))
251 .02
Noting that, when X, X₂ = 1, we have y = = 1- 6.. P(Y= 1) 0) = hy When at least one of X, and X₂ 0, we have Y = 1 (/, so P(Y = = = =) = 1) = k1. Since fi simply evaluates whether the expectation Y given X1 and X₂ greater than 0.5, if k1 < 0.5, we have (1, = and fi 1) (1, = (0, = 0. Thus, the expected accuracy is:
P(f(X1, = =) 0.25 Σ
=
(21, = 2
= 0.25 Σ
P(E1 = 0)
P(e1 0)
- k
14

---

Similarly,
= =Y) =
Σ
P(m, 22) = Y 1mm x2)
22
Σ
P(=))((mozy)P((f((m,, = Y mmm
=
0.5 = 1/ = 1)2(( Y m,, 222) + Pm = 1| = 0) Y 1\m, 22)
+ Pm = 0/2r 1)P(()(((( m2, P(m 0 [122 - 0) = -\m,
We can compute each term in the conditional probability table 22 as follow:
Pmmm- = 0/ = 0) = Σ
P(m = ø2 = 0, 9,)))( = 0)
y
= (1 - k2) - k1) + k
Pmmm 1/22 = 0) = Σ Pm = 1|22 = 0, y) 1)) 0)
= 1:2 - k1) + - k1
P(m 0/002 = 1) Σ
Pm = 0/ (,y))(y/(r22) = 1)
91
+ 0.5(( k2(0..k)
k1)) + - (0.55 k1) + 0.5k1)
0.5
P(m 1/ 1) = Σ P(m = 1|22 = 1, = 1)
(1 (0.5kz 0.5(1 A1)) + (0.5 kn) 0.5k1)
= 0.5
Plugging these values in, we have
IP ((f2(M,X)) =Y) = 0.5
(1 - 11 - k2 + 2kg 22
Y Y|m = 0, x2 = 0)
+ (k2 + 1 2k Y/m 1, 0)
+ 0.55((f2(m x22 =0 = 1)
+ 0.5P((f(m, x2) Y| = 1,
2 0.5
(1 - h1 - h2 + Ak)
Ylm = 0)
+ (k2 + k1 2k 2k5 = Y|m = 1)
+ 0.5P((22m, Y =)
+ 0.5P(f2(m, m m/m==1
0.5(1 - k1 - k2 + 2k + 122 + 4 - 2k + 0.5 - -k2)
1-k2
Thus, we have (X,, X²) = Y) = 11 bi and PM2( X2) Y) 2 - kg Since K2 < ki, we have
k2 ki
1- k2 > 1 - ki
P(2)(M, - Y) P(f Y),
required, where the last step follows because IP (X1, = Y) 2 1-12-
15

---

A.1 Alternative Proof
While the above offers a proof of Proposition 3.1 one might wonder whether the proposition holds outside of the case we constructed. In the previous proof. we aimed for case that resulted in an easy follow proof. Here, we prove Proposition B.T by second construction, to show that imputation can be worse than missingness-an as-a-value even if there is more noise in the informative missingness than in the data itself, and even when missingness relatively rare.
Proof. Consider case similar to that used for the proof in proposition 3.1, where we add an additional variable X3 and have the noise for the missingness be higher than any other noise.
Y X e1/iE1 Bern 12
Y E2/, 62 (e) 00
with probability with probability
M
X3 Y -eal, ~ Bern
We also adjust the probabilities for X1 and X₂ being true so that this is a balanced classification problem: X1, ~ Bern( 1 ) so X1X2 ~ ())
Note that we now also have missingness at well under 50% of the data (misssngness happens a quarter of the time).
The bayes optimal model with perfect imputation of X,, and no access M, is still just to predict in accordance with Xi X2 for PPX,, [X,XX, X2, Y)
T2 When X, X3. all information available
11
suggests X, X2 correct. When X,X2 * X3, we still have the bayes optimal prediction aligning with X2: P(Y = 1| 1, X3 0) 11 0.5.
# > 0.5 and P(Y 0 X2X2 = 0, X3 1)
21
we instead only have access to X, when not missing, but we also know when X, missing (i.e. we know M). then the following approach will perform better than the above model:
[Xs. X2x23
if =
Y
MM
When M = 1, we make additional errors relative to the previous approach at rate
PPM = (F(( r) ((X)X2)) +Y))
- 122
1 1
4
1
528
When M 0, we improve our classifier's accuracy by:
P Plmputation model is wrong, model with missingness right and M 0)
P(Imputation model is right, model with missingness is wrong and M 0)
P(X,X2 Y X3,, M 0) P(X,
-) X, M = 0)
= =P(X,X,=11
= 0, M = 0)
= 0,Y 1,M = 0)
=P(X,X2 1)JP(Y 0|X, 1)IP(X, = 0/Y - 0 = 0)
- -P(X,X2 = 11)P(Y = = -) 1:0M-00: = = 1))
1 1 107 111
11 212118 15
2112
528
16

---

That is, the proportion of cases the classifier that uses missingness will gain is greater than the proportion it will lose relative to the imputation approach. So, the model that uses missingness outperforms the imputation model.
17

---

B Proof of Corollary 3.2
Corollary B.1. Let Rif, X, Y) denote the risk of a model for data X, Y, and R the optimal risk. Let I: (R U NA)a Rd denote the oracle imputation function of Proposition 3.1 Under perfect imputation, it is possible for there to be no Bayes optimal model built on imputed data. That is,
2XX, Y) A R(f o 1, = =
Proof. Recall that, in Proposition A.I we constructed distribution of X, Y such that
R(f,,X,Y) > R(f,,x,,)
where
fi arg min I(f,
f2 = arg min Rf
By definition of R", we have that R(f2, X, Y) 2 R", immediately yielding
R(f,, X, > R(f.. X,Y) 2 R
18

---

C Proof of Theorem 3.4
Theorem 3.4 states:
Theorem C.1. Consider any GAM g R
TR, parameterized by 3, with shape functions defined as
linear combinations over boolean features (either thresholds fz( By) = ) 1 lns, St,,ul
or a feature that was originally boolean). Suppose some observations are missing boolean feature and that this feature imputed such that the modeled probability of wiS being true, P(z,,
1 -xx,,-b) (where Xi.-b refers to all covariates except b) is an affine function h Xi,-b [0,
For any parameterization B of 9. let [Elg(x; 3)] = IP( 1|x,,-))g((x ( ;++t):37 (3
O (x),()) (x) where x(0-) denotes x4 with Fi,b and (0-) denotes x; with Fi,6 Then, there exists a model in the model class M-GAM (which does not use imputations), that recovers this score Exg( 3)] for all i.
Proof.
C.1 Model Family Definition for (xg B)]
The general GAM, as per equation 1 is (xx;; () Bo + E (Ft,s188)) functions incorporated,
with the shape
d len(t;))
gxx: B) So
+
Bz (mx, Sty.al
Rm
where 1 (zy, st. values corresponding to missing feature j are replaced with some imputed value based on GAM. To highlight the sometimes sometimes-missing boolean feature b, may rewrite this
len(t;)
g(x;8) :30 +
Σ 57h
Σ
Bi (as, (os, 36 (x..o) + 1 (mral
For examples x, with feature b missing, the value [p),o must be imputed prior to being used. Under the setting of this proof, the feature is imputed by modeling the probability as an affine function of an additive score. Without loss of generality, that means there exists some additive model with
Stgu) for some set real-valued coefficients C, with corresponding probability Fi,b given, WLOG, by:
score: sx),-o) = Co + Ejzb Eket
s(x; b) min, (s(X,, P( = 11x = max Xv b)) mina (s(x;, + d
where >> 0, d > 0, + d 1. Note that set Co 0 without loss of generality because it will appear in both terms in the numerator and both terms in the denominator, having no impact on the overall probability.
The values max; (s(x,,_) and min. (s(xp,-o)) denote, respectively, the maximum and minimum of the score function over any possible x;.
If take the expectation of the score function the probability distribution learned by the imputing additive model, we get:
len(t,) Σ j'sb 1=1
g(x;33) = 3o
+ Bo1 + BB 1 404mmattee 1 1(x
"Note that sufficient Showing that this theorem holds for M-GAM in the less expressive case where there is only a single reason for missingness, also shows that the theorem holds when there is added expressiveness to the model class. As such, we do not include notation for distinct missingness reasons the proof.
that this this theorem holds single missingness reason,
19

---

(Recalling that 1 [msss] is defined to always be false when 25,6 is missing, and maeat(a, v) is true iff 2i,b is missing.)
We can put the #000 term back in the summation over thresholds and have:
d len(tz)
g(x;33)= 30 +
Σ
B (axljan Sttuel
+ B6 (044mmtterss))0p(((.. Par, = 1(xx,-b)
k=1
Simplifying further:
len(t,)
8(x b) ming ming((((; (((xx,)b))) + + Bg [meat(2)
g(x;1)) = :30
Σ
Bz.k a.. Stziel
max (s(x,r -b)r - miny (s (x)x
len(t,z)
s(x_6) ming (s(x),-b)) + So (mcat(rr.))000f max, (8(x),_))) miny (s(x),-b))
g(x;; B) 30
Σ
By l (taa))
+ 961 (m
len(t,)
s(x_6) + Aba max (s(x,, miny (s(xp,-b))
g(x;1) (3 = 30 Σ
By (os
ming (s(x,,-b))
max, ( min (s(x),-b)) Bo x)
ltn)
o(x;; (3 = 30 +
B5 (ta)))
A=
ABQ
+ s(x b)1 (m
max, (s(xx, min (s(x
Aba ming (s(x,,
[m)at(z..)
Bbd
(ax min, (s(x),-b))
can return to our shape function notation and write:
0(x;;8) Ba +
3ya =10 max (s(X;, miny (s(Xx, (s(x,,,6))
3,a ms (s(x,,
Bad
max; (s(x,_)))) minz (8(xe,-b))
C.2 M-GAM Model Family Definition
Recall from section
Equation
3.3 that the form of M-GAM is:
Imiss (x;; B, Juiss σ)) Bo h ( Bai
Fmiss)
Σ
hi "
='=1
Filling in the shape functions, have:
20

---

Gmiss Xx; (x; B, Bmiss ,,) = 30 + Σ
15 +
Σ
gmiss Jam
[msut([Fs, (=m)))
==1
(6)
len(t;)
O k.m mmoaa(es,)==) and Sty.
j=1 j=ly'=1 m=1
k=
C.3
Show that any linear imputation model has an equivalent representation as a missingness term model
To show this, sufficient to show that for any coefficients of a linear imputation model, there exists a parameterization M-GAM such that Equation (6 is equivalent Equation (5).
Start from Equation
Gmiss (x; B, Beiss .) =
Σ
Σ
Bmiss 1 (1; =
m==
len(ts)
a J.m meat(ar, )=m and Sty Styraal
j=1 =
Rearranging:
Imiss (x;i B, Jmiss .) = Bu + Σ (2,,,
5=1
f.m (moat(as))=
j=1
len(t;)
aj moat(os, and 2t.
j=1 f
We can pick the non non-missing coefficients for Equation 6 to match those from Equation 5 leaving
Bo + f = 3o + ΣØ f
Now pick 3miss 0 except when b. and Bmins
BBd
3sa (s (a(x, min, (=(x.1_))))
Then
we have:
d Umiss (X;; B, Jmsss a) 30 + Σ Ba (2 B3)
j=1
35a miin (s(pp.- max (s(xr, miny (s(x,,_b))
(meat(22,))=m)) =m]
Brd
len(t,)
ājj' k.m 1 (mcat(rs,) and xn Styr
1=1 11= m=1
k=1
21

---

Gmiss (x B, Busss .)) Bu Σ (z BB)
1=1
d
len(t;)
aid J.m (aml[mmati (mce(maa((ss))) x, Styall
k=1
35a (s
1 [moat(22,)=ml
(Byd
maxy (s(x), ming (s(X),
d Ekm(( 2/'=1 m=1 k= ab, k.m
Now, parameterize
to match
8ya s(r-b) (e(x, a)) min. (=(x
by, for each j', setting
Bba
k,, = Cy max, (s(xy,-b)) ming (s(xy,-b))
Set Rj = 0 for all j # b. Then we have:
Gmiss (xii B, gniss a)) 30 + Σ fz
j=1
35a (s(niinnnn((ssss-e
1 [moat(2x,))=)
Byd
maxy (s(x,v -b)) miny s(xxx,))))
Bba
[mmad(tzaa)1))((( max, (s(X++, miny (s(Xx.-b))
Now, as required, we have shown that Equations and 5 are equivalent.
22

---

D Experimental Details
D.1 Conditional Probability Table for Added MAR Missingness
In Section 4.1
we evaluated the predictive power of M-GAM when additional synthetie missingness was added to each dataset we studied. Let X denote the variable missingness is being added to, X2 another column from the dataset used determine whether to add missingness, and Y our target outcome. Let M denote whether synthetic missingness is added to X1, and let Qx2 (p) denote the p-th quantile of Xg. For target missing rate r, Table T shows the conditional probability being missing given each value of Y and X2. Note that missingness does not depend on X1. making this an MAR setting.
X1
Y=0 PMM-1YX000
Y=1
PM
X2 Qx, P(M = [M=-Y,X, X =0
Table 1: The conditional probability table obeyed when adding synthetic missingness.
D.2 Data Processing
We follow the structure from (Shadbahr et al.) 2023) for running our imputation baselines and selecting train/test splits. To collect the data for MIMIC, we used the process in (Johnson et al., 2018).
and additional steps by (Zhu et al. 2023) to convert the data to a single tabular dataset.
D.3 Detailed Experimental Setup
For every GAM we fit (M-GAM, FastSparse, and non-LO GAMs), we created an indicator for each of 8 quantiles (the 0.125 quantile, the 0.25 quantile, and on). On FICO, include missingness indicator and interaction terms as appropriate for four encodings of missingness: the three types in the original dataset (no information, no usable information, and report available) and an added indicator which is true for any type of missingness. All other datasets contain only one type of missingness.
We fit M-GAMs using FastSparse (Liu et al. 2022), p. and in all cases set the "max support size" variable to 100. This prevents the algorithm from exploring models with greater than 100 non-zero coefficients. For all experiments that did not report complete sparsity versus accuracy curves, we used 5-fold cross validation to select the value for the to sparsity penalty. We searched over the following set of values for for each GAM: 20, 10, 5, 2, 1, 0.5, 0.4, 0.2, 0.1, 0.05, 0.02, 0.01, and 0.005. We optimized for AUC Breast Cancer and accuracy all other datasets when using cross validation. We fit all non-sparse GAM's using SKLearn's implementation of logistic regression over binned data.
We evaluated the performance of variety of classifiers on all datasets in Section 4.4 For each datasets, we used MICE to impute 10 distinct datasets, and fit variety of predictive models (a logistic regression, a decision tree, a shallow neural network, and an XGBoost classifier [Chen & Guestrin| (2016) on these datasets. For each baseline classifier, we also provide accuracy for model fit with and without missingness indicators added via the SMIM procedure Van Ness al. (2023). We used cross validation to select hyperparameters separately for each imputed training dataset, and ensembled the 10 models for each model class to produce a single predictive model. Cross validation was performed using 5 folds via GridSearchCV from SKLearn (Pedregosa et al. a 2011)) and the SKLearn implementation was used for each model class considered other than XGBoost. The hyperparameters considered are:
AdaBoost model (Freund & Schapire 1197], random forest (Breiman 2001).
Logistic regression: ["C*:[0.01, 0.1, 1, 10], "penalty": ("12"),"mxx "max iter": [10,000], "tol": [5e-2]}
Random forest: ("n_estimators""125 [25, 50, 100, 200], "criterion": "cntropy"]
AdaBoost: ("n_estimators"" 25, 50, 100, 200]}
Decision tree: ("max_depth"iIe 7, 9, None], "criterion":"" (""gini",, "cntropy"))
23

---

Neural Network: {"hidden layer s(02s=:[(50.). (100,), (200.), (50, 50), (50, 100), (100, 100), (100, 200), (200, 200)], "tol": [5e-2], "max iter": [1000]]
XGBoost: 'n eeeeeetmtttrs:: :[100, 500, 1000), "gamma":: :10, 0.1J, 'lambda': :[-5, 2], "al- p.a':[.5, 1, }
We repeat this procedure for distinct train-test splits for each dataset considered.
D.4 Computational Resources
All experiments were performed on an institutional computing cluster. All experiments that involved timing conducted using one Tensor TXR231- 1000-0000R D126 Intel(R) Xeon(R) CPU E5-2640 y4 @ 2.40GHz (512GB RAM 40 cores), except for MIWAE timing experiments, which use one NVIDIA Tesla P100 GPU. When runtime was reported, experiments were run whatever hardware was available on the cluster at that time.
24

---

E
Additional Experiments
E.1
Additional Datasets
Throughout this section of the appendix, we will consider two new datasets in addition to the four datasets introduced the main body of the paper (FICO, Breast Cancer, MIMIC, and Pharyngitis).
We add the Chronic Kidney Disease Rubini et al. (2015) and Heart Disease Janosi et al. (988) datasets from UCL, refered to simply as CKD and Heart Disease respectively. CKD consists of 400 samples with 24 features, and involves prediction of chronic kidney disease using medical features.
Heart Disease concerns predicting heart disease using medical and demographic features, with 303
samples and 13 features. Note that the outcome in Heart Disease is integer between 0 and 4; binarize this label such that we classify no heart disease (0) versus any heart disease (1, 2, 3, 4). Both of tshese datasets contain only one missingness encoding.
E.2 Evaluation of Alternative Imputation Methods
In the main body of this paper. we focused our evaluation of baseline classifiers on impute-then thenppe--ee---eeictt using the MICE method for multiple imputation. However, a wide variety of multiple imputation methods are available. Using all six datasets, we evaluate the runtime and accuracy of four imputation methods: MICE (Van Buuren Oudshoorn 1999). MIWAE (Mattei Frellsen 2019), mean value imputation, and MissForest (Stekhoven & Bühlmann 2012). We impute ten alternative datasets for each ten distinct train-test splits for both FICO and Breast Cancer. For a given train-test split, we fit model from each of the model classes described in D.3 for each imputed training dataset. We ensemble these ten models to produce a single predictive model per train-test split, and evaluate the test accuracy (FICO) test AUC (Breast Cancer) of this ensembled model.
Figure [8 contains box-and- and-whiskers plots for the accuracy of each method considered across all six datasets. The imputation method does not generally have substantial impact on the performance of the resulting impute-then then-predict classifier. We also see that, across the two datasets not considered in the main body of the paper (CKD and Heart Disease) M-GAM continues to provide comparable accuracy to all baseline methods.
Figure D shows the time required to produce a predictive model under each method for each dataset. As in the main body of the paper, we show the sum of the time required impute data and the time required to produce the most accurate model. We see that, despite resulting in models with comparable accuracy to those produced by MICE and Mean imputation, MissForest and MIWAE take longer to compute on the majority of datasets. On all six datasets datasets, M-GAM is at least an order of magnitude faster than all three multiple imputation methods, although mean imputation tends to be fastest.
E.3 Scalability of M-GAM and Imputation Methods
This section studies how well each method scales in terms of the number of samples in the dataset. For each dataset, we take subsamples of increasing size (25%, 50%, 75%. and 100% of samples in each datasct) and run each seeute-then-preee predict procedure, as well as M-GAM over 10 distinct train-test splits. Figure reporrss the total time taken to produce model for each imputation method and M-GAM on each dataset/subsample combination. We find that M-GAM scales no worse than any of the imputation alternatives in terms of runtime.
E.4 Evaluation of Different Thresholds
Throughout the main body of this work, we reported results using 8 evenly spaced quantiles to threshold our input variables for both M-GAM and FastSparse GAMs fit on imputed data. In this section, we evaluate the sparsity versus accuracy curve for M-GAM under different binning strategies. In particular, we evaluate the performance of M-GAM on FICO with 4, 8, 16, and 32 evenly spaced quantiles. Figure IT shows the results of this analysis. As the number of quantiles increased. M-GAM remained sparse despite the exploding number of interaction terms, and fact for quantiles the interaction terms lead to especially sparse and accurate model. Beyond these observations, we found that reasonable changes the number of thresholds we consider did not significantly impac performance.
25

---

M-GAM (Indicators
sontrreCiosstrrr (SHIN)
Fordem SMIM:
GAM Interactions)
Log silRegression
Randem
AdsBoostCassttee (No SMIM)
Log sitRegression (SHIM)
XGaClessfier 1No SM M)
adaBoostClesstfe (SMI))
MLPClassifle (No SHIN)
XGBClesafer (SMIN)
DoceiionrreClaasfteee
MLFClesstfee
MissForest
0.84
0.82
0.78
0.75
0.72
1.00
0.85
0.80
0.73
0.72
0.71
0.68
0.9
Test Sccurao
0.5
0.5
Pharyngitts
0.6
0.5
0.4
0.90
0.89
MIMIC
0.87
Figure 8: Test accuracy of M-GAM compared against variety of baselines for four imputation methods and six datasets. Each column corresponds to a different imputation method, and each row to different dataset.
26

---

Breast Cancer
CKD
FICO
Time (Seconds)
10
10a
10-1
Heart Disease
MIMIC
Pharyngitis
Time (Seconds)
10a
10a
10-1
M-GAM (Ind)
M-GAM (Int)
Mean
MissForest
MICE
MIWAE
M-GAM (Ind)
M-GAM (Int) Mean
MICE MissForest
MIWAE
M-GAM (Ind)
M-GAM (Int)
Mean
MICE MissForest MIWAE
Figure 9: Runtime of different methods on Breast Cancer, CKD, FICO, Heart Disease, MIMIC, and Pharyngitis. For each imputation method, we report the total time required to impute missing data and fit the best performing impute-then-prres M-GAM (Ind) an M-GAM with indicators and M-GAM (Int) is an M-GAM with interaction terms. Error bars report standard error of total runtime over 10 train-test splits.
classifier for that dataset and imputation method.
E.5
Evaluation of MICE with Different Numbers of Imputations
Since MICE is a multiple imputation method, we needed to choose how many datasets we allow MICE to impute for each of experiments. In this section, evaluate the runtime versus test accuracy for models built on various numbers of imputed datasets for FICO. We evaluated each non-GAM baseling model considered in the main paper when ensembled over 5, 10, 20, and 30 MICE imputed datasets.
shows the accuracy versus runtime for each number of imputations. In Figure 12 we see that there is slight improvement in the accuracy our classifiers when increasing from to 5 imputations, but significant performance gain for any larger numbers of imputed datasets. As such, we opted to use the moderately fast and performant choice imputations.
Figure 12
E.6 Extension of Sparsity/Accuracy Results to Further Datasets
We focus on the FICO and Breast Cancer datasets for much of the main paper, alongside Pharyngitis and MIMIC In Figure 13 we show the superset of our sparsity-accurant accuracy results that includes the two UCI repository datasets, Heart Disease Janosi et al. (1988) and CKD Rubini et al. (2015). In Figure 14 we show the superset of our results for the data with added MAR missingness, for all datasets.
E.7
Evaluation of Alternative Distinct Missingness Encodings
As we discuss in the main text, allowing our model to encode different reasons for missingness allows the ability to handle multiple reasons for missingness, improving the model's power. However, using only single, overall reason for missingness could potentially allow for handling a larger set of missing data cases with fewer coefficients. These two encodings are not mutually exclusive; we could augment in both ways, as described in Equation E. For this reason, we investigate sparsity and test accuracy the FICO dataset across range of choices for whether to use distinct encodings,
27

---

(becords)
307
10"4
(Seconds)
307
10"4
Tire (Seconds)
307
10-4
(Seconds)
107
10-4
Tite (Seconds)
107
10-4
Pheryngitiis
Pharyngibs
Pheryngitss
Pharynghtis
(Seconds)
207
10
M-GAM (tnd)
M-GAM (ir)
Missfrrest MINAEE
M-GAM (tne)
M-GAM (tnt)
Hear
Mace
Misfoorst
M-GAU (tntl GAM
Menn AnCE Milnfoo Amoste
M-GAM (1me))
M-GAM lint)
MecE Minioorent
MiRiNE
Figure 10: Runtime different methods over 25%, 50%, 75%, and all of Breast Cancer, CKD, FICO, Heart Disease, MIMIC, and Pharyngitis. For each imputation method, report the total time required to impute missing data and fit the best performing dataset and imputation method. M-GAM (Ind) is M-GAM with indicators and M-GAM (Int) is an M-GAM with interaction terms. Error bars report standard error total runtime over train-test splits.
souute-teen-reee
classifier for that
28

---

Test Performance vs Sparsity
Quantiles
16 Quantiles
0.75
0.75
Test Accuracy
0.74
0.74-
Test Accuracy
0.72
0.72
0.71
0.71
0.70
20
40
80
100
0.70
20
40
220
240
Quantiles
32 Quantiles
0.75
0.75
Test Accurary
Test Accuracy
0.73
0.73
0.72
0.72-
0.71
0.71
0.70
20
40
120
140
160
0.70
20
40
360
380
400
Nonzero Coefficients
Nonzero Coefficients
(Indicators Only) M-GAM Interactions)
GAM MMI
MICE
MissForest
Figure 11: The sparsity versus accuracy plot for four distinct binning strategies for M-GAM on the FICO dataset.
0.75
Number of Imputations
0.70
Test Accuracy
0.65
0.60
Model Type
LagisticRegression
GAM (Imputation)
RandomFForettllasiffer
AdaBcostClassifier
0.50
DecisionTreCCassffee
MLPClassifier
0.45
100
200
300
400
500
Runtime (Seconds)
Figure 12: Test accuracy versus runtime for models built on different numbers of MICE imputed datasets. Each color represents a different number of imputed datasets, and each shape represents different ensembled model fit on these datasets.
29

---

Test Performance vs Sparsity
FICO
Pharyngitis
0.75
Test Accuracy
Test Accuracy
0.70
0.72
0.65
0.70
0.60
20
40
150
20
40
Breast Cancer
MIMIC
0.910
0.76
Test Accuracy
0.905
Test AUC
0.74
0.72
0.895
0.70
20
70
80
90
0.890
20
60 360
350
CKD
Heart Disease
1.00
0.90
Test Test Accuracy
0.98
Test Accuracy
0.85
0.96
0.80
0.94
0.75
0.92
5
10
100
125
0.70
0
20
40
60
Nonzero Coefficients
Nonzera Coefficients
M-GAM (Indicators Only)
SMIM
GAM MICE
MIWAE
M-GAM (w/ Interactions)
GAM w/ MVI
MissForest
Figure 13: Test accuracy sparsity for M-GAMs relative to competitor GAMs 6 datasets.
overall encodings, or some combination of the two. We explore a variety of combinations in Figure allowing nonzero values for different subsets (specific indicators), a overall (overall indicators), (specific interactions), and goverall.miss (overall interactions). We find no dramatic differences across these choices.
15
Definition E. Given parameters o, a overall gmiss goverall.miss, and 3, an M-GAM is defined as
d
Jmiss (x;; B, Bmiss goverall_miss ovenal = Bo +
Σ
hy (Zi.) Bg, Biiis gpvenillmiss
d hig (Is.j, Di,j aggsr aj
overally
where
len(t,) Σ m=1
hig Rigs o.jr gperall
Oij's k.m |meat(ss and Styral
k=
(en(t,)
overall 1 [mcct((r, and x, J.t k
and
(24.53 3z. Byiiss, g govenal.miss) = (x
Σ
gmiss Imcat(2)))==))
+ goverall (mcaa(rr,)))
30

---

Performance under Added Synthetic MAR Missingness Rate & 0.25
Rate & 0.5
0.85-
Test Accuracy
FICO
0.80
0.83
0.75
0.75
0.700
0.70
20
135
0.85
0.85
Brad Cancer
Text AUC
0.50
0.80
0.75
0.75-
0.70
50
0..70
50
0.75
0.75
Text Accuracy
Phayynglis
0.70
0.70
0.65
0.65
0.60
0.90
0.93
0.93-
Tedt Accurayy
MMII
0.322
0.332
0.90
0.90
0.80
350
0..50
300
825
350
Test Accuracy
0.98
0.98
CKD
0.316
0.98
0.94
10
128
0.93
Heart Diarae
Test Acaracy
0.85
0.85-
0.80
0.80
0.75
0.75-
0.70
0.70
# Norzero Confficients
# Nonzero Coefficintts
(Indicators Only)
M-GAM (w/ Interactions)
Figure 14: Test accuracy vs sparsity for M-GAMs relative to competitor GAMs 6 datasets with added missingness.
31

---

Test Performance Indicator: Specific
Sparsity for Distinct Missingness Encodings
Indicator: Specific Overall
Indicator: Overall
0.75
0.75
0.75
0.74
0.74
0.74
Interaction: Specific
Test Accuracy
0.73
0.73
0.73
0.70
0.70
0.70
0.75
0.75
0.75
Interaction: Specific Overall
0.74
0.74
0.74
Test Accuracy
0.73
0.73
0.72
0.72
0.72
0.71
0.71
0.71
0.70
0
20
40
0.70
20
40
0.70
20
40
0.75
0.75
0.75
0.74
0.74
0.74
Interaction: Overall
Test Accuracy
0.73
0.73
0.73
0.72
0.72
0.72
0.71
0.71
0.71
0.70
0
0.70
0.70 0
Nonzero Coefficients
# Nonzero Coefficients
Nonzero Coefficients
M-GAM (Indicators Only)
MICE
MIWAE
MissForest
M-GAM (w/ Interactions)
Figure 15: Results on the FICO dataset using different choices of missingness augmentation for indicators and interactions. "Specific" refers the distinct missingness used throughout the text. "Overall" refers to augmenting our matrix while treating missing data as all having single missingness reason.
32

---

Additional M-GAM Visualizations
In this section, we visualize three additional M-GAM, one with and one without missingness interactions on FICO (Figures and and 18 respectively), and one without missingness interactions on
Breast Cancer (Figure These figures are best viewed digitally.
Note that several shape functions in Figure 19 are simply flat lines; this is because several variables in Breast Cancer (e.g.. "M Stage" and "Overall Patient Receptor Status Triple Negative") binary.
G License Information for Used Assets
In this section, we provide license information for every external asset used in this paper. The Breast Cancer, CKD, Heart Disease, and Pharyngitis datasets are available under creative commons. FICO is used under its own license, the details of which can be found here: https: htt////commmt/ www.llennneeeeeeeeeeeeeee
fico. com s/ expl waaamaale-nachnn-eeaaiiiii available under the MIT license. We use code from Liu et al. (002) under the MIT license, and the code from Shadbahr al. (2023) under BSD 3-Clause.
MIMIC-III
33

---

(No Usable Information)
Missing Variable 3: NumSatisface fatttisaaaotiiiit TacooyyTareees Trades (Not Investigated or Not Found)
Missing Variable 8: ASinceMostRecenn
No Missing Data
-0.4
0.8
0.44
0.8
-0.4
0.8
40
40
ExtemalRisk Extimale
Vartable 1:
80
80
Missingness Dependent
ReseentDeeqeeee77000
Moin
Vanabll 2:
75
75
0
Variabble AumSatiifactoryy Trades
20
20
Verieble 4: AverageMMiniilee
400
Varieble 5 MaxDelk2Putkk RecLael
Not Missingness Dependent
7.5
Viriiable 6: NetFraccion Revolving
150
Variable Percerainstall Traani Trades
100
Figure 16: expanded version of Figure 2 with variable names included. "MSinneMostReeemmDDda is the number of months since the individual's 's last delinquent payment and "MSinceMostRecemmin gexc17day" is the number of months since the individual's 's last inquiry, excluding those within the last week; all features are described in the FICO challenge (FICO et al. 2018) data documentation.
34

---

Missing None
Mosing
Miwehhoolhoo eggodhllgp
Figure 17: A visualization of a M-GAM without interaction terms on FICO. The shape functions on the left are selected based which variables missing, with the relevant missing variable noted to the left. The shape functions on the right are used in all cases.
Maaing
Heaiig Udbrandeccbo Bryesabiiyy
Figure 18: An additional visualization of M-GAM with interaction terms FICO. The shape functions within left set of brackets are selected based on which variables are missing, with the relevant missing variable noted to the left. The shape functions in the right set of brackets applied in all cases.
Missing Nana
Missing Metastaric Recumenee Time
Figure 19: A visualization of a M-GAM without interaction terms on Breast Cancer. The shape functions within the left set of brackets are selected based on which variables missing, with the relevant missing variable noted to the left. The shape functions in the right set of brackets are applied in all cases.
35

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The primary claims made in the abstract and introduction are that our method improves upon sparsity without sacrificing accuracy. We also claim that we improve performance synthetic cases, and not substantially harm runtime. All these claims are directly supported by experimental section.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We discuss that the lo penalty we impose does not perfectly impose the prior we would like, and that missing data methods as a whole are vulnerable to distribution shift.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well well-specification asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairnsss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
36

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: A complete proof for each theoretical claim in the paper can be found in the appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: A thorough description of our experimental framework is provided in the appendix. We also provide references each dataset we use, and will release the code upon publication.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibilityy can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily the architecture clearly and fully.
model architecture, the paper should describe
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibiliity In the case of closed-source models, it may be that access to the model is limited in
37

---

some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
All datasets used are openly accessible (although MIMIC-II requires an application process) and code will be released. We also thoroughly describe each experimental setup in the appendix.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co
public/guides/Codees wtw.:/ddse/oeeeeuuaasssssooooooo
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/tipps.cc/ublic/guiieeee gwwpe//ddee/eee CCodeSubeeiieeeee ssi onyolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
The procedure and random seed used split data, all hyperparameters, selection methods, and implementation details are provided in the code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
All experimental figures include uncertainty quantification through error bars.
Guidelines:
The answer NA means that the paper does not include experiments.
38

---

The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
show in tables or
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
The appendix provides details around all computational hardware used. Timing experiments are provided throughout the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics httpr//rrrr/////p......ppp//p//p
Answer: [Yes]
We have reviewed and are compliance with the code of ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
We discuss the broader impacts of this work in the conclusion.
39

---

Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considcrations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
This work poses substantial risks for misuse, beyond those associated with any predictive model.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
The license under which each asset is used is given in the appendix, and comply with the license for each asset.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
40

---

The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, datasets.paperrstt wwppppsipwaaaae thcode com datasets
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is available online, the authors are encouraged reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Complete code associated with our framework and experiments will be released.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Neither crowdsourced research nor research with human subjects was conducted.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
No such research was involved in this work.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
41

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
42


Reviews:
[{"review_id": "WspTOY0Rof", "reviewer": "Reviewer_i65A", "summary": "The paper \"Interpretable Machine Learning for Datasets with Missing Values\" proposes generalized additive models incorporating missingness indicators and their interaction terms with sparseness ensured by l0 regularization. Therefore, the authors combine GAMs with the Missing indicator method and l0 regularization. The method is compared on real-world data sets with missing values and added synthetic missingness and against a variety of techniques used on imputed data with and without selective addition of missingness indicators.", "questions": "Proof only works since the unobserved noise for k1 and k2 is chosen as it is. \"Let k_2<k_1<0.5\" \nTo me this looks like circular reasoning. If I assume the noise is lower for knowing the missingness relationship with the predictor Y, than my uncertainty with the oracle -> of course the one with lower noise with reach higher probability. If the assumption is switched or equal signs added the proposition will fall apart. It seems the proposition needs to be far more limited than it is right now. \nIn 108 should it than not say \"Corollary 3.2 states that perfectly imputing missing data can reduce the best possible performance of\na predictive model, if the noise of the assumed predictor dependent missingness mechanism is lower than the measurement noise.\"?\nI find this not very surprising. Adding correct information with higher certainty should always be better. Adding correct certain models of the missingness mechanism is of course preferable to high noise imputation. \nWhy did the authors impute the data for RF? If C4.5 is used than the missingness can be handled without explicit imputation by change of the impurity function. There are also internal impute proposals: Stekhoven & Bühlmann (2012), \"MissForest—non-parametric missing value imputation for mixed-type data\", Bioinformatics, 28, 1.", "limitations": "see above. I think their proposition fails to emphasize a major assumption to their claim.", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The manuscript is well-written and structured. \nUtilizing models of missingness into the ML process is a nice idea.", "weaknesses": "I would have run 2 distinct experiments at least. One with synthetic missingness where the ground truth is known and the missigness can be increased. Additionally I would have liked to see also a mix of MCAR and MAR to demonstrate the results when the assumed mechanism is incorrectly chosen. \nThe authors emphasize the interpretability enabled by M-GAM and figures are shown. It would have been nice to discuss the interpretation in the text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We hope we have addressed your primary concern below, and look forward to any further discussion.\n\n>Proof only works since the unobserved noise for k1 and k2 is chosen as it is... If the assumption is switched or equal signs added the proposition will fall apart. It seems the proposition needs to be far more limited than it is right now.\n\n>In 108 should it than not say \"Corollary 3.2 states that perfectly imputing missing data can reduce the best possible performance of a predictive model, if the noise of the assumed predictor dependent missingness mechanism is lower than the measurement noise.\"?...\n\nProposition 3.1 and Corollary 3.2 are existence claims, which we prove by construction. It is valid and standard to prove such claims by constructing a single viable Data Generating Process (DGP); such a proof does not imply that DGP is the only case where our claim holds. In fact, in situations like the one you've named, where missingness information is higher noise than the rest of the data, there still exist cases where missingness information is needed to achieve the best possible model - we just chose not to focus on them to keep the proof simple. As an example, we have constructed a case where the noise in the missingness indicator is the greatest (k_1<...<k_2<0.5), but the Bayes’ optimal model with missing data is still superior to that with perfect imputation: \n\nConsider a case similar to that used for the proof in proposition 3.1, where we add an additional variable $X_3$ and, as requested, have the noise for the missingness be higher than any other noise. \n\n$$Y = |X_1X_2 - \\epsilon_1|, \\epsilon_1 \\sim \\textrm{Bern}(\\frac{1}{6})$$\n$$M = |Y - \\epsilon_2|, \\epsilon_2 \\sim \\textrm{Bern}(\\frac{1}{4})$$\n$$X_3 = |Y - \\epsilon_3|, \\epsilon_3 \\sim \\text{Bern}(\\frac{1}{5})$$\n\nWe also adjust the probabilities for $X_1$ and $X_2$ being true so that this is a balanced classification problem: $X_1, X_2 \\sim \\textrm{Bern}(\\frac{1}{\\sqrt{2}})$, so $X_1X_2 \\sim \\textrm{Bern}(\\frac{1}{2})$\n\nThe complete proof for this case is available at the bottom of this rebuttal.\n\n>Why did the authors impute the data for RF? If C4.5 is used than the missingness can be handled without explicit imputation by change of the impurity function. There are also internal impute proposals: Stekhoven & Bühlmann (2012), \"MissForest—non-parametric missing value imputation for mixed-type data\", Bioinformatics, 28, 1.\n\nAvoiding explicit imputation is a valid alternative baseline, which we have added, as shown in Figure 1 of our shared response. The results of this experiment align with those for previous baselines: random forests and decision trees without imputation do not outperform M-GAM. \nAdditionally, we note that MissForest is used as an imputation method in Section E.2 of the supplement. We find that MissForest imputation (including MissForest used with a random forest model) does not outperform any of our other impute-then-predict random forest baselines. \n\nWeaknesses:\n>I would have run 2 distinct experiments at least. One with synthetic missingness where the ground truth is known and the missigness can be increased. Additionally I would have liked to see also a mix of MCAR and MAR to demonstrate the results when the assumed mechanism is incorrectly chosen.\n\nWe don’t actually make any strict assumptions about the missingness mechanism. We regularize towards MCAR through our $\\ell_0$ regularization, but this is not a strict assumption (for example, we still handle the MAR setting in figure 3). As per your suggestion to include MCAR missingness, we’ve included a plot where we switch to MCAR instead of MAR missingness for the experiment done in figure 3 (plot included in the general response). We still handle missingness as well as our competitors, though there is no informative missingness that we can take advantage of in this setting.\n \n>The authors emphasize the interpretability enabled by M-GAM and figures are shown. It would have been nice to discuss the interpretation in the text.\n\nWe agree that additional discussion of how to interpret an M-GAM would be appropriate – we will add more discussion around Figure 2, and several more visualizations will be added to the supplement.\n\n---\n\nProof: \n\nThe bayes optimal model with perfect imputation of $X_1$, and no access to $M$, is still just to predict in accordance with $X_1X_2$ for $P(X_1X_2=Y)=\\frac{5}{6}$. When $X_1X_2=X_3$, all information available suggests $X_1X_2$ is correct. \nWhen $X_1X_2\\neq X_3$, we still have the bayes optimal prediction aligning with $X_1X_2$: $P(Y=1|X_1X_2=1,X_3=0) =\\frac{5}{9}>0.5$ and $P(Y=0|X_1X_2=0,X_3=1)=\\frac{5}{9}>0.5$.\n\nIf we instead only have access to $X_1$ when it is not missing, but we also know when $X_1$ is missing (i.e. we know $M$), then the following approach will perform better than the above model:\n$$Y=\\begin{cases}\nX_3,&\\textrm{if $M=1$}\\\\\\\\\nX_1X_2X_3,&\\textrm{if $M=0$}\n\\end{cases}$$\nWhen $M=1$, we make additional errors relative to the previous approach at rate \n$$P(M=1)(P((X_3)\\neq Y)-P((X_1X_2)\\neq Y))=\\frac{1}{2}(\\frac{1}{5}-\\frac{1}{6})=\\frac{1}{60}$$\nWhen $M=0$, we improve our classifier's accuracy by:\n\\begin{align*}\n&P(\\text{Imputation model is wrong, model with missingness is right}|M=0)\\\\\\\\ \n&-P(\\text{Imputation model is right, model with missingness is wrong}|M=0)\\\\\\\\\n=&P(X_1X_2\\neq Y=X_1X_2X_3)-P(X_1X_2=Y\\neq X_1X_2X_3)\\\\\\\\\n=&P(X_1X_2=1,X_3=0,Y=0,M=0)-P(X_1X_2=1,X_3=0,Y=1,M=0)\\\\\\\\\n=&P(X_1X_2=1)P( Y=0|X_1X_2=1)P(X_3=0|Y=0)P(M=0|Y=0)\\\\\\\\\n&-P(X_1X_2=1)P(Y=1|X_1X_2=1)P(X_3=0|Y=1)P(M=0|Y=1)\\\\\\\\\n=&\\frac{1}{2}\\frac{1}{6}\\frac{4}{5}\\frac{3}{4}-\\frac{1}{2}\\frac{5}{6}\\frac{1}{5}\\frac{1}{4}\\\\\\\\\n=&\\frac{7}{240}\\\\\\\\\n\\>&\\frac{1}{60}\n\\end{align*}\nSo the model that uses missingness outperforms the imputation model.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zo70pnUBmp", "reviewer": "Reviewer_fLWT", "summary": "This paper introduces M-GAM, which incorporates concept of missingness into Generalized Additive Model(GAM). Since GAM represents arbitrary function with sum of univariate functions which take each input feature as input, M-GAM maintains sparsity and interpretability for inference with missing data.", "questions": "- why M-GAM especially good on MAR setup?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Adapting GAM as tool for modeling under missingness is new and useful. Common approach for integrating missingness information is concatenating missingness indicator to input feature, but this makes exhaustive expansion of input dimension which can potentially harm the inference in many aspects. So I think this approach is smart and efficient\n- it is faster than other baselines with reasonable performance\n- abundant experiments that support the authors claim and detailed description of experimental setup", "weaknesses": "- I think the main weakness of M-GAM is its performance which just barely match other baselines' performance. Since it is trained end-to-end manner, we can expect more performance gain than impute-then regress methods since it directly uses supervision during training. Since informative missingness is rare in real world dataset, predictive performance of M-GAM  on real world dataset can harm the applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review. We appreciate your recognition of the novelty of this work and its strong experimental backing. We look forward to discussing any ongoing questions or concerns you may have.\n\n> I think the main weakness of M-GAM is its performance which just barely match other baselines' performance. Since it is trained end-to-end manner, we can expect more performance gain than impute-then regress methods since it directly uses supervision during training. Since informative missingness is rare in real world dataset, predictive performance of M-GAM on real world dataset can harm the applicability.\n\nThe goal of this work is to improve interpretability without harming performance, and we accomplished that. M-GAM is consistently among the most performant models on each dataset. Sometimes, it even improves both performance and interpretability. By producing an interpretable model, we allow practitioners to easily identify confounded reasoning and use the model more responsibly. This ability to troubleshoot helps improve overall accuracy of the system during and after deployment, not just performance on a static dataset.\n\n> why M-GAM especially good on MAR setup?\n\nM-GAM is particularly effective in the MAR setup used in the semi-synthetic example because the value is MAR with respect to the label. That is, in that semi-synthetic case, there is some signal about the outcome embedded in whether or not a feature is missing. \n\nThis can happen in a variety of settings; for example, imagine conducting a survey to help predict whether individuals care about privacy or not. If we ask for a home address and an individual doesn’t answer the question, resulting in missing data, that might be a strong indicator that they do care about privacy.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WWAmTne4sH", "reviewer": "Reviewer_fBin", "summary": "The paper introduces M-GAM, a novel extension of Generalized Additive Models (GAMs) designed to maintain interpretability while handling datasets with missing features. By incorporating missingness indicators and their interaction terms through ℓ0 regularization, M-GAM balances accuracy and sparsity. The model provides comparable or superior performance to traditional imputation methods while avoiding the complexity and overfitting issues associated with the naive inclusion of missingness indicators.", "questions": "- l.27, it seems strange to contrast the choice of (not) using missingness with the choice of using l0 regularization. They seems to be rather orthogonal choices.\n- Why l0 and not some other regularization?\n- It seems questionable to encode reasons for missingness with natural numbers, as these imply an ordering.\n- It looks like all other methods have better mean accuracy than M-GAM in Figure 6, Breast Cancer dataset. Is this correct? If so, the text should discuss this rather than glossing over it.", "limitations": "- The authors state interpretability as one of M-GAM's main advantages. I would like to see this made explicit, e.g., with an example, especially in comparison to existing methods.\n- Empirical performance is not great.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Very clear explanation in section 3.\n- Sparsity and interpretability are highly relevant for practitioners.", "weaknesses": "- The title should convey that the study is limited to GAMs\nRuntime comparison in 4.3 is a bit vacuous because it depends on the exact implementation, etc. Maybe looking at computational complexity would be more interesting.\n- Empirical evaluation is somewhat limited", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thoughtful review – we appreciate your recognition that sparsity and interpretability are particularly important for practitioners. We hope we have addressed each of your concerns below, and look forward to any ongoing discussion.\n\n> The title should convey that the study is limited to GAMs \n\nThis is a fair point – we plan to update the title to something like  “Interpretable Generalized Additive Models for Datasets with Missing Values”. GAMs are currently among the most popular forms of interpretable ML models [1].\n\n[1] https://github.com/interpretml/interpret\n\n> Runtime comparison in 4.3 is a bit vacuous because it depends on the exact implementation, etc. Maybe looking at computational complexity would be more interesting.\n\nWhile computational complexity would provide an interesting additional kind of comparison, we believe practical runtime is the more relevant metric here because it is more reflective of what users will experience. This is particularly true for methods like FastSparse, where computational tricks substantially improve the algorithm’s practical runtime without necessarily affecting its big O runtime. \nWhile runtime can vary with implementation, we deliberately used implementations that are well established in prior work whenever possible. As such, the results in 4.3 should be fairly reflective of what an actual user would experience, as they would likely use these same implementations.\n\n> Empirical evaluation is somewhat limited\n\nWe are unsure in what way this is meant, but note that appendix E contains a variety of additional empirical results, including evaluation over six datasets using a wide variety of baseline methods. Together, sections D and E of the appendix are longer than the entire main paper and consist of empirical evaluation.\n\n> l.27, it seems strange to contrast the choice of (not) using missingness with the choice of using l0 regularization. They seems to be rather orthogonal choices.\n\nYes, they are orthogonal choices. We think this is just ambiguous phrasing – we meant to contrast handling missingness alongside $\\ell_0$ regularization with handling missingness without $\\ell_0$ regularization. We will update the wording to make this clearer.\n\n> Why l0 and not some other regularization?\n\n$\\ell_0$ regularization is beneficial because it produces very sparse models, and this paper is focused on interpretability. In the context of M-GAMs, this means fewer variables will contribute to model predictions and therefore fewer shape functions will need to be presented. This makes it easier for users to interpret model predictions.\n\n> It seems questionable to encode reasons for missingness with natural numbers, as these imply an ordering.\n\nGood point, we’ll change it to letters (or another notation you suggest that implies no ordering). This does not actually affect the implementation since M-GAM is only exposed to the binary indicator variable for each missingness reason. If this is referring to the FICO description from the dataset description, it was not our choice to use the stated missingness encodings; those are from the creators of the dataset.\n\n> It looks like all other methods have better mean accuracy than M-GAM in Figure 6, Breast Cancer dataset. Is this correct? If so, the text should discuss this rather than glossing over it.\n\nThis is true, but reflects an oversight in creating Figure 6. Breast Cancer and MIMIC are both heavily imbalanced, and as such performance is better measured via AUC, which is done in other plots. Accuracy is not reflective of quality for imbalanced datasets. Under this metric, it is not true that the median AUC for M-GAM is worse than all other methods, as shown in the combined response. \nThe class balance for each of the four datasets is as follows\n\nBreast Cancer: 78.2% negative\n\nMIMIC: 89.6% negative\n\nPharyngitis: 54.9% negative\n\nFICO: 47.8% negative\n\n> The authors state interpretability as one of M-GAM's main advantages. I would like to see this made explicit, e.g., with an example, especially in comparison to existing methods.\n\nWe intended for Figures 1 and 2 to serve as such examples, but believe that additional examples on the datasets we study might help prove this point. We will prepare such examples, and add them to the appendix.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fH6cS0OYNA", "reviewer": "Reviewer_63xg", "summary": "The paper presents M-GAM, a novel generalized additive model that that incorporates missingness indicators while maintaining a sparse model via l_0 regularization. Results shows that on augmented datasets with missing at randomness M-GAM provides better performance; on real-world (not augmented) datasets, M-GAM achieves similar performance but is faster than impute-then-predict.", "questions": "I would appreciate the authors response to the weaknesses listed above. In addition I was wondering if the authors have considered incorporating this mechanism into one of the recent neural GAM approaches, e.g., Neural Additive Models [] which will allow potentially for better scalability?", "limitations": "Limitations are adequately addressed.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Novel approach for dealing with missing values in GAMs while keeping the model sparse.\n- Interesting theoretical results to support the work including\n- The paper is generally clear and written well.", "weaknesses": "- My understanding is that the setting considered in this paper seems to be limited to GAMs with main effects and not higher-order GAMs (in particular, GAM with pairwise interaction effects, GA2M [Lou et al., KDD 2013]). \n- There has been significant work on handling with missing values in decision trees (that are also a widely used interpretable model). There is very little both in terms of discussion (e.g., are the approaches there applicable to GAMs, does the proposed approach share some similarities to existing approaches in decision trees) as well as in terms of experimental results (how do decision trees with missing values compare to GAM with missing values, at the moment only SMIM with decision tree is considered and not approaches that are inherent to decision trees).\n- The experimental results on real datasets do not demonstrate strong improvement in performance (but there is improvement in other factors, e.g., runtime compared to impute-then-predict).\n- The analysis on other models, including decision trees (with and without SMIM) does not include runtimes so it is not clear if the new model is also significantly faster from, say, decision trees or logistic regression or random forests with SMIM.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate your review, and your recognition of this work’s novelty, theoretical foundations, and clear writing. We have responded to each of your criticisms below, and look forward to any continued discussion.\n\n> My understanding is that the setting considered in this paper seems to be limited to GAMs with main effects and not higher-order GAMs (in particular, GAM with pairwise interaction effects, GA2M [Lou et al., KDD 2013]).\n\nThis is generally correct, although M-GAMs could naturally be extended to include interaction effects of an arbitrary order – both between missingness and between observed values, and between different combinations of observed values. However, we note that, even in Lou et al. 2013, higher order GAMs showed substantially improved classification performance relative to main effect GAMs only on digit recognition tasks (“Letter”, “Gisette”) and not on datasets that would be treated as tabular today. We did experiment with additional interaction terms, along the lines of GA2M, but did not find any scenarios where this improved performance. As such, we did not feel comfortable including it in the paper.\n\n>There has been significant work on handling with missing values in decision trees (that are also a widely used interpretable model). There is very little both in terms of discussion (e.g., are the approaches there applicable to GAMs, does the proposed approach share some similarities to existing approaches in decision trees) as well as in terms of experimental results (how do decision trees with missing values compare to GAM with missing values, at the moment only SMIM with decision tree is considered and not approaches that are inherent to decision trees).\n\nThis is a fair point. We will add discussion of this work in the related work section, and have added comparisons to some decision tree based methods that explicitly handle missingness as implemented in SKLearn, as shown in the shared response. In general, because these papers introduce changes that are specific to decision trees, such as default traversal paths when data is missing, they are not directly applicable to M-GAM. We found that decision trees with this explicit handling improved runtime and offered interpretability, but generally had poorer performance than M-GAM. Random forests offered comparable performance and runtime, at the cost of interpretability.\n\nIn summary, we’re happy to address this work in related work, but it does not decrease the value of our own approach, which performs well and is separate from the decision-tree-inherent methods you’ve mentioned. \n\n> The experimental results on real datasets do not demonstrate strong improvement in performance (but there is improvement in other factors, e.g., runtime compared to impute-then-predict).\n\nWhile this is true, the fact that M-GAM has comparable performance to other methods on real data means that our runtime and interpretability gains come with no substantial costs. M-GAM quickly provides a sparse, transparent model that is as accurate as complex impute-then-predict baselines, which we see as a strength.\n\n> The analysis on other models, including decision trees (with and without SMIM) does not include runtimes so it is not clear if the new model is also significantly faster from, say, decision trees or logistic regression or random forests with SMIM.\n\nWe have added some such comparisons in the global response. Please note that the SMIM framework consists of both imputing missing data and providing missingness indicators, meaning that it is no faster than the given runtimes with imputation. Additionally, these methods that use imputation introduce complexity, sacrificing interpretability in a way that M-GAM avoids.\n\n> I would appreciate the authors response to the weaknesses listed above. In addition I was wondering if the authors have considered incorporating this mechanism into one of the recent neural GAM approaches, e.g., Neural Additive Models [] which will allow potentially for better scalability?\n\nWe hadn’t previously, but it's an interesting idea. However, based on the results presented in [1], it seems that NAM’s will not actually scale better than M-GAMs. Our current $\\ell_0$ regularized method is already quite fast.  If we were to discard our approach in favor of a Neural GAM with some other sparsity approach, we don’t anticipate benefits to scalability because such an approach  would effectively need to fit neural nets for up to (# features)$^2$ shape functions to cover missingness interactions, and a sparsity-regularized neural GAM can take quite a while even for a simple dataset. For example, table 5 of [1] shows a neural GAM taking ~30 seconds on a 6172 sample, 13-feature dataset, COMPAS, which is slower than our method on the much more computationally difficult FICO dataset (10,459 samples, 23 features), even when we augment the FICO dataset with missingness interactions.\n\n[1]  Shiyun Xu, Zhiqi Bu, Pratik Chaudhari, Ian J. Barnett. Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity. ECML PKDD 2023", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LfmDrkRujX", "reviewer": "Reviewer_W4vf", "summary": "The paper discusses challenges posed by missing data in important datasets for machine learning models. Existing methods like imputation or using indicator variables for missingness can compromise model interpretability or introduce complexity and reduced sparsity. The authors propose M-GAM, a sparse, generalized additive modeling approach. M-GAM addresses these issues by incorporating missingness indicators and their interactions, while maintaining sparsity through l0 regularization. They demonstrate that M-GAM achieves comparable or better accuracy than existing methods while significantly improving sparsity compared to imputation or straightforward inclusion of indicator variables.", "questions": "N/A", "limitations": "N/A", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The paper is well-written\n\n- The proposed method is novel\n\n- Extensive experiments are conducted to robustly support the claims.", "weaknesses": "- The proposed method is constrained by its reliance on l_0 regularization.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review! We appreciate your recognition that this work is well written, novel, and contains extensive experiments, and we are open to discussing any additional concerns you might have.\n\n> The proposed method is constrained by its reliance on l_0 regularization.\n\nWhile $\\ell_0$ regularization has historically been difficult to optimize, a key strength of our paper is that it leverages the fast optimization framework from Liu et al. 2022, which introduces several computational tricks that make it quite manageable. This lets us gain the substantial benefits of $\\ell_0$ regularization (extreme sparsity in a setting that is prone to producing dense models) without suffering substantial runtime costs.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their thoughtful reviews. We have responded to each reviewer separately. Attached to this message, please find figures used in responses to individual reviewers. 

We look forward to further discussion in the days to come.

Meta-review: Paper proposes a novel variant of generalized additive model (GAM) for missing features (M-GAM) to provide comparable accuracy to popular imputation approach, keeping computational cost slow and producing sparse and interpretable model. Both theoretical and experimental analysis is done, showing the usefulness of the proposed model in certain benchmark problems.

Based on comments and scores, all the reviewers are on the positive side of margin. The strengths of the paper are related to novel and useful idea (Reviewers i65A, fLWT, 63xg, W4vf), well-written and structured content (Reviewers i65A, fBin, 63xg, W4vf), valuable method/model (Reviewer i65A), e.g., for practitioners (Reviewers fBin), and extensive experiments (Reviewers fLWT and W4vf). On weaknesses/limitations side, there have been some critics on the experiments (in general Reviewer fBIN, baseline methods Reviewer 63xg, 
and e.g., related to missing of synthetic examples with ground truth and where missingness is controlled, Reviewer i65A), performance of proposed method (Reviewers fLWT and fBin), detailed discussion of interpretability of the model (Reviewers i65A and fBin), and theoretical background / claims (Reviewer i65A).

Authors have provided some response to critics in rebuttal, including: 1) additional experiments against baselines of decision trees (DT) and random forest (RF),  and AUC validation metrics for imbalanced datasets (in global rebuttal file, Figure 1), 2) additional computational time analysis of DT and RF (in global rebuttal file, Figure 2), and 3) additional MCAR experiment (in global rebuttal file, Figure 3). Furthermore, more discussion of interpretability around Figure 2 in original manuscript (in discussion with Reviewer i65A) and discussion about DT handling missing values to related work (in discussion with Reviewer 63xg) are to be added. Overall, paper brings valuable information to community and interesting tool for practitioners. Based on these, I would recommended paper to be accepted as 
a poster. Authors are encouraged to revise the final paper, including all the improvements and suggestion raised during the rebuttal discussion.
Decision: Accept (poster)