NODE-GAM: N EURAL G ENERALIZED A DDITIVE M ODEL FOR I NTERPRETABLE D
EEP L EARNING
Anonymous authors
Paper under double-blind review
A BSTRACT
Deployment of machine learning models in real high-risk settings (e.g. healthcare) often depends not only on model’s accuracy but also on its fairness, robustness and interpretability.
Generalized Additive Models (GAMs) are a class of interpretable models with a long history of use in these high-risk domains, but they lack desirable features of deep learning such as differentiability and scalability.
In this work, we propose a neural GAM (NODE-GAM) and neural GA 2 M (NODE-GA 2 M) that scale well and perform better than other GAMs on large datasets, while remaining interpretable compared to other ensemble and deep learning models.
We demonstrate that our models ﬁnd interesting patterns in the data.
Lastly, we show that we improve model accuracy via self-supervised pre-training, an improvement that is not possible for non-differentiable GAMs.
I NTRODUCTION
As machine learning models become increasingly adopted in everyday life, we begin to require models to not just be accurate, but also satisfy other constraints such as fairness, bias discovery, and robustness under distribution shifts for high-stakes decisions (e.g., in healthcare, ﬁnance and criminal justice).
These needs call for an easier ability to inspect and understand a model’s predictions.
Generalized Additive Models (GAMs) (Hastie & Tibshirani, 1990) have a long history of being used to detect and understand tabular data patterns in a variety of ﬁelds including medicine (Hastie &
Tibshirani, 1995; Izadi, 2020), business (Sapra, 2013) and ecology (Pedersen et al., 2019).
Recently proposed tree-based GAMs and GA 2 Ms models (Lou et al., 2013) further improve on original GAMs (Spline) having higher accuracy and better ability to discover data patterns (Caruana et al., 2015).
These models are increasingly used to detect dataset bias (Chang et al., 2021) or audit black-box models (Tan et al., 2018a;b).
As a powerful class of models, they still lack some desirable features of deep learning that made these models popular and effective, such as differentiability and scalability.
In this work, we propose a deep learning version of GAM and GA 2 M that enjoy the beneﬁts of both worlds.
Our models are comparable to other deep learning approaches in performance on tabular data while remaining interpretable.
Compared to other GAMs, our models can be optimized using GPUs and mini-batch training allowing for higher accuracy and more effective scaling on larger datasets.
We also show that our models improve performance when labeled data is limited by self-supervised pretraining and ﬁnetuning, where other non-differentiable GAMs cannot be applied.
Several works have focused on building interpretable deep learning models that are effective for tabular data.
TabNet (Arik & Pﬁster, 2020) achieves state-of-the-art performance on tabular data while also providing feature importance per example by its attention mechanism.
Although attention seems to be correlated with input importance (Xu et al., 2015), in the worst case they might not correlate well (Wiegreffe & Pinter, 2019).
Yoon et al.
(2020) proposes to use self-supervised learning on tabular data and achieves state-of-the-art performance but does not address interpretability.
NIT (Tsang et al., 2018) focuses on building a neural network that produces at most K-order interactions and thus include GAM and GA
M.
However, NIT requires a two-stage iterative training process that requires longer computations.
And their performance is slightly lower to DNNs while ours are overall on par with it.
They also do not perform puriﬁcation that makes GA 2 M graphs unique when showing them.
The most relevant approaches to our work are NODE (Popov et al., 2019) and NAM (Agarwal et al., 2020).
Popov et al.
(2019) developed NODE that mimics an ensemble of decision trees but permits differentiability and achieves state-of-the-art performance on tabular data.
Unfortunately, NODE suffers from a lack of interpretability similarly to other ensemble and deep learning models.
On the other hand, Neural Additive Model (NAM) whose deep learning architecture is a GAM, similar to our proposal, thus assuring interpretability.
However, NAM can not model the pairwise interactions and thus do not allow GA 2 M.
Also, because NAM builds a small feedforward net per feature, in high-dimensional datasets NAM may require large memory and computation.
Finally, NAM requires training of 10s to 100s of models and ensemble them which incurs large computations and memory, while ours only trains once; our model is also better than NAM without the ensemble (Supp.
A).
To make our deep GAM scalable and effective, we modify NODE architecture (Popov et al., 2019) to be a GAM and GA 2 M, since NODE achieves state-of-the-art performance on tabular data, and its tree-like nature allows GAM to learn quick, non-linear jumps that better match patterns seen in real data (Chang et al., 2021).
We thus call our models NODE-GAM and NODE-GA 2 M respectively.
One of our key contributions is that we design several novel gating mechanisms that gradually reduce higher-order feature interactions learned in the representation.
This also enables our NODE-GAM and
NODE-GA 2 M to automatically perform feature selection via back-propagation for both marginal and pairwise features.
This is a substantial improvement on tree-based GA 2 M that requires an additional algorithm to select which set of pairwise feature interactions to learn (Lou et al., 2013).
Overall, our contributions can be summarized as follows:
• Novel architectures for neural GAM and GA 2 M thus creating interpretable deep learning models.
• Compared to state-of-the-art GAM methods, our NODE-GAM and NODE-GA 2 M achieve similar performance on medium-sized datasets while outperforming other GAMs on larger datasets.
• We demonstrate that NODE-GAM and NODE-GA 2 M discover interesting data patterns.
• Lastly, we show that NODE-GAM beneﬁts from self-supervised learning that improves performance when labeled data is limited, and performs better than other GAMs.
We foresee our novel deep learning formulation of the GAMs to be very useful in high-risk domains, such as healthcare, where GAMs have already proved to be useful but stopped short from being applied to new large data collections due to scalability or accuracy issues, as well as settings where access to labeled data is limited.
Our novel approach also beneﬁts the deep learning community by adding high accuracy interpretable models to the deep learning repertoire.
B ACKGROUND
GAM and GA 2 M: GAMs and GA 2 Ms are interpretable by design because of their functional forms.
Given an input x ∈ R D , a label y , a link function g
(e.g. g is log p 1 − p in binary classiﬁcation), main effects f j for each feature j , and feature interactions f jj (cid:48) , GAM and GA 2 M are expressed as:
GAM : g ( y
[Equation]
D (cid:88)
[Equation]
= f [Equation] ( x j ) [Equation]
GA [Equation] M
[Equation] g ( y [Equation]
D (cid:88)
j [Equation] f
[Equation] [Equation]
D (cid:88)
j = (cid:88) j (cid:48) >
j f jj (cid:48) ( x j , x j (cid:48) ) .
Unlike full complexity models (e.g. DNNs) that have y = f
( x 1 , ..., x j ) , GAMs and GA 2 M are interpretable because the impact of each feature f j and each feature interaction f jj (cid:48) can be visualized as a graph (i.e. for f j , x-axis shows x j and y-axis shows f j ( x j ) ).
Humans can easily simulate how they work by reading f j s and f jj (cid:48) off different features from the graph and adding them together.
GAM baselines: We compare with Explainable Boosting Machine (EBM) (Nori et al., 2019) that implements tree-based GAM and GA 2 M.
We also compare with splines proposed in the 80s (Hastie & Tibshirani, 1990) using Cubic splines in pygam package (Servén & Brummitt, 2018).
Neural Oblivious Decision Trees (NODEs): We describe NODEs for completeness and refer the readers to Popov et al.
for more details.
NODE consists of L layers where each layer has m differentiable oblivious decision trees (ODT) of equal depth C .
Below we describe a single ODT.
Differentiable Oblivious Decision Trees: An ODT works like a traditional decision tree except for all nodes in the same depth share the same input features and thresholds, which allows parallel computation and makes it suitable for deep learning.
Speciﬁcally, an ODT of depth C compares C chosen input feature to C thresholds, and returns one of the 2 C possible responses.
Mathmatically, for feature functions F c which choose what features to split, splitting thresholds b c , and a response vector R ∈ R 2 C , the tree output h ( x ) is deﬁned as:
[Equation]
Here I is the indicator function, ⊗ is the outer product, and · is the inner product.
Both feature functions F c and I prevent differentiability.
To make them differentiable, Popov et al.
(2019) replace F c ( x ) as a weighted sum of features:
[Equation]
Here F c ∈ R D are the logits for which features to choose, and entmax α (Peters et al., 2019) is the entmax transformation which works like a sparse version of softmax such that the sum of the output equals to 1 .
They also replace the I with entmoid which works like a sparse sigmoid that has output values between 0 and 1 .
Since all operations are differentiable (entmax, entmoid, outer and inner products), the ODT is differentiable.
Stacking trees into deep layers: Popov et al.
(2019) follow the design similar to DenseNet where all tree outputs h ( x ) from previous layers (each layer consists of total I trees) become the inputs to the next layer.
For input features x , the inputs x l to each layer l becomes:
[Equation]
And the ﬁnal output of the model ˆ y ( x ) is the average of all tree outputs h 1 , ..., h L of all L layers:
[Equation]
O UR MODEL DESIGN
GAM design: See Supp.
B for a complete pseudo code.
To make NODE a GAM, we make three key changes to avoid any feature interactions in the architecture (Fig. 1).
First, instead of letting
F c ( x ) be a weighted sum of features (Eq. 2), we make it only pick 1 feature.
We introduce a temperature annealing parameter T that linearly decreases from 1 to 0 for the ﬁrst S learning steps to make entmax α ( F c /T ) gradually become one-hot:
[Equation]
Second, within each tree, we make the logits F c the same across depth C i.e. F 1 = · · · = F C = F to avoid any feature interaction within a tree.
Third, we avoid the DenseNet connection between two trees that focus on different features j, j (cid:48) , since they create feature interactions between features j and j (cid:48) if two trees connect.
Thus we introduce a gate that only allows connections between trees that take the same features.
Let G i =
entmax
α
( F i /T ) of the tree i .
For tree i in layer l and another tree ˆ i in layer ˆ l for ˆ l < l , the gating weight g l ˆ ii and the feature function F li for tree i become:
[Equation]
Since G becomes gradually one-hot by Eq.
5, after S steps g ˆ ii would only become 1 when G ˆ i
= G i and 0 otherwise.
This enforces no feature interaction between tree connections.
Attention-based GAMs (AB-GAMs): To make the above GAM more expressive, we add an attention weight a l ˆ ii in the feature function F li ( x ) to decide which previous tree to focus on:
[Equation]
To achieve this, we introduce attention logits A li for each tree i that after entmax it produces a l ˆ ii :
[Equation]
This forces the attention of a tree i that (cid:80) ˆ
i a l ˆ ii = 1 for all ˆ i that g l ˆ ii = 1 and a l ˆ ii = 0 when g l ˆ ii = 0 .
The attention logits A requires a large matrix size [ I , ( l − 1) I ] for each layer l > 1 which explodes the memory.
We instead make A as the inner product of two smaller matrices such that A = BC where
B is of size [ I , E ] and C is of size [ E , ( l − 1) I ], where E is a hyperparameter for the embedding dimension of the attention.
Last Linear layer: Lastly, instead of averaging the outputs of all trees as the output of the model (Eq. 4), we add the last linear layer to be a weighted sum of all outputs
[Equation]
Note that in self-supervised learning, w li has multiple output heads to predict multiple tasks.
Regularization: We also include other changes that improves performance.
First, we add Dropout (rate p 1 ) on the outputs of trees h li ( x l ) , and Dropout (rate p 2 ) on the ﬁnal weights w li .
Also, to increase diversity of trees, each tree can only model on a random subset of features ( η ), an idea similar to Random Forest.
We also add an (cid:96) 2 penalization ( λ ) on h li ( x l ) .
In binary classiﬁcation task where labels y are imbalanced between class 0 and 1 , we set a constant as log p ( y ) 1 − p ( y ) that is added to the ﬁnal output of the model such that after sigmoid it becomes the p ( y ) if the output of the model is 0 .
We ﬁnd it’s crucial for (cid:96) 2 penalization to work since (cid:96) 2 induces the model to output 0 .
NODE-GA 2 Ms — extending NODE-GAMs to two-way interactions: To allow two-way interactions, for each tree we introduce two logits F 1 and F 2 instead of just one, and let F c =
F ( c − 1) mod 2+1 for c > 2 ; this allows at most 2 features to interact within each tree (Fig. 7).
Besides temperature annealing (Eq. 5), we make the gating weights g ˆ ii = 1 only if the combination of F 1 , Fis the same between tree
ˆ i and i (i.e. both trees
ˆ i
and i focus on the same 2 features).
We set g ˆ ii as:
[Equation]
We cap the value at 1 to avoid uneven ampliﬁcations as g ˆ ii = 2 when G 1 i =
G 2 i =
G 1ˆ i =
G 2ˆ i .
Data Preprocessing and Hyperparameters: We follow Popov et al.
to do target encoding for categorical features, and do quantile transform for all features to Gaussian distribution (we ﬁnd
Gaussian works better than Uniform).
We use random search to search the architecture space for
NODE, NODE-GAM and NODE-GA 2 M.
We use QHAdam (Ma & Yarats, 2018) and average the most recent 5 checkpoints (Izmailov et al., 2018).
In addition, we adopt learning rate warmup (Goyal et al., 2017), and do early stopping and learning rate decay on the plateau.
More details in Supp.
F.
Extracting shape graphs from GAMs: We follow Chang et al.
(2021) to implement a function that extracts main effects f j from any GAM model including NODE-GAM, Spline and EBM.
The main idea is to take the difference between the model’s outputs of two examples ( x 1 , x 2 ) that have the same values except for feature j .
Since the intercept and other main effects are canceled out when taking the difference, the difference f ( x 2 ) − f ( x 1 ) is equal to f j ( x 2 j ) − f j ( x 1 j ) .
If we query all the unique values of x j , we get all values of f j relative to f j ( x 1 j ) .
Then we center the graph of f j by setting the average of f j ( x j ) across the dataset as 0 and add the average to the intercept term f 0 .
Extracting shape graphs from GA 2 Ms: Designing a black box function to extract from any
GA 2 M is non-trivial, as each changed feature x j would change not just main effect term
f j but also every interactions ∀ j (cid:48)
f jj (cid:48) that involve feature j .
Instead, since we know which features each tree takes, we can aggregate the output of trees into corresponding main f j and interaction terms f jj (cid:48) .
Note that GA 2 M can have many representations that result in the same function.
For example, for a prediction value v associated with x 2 , we can move v to the main effect f 2 ( x 2 ) = v , or the interaction effect f 23 ( x 2 , · ) = v that involves x 2 .
To solve this ambiguity, we adopt "puriﬁcation" (Lengerich et al., 2020) that pushes interaction effects into main effects if possible.
See Supp.
C for details.
R ESULTS
We ﬁrst show the accuracy of models in Sec.
and interpret 2 datasets, Bikeshare and MIMIC2, in
Sec. 4.2.
In Sec. 4.3, we show that NODE-GAM beneﬁts from self-supervised pre-training and often outperforms other GAMs when labels are limited.
4. A RE NODE-GAM AND NODE-GA 2 M ACCURATE ?
We compare our performance on 6 popular binary classiﬁcation datasets (Churn, Support2, MIMIC2,
MIMIC3, Income, and Credit) and 2 regression datasets (Wine and Bikeshare).
These datasets are medium-sized with 6k-300k samples and 6-57 features (Table 5).
We use 5-fold cross validation to derive the mean and standard deviation for each model.
We use 80-20 splits for training and val set.
To compare models across datasets, we calculate 2 summary metrics: (1) Rank: we rank the performance on each dataset, and then compute the average rank across all 9 datasets (the lower the rank the better).
(2) Normalized Score (NS): for each dataset, we set the worst performance for that dataset as 0 and the best as 1 , and scale all other scores linearly between 0 and 1 .
In Table 1, we show the performance of all GAMs, GA 2 Ms and full complexity models.
First, we compare 4 GAMs (here NODE-GA 2 M-main is the puriﬁed main effect from NODE-GA 2 M).
We ﬁnd all 4 GAMs perform similarly and the best GAM in different datasets varies, with Spline as the best in Rank and NODE-GAM in NS.
But the differences are often smaller than the standard deviation.
Next, both NODE-GA 2 M and EBM-GA 2 M perform similarly, with NODE-GA 2 M better in Rank and EBM-GA 2 M better in NS.
Lastly, within all full complexity methods, XGB performs the best
[Table] [Table]
Epsilon
[Table]
- - 0.5% - - 3.6% - - - with not much difference from NODE and RF performs the worst.
In summary, all GAMs perform similarly.
NODE-GA 2 M is similar to EBM-GA 2 M, and slightly outperforms full-complexity models.
In Table 2, we test our methods on 6 large datasets (all have samples > 500K) used in the NODE paper, and we use the same train-test split to be comparable.
Since these only provide 1 test split we report standard deviation across multiple random seeds.
First, on a cluster with 32 CPU and 120 GB memory, Spline goes out of memory on 5 out of 6 datasets and EBM also can not be run on dataset Epsilon with 2k features, showing their lack of ability to scale to large datasets.
For 5 datasets that EBM can run, our NODE-GAMs runs slightly better than EBM.
But when considering GA 2 M,
NODE-GA 2 M outperforms EBM-GA 2 M up to 7.3% in Higgs and average relative improvement of 3.6%.
NODE outperforms all GAMs and GA 2 Ms substantially on Higgs and Year, suggesting both datasets might have important higher-order feature interactions.
[Figure]
[Figure]
4. S HAPE GRAPHS OF NODE-GAM AND NODE-GA 2 M: B IKESHARE AND MIMIC
In this section, we highlight our key ﬁndings, and show the rest of the plots in Supp.
H.
Bikeshare dataset: Here we interpret the Bikeshare dataset.
It contains the hourly count of rental bikes between the years 2011 and 2012 in Capital bikeshare system located in Washington, D.C.
Note that all 4 GAMs trained on Bikeshare are equally accurate with < 0 .
1% error difference (Table 1).
In Fig.
2, we show the shape plots of 4 features: Hour, Temperature, Month, and Week day.
First,
Hour (Fig.
2a) is the strongest feature with two peaks around 9 AM and 5 PM, representing the time that people commute, and all 4 models agree.
Then we show Temperature in Fig.
2b.
Here temperature is normalized between 0 and 1, where 0 means -8°C and 1 means 39°C.
When the weather is hot (Temp > 0.8, around 30°C), all models agree rental counts decrease which makes sense.
Interestingly, when it’s getting colder (Temp < 0.4, around 11°C) there is a steady rise shown by NODE-GAM, Spline and EBM but not NODE-GA 2 M (blue).
Since it’s quite unlikely people rent more bikes when it’s getting colder especially below 0°C, the pattern shown by GA 2 M seems more plausible.
Similarly, in feature Month (Fig. 2c), NODE-GA 2 M shows a rise in summer (month 6 − 8 ) while others indicate a strong decline of rental counts.
Since we might expect more people to rent bikes during summer since it’s warmer, NODE-GA 2 M might be more plausible, although we might explain it due to summer vacation fewer students ride bikes to school.
Lastly, for Weekday (Fig. 2d) all 4 models agree with each other that the lowest number of rentals happen at the start of the week (Sunday and Monday) and slowly increase with Saturday as the highest number.
In Fig.
3, we show the 4 feature interactions (out of 67 ) from our NODE-GA 2 M.
The strongest effect happens in Hr x Working day (Fig. 3(a)): this makes sense since in working day (orange), people usually rent bikes around 9 AM and 5 PM to commute.
Otherwise, if the working day is 0 (blue), the number peaks from 10AM to 3PM which shows people going out more often in daytime.
In Hr x Weekday (Fig. 3(b)), we can see more granularly that this commute effect happens strongly on
Monday to Thursday, but on Friday people commute a bit later, around 10 AM, and return earlier, around 3 or 4 PM.
In Hr x Temperature (Fig. 3(c)), it shows that in the morning rental count is high when it’s cold, while in the afternoon the rental count is high when it’s hot.
We also ﬁnd in Hr x
Humidity (Fig. 3(d)) that when humidity is high from 3-6 PM, people ride bikes less.
Overall these interpretable graphs enable us to know how the model predicts and ﬁnd interesting patterns.
MIMIC2: MIMIC2 is the hospital ICU mortality prediction task (Johnson et al., 2016a).
We extract 17 features within the ﬁrst 24 hour measurements, and we use mean imputation for missingness.
[Figure] [Figure]
We show the shape plots (Fig. 4) of 4 features: Age, PFratio, Bilirubin and GCS.
In feature Age (Fig.. 4(a)), we see the all 4 models agree the risk increases from age 20 to 90 .
Overall NODEGAM/GA 2 M are pretty similar to EBM in that they all have small jumps in a similar place at age 55, 80 and 85; spline (red) is as expected very smooth.
Interestingly, we see NODE-GAM/GA 2 M shows risk increases a bit when age < 20.
We think the risk is higher in younger people because this is generally a healthier age in the population, so their presence in ICU indicates higher risk conditions.
In Fig. 4(b), we show PFratio: a measure of how well patients oxygenate the blood.
Interestingly,
NODE-GAM/GA 2 M and EBM capture a sharp drop at 332 .
It turns out that PFratio is usually not measured for healthier patients, and the missing values have been imputed by the population mean 332 , thus placing a group of low-risk patients right at the mean value of the feature.
However, Spline (red) is unable to capture this and instead have a dip around 300-600.
Another drop captured by
NODE-GAM/GA 2 M from 400 − 500 matches clinical guidelines that > 400 is healthy.
Bilirubin shape plot is shown in Fig. 4(c).
Bilirubin is a yellowish pigment made during the normal breakdown of red blood cells.
High bilirubin indicates liver or bile duct problems.
Indeed, we can see risk quickly goes up as Bilirubin is > 2 , and all 4 models roughly agree with each other except for
Spline which has much lower risk when Billirubin is 80, which is likely caused by Spline’s smooth inductive bias and unlikely to be true.
Lastly, in Fig. 4(d) we show Glasgow Coma Scale (GCS): a bedside measurement for how conscious the patient is with 1 in a coma and 5 as conscious.
Indeed, we ﬁnd the risk is higher for patients with GCS=1 than 5, and all 4 models agree.
In Fig.
5, we show the 4 of 154 feature interactions learned in the NODE-GA 2 M. First, in Age x
Bilirubin (Fig. 5(a)), when Billirubin is high (>2), we see an increase of risk (blue) in people with age 18 − 70 .
Risk decreases (red) when age > 80.
Combined with the shape plots of Age (Fig. 4(a)) and
Bilirubin (Fig. 4(c)), we ﬁnd this interaction works as a correction effect: if patients have Bilirubin > 2 (high risk) but are young (low risk), they should have a higher risk than what their main (univariate) effects suggest.
On the other hand, if patients have age > 80 (high risk) and Bilirubin > 2 (high risk), they already get very high risk from main effects, and in fact the interaction effect is negative to correct for the already high main effects.
It suggests that Billirubin=2 is an important threshold that should affect risk adjustments.
Also in GCS x Bilirubin plot (Fig. 5(b)), we ﬁnd similar effects: if Bilirubin >2, the risk of GCS is correctly lower for GCS=1,2 and higher for 3-5.
In Fig. 5(c)
we ﬁnd patients with GCS=1-3 (high risk) and age>80 (high risk), surprisingly, have even higher risk (blue) for these patients; it shows models think these patients are more in danger than their main effects suggest.
Finally, in Fig.
5(d)
we show interaction effect GCS x PFratio.
We ﬁnd PFratio also has a similar threshold effect: if
PFratio > 400 (low risk), and GCS=1,2 (high risk), model assigns higher risk for these patients while decreasing risks for patients with GCS=3,4,5.
Churn
Support
MIMIC
R e l a ti v e
I m p ( % )     −20 − −10 −
SS EBM
Spline    10 − − −
SS EBM
Spline     −
SS EBM
Spline
Labeled data ratio(%)
Labeled data ratio(%) Labeled data ratio(%)
MIMIC
Income
Credit
[Figure]
v e I m p ( % )    10 −30 − −20 − −10 −
SS EBM
Spline    10 −7 − −5 −4 − −2 −
SS EBM
Spline 2.5 5.0 7.5 10.0 12.5 15.0 17.5 20.
− − − −
SS EBM
Spline
Labeled data
[Figure] [Figure]
[Figure]
4. S ELF - SUPERVISED P RE - TRAINING
By training GAMs with neural networks, it enables self-supervised learning that learns representations from unlabeled data which improves accuracy in limited labeled data scenarios.
We ﬁrst learn a
NODE-GAM that reconstructs input features under randomly masked inputs (we use 15% masks).
Then we remove and re-initialize the last linear weight and ﬁne-tune it on the original targets under limited labeled data.
For ﬁne-tuning, we freeze the embedding and only train the last linear weight for the ﬁrst 500 steps; this helps stabilize the training.
We also search smaller learning rates [ 5 e − 5 , 1 e − 4 , 3 e − 4 , 5 e − 4 ] and choose the best model by validation set.
We compare our self-supervised model ( SS ) with 3 other baselines: (1) NODE-GAM without self-supervision ( No-SS ), (2) EBM and (3) Spline.
We randomly search 15 attention based AB-GAM architectures for both SS and No-SS.
In Figure 6, we show the relative improvement over the AUC of No-SS under variously labeled data ratio 0 .
5% , 1% , 2% , 5% , and 10% (except Credit which 0 .
5% has too few positive samples and thus crashes).
And we run 3 different train-test split folds to derive mean and standard deviation.
Here the relative improvement means improvement over No-SS baselines.
First, we ﬁnd NODE-GAM with self-supervision (SS, blue) outperforms No-SS in 6 of 7 datasets (except Income) with MIMIC having the most improvement (6%).
This shows our NODE-GAM beneﬁts from self-supervised pre-training.
SS also outperforms EBM in 3 out of 6 datasets (Churn, MIMIC2 and Credit) up to 10% improvement in Churn, demonstrating the superiority of SS when labeled data is limited.
L IMITATIONS , D ISCUSSIONS AND C ONCLUSIONS
Although we interpret and explain the shape graphs in this paper, we want to emphasize that the shown patterns should be treated as an association not causation.
Any claim based on the graphs requires a proper causal study to validate it.
In this paper, we assumed that the class of GAMs is interpretable and proposed a new deep learning model in this class, so our NODE-GAM is as interpretable to other GAMs.
But readers might wonder if GAMs are interpretable to human.
Hegselmann et al.
(2020) show GAM is interpretable for doctors in a clinical user study; Tan et al.
ﬁnd GAM is more interpretable than the decision tree helping users discover more patterns and understand feature importance better; Kaur et al.
(2020) compare GAM to post-hoc explanations (SHAP) and ﬁnd that GAM signiﬁcantly makes users answer questions more accurately, have higher conﬁdence in explanations, and reduce cognitive load.
In this paper we propose a deep-learning version of GAM and GA 2 M that automatically learn which main and pairwise interactions to focus without any two-stage training and model ensemble. Our GAM is also more accurate than traditional GAMs in both large datasets and limited-labeled settings.
We hope this work can further inspire other interpretable design in the deep learning models.
E THICS S TATEMENT
A potential misuse of this tool is to claim causal relationships of what GAM models learned.
For example, one might wrongfully claim that having asthma is good for pneumonia patients (Caruana et al., 2015).
This is obviously a wrong conclusion predicated by the bias in the dataset.
The interpretable models are a great tool to discover potential biases hiding in the data.
Especially in high-dimensional datasets when it’s hard to pinpoint where the bias is, GAM provides easy visualization that allows users to conﬁrm known biases and examine hidden biases.
We believe that such models foster ethical adoption of machine learning in high-stakes settings.
By providing a deep learning version of GAMs, our work enables the use of GAM models on larger datasets thus increasing GAM adoption in real-world settings.
R EPRODUCIBILITY S TATEMENT
We released our code anonymously in the supplementary with instructions and hyperparameters to reproduce our ﬁnal results.
Our datasets can be automatically downloaded to the directory with train and test fold using included code.
We report the details of our datasets in Supp.
E and hyperparameters for both our model and baselines in Supp.
F. We list the hyperparameters corresponding to the best performance in Supp.
G.
A C OMPARISON TO NAM (A GARWAL ET AL ., 2020) AND THE UNIVARIATE NETWORK IN NID (T SANG ET AL ., 2017)
First, we compare with NAM’s
(Agarwal et al., 2020) performance without and with the ensemble.
Since NAM requires an extensive hyperparameter search and to be fair with NAM, we focus on 3 of their 5 datasets (MIMIC2, COMPAS, Credit) and use their reported best hyperparameters to compare.
We ﬁnd that NODE-GAM is better than NAM without ensemble consistently across 3 datasets.
[Table]
As the reviewer points out, in the NID paper (Tsang et al., 2017) they also have a similar idea to
NAM that trains a univariate network alongside an MLP to model the main effect.
But there are two key differences between the univariate network in NID and NAM: (1) NAM proposes a new jumpy activation function - ExU - that can model quick, non-linear changes of the inputs as part of their hyperparameter selection, and (2) NAM uses multiple networks to ensemble.
To be thorough, we compare with NAM that only uses normal units to resemble the univariate network in NID.
We also considered whether removing the ensemble would disproportionately impact ExU activations since
ExU activations are more prone to overﬁtting.
We show the performance in Table 4.
We show the results in 2 of their 5 datasets (MIMIC-II, Credit) that ﬁnd ExU perform better than normal units (in other 3 datasets normal units perform better).
First, we ﬁnd that after ensemble normal units perform quite similar to ExU in MIMIC2 but worse in Credit.
And given that in 3 other datasets NAM already ﬁnds normal units to perform better, we think normal units and ExU probably have similar accuracy.
Besides, ensemble helps improve performance much more for ExU units but not so much for normal units, since ExU is a more low-bias high-variance unit that beneﬁts more from ensembles.
[Table]
COMPAS
[Table]
B P SEUDO - CODE FOR NODE-GAM
Here we provide the pseudo codes for our model in Alg.
We highlight our key changes that make NODE as a GAM in red, and the new architectures or regularization in blue.
We show a single
GAM decision tree in Alg. 1, a single GA 2 M tree in Alg. 2, model algorithm in Alg. 3, and the model update in Alg.
Algorithm Input: Input X ∈ R D , Temperature T ( T −→ 0), Previous layers’ outputs X p ∈ R P , Previous layers’ feature selection G p ∈ R P × D , Attention Matrix A ∈ R P
Hyperparameters: Tree Depth C , Column subsample ratio η
Trainable Parameters : Feature Selection Logits F ∈ R D , Split Thresholds b ∈ R C , Split slope
S ∈ R C , Node weights W ∈ R 2 C if η < 1 then n = max( Dη, 1) (cid:46) Number of subset features i = shufﬂe(range(D))[int(n):] (cid:46) Randomly choose features to exclude
F [i] = − inf (cid:46) Exclude features end if
G = EntMax ( F /T )
(cid:46) Go through EntMax to generate soft one-hot vector (Eq. 5)
K = X · G (cid:46) Pick 1 feature softly if X P is not None then g =
G P G ∈ R P (cid:46)
Calculate gating weights
g (Eq. 6) g (cid:48) =
g / (cid:80) g
if A is None else g · entmax α (log( g ) + A ) (cid:46) Attention-based GAM (Eq.8)
K = K + X p · g (cid:48)
(cid:46) Add previous outputs with g normalized toend if
H = EntMoid (( K − b ) / S ) ∈ R C
(cid:46)
Generate soft binary value
e = (cid:18)(cid:20)
H(1 − H 1 ) (cid:21) ⊗ · · · ⊗ (cid:20) ( H C ) (1 − H C ) {+(cid:21)(cid:
+}
) ∈ R 2 C (cid:46) Go through the decision tree h = e · W (cid:46) Select one weight value softly as the output Return: h , G (cid:46) Return tree response h and feature selection G
Input: Input X ∈ R D , Temperature T ( T −→ 0), Previous layers’ outputs X p ∈ R P , Previous layers’ feature selection G 1 p , G 2 p ∈ R P × D , Attention Matrix A ∈ R P
Hyperparameters: Tree Depth C , Column subsample ratio η
Trainable Parameters : Feature Selection Logits F 1 , F 2 ∈ R D , Split Thresholds b ∈ R C , Split slope S ∈ R C , Node weights W ∈ R 2 C if η < 1 and ﬁrst time running then n = max( Dη, 1) (cid:46) Number of subset features i =
shufﬂe(range(D))[n:]
(cid:46) Randomly exclude features
F 1 [i] = − inf , F 2 [i] = − inf (cid:46) Exclude features end if
G 1 =
EntMax ( F 1 /T ) , G 2 = EntMax ( F 2 /T )
(cid:46) Get soft one-hot vector (Eq. 5)
K 1 = X · G 1 , K 2 = X · G(cid:46) Pick 1 feature softly if X P is not None then g = min (( G 1 · G 1 P ) ×
( G 2 · G 2 P ) + ( G 1 · G 2 P ) ×
( G 2 · G 1 P ) , 1) (cid:46)
Gating weights (Eq. 10) g (cid:48) =
g / (cid:80) g
if A is None else g · entmax α (log( g ) + A ) (cid:46) Attention-based GAM (Eq.8)
K 1 = K 1 + X p · g (cid:48)
(cid:46) Add previous outputs with g normalized to
K 2 =
K 2 + X p · g (cid:48)
K =
[ K 1 , K 2 , K 1 , ..., K 2 ] ∈ R C (cid:46) Alternating between K 1 and Kend
if
H = EntMoid (( K − b ) / S ) ∈ R C
(cid:46)
Generate binary e
= (cid:18)(cid:20)
H(1 − H 1 ) (cid:21) ⊗ · · · ⊗ (cid:20) ( H C ) (1 − H C ) (cid:21)(cid:
) ∈ R 2 C (cid:46) decision h
= e · W (cid:46) output
Return: h , [ G 1 , G 2 ]
C P URIFICATION OF GA 2 M
Note that GA 2 M can have many representations that result in the same function.
For example, for a prediction value v associated with x 2 , we can move v to the main effect f 2 ( x 2 ) = v , or the interaction effect f 23 ( x 2 , · ) = v that involves x 2 .
To solve this ambiguity, we adopt "puriﬁcation" (Lengerich et al., 2020) that pushes interaction effects into main effects if possible.
To purify an interaction f jj (cid:48) , we ﬁrst bin continuous feature x j into at most K quantile bins with
K unique values x 1 j , ...x Kj and for x j (cid:48) as well.
Then for every x kj , we move the average a kj of interactions f jj (cid:48) to main effects f j : ∀
[Equation]
= [Equation] x kj , a kj =
[Equation]
This is one step to purify f jj (cid:48) to f j .
Then we purify [Table] [Table] so
[Table] all [Table] are close to 0 .
D NODE-GA 2 M FIGURES
Here we show the architecutre of NODE-GA 2 M in Figure 7.
E D ATASET D ESCRIPTIONS
Here we describe all 8 datasets we use and we summarize them in Table 5.
• Churn: this is to predict which user is a potential subscription churner for telecom company.
https://www.kaggle.com/blastchar/telco-customer-churn •
Support2: this is to predict mortality in the hospital by several lab values.
http:// biostat.mc.vanderbilt.edu/DataSets • MIMIC-II and MIMIC-III dataset (Johnson et al., 2016b): this is an ICU patient datasets to predict mortality of patients in a tertiary academic medical center in Boston, MA, USA.
• Income:
UCI Dua & Graff (2017).
This is a dataset from census collected in 1994, and the goal is to predict who has income >50K/year.
https://archive.ics.uci.edu/ ml/datasets/adult • Credit: this is to predict which transaction is a fraud.
The features provided are the coefﬁcient of PCA components to protect privacy.
https://www.kaggle.com/mlg-ulb/ creditcardfraud • Bikeshare (Dua & Graff, 2017): this is the hourly bikeshare rental counts in Washington D.C., USA.
https://archive.ics.uci.edu/ml/datasets/bike+ sharing
dataset • Wine (Dua & Graff, 2017): this is to predict the wine quality based on a variety of lab values.
https://archive.ics.uci.edu/ml/datasets/wine+quality
For 6 datasets used in NODE, we use the scripts from NODE paper ( https://github.com/
Qwicen/node ) which directly downloads the dataset.
Here we still cite and list their sources:
Click: https://www.kaggle.com/c/kddcup2012-track • Higgs: UCI (Dua & Graff, 2017) https://archive.ics.uci.edu/ml/ datasets/HIGGS • Epsilon: https://www.k4all.org/project/large-scale-learning-challenge/ • Microsoft: https://www.microsoft.com/en-us/research/project/ mslr/ [Table] PASCAL
[Table] Microsoft [Table] Yahoo
Ads
[Table]
Yahoo LETOR dataset
Year
Music
[Table]
Million Song Dataset
• Yahoo
https:/
webscope
.sandbox.yahoo.com
/catalog
.php?
datatype
=c .
• Year (Dua & Graff, 2017): https://archive.ics.uci.edu/ml/datasets/
yearpredictionmsd E. P REPROCESSING
For NODE and NODE-GAM/GA 2 M, we follow Popov et al.
(2019) to do target encoding for categorical features, and do quantile transform 1 with 2000 bins for all features to Gaussian distribution (we ﬁnd Gaussian performs better than Uniform).
We ﬁnd adding small gaussian noise (e.g. 1e-5) when ﬁtting quantile transformation (but no noise in transformation stage) is crucial to have meanand standard deviation close to 1 after transformation.
F H YPERPARAMETERS S ELECTION
In order to tune the hyperparameters, we performed a random stratiﬁed split of full training data into train set (80%) and validation set (20%) for all datasets.
For datasets we compile of medium-sized (Income, Churn, Credit, Mimic2, Mimic3, Support2, Bikeshare), we do a 5-fold cross validation for 5 different test splits.
For datasets in NODE paper (Click, Epsilon, Higgs, Microsoft, Yahoo, Year), we use train/val/test split provided by the NODE paper author.
Since they only provide 1 test split, we report standard deviation by different random seeds on these datasets.
For medium-sized datasets, we only tune hyperparameters on the ﬁrst train-val-test fold split, and ﬁx the hyperparameters to run the rest of 4 folds. This means that we do not search hyperparameters per fold to avoid computational overheads.
All NODE, NODE-GAM/GA {+
+} M are run with 1 TITAN
Xp GPU, 4 CPU and 8GB memory.
For EBM and Spline, they are run with a machine with 32 CPUs and 120GB memory.
Below we describe the hyperparameters we use for each method:
F. EBM
For EBM, we set inner_bags=100 and outer_bags=100 and set the maximum rounds as 20k to make sure it converges; we ﬁnd EBM performs very stable out of this choice probably because we set total bagging as 10k that makes it stable; other parameters have little effect on ﬁnal performance.
For EBM GA 2 M, we search the number of interactions for 16 , 32 , 64 , 128 and choose the best one on validation set.
On large datasets we set the number of iterations as 64 as we ﬁnd it performs quite well on medium-sized datasets.
1 sklearn.preprocessing.quantile_transform
F. S PLINE
We use the cubic spline in PyGAM package (Servén & Brummitt, 2018) that we follow Chang et al.
(2021) to set the number of knots per feature to a large number 50 (we ﬁnd setting it larger would crash the model), and search the best lambda penalty between 1e-3 to 1e3 for 15 times and return the best model.
F. NODE, NODE-GA 2 M AND NODE
We follow NODE to use QHAdam (Ma & Yarats, 2018) and average the most recent 5 checkpoints.
In addition, we adopt learning rate warmup at ﬁrst 500 steps.
And we early stop our training for no improvement for 11 k steps and decay learning rate to 1/5 if no improvement happens in 5 k steps.
Here we list the hyperparameters we ﬁnd works quite well and we do not do random search on these hyperparameters:
• optimizer : QHAdam (Ma & Yarats, 2018)
(same as NODE paper) • lr_warmup_steps :• num_checkpoints_avged :•
temperature_annealing_steps ( S ): 4k • min_temperature : 0.01 (0.01 is small enough for making one-hot vector.
And after S steps we set the function to produce one-hot vector exactly.)
• batch_size : 2048, or the max batch size that ﬁts in GPU memory with minimumMaximum training time : 20 hours.
This is just to avoid model training for too long.
We use random search to ﬁnd the best hyperparameters which we list the range in below.
We list the random search range for NODE:
• num_layers : {2, 3, 4, 5} • total tree counts ( = num_trees × num_layers ): {500, 1000, 2000, 4000} • depth : {2, 4, 6} • tree_dim : {1, 2, 3} • output_dropout ( p 1 ): {0, 0.1, 0.2} • colsample_bytree : {1, 0.5, 0.1, 1e-5} • lr : {0.01, 0.005} • l2_lambda : {0., 1e-7, 1e-6, 1e-5} • add_last_linear
(to add last linear weight or not): {0, 1} • last_dropout
( p 2 , only if add_last_linear =1): {0, 0.1, 0.2, 0.3} • seed : uniform distribution [1, 100]
For NODE-GAM and NODE-GA 2 M, we have additional parameters:
• arch : {GAM, AB-GAM} • dim_att (dimension of attention embedding E ): {8, 16, 32}
We show the best hyperparameters for each dataset in Section G.
F. XGB OOST
For large datasets in NODE, we directly report the performance from the original NODE paper.
For medium-sized data, we set the depth of xgboost as 3, and learning rate as 0 .
1 with n_estimators =50k and set early stopping for 50 rounds to make sure it converges. [Table]
F. R ANDOM F OREST (RF)
We use the default hyperparameters from sklearn and set the number of trees to a large number 1000.
G B EST HYPERPARAMETERS FOUND IN EACH DATASET
Here we report the best hyperparameters we ﬁnd for 9 medium-sized datasets in Table 6 (NODEGAM), Table 7 (NODE-GA 2 M), and Table 8 (NODE).
We report the best hyperparameters for large datasets in Table 9 (NODE-GAM) and Table 10 (NODE-GA 2 M).
H C OMPLETE SHAPE GRAPHS IN B IKESHARE AND MIMIC
We list all main effects of Bikeshare in Fig.
8 and top 16 interactions effects in Fig.
We list all main effects of MIMIC2 in Fig.
10 and top 16 interactions effects in Fig.
[Table] [Table]
[Table] [Table]
[Figure] [Figure]
[Figure] [Figure]
[Figure]
EBMSpline 0.
[Figure]
[Figure] [Figure]
[Figure] [Figure] [Figure]
[Figure]
Lymphoma (
[Figure] [Figure]
EBMSpline 0.
[Figure]