Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection
Authors: James Enouen
Yan Liu
Venue: NeurIPS 2022 Accept (2022)
Keywords: interpretability
additive models

Abstract: There is currently a large gap in performance between the statistically rigorous methods like linear regression or additive splines and the powerful deep methods using neural networks.  Previous works attempting to close this gap have failed to fully consider the exponentially growing number of feature combinations which deep networks consider automatically during training.  In this work, we develop a tractable selection algorithm to efficiently identify the necessary feature combinations by leveraging techniques in feature interaction detection.
Our proposed Sparse Interaction Additive Networks (SIAN) construct a bridge from these simple and interpretable models to a fully connected neural network.  SIAN achieves competitive performance against state-of-the-art methods across multiple large-scale tabular datasets and consistently finds an optimal tradeoff between the modeling capacity of neural networks and the generalizability of simpler methods.

Sparse Interaction Additive Networks via Feature Interaction Detection and Sparse Selection
James Enouen
Yan Liu Department of Computer Science University of Southern California Los Angeles, CA yanliu cs@usc.edu
Department of Computer Science University of Southern California Los Angeles, CA enouen@usc. edu
Abstract
There is currently large gap in performance between the statistically rigorous methods like linear regression or additive splines and the powerful deep methods using neural networks. Previous works attempting to close this gap have failed to fully investigate the exponentially growing number of feature combinations which deep networks consider automatically during training. In this work, we develop a tractable selection algorithm to efficiently identify the necessary fea- ture combinations by leveraging techniques in feature interaction detection. Our proposed Sparse Interaction Additive Networks (SIAN) construct bridge from these simple and interpretable models fully connected neural networks. SIAN achieves competitive performance against state-of-the-an methods across multiple large-scale tabular datasets and consistently finds an optimal tradcoff between the modeling capacity of neural networks and the generalizability of simpler methods.
Introduction
Over the past decade, deep learning has achieved significant success in providing solutions to challenging AI problems like computer vision, language processing, and game playing [17, 44, 37J. As deep-learning-basee AI models increasingly serve important solutions to applications in our daily lives, we are confronted with some of their major disadvantages. First, current limitations in our theoretical understanding of deep learning makes fitting robust neural networks balaneing act between accurate training fit and low generalization gap. Questions which ask how the test performance will vary given the choice of architecture, dataset, and training algorithm remain poorly understood and difficult to answer. Second, infamously known as blackbox models, deep neural networks greatly lack in interpretability. This continues to lead variety of downstream consequences applications: unexplainable decisions for stakeholders, the inability to distinguish causation from correlation, and misunderstood sensitivity adversarial examples.
In contrast, simpler machine learning models such as linear regression, splines, and the generalized additive model (GAM) [16] naturally win in interpretability and robustness. Their smaller number of parameters often have clear and meaningful interpretations and these methods rarely succumb to overfitting the training data. The main shortcoming of these simpler methods is their inability to accurately fit more complex data distributions.
There is a growing strand of literature attempting to merge the interpretability of additive models with the streamlined differentiable power of deep neural networks (DNNs). Key works in this direction such NAM and TODE-GAM have been able successfully model one-dimensional main effeets and two-dimensional interaction effects using differentiable training procedures [2, 8). Although many other works have found similar success in modeling one- and two-dimensional interactions, few have made practical attempts towards feature interactions size three or greater, which we will
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

refer to throughout as higher-order feature interactions. In this way, no existing works have been able to use the hallmark abilitt of neural networks to model higher-order interactions: amassing the influence of hundreds of pixels in computer vision and combining specific words from multiple paragraphs in language processing. towards the impressive differentiable power of neural networks by developing a simple but effective selection algorithm and an efficient implementation to be able to train neural additive models which fit higher-order interactions of degrees three and greater. The proposed Sparse Interaction Additive Networks (SIANs) consistently achieve results that are competitive with state-of-the-ar methods across multiple datasets. We summarize our contributions as follows:
this work, we bring interpretable models one step closer
We develop a feature interaction selection algorithm which efficiently selects from the expo- nential number of higher-order feature interactions by leveraging heredity and interaction detection. This allows us to construct higher-order neural additive models for medium-scale datasets unlike previous works in neural additive modeling which only consider univariate and bivariate functions.
We provide further insights into the tradeoffs faced by neural networks between better gener- alization and better functional capacity. By tuning the hyperparameters of our SIAN model, we can gradually interpolate from one-dimensional additive models to full-complexity neural networks. We observe fine-grained details about how the generalization gap increases add capacity to our neural network model.
We design a block sparse implementation of neural additive models which is able to greatly improve the training speed and memory efficiency of neural-based additive models. These improvements over previous NAM implementations allow shape functions to be computed in parallel, allowing us to train larger and more complex additive models. We provide code for our implementation and hope the tools provided can be useful throughout the additive modeling community for faster training of neural additive models." I
2 Related Work
The generalized additive model (GAM) [16] has existed for decades as a more expressive alternative to linear regression, replacing each linear coefficient with nonparametric function. -io-dimensional extensions appeared in the literature shortly after its introduction [46]. Over the years, an abundance of works have used the GAM model as an interpretable method for making predictions, with the choice of functional model typically reflecting the most popular method during the time period: regression splines, random forests, boosting machines, kernel methods, and most recently neural networks [18, 7, 21, 49). Two of the most prominent neural network based approaches are NAM and NODE-GA 2M [2, 8). The former stacks multilayer perceptrons build one-dimensional GAM; the latter connects differentiable decision trees to build a two-dimensional GAM. Both have demonstrated competitive performance and interpretable trends learned over multiple tabular datasets. Other neural network extensions [47, 33] increase the modeling capacity to higher-order interactions by first training a two-dimensional model and then training an additional blackbox neural network to fit the residual error. While this approach does have higher modeling capacity, it foregoes interpretable insights on the higher-order feature interactions and suffers the same inclination to overfit held naturally by deep neural networks.
Other works in additive modeling instead focus on extending the univariate GAM sparse additive models in the high-dimensional inference regime [24, 36, 29, 48). Further extensions of these methods to sparse bivariate models for high-dimensional inference also exist [43, 25]. These works extend classical nigh-dimensional inference techniques like LASSO and LARS from linear parameters to additive functions by shrinking the effect of minimally important variables and emphasizing sparse solutions. We note that, unlike these works, we do not use sparsity in the soft-constrained sense to shrink features from fixed selection. but instead adaptively use feature interaction measurements to hierarchically build a feasible set of interactions. The only existing work in additive modeling which uses hierarchy to induce sparsity in the same sense as this work is the GAMI-Net which uses a three stage procedure to select candidate pairs under a hierarchical constraint [49]. Extending their procedure to three or higher dimensions would require four or more stages of training and is left unexplored in their work.
Available htttab/.wom./hooeeee//eeeeeeeeee wtt.aeercccccccoiiaiiiiieeeetee.
2

---

Deep
(e) Sparse Interaction Additive Network
Intemction Solection Algoritim
Model from Interactions
Sei
11:33
11:43
(b)
Selected
[1.
[14
12.4
A-4
Feature
Internctions
II
R
Unrelooed hoam
-olendd Fouru
Figure 1: SIAN Pipeline Diagram. (a) We train a base DNN to be able to learn the feature interactions
from the dataset given the 4 input features of X1, X2. X3, X4. (b) We feed our DNN into the FIS algorithm to be able to select from the 24 = 16 possible subsets of {X1, X2, Xa, Xa). (c) We use selected feature subsets as the GAM array (of length 6 in the image) for our full STAN network. The empty set function f(J a constant we absorb into the last additive layer of (x)) Finally, we train our SIAN neural network using the specified architecture.
Although the theoretical extension to baree-dimensional additive models is clear, there is currently a lack of discussion surrounding the practical challenges faced when trying to model dinenssiiaii shape functions. One the few works to pursue practical implementation of higher-order GAMs real-world datasets is the work of SALSA [21]. This work to fit additive models of order three and higher. Their work also corroborates our finding that optimal performance is achieved by different orders for different datasets. However, the kernel-based approaches used in this work make it unsuitable beyond small-scale datasets with few samples. This makes our work one of the first train additive models of higher-order interactions which leverage the automatic differentiation and GPU optimization toolkits which have become commonplace in modern workstations.
specialized kernel function
3 Methods
Notation We denote d-dimensional input E Rd with its i-th component its correspond- ing output is denoted y E R. We consider one-dimensional y as in regression and binary classification. We will use f(x:) denote the function model used to approximate y, implicitly considering the additive noise model y = f (Ix + e for some noise term E. We denote subset the set of features by I fir..... ( C [d] {1, (1..... d}. Its cardinality is denoted [Z], its complement !I, and its
power set P(Z). For Rd we define æτ Rd such that:
(xz); {
Tq if i otherwise
3.1 Generalized Additive Models
We first consider the generalized additive model (GAM) [16], which extends linear regression by allowing each input feature to have a nonlinear relationship with the output.
g fo + (x + +... fa(wa)
(1)
Each of the f, 'reshapes' their respective feature z; and then adds the reshaped feature the total prediction. These f. are hence called shape functions and were traditionally fit using regression splines. The function g is the link function which will be the identity function for regression and the inverse-sigmoid g(v) log( - for classification. fe is normalizing constant. This original
formulation where each shape function considers only one feature will further refer as GAM-1.

---

Feature Interactions In order to extend this definition we must consider the interplay which occurs 'non-additive feature interaction' between features S [d] between multiple imput features. for the function is said to exist when the function ʃ cannot be decomposed into sum of [ arbitrary subfunctions such that each subunction Si excludes one of the interacting variables r;; f() * Eier (1.......... [15, 38, 41J. In other words, the entire feature set (r, : E I) must be simultaneously known to be able to correctly predict the output (z)
The goal of feature interaction detection is to uncover these groups of features which depend one another. For smooth functions, this can be quantitatively done by finding the sets I S [d] such that the interaction strength, w (I), is positive and large.
a f(x)
(I) := Ez
> O.
(2)
ari, Oris 0rigzi
We may now adjust the GAM definition to capture feature interactions by considering a set of T specified interactions, {-+]-= where each Ie S [d] is an interaction of size [Tz|:
((y) = So + Ji(ss)) Σ Ja (1T,)
1
The third term extends GAMs to full capacity models which can represent nonlinear dependencies of arbitrary feature sets. For instance, if our set of interactions (I,)²= includes the complete feature set [d]. then our model has exactly the same capacity as the underlying functional model we choose for the shape functions (splines, random forests, deep neural networks, etc.) An abundance of previous works have used this extended version of the GAM model, often called the Smoothing Spline ANOVA model or the functional ANOVA model [7, 36, 29, 46].
Throughout this work, we will use GAM-K to refer to GAM whose highest order interaction in {I,)=1 is of cardinality K. (i.e. [Z,| S K Vt E [T]) For instance, we will call the VODE-GA'M model [8] GAM-2 model since the interaction sets are all possible feature pairs: {{i,)
j for j E [d]]. We will refer to our SIAN networks as SIAN-K using the same convention.
3.2 Feature Interaction Selection
A key concern of using neural networks to fit the shape functions is keeping the number of networks low enough that our training time computation is kept reasonable. While this is typically not problem for the GAM-1, this can quickly become an issue for the GAM-2. For instance, if we have an imput variable æ with 30 features, then including all pairwise functions would need cover the (30) 435 possible pairs. Although learning 465 linear coefficients is reasonable, training hundreds of neural networks becomes less so. Moreover, this quantity only grows exponentially as we increase K higher-order interactions (3, 4, 5, etc.) In an effort combat this growing complexity, we introduce a Feature Interaction Selection (FIS) algorithm which depends on two key ingredients: an interaction detection procedure and heredity condition.
Feature Interaction Detection In recent years, there has been growing body of work focused on detecting and measuring the strength of feature interactions from large-scale data. Three of the most popular and generally applicable of these methods are the Shapley Additive Explanations (SHAP) [28, 26, 13], Integrated Hessians [40, 19], and Archipelago [42]. For our experiments, we primarily use an adaptation of Archipelago for higher-order interactions because of its compatibility with blackbox models. In contrast, Integrated Hessians is only applicable to sufficiently smooth networks (ReLU networks are not compatible) and SHAP only has fast implementations available for tree-based approaches. Moreover, although both SHAP and Integrated Hessians have clear ideological extensions to higher-order interactions, there are no currently available implementations. A detailed discussion surrounding Archipelago and other detection techniques can be found Appendix C.
Heredity The practice of only modeling the pairwise interaction effect {i.j} C [d] for some features i E [d] when both of the main effects {i) and (j} are already being modeled has a long history throughout statistics [34, 11, 5]. There are two main versions of this hierarchical principle explored in the literature: strong heredity and weak heredity. If we are given that w ((iij]) > O, strong heredity implies that both ((i))) > 0 and w(()) O whereas weak heredity implies that

---

((f+)) > 0 or (())) > 0. For our definition of feature interaction, we have that strong heredity holds; however, our algorithm will instead focus on a computational version of heredity which asks that 7 percent of subsets are above threshold 0 in order for pair (triple, tuple, etc.) to be considered as a possible interaction.
Algorithm In Algorithm 1, we detail how we use Archipelago to build our FIS algorithm. The visual overview of the SIAN pipeline is also de- picted in Figure above. We start by training reference DNN which is required for the in- ductive insights generated by Archipelago. We then pass the trained function approximator to the FIS algorithm along with hyperparameters K' (interpretability index), (heredity strength threshold), and (interaction strength thresh- old). This procedure efficiently searches and ranks the 2d possible feature subsets to produce final set of interactions which we the SIAN model architecture. Finally, we train
Algorithm 1 Feature Interaction Selection (FIS)
Inputs: Trained prediction model f(x), and vali- dation dataset X = (xx(1), ;-..; ........V)))) xv{)
Parameters: Cutoff index K, cutoff threshold r,
strength threshold 0
Output: I. a family of feature interactions with index most K and strength above 0
1: Set I <
2: Set J <- {(i) E
specify
3: k x-11
4: while k < K do
our SIAN neural network using typical gradient
5: for J in J do
descent methods. In Appendix A, we provide further theoretical discussion in which we prove exact recovery of the true feature interactions and show how this sparse selection leads to prov- ably lower generalization gaps in toy additive
6:
(x(X") +0
7:
for (0)) x do
w)(x(")) t Archipelago x())
8:
9:
wx(X') t w)(X') + ()(((o))
10:
end for
noise model.
11: w)(X*) t wx(()) px-
12: ifw)(X") > 0 then
3.3 Block Sparse Implementation
I IU 14: end if
13:
In order to improve the time and memory ef- ficiency of the SIAN model, we implement 16:
15:
end for
J' + {J J P((d): +1]
block sparse construction for neural additive
17:
for J in J' do
models. The default scheme of SIAN is to use network module for each the shape functions
18:
v(J)= р Eigulleer ((--)=)) end for
and additively combine the output features, fol-
20:
J+ (J:))
lowing the implementation strategies of NAM
21:
k k+
and other popular neural additive models. How-
22:
end while
ever, since each shape function network is com-
23: return
puted sequentially, this greatly bottlenecks the computation speed of the model. We instead construct large, block sparse network which computes the hidden representations of all shape function networks simultaneously, leveraging the fact that each shape subnetwork has the same depth. Using this block sparse network allows for shape features to be computed in parallel, leading to a significant improvement in training speed. The main consequence of this design a higher footprint in memory; therefore, we also develop a compressed sparse matrix implementation of the network which has greatly reduced memory footprint for saving network parameters. SIAN is able interchange between these different modes with minimal overhead, allowing for faster training in the block sparse module, lower memory footprint in the compressed module, and convenient visualization of shape functions in the default module. We provide further numerical details of our gains in Section 5.
Datasets
Our experiments focus on seven machine learning datasets. Two are in the classification setting, the MIMIC-III Healthcare and the Higgs datasets [20, 3]. The other five are in the regression setting, namely the Appliances Energy, Bike Sharing, California Housing Prices, Wine Quality, and Song Year datasets [6. 14, 22, 12, 4]. More details about each dataset are provided in Table 1. We evaluate the regression datasets using mean- mean-squared error (MSE). We measure the performance on the classification datasets using both the area under the receiver operating characteristic (AUROC) and
5

---

the area under the peecision-recall curve (AUPRC) metrics. We report both metrics for the MIMIC dataset since the positive class is only 9% of examples and report only AUROC for the Higgs dataset since it is relatively well-balanced.
4.1 Experiment Details
For the baseline DNNs we are using hidden layer sizes [256, 128,64] with ReLU activations. For the GAM subnetworks are using hidden layer sizes [2,8] with ReLU activations. We use L1 regularization of size 5e-5. In the main results section, we report the results for each K 21, 5) using only single value of and f. The hyperparameter selected from a handful of potential values using a validation set. We train all networks using Adagrad with learning rate of 5e-3. All models are evaluated on held-out test dataset over five folds of training-vallen validation split unless three folds are specified. Three folds are used for NODE-GAM on all datasets as well as Song Year and Higgs for all models. We respect previous testing splits when applicable, otherwise we subdivide the data using an 80-20 split to generate a testing set. In addition to NODE-GA2M, we compare against the interpretable models LASSO and GA EBM well as the popular machine learning models of support vector machines (SVM), random forests (RF), and extreme gradient boosting (XGB) [10, 30].
taken to be 0.5 throughout and 0 was
Table I: Real-world datasets. n is the number of sam- ples, d is the number of features, and p the percent- age of data samples with the positive class label.
Table 2: MIMIC-III Performance.
Model
AUROC (f)
AUPRC (1)
SAPS II
0.792
0,281
SOFA
0.703
0,225
Dataset
π
p
LASSO
0.568
0.396
Higgs Boson
11, 000, 000
28
53.0%
GA"M EBM
0.840
0.375
NODE-GA"M
0.826
0.345
MIMIC-III
32, 254
30
9.2%
SIAN-1
0.848
0.409
Energy Appliances
19, 735
30
SIAN-2
0.855
0.423
0.856
Bike Sharing
17,379
13
SIAN-S
0.856
0.425
California Housing
640
KF
0.821
0.369
Wine Quality
6.497
12
0.831
0.404
XGB
0.813
0,382
Song Year
515,345
90
DNN
0.844
0.382
5 Results
Across seven different datasets, SIAN achieves an average rank of 3.00 out of the 8 models consider. The next best performing model, NODE-GA2M, has an average rank of 3.71 out of The third best performing model, DNN, has an average rank of 3.86 out of 8. We find that SIAN achieves consistent performance by being able to adapt to both the lww-dimensional datasets and the high-dimensional datasets, finding balance between good training fit and good generalization.
Table 3: Test metrics for six of seven datasets. (+)(() indicates higher/lower better, respectively.
Model
Appliunces Energy
Bike Sharing (1)
California Housing (3)
Wine Quality (0)
Song Year (3)
Higgs Boson (1))
LASSO
0,74020.002
1.055=0.001
0,478:0.0000
0.375=0.0022
1:000000008
0.635=0.000
GA EBM
1.053:0.138
0.124:20.044
0.26510.002
0.49810.004
0.894:0.001
0.698=0.001
NODE-GA 'M
I 1.04420.066
0.111==0.006
0.222::0.005
0.521=000009
0.806=0.001
0.811=0.000
SIAN-1
0.718
0.387+0.035
0.378+0.0077
0.551+00004
0.860=0.001
0.771=0.001
SIAN-2
0.763-10.009
0.127-10.008
0.30210.002
0.497:00.003
0.842-00.02
0.795=0.001
SIAN-3
0.800+000066
0.125=00013
0.278+0.001
0.497=00000
0.81==0.001
0.798-0000
SIAN-S
0.801:10.011
0.149=00011
0.272+0.003
0.48420.0066
0.811=0.001
0.802=0.001
RF
L11410.195
0,206 0..009
0.201
0.409
0.994
0.634_0.002
0.740+0.008
0.262+0.001
0.457=0.008
0.940-0.012
0.698
XGB
1118:201119
0.157=00.03
0.229:0.002
0,465=0.014
0.88120.000
0.740=0.000
DNN
0.945+0.054
0,374++0.017
0.283+0.0005
0.495+00.007
0.791-00.002
0,823-00 0.000
For the MIMIC dataset, we can see the models' performances in terms of both AUROC and AUPRC in
Table 2. addition to the models we previously described, we also compare against the interpretable medical baselines of SOFA and SAPS II which are simple logic-based scoring methods [23, 45). We see that the machine learning methods improve over the baseline performance achieved by the SAPS and SOFA methods.

---

In Table 3, we can see the combined results for our six other datasets. First, in the Appliances Energy dataset, we see that the SIAN-I performs the best, with LASSO and SVM trailing slightly behind. The success of one-dimensional methods on this dataset could be indicative that many of the dataset's trends are one-dimensional. As we increase the dimension of the SIAN, the test error slowly increases as the generalization gap grows; the full-complexity DNN has even worse error than all SIAN models.
Second, in the Bike Sharing dataset, we see that the best performing model the NODE-GA2M, with the EBM, SIAN-2, and SIAN-3 trailing only slightly behind. All of these methods are two or three dimensional GAM models, again hinting that a significant portion of the important trends in this dataset could be bivariate. Indeed, the most important bivariate trend accounts for more than 50% of the variance the dataset, as we explore Figure below.
For the remaining datasets, we see that the performance of the SIAN model improves as we add higher and higher-order feature interactions. For the California Housing dataset, find the best performance using the differentiable tree method of NODE-GA2M. For the Wine Quality dataset, we find the best performance using the random forest algorithm. For the larger-scale datasets of Song Year and Higgs Boson, we find that the best performance is still obtained by a full-complexity deep neural network. These two are the only datasets where SIANs of order five less are not sufficient to outperform vanilla deep neural networks, implying there are important feature interactions of degree greater than five.
5.1 Training Speed and Storage
In Table 4 below we see how our STAN network compares against other popular differentiable additive models in both training time and size on disk. For fair comparison we do not utilize our interaction selection algorithm for SIAN in this section, instead training SIAN-2 with all possible pairs. We see that our implementation of different modes for the SIAN architecture allows us to outperform both NAM and faster training on GAM-2 models. We reiterate that because the overhead for switching between modes is negligible, SIAN enjoys the benefits of all modes: training quickly in the block sparse mode and saving succinetly in the compressed mode.
with 2-8x faster training GAM-1 models and 10-80x
Table 4: Training Time and Memory Size. Experiments are run with a machine using a GTX 1080
GPU and 16GB of RAM. The average wall-clock time over three runs reported.
STAN-2
(block- (comp-
NODE-
STAN-1
(block-
(comp- ressed)
NAM
(default)
sparse)
ressed)
GA2M
(default)
sparse)
Wine
17.57
0.69
55.19
3.03
0.66
5.45
Training Time
Bike
159.65
5.38
60.75
25.65
4.58
12.09
(minules)
House
88.67
6.09
58.03
22.72
5.88
14.74
Wine
7,113
7,113
537
1,040
182
182
86
1,203
Size on Disk (KB)
Bike
9.727
9,666
626
1,040
218
212
92
308
House
1,526
1,526
251
1,040
83
83
59
1.921
5.2
Beyond ReLU Networks
The FIS algorithm we describe can be applied to other combinations of FID algorithm functional model besides Archipelago + ReLU Neural Networks. To demonstrate the general applicability of our scheme, we replace the continuous, piecewise-linear functions of ReLU neural networks with the piecewise-constant differentiable decision trees using NODE-GAM, We extend the original implementation of NODE-GAM to handle feature triplets, extending the method to trivariate function or GAM-3 model. We run our FIS algorithm using = 3 on both the Housing and Wine datasets to fit GAM-3 models using the inductive biases of the NODE architecture.
Extending from GODE-GA2M to PODE-GA3M is able to improve performance from 0.222 to 0.175 on the Housing dataset, 21.20 11.2% further improvement over the state of the art method. The same extension is only able to deliver 4.8% improvement from 0.521 to 0.496 on the Wine dataset; however, both the SIAN-5 and NODE-GA3M are able to improve performance over all previously available additive models. These two examples demonstrate the ability of our FIS pipeline to be applied to more general machine learning techniques to model higher-order interactions.
7

---

(2)
Bike Sharing Interactions
Tergengaang
tipres
tripits part singles
MSE
Tartee
SiNcs Rortod
Hot olay Weslond Work Day
Feature Interaction Strnnghh [kg]]
Number Functions (log)
Torperstar
Figure 2: vertical scaling by degree to display in single plot. (b) We plot the training and validation performance (darker and lighter, respectively) average to reduce variance instcad of training multiple networks of each size. The blue star the performance the shape functions in 2d. (c) The most important shape function showing the interaction between hour of the day and work day. (d) The next shape function showing temperature's 's effect on bikes rented.
Analysis of Bike Sharing. (a) We plot the histogram of feature interaction strength, adjusting the
add feature interactions SIAN. We
exponentially weighted moving
Los Angeles
San Francisco
Low Populatton (500 people)
High Population (10,000 people)
Latitdde
(A)
M4
4200
<D Longitude
Dlock-Lavell Popul al
Figure 3: 3D feature interaction for California Housing. The effect of the location only for the entire state of California and zoomed in to three selected locations. (b) The effect of only the block-level population. (c) The three-dimensional interaction effect of block-level population with location. We plot the effect for both a low and high population to highlight the differences in effect for each location. Note the different scales panel.
Discussion
In this section, we further explore the feature interactions learned by SIAN and FIS across multiple datasets. We provide multiple visualizations of the shape functions learned by SIAN to get a sense of the diverse analysis and interpretations which are made possible by additive models. Further discussion and graphics are provided in Appendix and
In Figure 2a, we visualize the interaction strength measured for each of the singles, pairs, and triples of features from the Bike Sharing dataset. The four top rated interactions (three of which are visually separated from the main body of the histogram) are, from right to left, ["hour", "hour X workday", "workday". "temperature"), In 2b we see the effect gradually adding feature interactions to our model. We see there is steep jump in performance when we are able to model the first pair, depicted 2c. Together with 2d, these two visualizations alone explain 84% of the variance in the Bike Sharing dataset. Importantly, these two trends are interpretable and agree with our intuition about when people are more likely to bike: on work days, we see peak spikes at 8 a.m. and 5 p.m., corresponding to the beginning and end of work hours; weekends and holidays, there is steady demand of bikes throughout the afternoon. There are also more bikers during warmer temperatures.
In Figure 3, we set out to visualize one of the three dimensional interactions which occurs between the three features of latitude, longitude, and population. 3a, we see the rise in housing price along the coast of California, especially around the metropolitan areas of Los Angeles and San Francisco. 3b, we see that house price does not monotonically increase with population as we might expect for higher population densities. A detailed look into the dataset reveals that the 'population' feature being used is accumulated the census block level, creating an inverse relationship with population density areas like LA and SF are subdivided more than their suburban and rural counterparts. Although it is

---

possible location and population density might have independent effects on housing price, the dataset nevertheless induces an interaction between location and population. 3c, we attempt to visualize this 3D interaction by viewing subsamples across two population sizes and over three regions. We see the network has learned differentiate the urban regions of LA and SF from the rural coast of northern California, where high population becomes indicative of higher population densities and higher housing prices. We note that the trends learned by SIAN tends to be very continuous, which is potential shortcoming in learning fine-grained block-level information in cities like Los Angeles and San Francisco. In light of these concerns, we run experiments on three-dimensional extension of the piecewise constant, discontinuous NODE-GAR model in section 5.2.
In Figure 4, we see two example trends from the SIAN trained to pre- dict mortality given hospital data. On the left, we see that health risk gradu- ally increases with age. On the right, the trend with respect to the Glasgow Coma Scale indicator which is a measure of alertness and con- sciousness. The lowest and highest scores correspond severe head in- jury and full alertness, respectively. We see that very low GCS corresponds to high risk and that the risk decreases GCS increases. At the score of 13, however, we see that an increase to 14 or 15 actually increases mortality risk, defying the intuition that risk should be monotone in the GCS score. It is highly likely that this dip in mortality risk comes from the special care given to patients with GCS scores below 15, compared to their counterparts with perfect scores.
Figure 4: MIMIC-III. Age Risk and GCS Risk
This trend illustrates a phenomenon occurring throughout machine learning applications healthcare where correlation and causation are conflated with one another. Similar results have been found linking asthma to a decreased risk of death from pneumonia [7]. These issues might go unnoticed in black-box models whereas interpretable models uncover and resolve such discrepancies before deployment. While discovering and correcting interpretable trends brings clear ad advantages, is also possible for these trends to be misinterpreted by non-experts causal discoveries can not only lead to physical harm in the domain of medicine, but also to larger social harm in broader Al systems and applications.
causal relationship. Such false
7 Limitations and Future Work
A primary limitation of the current work is its focus on multilayer perceptrons whereas modern state-of th-t-off-theart results are dominated by geometric deep learning and transformer architectures. Extending this procedure to more general architectures is key direction for bringing interpretability domains like computer vision and natural language. Such domains can further customize the FIS algorithm to respect specific structures like spatial locality and knowledge graph semantics.
Another important direction of research includes better theoretical understanding how the benefits of SIAN scale with the dimensionality of the dataset and the number of samples provided, providing yet another lens study the implicit biases deep neural networks. Further, developing theory which accurately models the empirically observed distributions of feature interactions in real-world data, especially in the presence of heteroscedastic noise and correlated features, would be of great interest for this direction.
Multiple experiments confirm that SIAN produce powerful and interpretable machine learning models which match the performance of sfate-of-the-ar benchmarks like deep neural networks and NODE-GA2M. Further experiments show that FIS be applied more general machine learning algorithms. Hopefully, future work will be able to further clarify this sparse interaction perspective and help deepen understanding of the generalization performance of neural networks.

---

References
[1] A. Abdul, C. von der Weth, M. Kankanhalli, and B. Y. Lim. Cogam: Measuring and moderating cognitive load in machine learning model explanations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, CHI '20, page 1-14, New York, NY, USA, 2020. Association for Computing Machinery.
[2] R. Agarwal, N. Frosst, Zhang, R. Caruana, and G. E. Hinton. Neural additive models: Interpretable machine learning with neural nets. CoRR, abs/2004.13912, 2020.
[3] P. Baldi, P. Sadowski, and D. Whiteson. Searching for exotic particles in high-energy physics with deep learning. Nature communications, 5:4308, 07 2014.
[4] T. Bertin-Mahieux, D. P. Ellis, B. Whitman, and P. Lamere. The million song dataset. In Proceedings of the 12th International Conference on Music Information Retrieval (ISMIR 201 I), 2011.
[5] J. Bien, J. Taylor, and R. Tibshirani. A lasso for hierarchical interactions. Annals of stafistics, 41(3):1111,000
[6] L. M. Candanedo, V. Feldheim, and D. Deramaix. Data driven prediction models of energy use of appliances low-energy house. Energy and Buildings, 140:81-97, 2017.
[7] R. Caruana, Y. Lou, J. Gehrke, P. Koch, M. Sturm, and N. Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In P Proceedings the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1721-1730. ACM. 2015.
[8] C.-H. Chang, R. Caruana, and A. Goldenberg. NODE-GAM: Neural generalized additive model for interpretable deep learning. In International Conference Learning Representations, 2022.
[9] C.-H. Chang. S. Tan, Lengerich, A. Goldenberg, and R. Caruana. How interpretable and trustworthy are gams? In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining, KDD '21, page 95 105, New York, NY, USA, 2021. Association for Computing Machinery.
[10] T. Chen and C. Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785-794 ACM, 2016.
[11] H. A. Chipman. Bayesian variable selection with related predictors. Canadian Journal of Statistics-revue Canadienne De Statistique, 24:17-36, 1995.
[12] P. Cortez, A. Cerdeira, F. Almeida, T. Matos, and J. Reis. Modeling wine preferences by data mining from physicochemical properties. Decision Support Systems, 47:547-553, 11 2009.
[13] K. Dhamdhere, A. Agarwal, and M. Sundararajan. The shapley taylor interaction index. arXiv preprint arXiv: 1ww..9902...222. 2019.
[14] H. Fanaee-T and J. Gama. Event labeling combining ensemble detectors and background knowledge. Progress in Artificial Intelligence, pages 1-15, 2013.
[15] H. Friedman and B. E. Popescu. Predictive learning via rule ensembles. The Annals of Applied Statistics, pages 916-954, 2008.
[16] J. Hastie and J. Tibshirani. Generalized additive models, 1990.
[17] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition, 2015.
[18] G. Hooker. Generalized functional anova diagnostics for high-dimensional functions of depen-
dent variables. Journal of Computational and Graphical Statistics, 16(3):709-732, 2007.
[19] D. Janizek, Sturmfels, and S.-I. Lee. Explaining explanations: Axiomatic feature interac- tions for deep networks. arXiv preprint 041000000000221388.. 2020.
10

---

[20] A. E. Johnson, T. J. Pollard, L. Shen, L. wei H. Lehman, M. Feng. M. Ghassemi, B. Moody, Szolovits, L. A. Celi, and R. G. Mark. Mimic-iii, freely accessible critical care database. Scientific Data, 3( 3(160035), 2016.
[21] K. Kandasamy and Y. Yu. Additive approximations in high dimensional nonparametric re- gression via the salsa. In M. F. Balcan and K. Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 69-78, New York, New York, USA, 20-22 Jun 2016. PMLR.
[22] R. Kelley Pace and Barry. Sparse spatial autoregressions. Statistics Probability Letters, 33(3):291-297, 1997.
[23] J.-R. Le Gall, S. Lemeshow, and F. Saulnier. A new simplified acute physiology score (saps ii) based on a european/north american multicenter study. JAMA, 270(24), 1993.
[24] Y. Lin and H. Zhang. Component selection and smoothing in multivariate nonparametric regression. The Annuly of Statistics, 34(5):2272-229) 2006.
[25] G. Liu, H. Chen, and H. Huang. Sparse shrunk additive models. In H. D. III and A. Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Reseurch, pages 6194-6204, PMLR, 13-18 Jul 2020.
[26] S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, and S.-I. Lee. From local explanations to global understanding with explainable ai for trees. Nature Machine Intelligence, 2(1):2522-5839, 2020.
[27] S. M. Lundberg, G. Erion, H. Chen, A. DeGrave, J. M. Prutkin, B. Nair, R. Katz, J. Himmelfarb, N. Bansal, and S.-I. Lee. From local explanations to global understanding with explainable ai for trees, 2020.
[28] S. M. Lundberg and S.-1. Lee. A unified approach to interpreting model predictions. In I. Guyon, U. V. Luxburg, S. Bengio, Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 4765-4774. Curran Associates, Inc., 2017.
[29] L. Meier, S. van de Geer, and P. Bühlmann. High-dimensional additive modeling. The Annals of Statistics, 37(6B):3779 - 3821, 2009.
[30] H. Nori, S. Jenkins, P. Koch, and R. Caruana. Interpretml: A unified framework for machine learning interpretability. arXiv preprint
2019.
[31] R. O'Doneell. Some topics in analysis of boolean functions. In Proceedings of the Fortieth Annual ACM Symposium on Theory of Computing, STOC '08, page 569-578, New York, NY, USA, 2008. Association for Computing Machinery.
[32] R. O'Donnell. Analysis of Boolean Functions. Cambridge University Press, 2014.
[33] L. O'Neill, S. Angus, S. Borgohain, N. Chmait, and D. L. Dowe. Creating powerful and interpretable models with regression networks, 2021.
[34] J. I. Peixoto. Hierarchical variable selection in polynomial regression models. The American Statistician, 111433311-313, 1987.
[35] S. Raskhodnikova and G. Yaroslavtsev. Learning pseudo-boolean -dnf and submodular functions. Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, 08 2012.
[36] P. Ravikumar, J. Lafferty, H. Liu, and L. Wasserman. Sparse additive models. Journal of the Royal Statistical Society. Series B (Statistical Methodology). 7105):1009 2009.
[37] D. Silver and et 2017.
Mastering the game of go without human knowledge. Nature, 550:354-333
[38] D. Sorokina, R. Caruana, M. Riedewald, and D. Fink. Detecting statistical interactions with additive groves of trees. In Proceedings of the 25th international conference on Machine learning, pages 1000-1007. ACM, 2008

---

[39] P. Stobbe and A. Krause. Learning fourier sparse set functions. In N. D. Lawrence and M. Girolami, editors. Proceedings of the Fifteenth International Conference on Artificial Intelligence and Statistics, volume 22 of Proceedings of Machine Learning Research, pages 1125-1133, La Palma, Canary Islands, 21-23 Apr 2012. PMIR.
[40] M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference Machine Learning- Volume 70, pages 3319-333333 3328. JMLR. org, 2017.
[41] M. Tsang, H. Liu, S. Purushotham, P. Murali, and Y. Liu. Neural interaction transparency (nit): Disentangling learned interactions for improved interpretability. Advances in Neural Information Processing Systems, pages 5804-5813, 2018.
[42] M. Tsang, S. Rambhatla, and Y. Liu. How does this interaction affect me? interpretable attribution for feature interactions. arXiv preprint arXiv:2006. 10965, 2020.
[43] H. Tyagi, Kyrillidiss Gärtner, and A. Krause. Learning sparse additive models with interactions in high dimensions. In A. Gretton and C. C. Robert, editors, Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, volume 51 of Proceedings of Machine Learning Research, pages 111-120, Cadiz, Spain, 09-11 May 2016. PMLR.
[44] A. Vaswani and et al. Attention is all you need. CoRR, ab./1706.03762, 2017.
[45] J. L. Vincent, R. Moreno, J. Takala, S. Willatts, A. D. Mendonça, H. Bruining, C. K. Reinhart, M. Suter, and L.. G. Thijs. The sofa (sepsis-related organ failure assessment) score to describe organ dysfunction/failure. Intensive Care Med., 22(7):707-710, 1997.
[46] G. Wahba, Y. Wang, C. Gu, Kleins, and B. Kle. Smoothing spline anova for exponential families. In The Annals of Statistics, 1994.
[47] T. Wang, J. Yang, Y. Li, and B. Wang. Partially interpretable estimators (pie): Black-box-refined interpretable machine learning, 2021.
[48] S. Xu, Z. Bu, P. Chaudhari, and I. J. Barnett. Sparse neural additive model: Interpretable deep learning with feature selection via group sparsity, 2022.
[49] 7. Yang, A. Zhang, and A. Sudjianto. Gami-net: An explainable neural network based generalized additive models with structured interactions, 2020.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [Yes]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits. hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g.. with respect the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (c.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "lME5n4dCg5", "reviewer": "Reviewer_FZvZ", "summary": "This paper proposed Sparse Interaction Additive Networks (SIAN), which can effectively tackle the exponential number of high-order feature interactions and scale up to larger scale datasets. By leveraging heredity and interaction detection, SIAN achieves competitive performance across multiple datasets. The work also provides further insights into the generalization and capacity trade-off and a block sparse implementation of neural additive models, which achieve better training speed and memory efficiency of neural-based additive models.", "questions": "How do the training speed and storage change along with the increasing of feature intersection orders in SIAN ?", "limitations": "The authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "#### Strength:\n\n1. The paper is well organized and easy to follow.\n2. Sufficient implementation details are provided for reproduction.\n3. The experiment section consists of both quantitative results and several visualization analyses.\n\n#### Weakness:\n\n1. As shown in Table 2, SIAN-1 performs the best in Appliances Energy dataset, while SIAN-2/3 is better in Bike Sharing and SIAN-5 in the rest. So how to effectively choose the feature intersection order used in SIAN for better performance.\n2. When we increase the intersection order of additive models, will it reduce the model's interpretability to some extent, which may degrade the value of performance improvement.\n3. Letter case should remain consistent, e.g., sian-1/2/3/5 in Table2, but SIAN in the main text.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "jSfws9BiuJk", "reviewer": "Reviewer_Q2gD", "summary": "This paper proposes Sparse Interaction Additive Networks (SIANs). On an array of tabular datasets, SIANs perform comparably to neural networks and offer more interpretability than neural networks. ", "questions": "1. In Tables 2, 3, and 4, are DNN baselines here also the reference DNNs used in Feature Interaction Selection (FIS)?\n\n2. Would it be possible to have a discussion / empirical comparison between the interpretability of SIANs and DNNs undergone posthoc feature-attribution? As a concrete example, the authors can train a standard DNN on the California Housing dataset and visualize key features crucial to its performance. Are these features meaningful as well? And do they give rise to the same interpretaion as by SIANs in Figure 3?  ", "limitations": "In my opinion, the interpretability of SIANs comes at the cost of being only able to deal with tabular datasets. In fact, even for tabular datasets, it is unclear to me whether the interpretability of SIANs is indeed greater than equipping conventional DNNs with feature attribution methods -- I encourage the authors to provide such a comparison. ", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strength**. The paper is very well written. The goal of the paper -- scaling up interpretable additive models while making their training to be tractable is very well motivated.   \n\n**Weakness**. In my opinion, this paper can be further improved by incorporating a discussion / empirical comparison between the interpretability of DNNs and SIANs. To my understanding, the main advantage of SIANs is not their data-fitting ability (DNNs are better in these perspectives) but their interpretability. However, there has been a thrust of research in the post-hoc interpretation of neural networks such as saliency maps. These DNN-based saliency maps (or other attributed features) give rise to meaningful interpretation while being very flexible: They can be applied to DNNs in a model-agnostic way. The applicability of SIANs, however, is currently confined to tabular datasets and MLPs. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "i4AOlX6HnQA", "reviewer": "Reviewer_wCbJ", "summary": "The paper discusses how to generalize neural additive models to higher-order interactions and how to do them efficiently. This is done in three steps: 1) a general DNN is trained; 2) a feature interaction algorithm based on higher-order derivatives with a large norm is used to choose which features to focus on for learning neural additive models. These are then applied to an additive network to learn the final features. Experimental results demonstrate reasonable performance on regression and classification datasets", "questions": "Q1: Table 2 is a bit confusing. Both regression and classification metrics and datasets are listed, and for one higher is better, for the other lower is better. It would be clearer to clearly indicate which is which, as the reader might have a hard time switching context between Section 4 and it. Also, it is not all datasets since MIMIC III is in Table 3.\n\nQ2: It might be nice to show the visualization in the discussion via some \"automated\" process, as it would illustrate that these type of data analysis/visualization can be done automatically (what are the shared components in these post-processing steps that can be written into programs easily?). The results in the discussion section are nice, but it is also possible to produce these with some \"manual\" exploratory data analysis methods. ", "limitations": "Yes", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- A sound solution for addressing the efficiency issues in selecting higher-order interactions for generalized additive models. \n- Extensive experiments on various classification and regression datasets. \n- The explanation of MIMIC III is particularly interesting. \n- Block sparse implementations massive improves training, and compression reduces memory cost.\n\nWeaknesses:\n- Relatively weak direct technical novelty as it mostly combines Archipelago and generalized additive models. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept