D ETECTING S TATISTICAL I NTERACTIONS F ROM N EURAL N ETWORK W EIGHTS
Anonymous authors
Paper under double-blind review
A BSTRACT
We develop a method of detecting statistical interactions in data by directly interpreting the trained weights of a feedforward multilayer neural network.
By structuring the neural network to statistical properties of data and applying sparsity regularization, we are able to leverage the weights to detect interactions with similar performance to the state-of-the-art without searching an exponential solution space of possible interactions.
We obtain our computational savings by ﬁrst observing that interactions between input features are created by the non-additive effect of nonlinear activation functions and that interacting paths are encoded in weight matrices.
We use these observations to develop a way of identifying higher-order interactions with a simple traversal over the input weight matrix.
In experiments on simulated and real-world data, we demonstrate the performance of our method and the importance of discovered interactions
I NTRODUCTION
Despite their predictive capability, neural networks have traditionally been difﬁcult to interpret, preventing their adoption in many application domains.
Healthcare and ﬁnance are examples of such domains, where understanding a machine learning model is paramount when using it to make critical decisions (Caruana et al., 2015; Goodman & Flaxman, 2016).
This is because models can learn unintended patterns from data, and the risks associated with depending on these models can be consequential for stakeholders (Varshney & Alemzadeh, 2016).
Existing approaches to interpreting feedforward neural networks have focused on explanations of individual feature importance, for example by computing input gradients (Hechtlinger, 2016;
Ross et al., 2017) or by using post-hoc means (Ribeiro et al., 2016).
Owing to the importance of interpretation, we add to the existing approaches by introducing a way of ﬁnding feature groupings that neural networks model, in this case the statistical interactions.
Statistical interactions carry great importance in natural phenomena, where features often have joint effects with other features on predicting an outcome.
This is different than correlation because correlations do not involve outcome variables.
The discovery of interactions can be very useful for science, where for example, physicists may want to better understand what joint factors provide evidence for new elementary particles.
Moreover, interpreting interactions can also be useful for validating machine learning models.
For example, doctors may want to know what interactions are accounted for in risk prediction models, to compare against known interactions from scientiﬁc literature.
In this work, we developed a simple and efﬁcient algorithm that proposes statistical interactions of variable order in data, by accounting for all weights of a feedforward network that is fully-connected across input features.
Our approach is efﬁcient because it avoids searching over an exponential solution space of interaction candidates, which is achieved by making an approximation of hidden unit importance at the ﬁrst hidden layer via all weights above and doing a 2D traversal of the input weight matrix.
We provide theoretical justiﬁcation that interactions between features must be created at hidden units and that our hidden unit importance approximation satisﬁes bounds on hidden unit gradients.
We propose our framework, Neural Interaction Detector ( NID ), which generates a ranking of interaction candidates solely by interpreting the weights of a feedforward network.
TopK true interactions are then determined by ﬁnding a cutoff on the ranking using a special form of generalized additive model, which accounts for interactions of variable order (Wood, 2006; Lou et al., 2013).
We show via proof that our NID framework satisﬁes desirable properties on the rank order of higher-order interactions.
In experiments on simulated and real-world data, we evaluate the performance of our approach, the results of which show similar interaction detection performance compared to the state-of-the-art while taking orders of magnitude less time.
Our contributions are as follows: 1) we demonstrate a new interpretation of neural network weights on the statistical interactions that feedforward networks capture, and 2) we propose an accurate and efﬁcient algorithm that detects statistical interactions of variable order in data, without searching an exponential solution space of interaction candidates.
Roadmap:
We ﬁrst review related works and deﬁne notations in Section 2.
In Section 3, we examine and quantify the interactions encoded in a neural network, which leads to our framework for interaction detection detailed in Section 4.
Finally, we study our framework empirically and demonstrate its practical utility on real-world datasets in Section 5.
R ELATED W ORK AND N OTATIONS 2. I NTERACTION D ETECTION
Statistical interaction detection has been a well-studied topic in statistics, dating back to the 1920s when two-way ANOVA was ﬁrst introduced (Fisher, 1925).
Since then, two general approaches emerged for conducting interaction detection.
One approach has been to conduct individual tests for each combination of features (Lou et al., 2013).
The other approach has been to pre-specify all interaction forms of interest, then use lasso to simultaneously select which are important (Tibshirani, 1996;
Bien et al., 2013).
Notable approaches such as ANOVA and Additive Groves (Sorokina et al., 2008) belong to the ﬁrst group.
Two-way ANOVA has been a standard method of performing pairwise interaction detection that involves conducting hypothesis tests for each interaction candidate by checking each hypothesis with F-statistics (Wonnacott & Wonnacott, 1972).
Besides two-way ANOVA, there is also threeway ANOVA that performs the same analyses but with interactions between three variables instead of two; however, four-way ANOVA and beyond are rarely done because of how computationally expensive such tests become.
Speciﬁcally, the number of interactions to test grows exponentially with interaction order.
Additive Groves is another method that conducts individual tests for interactions and hence must face the same computational difﬁculties; however, it is special because the interactions it detects are not constrained to any functional form e.g. multiplicative interactions.
The unconstrained manner by which interactions are detected is advantageous when the interactions are present in highly nonlinear data (Sorokina et al., 2007; 2008).
Additive Groves accomplishes this by comparing two regression trees, one that ﬁts all interactions, and the other that has the interaction of interest forcibly removed.
In interaction detection, lasso-based methods are popular in large part due to how quick they are at selecting interactions.
One can construct an additive model with many different interaction terms and let lasso shrink the coefﬁcients of unimportant terms to zero (Tibshirani, 1996).
While lasso methods are fast, they require specifying all interaction terms of interest.
For pairwise interaction detection, this requires O ( p 2 ) terms (where p is the number of features), and O (2 p ) terms for higherorder interaction detection.
Still, the form of interactions that lasso-based methods capture is limited by which are pre-speciﬁed.
Our approach to interaction detection is unlike others in that it is both fast and capable of detecting interactions of variable order without limiting their functional forms.
The approach is fast because it does not conduct individual tests for each interaction to accomplish higher-order interaction detection.
This property has the added beneﬁt of avoiding a high false positive-, or false discovery rate , that commonly arises from multiple testing (Benjamini & Hochberg, 1995).
2. I NTERPRETABILITY
The interpretability of neural networks has largely been a mystery since their inception; however, many approaches have been developed in recent years to interpret neural networks in their traditional feedforward form and as deep architectures.
Feedforward neural networks have undergone multiple advances in recent years, with theoretical works justifying the beneﬁts of neural network depth (Telgarsky, 2016; Liang & Srikant, 2016) and new research on interpreting feature importance from input gradients (Hechtlinger, 2016; Ross et al., 2017).
Deep architectures have seen some of the greatest breakthroughs, with the widespread use of attention mechanisms in both convolutional and recurrent architectures to show where they focus on for their inferences (Itti et al., 1998;
Mnih et al., 2014; Xu et al., 2015).
Methods such as feature map visualization (Yosinski et al., 2015;
Zhang et al., 2017), de-convolution (Zeiler & Fergus, 2014), saliency maps (Simonyan et al., 2013), and many others have been especially important to the vision community for understanding how convolutional networks represent images. [Figure]
Unlike previous works in interpretability, our approach interprets a multilayer feedforward network based on statistical groupings of features that are modeled, speciﬁcally statistical interactions between features with respect to an outcome variable, y .
2. N OTATIONS
Vectors are represented by boldface lowercase letters, such as x , w ; matrices are represented by boldface capital letters, such as W .
The i -th entry of a vector w is denoted by w i , and element ( i, j ) of a matrix W is denoted by W i,j .
The i -th row and j -th column of W are denoted by W i, : and W : ,j , respectively.
For a vector w ∈ R n , let diag( w ) be a diagonal matrix of size n × n , where { diag( w ) } i,i = w i .
For a matrix W , let | W | be a matrix of the same size where | W |
i,j = | W i,j | .
Let [ p ] denote the set of integers from 1 to p .
An interaction , I , is a subset of all input features [ p ] with |I| ≥ 2 .
For a vector w ∈ R p and I ⊆
[ p ] , let
w I ∈
R |I| be the vector restricted to the dimensions speciﬁed by I .
Feedforward Neural Network 1 Consider a feedforward neural network with L hidden layers.
Let p (cid:96) be the number of hidden units in the (cid:96) -th layer.
We treat the input features as the 0 -th layer and p
= p is the number of input features.
There are L weight matrices W ( (cid:96) )
∈
R p (cid:96) × p (cid:96) − 1 , (cid:96) = 1 , 2 , . . .
, L , and L + 1 bias vectors b ( (cid:96) )
∈
R p (cid:96) , (cid:96) = 0 , 1 , . . .
, L .
Let φ ( · ) be the activation function (non-linearity), and let w y ∈ R p L and b y ∈ R be the coefﬁcients and bias for the ﬁnal output.
Then, the hidden units h ( (cid:96) ) of the neural network and the output y with input x ∈ R p can be expressed as:
[Equation]
We can construct a directed acyclic graph G =
( V, E ) based on non-zero weights, where we create vertices for input features and hidden units in the neural network and edges based on the non-zero entries in the weight matrices.
See Appendix A for a formal deﬁnition.
F EATURE I NTERACTIONS IN N EURAL N ETWORKS
A statistical interaction describes a situation in which the joint inﬂuence of multiple variables on an output variable is not additive (Dodge, 2006; Sorokina et al., 2008).
Let x i , i ∈
[ p ] be the features and y be the response variable, a statistical interaction
I ⊆
[ p ] exists if and only if E [ y | x ] , which is a function of x = ( x 1 , x 2 , . . .
, x p ) , contains a non-additive interaction between variables x I :
Deﬁnition 1 (Non-additive Interaction) .
Consider a function f ( · ) with input variables x i , i ∈ [ p ] , and an interaction I ⊆
[ p ] .
Then I is a non-additive interaction of function f ( · ) if and only if there does not exist a set of functions
f i ( · ) , ∀ i ∈ I where f i ( · ) is not a function of x i , such that f
[Equation]
x [Equation] = [Equation]
For example, in x 1 x 2 +sin ( x 2 + x 3 + x 4 ) , there is a pairwise interaction { 1 , 2 } and a higher-order 3-way interaction { 2 , 3 , 4 } , where higher-order denotes |I| ≥ 3 .
Note that d -way interaction can only exist if all its corresponding ( d − 1) -interactions exist (Sorokina et al., 2008).
For example, the interaction { 1 , 2 , 3 } can only exist if interactions { 1 , 2 } , { 1 , 3 } , and { 2 , 3 } also exist.
We will often use this property in this paper.
In feedforward neural networks, statistical interactions between features, or feature interactions for brevity, are created at hidden units with non-linear activation functions, and the inﬂuences of the interactions are propagated layer-by-layer to the ﬁnal output (see Figure 1).
In this section, we propose a framework to identify and quantify interactions at a hidden unit for efﬁcient interaction detection, then the interactions are combined across hidden units in Section 4.
3. F EATURE I NTERACTIONS AT I NDIVIDUAL H IDDEN U NITS
In feedforward neural networks, any interacting features must follow strongly weighted connections to a common hidden unit before the ﬁnal output.
That is, in the corresponding directed graph, interacting features will share at least one common descendant.
The key observation is that nonoverlapping paths in the network are aggregated via weighted summation at the ﬁnal output without creating any interactions between features.
The statement is rigorized in the following proposition and a proof is provided in
Appendix A.
The reverse of this statement, that a common descendant will create an interaction among input features, holds true in most cases.
Proposition 2 (Interactions at Common Hidden Units) .
Consider a feedforward neural network with input feature x
i , i ∈
[ p ] , where y
= ϕ
( x 1 , . . .
, x p ) .
For any interaction I ⊂ [ p ] in ϕ ( · ) , there exists a vertex v I in the associated directed graph such that I is a subset of the ancestors of v I at the input layer (i.e., (cid:96) = 0 ).
In general, the weights in a neural network are nonzero, in which case Proposition 2 blindly infers that all features are interacting.
For example, in a neural network with just a single hidden layer, any hidden unit in the network can imply up to 2 (cid:107) W j, : (cid:107)
potential interactions, where (cid:107) W j, : (cid:107) 0 is the number of nonzero values in the weight vector W j, : for the j -th hidden unit.
Managing the large solution space of interactions based on nonzero weights requires us to characterize the relative importance of interactions, so we must mathematically deﬁne the concept of interaction strength.
In addition, we limit the search complexity of our task by only quantifying interactions created at the ﬁrst hidden layer, which is important for fast interaction detection and sufﬁcient for high detection performance based on empirical evaluation (see evaluation in Section 5.2 and Table 2).
Consider a hidden unit in the ﬁrst layer: φ (cid:0) w (cid:62) x
+ b (cid:1) , where w is the associated weight vector and x is the input vector.
While having the weight w i of each feature i , the correct way of summarizing feature weights for deﬁning interaction strength is not trivial.
For an interaction I ⊆
[ p ] , we propose to use an average of the relevant feature weights w I as the surrogate for the interaction strength: µ ( | w I | ) , where µ ( · ) is the averaging function for an interaction that represents the interaction strength due to feature weights.
We provide guidance on how µ should be deﬁned by ﬁrst considering representative averaging functions from the generalized mean family: maximum value, root mean square, arithmetic mean, geometric mean, harmonic mean, and minimum value (Bullen et al., 1988).
These options can be narrowed down by accounting for intuitive properties of interaction strength : 1) interaction strength is evaluated as zero whenever an interaction does not exist (one of the features has zero weight); 2) interaction strength does not decrease with any increase in magnitude of feature weights; 3) interaction strength is less sensitive to changes in large feature weights.
While the ﬁrst two properties place natural constraints on interaction strength behavior, the third property is subtle in its intuition.
Consider the scaling between the magnitudes of multiple feature weights, where one weight has much higher magnitude than the others.
In the worst case, there is one large weight in magnitude while the rest are near zero.
If the large weight grows in magnitude, then interaction strength may not change signiﬁcantly, but if instead the smaller weights grow at the same rate, then interaction strength should strictly increase.
As a result, maximum value, root mean square, and arithmetic mean should be ruled out because they do not satisfy either property 1 or 3.
3. M EASURING THE I NFLUENCE OF H IDDEN U NITS
Our deﬁnition of interaction strength at individual hidden units is not complete without considering their outgoing paths, because an outgoing path of zero weight cannot contribute an interaction to the ﬁnal output.
To propose a way of quantifying the inﬂuence of an outgoing path on the ﬁnal output, we draw inspiration from Garson’s algorithm
(Garson, 1991; Goh, 1995), which instead of computing the inﬂuence of a hidden unit, computes the inﬂuence of features on the output.
This inﬂuence is calculated via cumulative matrix multiplications of the absolute value of neural network weight matrices.
In the following, we propose our deﬁnition of hidden unit inﬂuence, then prove that this deﬁnition upper bounds the gradient magnitude of the hidden unit.
To represent the inﬂuence of a hidden unit i at the (cid:96) -th hidden layer, we deﬁne the aggregated weight z
( (cid:96) )
i , where z ( (cid:96) )
∈
R p (cid:96) and
[Equation]
We show that this deﬁnition upper bounds the gradient magnitudes of hidden units by proving that it computes Lipschitz constants for the corresponding units.
Gradients have been commonly used as variable importance measures in neural networks, especially input gradients which compute directions normal to decision boundaries (Ross et al., 2017; Goodfellow et al., 2014;
Simonyan et al., 2013).
Thus, an upper bound on the gradient magnitude approximates how important the variable can be.
The full proof is shown in Appendix C.
Lemma 3 (Neural Network Lipschitz Estimation) .
Let the activation function φ ( · ) be a 1 -Lipschitz function.
Then the output y is z ( (cid:96) )
i -Lipschitz with respect to h ( (cid:96) )
i .
3. Q UANTIFYING I NTERACTION S TRENGTH
We now combine our deﬁnitions from Sections 3.1 and 3.2 to obtain the interaction strength ω
i ( I ) of a potential interaction I at the i -th unit in the ﬁrst hidden layer h (1) i ,
[Equation]
Note that ω
i ( I ) is deﬁned on a single hidden unit, and it is agnostic to scaling ambiguity within a
ReLU based neural network.
In Section 4, we discuss our scheme of aggregating strengths across hidden units, so we can compare interactions of different orders.
I NTERACTION D ETECTION
In this section, we propose our feature interaction detection algorithm NID , which can extract interactions of all orders without individually testing for each of them.
Our methodology for interaction detection is comprised of three main steps: 1) structure the feedforward network to model univariate effects explicitly, 2) interpret trained weights to obtain a ranked shortlist of interaction candidates, and 3) determine a cutoff for the top K interactions
. 4. A RCHITECTURE
Data often contains both statistical interactions and main effects (Winer et al., 1971).
Main effects describe the univariate inﬂuences of variables on an outcome variable.
In fully-connected networks, main effects must pass through hidden units, which may errantly create statistical interactions between what are exclusively main effect variables and other variables.
We minimize the opportunity for such spurious interactions being modeled by ﬁrst structuring a neural network to account for main effects, that is, we separately model main effects using networks with univariate inputs for each input variable, and we sum up these networks and the standard fully-connected network together at their respective outputs (see Figure 2).
We refer to this additive model networks as MLP-M .
When we jointly train MLP-M via backpropagation, we apply L1 regularization on the interaction network and not on the main effect networks to push the modeling of main effects away from the interaction network as much as possible.
Input: input-to-ﬁrst hidden layer weights W (1) , aggregated weights z (1)
Output: ranked list of interaction candidates {I i } mi = 1: d ← initialize an empty dictionary mapping interaction candidate to interaction strength 2: for each row w (cid:48) of W (1) indexed by r do 3: for j = 2 to p do 4:
I ← sorted indices of top j weights in w (cid:48) 5: d [ I ] ← d
[ I ] + z (1) r µ ( | w (cid:48)I | ) 6: {I i } mi =1 ← interaction candidates in d sorted by their strengths in descending order
4. R ANKING I NTERACTIONS OF V ARIABLE O RDER
[Figure] We design a greedy algorithm that generates a ranking of interaction candidates by only considering, at each hidden unit, the top-ranked interactions of every order, where 2 ≤ |I| ≤ p , thereby drastically reducing the search space of potential interactions while still considering all orders.
We ﬁrst train our architecture MLP-M with L1 regularization on the interaction network to suppress unimportant weights, after which we use our greedy strategy to traverse the regularized input weight matrix W (1) (Figure 2).
The greedy algorithm (Algorithm 1) selects only top-ranked interaction candidates per hidden unit based on their interaction strengths.
By selecting topranked interactions of every order and summing their respective strengths across hidden units, we obtain ﬁnal interaction strengths, allowing arbitrary-order interaction candidates to be ranked relative to each other.
For this algorithm, we set the averaging function µ ( · ) = min ( · ) based on its performance in experimental evaluation (Section 5.1).
Since a higher-order interaction only exists if its subset interactions also exist (Section 3), the subset interactions are redundant in the presence of the higher-order interaction.
Thus, Algorithm 1 assumes that there are at least as many ﬁrst-layer hidden units as there are the true number of such higher-order interactions and non-redundant pairwise interactions.
In addition to efﬁciency, a beneﬁt of Algorithm 1’s greedy strategy is that it automatically improves the ranking of a higher-order interaction over its redundant subsets.
This allows the higher-order interaction to have a better chance of ranking above any false positives and being captured in the cutoff stage.
We justify this improvement by proving Theorem 4 under a mild assumption.
Theorem 4 (Improving the ranking of higher-order interactions) .
Let R be the set of interactions proposed by Algorithm 1, let I ∈
R be a d -way interaction where d ≥ 3 , and let S be the set of subset ( d − 1 )-way interactions of I where |S| = d .
Assume that for any hidden unit j which proposed s ∈ S ∩ R , I will also be proposed at the same hidden unit, and w j ( I ) > 1 d w j ( s ) .
Then, one of the following must be true: a) ∃ s ∈ S ∩ R ranked lower than I , i.e., w ( I ) > w ( s ) , or b) ∃ s ∈ S where s / ∈ R .
The full proof is included in Appendix D. Under the noted assumption, the theorem in part a) shows that a d -way interaction will improve over one its d − 1 subsets in rankings as long as there is no sudden drop from the weight of the ( d − 1) -way to the d -way interaction at the same hidden units.
We note that the improvement extends to b) as well, when d = |S ∩ R| > 1 . [Table]
4. C UTOFF ON I NTERACTION R ANKING
In order to predict the true top- K interactions {I i } Ki =1 , we must ﬁnd a cutoff point on our interaction ranking from Section 4.2.
We obtain this cutoff by constructing a Generalized Additive Model ( GAM ) with interactions:
[Equation]
where g i ( · ) captures the main effects, g (cid:48) i ( · ) captures the interactions, and both g i and g (cid:48)
i are small feedforward networks trained jointly via backpropagation.
We refer to this model as MLP-Cutoff .
We gradually add top-ranked interactions to the GAM , increasing K , until GAM performance on a validation set plateaus.
The exact plateau point can be found by early stopping or other heuristic means, and we report {I i } Ki =1 as the identiﬁed feature interactions.
4. P AIRWISE I NTERACTION D ETECTION
A variant to our interaction ranking algorithm tests for all pairwise interactions.
Pairwise interaction detection has been a standard problem in the interaction detection literature (Lou et al., 2013; Fan et al., 2016) due to its simplicity.
Modeling pairwise interactions is also the de facto objective of many successful machine learning algorithms such as factorization machines (Rendle, 2010) and hierarchical lasso (Bien et al., 2013).
We rank all pairs of features { i, j } according to their interaction strengths ω
( { i, j } ) calculated on the ﬁrst hidden layer, where again the averaging function is min ( · ) , and ω ( { i, j } )
= (cid:80) p 1 s =1 ω s ( { i, j } ) .
The higher the rank, the more likely the interaction exists.
E XPERIMENTS
In this section, we discuss our experiments on both simulated and real-world datasets to study the performance of our approach on interaction detection.
5. E XPERIMENTAL S ETUP
Averaging Function Our proposed NID framework relies on the selection of an averaging function (Sections 3.1, 4.2, and 4.4).
We experimentally determined the averaging function by comparing representative functions from the generalized mean family (Bullen et al., 1988): maximum value, root mean square, arithmetic mean, geometric mean, harmonic mean, and minimum value, intuitions behind which were discussed in Section 3.1.
In order to compare the averaging functions, we used a test suite of 10 synthetic functions, which consist of a variety of interactions of varying order and overlap, as shown in Table 1.
We applied our proposed greedy ranking algorithm, Algorithm 1, to each of the test functions in the test suite over 10 trials, and we counted the total number of correct interactions that are ranked before any false positives.
In our evaluation, we ignore predicted interactions that are subsets of true higher-order interactions because the subset interactions are redundant (Section 3).
As shown in Figure 3, the number of true top interactions we recover is highest with the averaging function, minimum value, which we will use in all of our experiments.
A simple analytical study on a bivariate hidden unit, provided in Appendix B, also suggests that the minimum value is closely correlated with interaction strength. [Figure]
Neural Network Conﬁguration
In our experiments, we trained feedforward neural networks that combined separate networks for capturing feature interactions and main effects for both MLP-M and MLP-Cutoff (Figure 2, Section 4.3).
The interaction and main effect networks are summed at their outputs, allowing joint training via backpropagation.
In all of our experiments, the feature interaction component of our neural network consisted of four hidden layers with ﬁrstto-last layer sizes of: 140, 100, 60, and 20 units.
The main effect components each had three hidden layers with sizes of: 10, 10 and 10 units.
All networks used ReLU activation.
The objective functions were mean-squared error for regression and cross-entropy for classiﬁcation tasks.
On the synthetic test suite, MLP-M was trained with L1 constants in the range of 5e-6 to 5e-4, based on parameter tuning on a validation set.
On real-world datasets, L1 was ﬁxed at 5e-5.
MLP-Cutoff used a ﬁxed L2 constant of 1e-4 in all experiments involving cutoff.
Early stopping was used to prevent overﬁtting.
In all experiments, we obtained interaction rankings from MLP-M , which was trained to ﬁt data.
Datasets We study our interaction detection framework on both simulated and real-world experiments.
For simulated experiments, we used a test suite of synthetic functions, as shown in Table 1.
The test functions were designed to have a mixture of pairwise and higher-order interactions, with varying order, strength, nonlinearity, and overlap.
F 1 is a commonly used function in interaction detection literature (Hooker, 2004; Sorokina et al., 2008; Lou et al., 2013).
All features were uniformly distributed between − 1 and 1 except in F 1 , where we used the same variable ranges as reported in literature (Hooker, 2004).
We use four real-world datasets, of which two are regression datasets, and the other two are binary classiﬁcation datasets.
The datasets are a mixture of common prediction tasks in the cal housing and bike sharing datasets, a scientiﬁc discovery task in the higgs boson dataset, and an example of very-high order interaction detection in the letter dataset.
Speciﬁcally, the cal housing dataset is a regression dataset with 21k data points for predicting California housing prices (Pace & Barry, 1997).
The bike sharing dataset contains 17k data points of weather and seasonal information to predict the hourly count of rental bikes in a bikeshare system (Fanaee-T & Gama, 2014).
The higgs boson dataset has 800k data points for classifying whether a particle environment originates from the decay of a Higgs Boson (Adam-Bourdarios et al., 2014).
Lastly, the letter recognition dataset contains 20k data points of transformed features for binary classiﬁcation of letters on a pixel display (Frey & Slate, 1991).
For all real-world data, we use random train/valid/test splits of 80 / 10 / 10 .
Baselines We compare the performance of NID to that of three baseline interaction detection methods.
Two-Way ANOVA (Wonnacott & Wonnacott, 1972) utilizes linear models to conduct signiﬁcance tests on the existence of interaction terms.
Hierarchical lasso (HierLasso) (Bien et al., 2013) applies lasso feature selection to extract pairwise interactions.
Additive Groves ( AG ) (Sorokina et al., 2008) is a nonparameteric means of testing for interactions by placing structural constraints on an additive model of regression trees.
AG is a reference method for interaction detection because it directly detects interactions based on their non-additive deﬁnition. [Table]
5. P AIRWISE I NTERACTION D ETECTION
As discussed in Section 4. [Table] our framework NID can be used for pairwise interaction detection.
To evaluate this approach, we used datasets generated by synthetic functions F 1 - F 10 (Table 1) that contain a mixture of pairwise and higher-order interactions, where in the case of higher-order interactions, we test for their pairwise subsets as in Sorokina et al.
; Lou et al.
AUC scores of interaction strength proposed by NID and each baseline are shown in Table 2, and NID ’s interaction strengths are visualized in Figures 4 and 5 for all synthetic and real-world datasets.
For the AUC evaluation, we ran ten trials of NID and AG on each dataset and removed two trials with the highest and lowest AUC scores.
When comparing the AUCs of NID and AG , we observe that their scores tend to close, except for F 5 , F 6 , F 8 , and F 9 , where NID performs signiﬁcantly better than AG .
This performance difference may be due to limitations on the model capacity of AG , which is tree-based.
In comparison to ANOVA and HierLasso , NID generally performs on par or better.
This is expected because ANOVA and HierLasso are based on quadratic models, which can have difﬁculty approximating the interaction nonlinearities that are present in the test suite.
There is one function, F 6 , where ANOVA and HierLasso outperform both NID and AG .
We can attempt to understand this function better through visualizations of interaction strength.
In Figure 4, heat maps of synthetic datasets show the relative strengths of all possible pairwise interactions, and ground truth is indicated by red cross-marks.
As shown, the interaction strengths are normally high at the cross-marks.
An exception is F 6 , where NID proposes weak or negligible interaction strengths at the cross-marks on the lower right region of the heat map.
ANOVA and
HierLasso are more sensitive to these interactions, perhaps because their underlying models are simpler.
Besides F 6 , F 7 also shows erroneous interaction strengths; however, comparative detection performance by the baselines is similarly poor.
Interaction strengths are also visualized on real-world datasets via heat maps (Figure 5).
For example, in the cal housing dataset, there is a high-strength interaction between x 1 and x 2 .
These variables mean longitude and latitude respectively, and it is clear to see that the outcome variable,
California housing price, should indeed strongly depend on geographical location.
We further observe high-strength interactions appearing the the heat maps of the bike sharing, higgs boson dataset, and letter datasets.
For example, all feature pairs appear to be interacting in the letter dataset.
The binary classiﬁcation task from the letter dataset is to distinguish letters A-M from N-Z using 16 pixel display features.
Since the decision boundary between A-M and N-Z is not obvious, it would make sense that a neural network learns a highly interacting function to make the distinction.
5. H IGHER -O RDER I NTERACTION D ETECTION
We use our greedy interaction ranking algorithm (Algorithm 1) to perform higher-order interaction detection without an exponential search of interaction candidates.
We ﬁrst visualize our higher-
[Figure]
generated by functions F 1 - F 10 (Table 1)
{+x
+}
{+x2 x3 x4 x5 x6 x7 x x1 x x3 x x5 x x7 x x
 x2 x3 x4 x5 x6 x7 x8 x9x10x11x12x13x14x x
 x2 x x4 x5 x x7 x8 x x
+}
{+x11 x x13 x14 x   x1 x2 x3 x4 x5 x6 x7 x8 x9x10x11x12x13x14x15x x1 x2 x3 x x5 x6 x7 x x9 x10 x11 x x13 x14 x
 x+} [Figure]
order interaction detection algorithm on synthetic and real-world datasets, then we show how the predictive capability of detected interactions closes the performance gap between MLP-Cutoff and
MLP-M .
Next, we discuss our experiments comparing NID and AG with added noise, and lastly we verify that our algorithm obtains signiﬁcant improvements in runtime.
We visualize higher-order interaction detection on synthetic and real-world datasets in Figures 6 and 7 respectively.
The plots correspond to higher-order interaction detection as the ranking cutoff is applied (Section 4.3).
The interaction rankings generated by NID are shown on the x -axes, and the blue bars correspond to the validation performance of MLP-Cutoff as interactions are added.
For example, the plot for cal housing shows that adding the ﬁrst interaction signiﬁcantly reduces RMSE.
We keep adding interactions into the model until reaching a cutoff point.
In our experiments, we use a cutoff heuristic where interactions are no longer added after MLP-Cutoff ’s validation performance reaches or surpasses MLP-M ’s validation performance (represented by horizontal dotted lines).
As seen with the red cross-marks, our method ﬁnds true interactions in the synthetic data of F 1 - Fbefore the cutoff point.
Challenges with detecting interactions are again mainly associated with Fand F 7 , which have also been difﬁcult for baselines in the pairwise detection setting (Table 2).
For the cal housing dataset, we obtain the top interaction { 1 , 2 } just like in our pairwise test (Figure 5, cal housing), where now the { 1 , 2 } interaction contributes a signiﬁcant improvement in MLP-Cutoff performance.
Similarly, from the letter dataset we obtain a 16-way interaction, which is consistent with its highly interacting pairwise heat map (Figure 5, letter).
For the bike sharing and higgs boson datasets, we note that even when considering many interactions, MLP-Cutoff eventually reaches the cutoff point with a relatively small number of superset interactions. This is because many subset interactions become redundant when their corresponding supersets are found.
[Table]
F
F F F
FØ x3x4x5x6x8x2x6x3x4x5x1x3x2x6x10x1x3x10x2x6x9x10x1x3x5x7x10x3x4x7x2x4x6x9x10x3x4x5x7x3x4x8x3x4x5x10x2x5x6x8x
[Figure]
[Figure] In our evaluation of interaction detection on real-world data, we study detected interactions via their predictive performance.
By comparing the test performance of MLP-Cutoff and MLP-M with respect to MLP-Cutoff without any interactions ( MLP-Cutoff Ø ), we can compute the relative test performance improvement obtained by including detected interactions.
These relative performance improvements are shown in Table 3 for the real-world datasets as well as four selected synthetic datasets, where performance is averaged over ten trials per dataset.
The results of this study show that a relatively small number of interactions of variable order are highly predictive of their corresponding datasets, as true interactions should.
We further study higher-order interaction detection of our NID framework by comparing it to AG in both interaction ranking quality and runtime.
To assess ranking quality, we design a metric, toprank recall , which computes a recall of proposed interaction rankings by only considering those interactions that are correctly ranked before any false positive.
The number of top correctly-ranked interactions are then divided by the true number of interactions.
Because subset interactions are redundant in the presence of corresponding superset interactions, only such superset interactions can count as true interactions, and our metric ignores any subset interactions in the ranking.
We compute the top-rank recall of NID , averaged across all tests in the test suite of synthetic functions (Table 1) with 10 trials per test function.
For each test, we remove two trials with max and min recall.
We conduct the same tests using the state-of-the-art interaction detection method AG , except with only one trial per test because AG is very computationally expensive to run.
In Figure 8a, we show top-rank recall of NID and AG at different Gaussian noise levels 4 , and in Figure 8b, we show runtime comparisons on real-world and synthetic datasets.
As shown, NID can obtain similar top-rank recall as AG while running orders of magnitude times faster. [Table] [Figure] [Table]
[Figure]
[Table]
[Figure] [Table] [Figure] L IMITATIONS
In higher-order interaction detection, our NID framework can have difﬁculty detecting interactions from certain forms of functions.
Speciﬁcally, functions in which interacting variables have interlinked structure can cause problems for our framework to separate such structure.
For example, a clique x 1 x 2 + x 1 x 3 + x 2 x 3 or a ring x 1 x 2 + x 2 x 3 + x 3 x 4 + x 4 x 5 + x 5 x 1 only contains pairwise interactions.
When detecting pairwise interactions (Section 5.2), NID often obtains an AUC of 1.
However, in higher-order interaction detection, the interlinked pairwise interactions are often confused for single higher-order interactions.
This issue could mean that our higher-order interaction detection algorithm fails to separate interlinked pairwise interactions encoded in a neural network, or the network approximates interlinked low-order interactions as higher-order interactions.
In experiments on real-world datasets, our framework sometimes detected spurious interactions or missed interactions as a result of correlations between features; however, correlations are known to cause these problems for any interaction detection method (Sorokina et al., 2008; Lou et al., 2013).
C ONCLUSION
We presented our NID framework, which detects statistical interactions in data without searching an exponential solution space of interactions candidates.
The framework detects interactions by interpreting the weights of a feedforward neural network, which is trained to ﬁt data.
Our core insight is that interactions between features must be modeled at common hidden units, and we directly decode the weights according to this insight.
In future work, we plan to detect feature interactions by accounting for common units in intermediate hidden layers of feedforward networks.
We would also like to use the perspective of interaction detection to interpret weights in other neural architectures, including deep networks.
A P ROOF AND D ISCUSSION FOR P ROPOSITION
Given a trained feedforward neural network as deﬁned in Section 2.3, we can construct a directed acyclic graph G =
( V, E ) based on non-zero weights as follows.
We create a vertex for each input features and hidden units in the neural network: V = { v (cid:96),i |∀ i, (cid:96) } , where v (cid:96),i be the vertex corresponding to the i -th hidden unit in the (cid:96) -th layer.
Note that the ﬁnal output y is not included.
We create edges based on the non-zero entries in the weight matrices, i.e., E = { ( v (cid:96) − 1 ,i , v (cid:96),j ) | W
(cid:96)j,i (cid:54) = 0 , ∀ i, j, (cid:96) } .
Note that under the graph representation, the value of any hidden unit is a function of parent hidden units.
We will also use vertices and hidden units interchangeably.
Proposition 2 (Interactions at Common Hidden Units) .
Consider a feedforward neural network with input feature x
i , i ∈
[ p ] , where y
= ϕ
( x 1 , . . .
, x p ) .
For any interaction I ⊂ [ p ] in ϕ ( · ) , there exists a vertex v I in the associated directed graph such that I is a subset of the ancestors of v I at the input layer (i.e., (cid:96) = 0 ).
Proof.
We prove Proposition 2 by contradiction.
Let I be an interaction where there is no vertex in the associated graph which satisﬁes the condition.
Then, for any vertex v L,i at the L -th layer, the value f i of the corresponding hidden unit is a function of its ancestors at the input layer
I i where I (cid:54)⊂ I i .
Next, we group the hidden units at the L -th layer into non-overlapping subsets by the ﬁrst missing feature with respect to the interaction I .
That is, for element i in I , we create a index set S i ∈
[ p L ] :
[Equation]
[Equation]
∈
[Equation]
] |
[Equation]
I j and
[Equation]
<
[Equation]
[Equation]
(cid:54)∈
[Equation]
Note that the ﬁnal output of the network is a weighed summation over the hidden units at the L -th layer:
[Equation]
Since that (cid:80) j ∈S i w yj f j (cid:0) x
I j (cid:1) is not a function of x i , we have that ϕ ( · ) is a function without the interaction I , which contradicts with our assumption.
The reverse of this statement, that a common descendant will create an interaction among input features, holds true in most cases.
The existence of counterexamples is manifested when early hidden layers capture an interaction that is negated in later layers.
For example, the effects of two interactions may be directly removed in the next layer, as in the case of the following expression: max { w 1 x 1 + w 2 x 2 , 0 } − max {− w 1 x 1 − w 2 x 2 , 0 } =
w 1 x 1 + w 2 x 2 .
Such an counterexample is legitimate; however, due to random ﬂuctuations, it is highly unlikely in practice that the w 1 s and the w 2 s from the left hand side are exactly equal.
B P AIRWISE I NTERACTION S TRENGTH VIA Q UADRATIC A PPROXIMATION
We can provide a ﬁner interaction strength analysis on a bivariate ReLU function: max { α 1 x 1 + α 2 x 2 , 0 } , where x 1 , x 2 are two variables and α 1 , α 2 are the weights for this simple network.
We quantify the strength of the interaction between x 1 and x 2 with the cross-term coefﬁcient of the best quadratic approximation.
That is,
[Equation]
Then for the coefﬁcient of interaction { x 1 , x 2 } , β 5 , we have that,
[Equation]
Note that the choice of the region ( − 1 , 1) × ( − 1 , 1) is arbitrary: for larger region ( − c, c ) × ( − c, c ) with c > 1 , we found that | β 5 | scales with c − 1 .
By the results of Proposition B, the strength of the interaction can be well-modeled by the minimum value between | α 1 | and | α 2 | .
Note that the factor before min {| α 1 | , | α 2 |} in Equation (2) is almost a constant with less than 20% ﬂuctuation.
C P ROOF FOR L EMMA
Lemma 3 (Neural Network Lipschitz Estimation) .
Let the activation function φ ( · ) be a 1 -Lipschitz function.
Then the output y is z ( (cid:96) )
i -Lipschitz with respect to h ( (cid:96) )
i .
Proof.
For non-differentiable φ ( · ) such as the ReLU function, we can replace it with a series of differentiable 1 -Lipschitz functions that converges to φ ( · ) in the limit.
Therefore, without loss of generality, we assume that φ ( · ) is differentiable with | ∂ x φ ( x ) | ≤ 1 .
We can take the partial derivative of the ﬁnal output with respect to h ( (cid:96) )
i , the i -th unit at the (cid:96) -th hidden layer:
[Equation]
where ˙ φ ( (cid:96) )
∈
R p (cid:96) is a vector that
[Equation]
We can conclude the Lemma by proving the following inequality:
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
The left-hand side can be re-written as
[Table]
[Equation]
[Table]
[Equation]
[Table]
[Equation]
The right-hand side can be re-written as
[Equation]
We can conclude by noting that | ∂ x φ ( x ) | ≤ 1 .
D P ROOF FOR T HEOREM
Theorem 4 (Improving the ranking of higher-order interactions) .
Let R be the set of interactions proposed by Algorithm 1, let I ∈
R be a d -way interaction where d ≥ 3 , and let S be the set of subset ( d − 1 )-way interactions of I where |S| = d .
Assume that for any hidden unit j which proposed s ∈ S ∩ R , I will also be proposed at the same hidden unit, and w j ( I ) > 1 d w j ( s ) .
Then, one of the following must be true: a) ∃ s ∈ S ∩ R ranked lower than I , i.e., w ( I ) > w ( s ) , or b) ∃ s ∈ S where s / ∈ R .
Proof.
Suppose for the purpose of contradiction that S ⊆ R and ∀ s ∈ S , w ( s ) ≥ w ( I ) .
Because w j ( I ) > 1 d w j ( s ) ,
[Equation]
Since ∀ s ∈ S , w ( s ) ≥ w ( I ) ,
[Equation]
Since S ⊆ R ,
|S ∩ R| = d .
Therefore,
[Equation]
which is a contradiction.