Fast Generic Interaction Detection for Model Interpretability and Compression

1 INTRODUCTION . Explainable machine learning is an active research field that focuses on providing interpretable models , transparent explanations , and confident decisions to practical AI systems . Investigating feature interaction is vital to model interpretability . Interaction detection should be able to reveal which subset of features influence the output jointly , and what the corresponding nonlinear transformation is . We aim to find the underlying interactions from data , such that we can interpret the model properly . To go one step further , we hope that new models can be built with the aid of the detected interaction knowledge economically to avoid heavy parameterization . In this paper , we first restrict ourselves to interaction detection . A novel method is derived directly from the most acknowledged definition of feature interaction ( Friedman et al. , 2008 ) . We further apply the obtained interaction knowledge to design a transparent and refined neural network ( NN ) . This approach fits well into the data science life cycle ( Yu & Kumbier , 2019 ) , which was applied , for instance in Tsang et al . ( 2020a ) , successfully for interpretable recommender system design . The main contributions of this paper include : 1 ) a fast and principled interaction detection method , 2 ) a lightweight and interpretable neural network model that can surpass its Teacher , 3 ) thorough theoretical analysis and performance evaluations with real datasets . More details are given below . 1 . We propose a generic interaction detection method based on a global statistical metric , namely the expected Hessian , Hij : = Ex [ ∂2F ( x ) ∂xi∂xj ] . Notably , our method is model-agnostic and applicable to any pre-trained learning model , F ( x ) , being for instance , a deep neural network model or a tree model . It is also flexible to use for multi-way interaction detection . 2 . To speed up the detection process , we evaluate the expected Hessian via adaptive sampling using the Upper Confidence Bound ( UCB ) algorithm ( Lai & Robbins , 1985 ) , which can significantly reduce the computational complexity . Besides , thorough analysis of the proposed interaction detection method are conducted . 3 . Using the detected interaction pairs , we further design a compressed but interpretable Student model which can surpass its Teacher by 26 % in terms of data fitting performance averaged over various datasets . The compressed model reduces its size over 100 times compared to the baseline fully-connected , over-parameterized neural network ( OverparaFC ) . 4 . We demonstrate the linkages between our compressed model and the classic alternating conditional expectation ( ACE ) model ( Breiman & Friedman , 1985 ) . 5 . We conduct large-scale performance evaluations and further explain the obtained model interpretability with some real datasets . The remainder of this paper is organized as follows . Section 2 introduces all related works . Our proposed interaction detection method is introduced in Section 3 . By exploiting the detection outcome , a new variant of lightweight and interpretable deep learning model is introduced in Section 4 . Experimental results are given in Section 5 . Finally , we conclude the paper in Section 6 . 2 RELATED WORKS . Interaction Detection : Early works adopt pure statistics for detecting feature interactions , and representatives include ANOVA and GUIDE ( Wonnacott & Wonnacott , 1990 ; Fisher , 1992 ; Loh , 2002 ) . These works have motivated a plethora of new methods with the aim to enhance the detection accuracy and/or efficiency . The first class of methods centered around the GA2M and tree models , see for instance Lou et al . ( 2013 ) ; Sorokina et al . ( 2008 ) ; Friedman et al . ( 2008 ) ; Lundberg et al . ( 2020 ) . The second class of methods were built on the so-called factorization machines ( Rendle , 2010 ) as well as its new variants ( Xiao et al. , 2017 ; Song et al. , 2019 ) with attention mechanism . The third class exploits the most recent advances in deep learning , including the Neural Interaction Detection ( NID ) and some new variants ( Tsang et al. , 2018a ; b ; Cui et al. , 2019 ) : Persistence Interaction Detection ( PID ) ( Liu et al. , 2020 ) , Integrated Hessians ( IH ) ( Janizek et al. , 2020 ) , Shapley interaction ( Zhang et al. , 2020 ; Sundararajan et al. , 2020 ) , etc . Although we have witnessed well improved interaction detection performance for many datasets over the decades , the above methods still lead to inconsistent detection results for some other datasets . The reasons are twofold . Firstly , the interaction strength is empirically defined , for instance , NID method computes the interaction strength via summarizing the neural network weights . Secondly , a specific deep learning model is required , for instance , an ` 1-regularized ReLU network is required by the latest NID and PID methods to maintain high accuracy . In contrast , our proposed method is derived directly from the definition of feature interaction ( Friedman et al. , 2008 ) and moreover is not confined to any specific learning model . Model Interpretability : There are two categories of approaches to address model interpretability , namely the transparency-based and post-hoc approaches ( Došilović et al. , 2018 ) . Transparency-based approaches require the model itself to be simple and interpretable , like linear models , decision trees , etc . One can directly read off the interpretations from their coefficients or decision rules . But often , they are less accurate due to limited representation power . In contrast , post-hoc approaches extract useful information from a pre-trained model , which is often complex and hard to interpret . Well-known methods such as LIME ( Ribeiro et al. , 2016 ) and SHAP ( Lundberg & Lee , 2017 ) fall in this category , but they did not take feature interaction into account . Recently proposed symbolic metamodel ( Alaa & van der Schaar , 2019 ) captures nonlinear interactions by approximating the black-box model with explicit symbolic expressions . There are also some other post-hoc approaches based on sensitivity analysis ( Cortez & Embrechts , 2013 ) , which return a quantification of feature importance νi = E [ ( ∂F ( x ) ∂xi ) 2 ] ( Kucherenko et al. , 2009 ) and interactions νij = E [ |∂ 2F ( x ) ∂xi∂xj |2 ] ( Roustant et al. , 2014 ) by input perturbation . Our work aims to combine the strengths of the two categories . Concretely , we first extract the interaction knowledge by a post-hoc method , and then build a transparent and interpretable learning model as illustrated in Figure 3 . Model Compression and Knowledge Distillation ( KD ) : Model compression ( Bucilua et al. , 2006 ) aims to learn a small compressed model ( Student ) from a large complex model ( Teacher ) with augmented training data produced by the Teacher . Compared to the Teacher , the Student can make similar or even better predictions . Knowledge distillation ( Hinton et al. , 2015 ) mainly deals with multi-class classification problems and extracts “ valuable information that defines a rich similarity structure over the data ” . Our work introduces a novel viewpoint of knowledge ( the interacted relationships ) and targets a lightweight but more accurate Student model . 3 PROPOSED INTERACTION DETECTION METHOD . Before diving into in-depth discussions of interaction detection , we need to formally define what feature interaction is . The textbook definition according to Friedman et al . ( 2008 ) is given below . Definition 3.1 ( Friedman & Popescu 2008 ) . A function F : Rp → R is said to exhibit an interaction between two of its variables xi and xj if the difference in the value of F ( x ) as a result of changing the value of xi depends on the value of xj . Equivalently , if Ex [ |∂ 2F ( x ) ∂xi∂xj |2 ] > 0 , namely the partial derivative w.r.t xj turns out to be dependent on xi , then we say xi and xj are interacted . Otherwise , xi and xj have no interaction , if F ( x ) can be expressed as the sum of two functions f\i and f\j ( Sorokina et al. , 2008 ) , namely , F ( x ) = f\i ( x1 , . . . , xi−1 , xi+1 , . . . , xp ) + f\j ( x1 , . . . , xj−1 , xj+1 , . . . , xp ) , where f\i ( j ) is irrespective to xi ( j ) . Similarly , higher-order ( multi-way ) interaction can be defined . In this paper , we mainly focus on pairwise interaction , while multi-way interaction is only briefly discussed due to space limitations . 3.1 INTERACTION STRENGTH MEASURE . Motivated by the above definition of interaction , it is natural to take advantage of the Hessian matrix H : = ∇2xxF ( x ) . The magnitude of its entry | ∂2F ( x ) ∂xi∂xj | contains rich information about the local curvature at a data point x . Note that F ( x ) is a regression function here , not a loss function . This idea was originally considered in the economics community ( Ai & Norton , 2003 ) and rediscovered for sensitivity analysis in Roustant et al . ( 2014 ) . The goodness of F ( x ) as an approximator can essentially influence the interaction detection performance , see our Theorem M.2 in the supplement . We define g ( x , i , j ) : = |∂ 2F ( x ) ∂xi∂xj |2 to measure the local interaction strength of the i-th and j-th features at point x . We then use f ( i , j ) : = Ex [ g ( x , i , j ) ] = Ex [ |∂ 2F ( x ) ∂xi∂xj |2 ] as a measure of the global interaction strength . If f ( i , j ) ≈ 0 , we say feature xi and xj have weak interaction ; otherwise , if f ( i , j ) is significantly larger than zero , then xi and xj have strong interaction . In this paper , we focus on the global interaction . 3.2 INTERACTION STRENGTH EVALUATION The above defined global interaction strength f ( i , j ) = Ex [ |∂ 2F ( x ) ∂xi∂xj |2 ] is mostly unavailable due to the unknown input distribution x ∼ P ( x ) . The Monte Carlo method can be used to approximate it by computing the sample mean over the training data ( Roustant et al. , 2014 ) . Analytical Evaluation : The Hessian matrix∇2xxF ( x ) for neural networks at a certain data point can be calculated analytically and efficiently , by using the automatic differentiation ( Paszke et al. , 2017 ) . However , using this analytical solution is problematic for some learning models , such as the ReLU network , Random Forest ( RF ) , etc. , see supplement A . For example , the landscape of a ReLU network is piece-wise linear as shown in Figure 1 , thus the exact Hessian is a zero matrix at almost every point . So , we turn to the following numerical evaluation for broader horizons . Numerical Evaluation : Finite difference method is a common way to approximate the Hessian on a given data point ( Campolongo & Braddock , 1999 ) , i.e. , ∂2F ( x ) ∂xi∂xj ≈ 1 4hihj [ F ( x + eihi + ejhj ) − F ( x + eihi − ejhj ) − F ( x + ejhj − eihi ) + F ( x− eihi − ejhj ) ] , ( 1 ) where ei is a one-hot vector with the i-th element being equal to one and the rest of elements being zeros . We also note that if the computation of gradient ∂F∂x is cheap ( e.g. , F is a neural network ) , then Hij can be approximated as 12h [ ∂F ( x+ejh ) ∂xi − ∂F ( x−ejh ) ∂xi ] to reduce computation , which is similar to the feature interaction score defined in Greenside et al . ( 2018 ) . The choice of perturbation size hi or hj ( abbr . hi ( j ) ) is critical . Generally , we do not want hi ( j ) to be too small ( incurring round-off error ) or too large ( incurring truncation error ) to get a good overall approximation of the derivative ( Jerrell , 1997 ; Baydin et al. , 2018 ) . For our problem , we particularly do not want hi ( j ) to be too small so that the four evaluated points ( shown in the numerator of Equation 1 ) lie on the same hyperplane ( e.g. , region A in Figure 1 ) , which makes the quantity in Equation 1 always zero . The following theorem reveals that choosing a sufficiently small hi ( j ) is not necessary . Theorem 3.1 . For any x and y , function F shows no interaction between them , i.e. , it can be decomposed as F ( x , y ) = a ( x ) + b ( y ) if and only if , for any h , k > 0 , F ( x + h , y + k ) − F ( x + h , y − k ) − F ( x− h , y + k ) + F ( x− h , y − k ) = 0 . The magnitude of the numerator in Equation 1 tells us whether F is locally separable for variables xi and xj at point x , see our proof in supplement B . The main issue of the finite difference method lies in the computational complexity for approximating the global interaction strength f ( i , j ) , especially when the function evaluation itself is expensive . Using all the training samples for finding just a few strongest interaction pairs can be a total waste of computation resources . In general , for a dataset with N samples in p-dimensional feature spaces , a total number of 4Np ( p− 1 ) /2 arithmetic evaluations of the surrogate regression function F are needed . If the Hessian happens to be sparse , that is , there are only a few interactions existing in the ground truth function , massive evaluations on those feature pairs with zero interaction strength should be avoided . 3.3 IDENTIFICATION OF THE k-STRONGEST PAIRWISE INTERACTIONS In practice , we do not have to obtain the interaction strengths for all interaction pairs , because many of them are simply too weak to have an impact on the output , and the top k strongest pairwise interactions are well sufficient for data modeling and prediction . Finding the top k strongest pairwise interactions fits perfectly into the best k-arms identification problem in the context of multi-armed bandits . Inspired by the recent work ( Bagaria et al. , 2018b ) , we can treat each entry of the Hessian Hij as an arm Ar ( see Figure 2 ) . Since Hessian is a symmetric matrix , we only need to consider n = p ( p− 1 ) /2 arms in the set { Ar : r ∈ [ n ] } . For example , if we choose to pull the arm for the i-th row and j-th column of the Hessian , we will evaluate the local interaction strength g ( xξ , i , j ) on a randomly selected training data point xξ . The strength g ( xξ , i , j ) can be regarded as a random reward , where xξ is uniformly drawn from the training data { x1 , . . . , xN } . For ease of notation , we let f ( r ; ξ ) : = g ( xξ , i , j ) as the random reward of the arm r , and define µr : = Eξ [ f ( r ; ξ ) ] as the true mean value . Our goal is to find the best k arms with the highest mean reward . This reformulation is reasonable since the following three essential assumptions for multi-armed bandits ( Slivkins et al. , 2019 ) are satisfied . a . Only the reward will be observed after each pull ; b . The reward for each arm is drawn independently and identically from its reward distribution ; c. The reward for each round is bounded . Since we only consider functions F ( x ) defined on a compact set , if F is a continuous function and twice differentiable , its first- and second-order derivatives are bounded accordingly . 3.4 DETECTING THE k-STRONGEST PAIRWISE INTERACTIONS WITH UCB ALGORITHM In this section , we briefly introduce the UCB algorithm ( Lai & Robbins , 1985 ) for finding the k-strongest interactions . Here , we design a sequence of estimators { f̂ ` ( r ) } for the mean reward of the arm Ar and construct confidence interval C ( ` ) . We let f̂ ` ( r ) denote the estimator after ` pulls of the arm Ar . Several assumptions need to be made before we give out the primary outcome . Assumption 1 . Finite m evaluations of one arm are sufficient to obtain an accurate reward . Assumption 2 . Estimators f̂ ` ( r ) are σr-subgaussian . Assumption 2 is satisfied because the reward is bounded ( Hoeffding ’ s lemma , 1963 ) . Here , σrs are pre-defined parameters . In the following analysis , we will choose σ = maxr { σr } , such that all f̂ ` ( r ) are σ-subgaussian . Let f̂ ` ( r ) : = 1 ` ∑ ` i=1 f ( r ; ξi ) , and we construct the 1− δ confidence intervals ( see supplement C ) of f̂ ` ( r ) as , C ( ` ) : = { √ 2σ2 log 2δ ` if ` ≤ m 0 if ` > m , ( 2 ) µr ∈ [ f̂ ` ( r ) − C ( ` ) , f̂ ` ( r ) + C ( ` ) ] , w.p . 1− δ , ( 3 ) where µr denotes the true expectation . Increasing number of pulls of one arm , the uncertainty of the reward for that arm will decrease . When the number of pulls reaches m , we can remove the uncertainty of the arm and set C ( ` ) = 0 . Let ` r ( t ) count the number of pulls of arm Ar till iteration t. To simplify the notations , we let µ̂r ( t ) = f̂ ` r ( t ) ( r ) and Cr ( t ) = C ( ` r ( t ) ) be the sample means and confidence intervals at iteration t. Now , we formally introduce the complete procedure in Algorithm 1 . The Algorithm 1 is initialized by evaluating all arms O ( log ( n ) ) times , then at each subsequent iteration , it picks the arm with the highest UCB , i.e. , maxj { µ̂j ( t ) +Cj ( t ) } , to evaluate ( see Figure 2 ) . If one arm has been evaluated for m times , we will its true mean value and set the uncertainty to zero . At the end of each iteration , if there exists one arm whose Lower Confidence Bound ( LCB ) is higher than all the other ’ s UCB , we will put it into the set of k best arms . In the following , we give a theoretical analysis on the complexity of this algorithm . We have to further define ∆ ( k ) i : = max ( 0 , µi∗k − µi ) , where i ∗ k is the index for the k-th best arm and ∆ ( k ) i indicates the gap of true values between the top-k arms and the rest . Theorem 3.2 . With probability 1−Θ ( 1n2 ) , given maximal pulling times m for each arm , Algorithm 1 returns the k-strongest interaction pairs in O ( ∑n i=1 log ( n ) ( σ2 log ( nm ) ( ∆ ( k ) i ) 2 ∧m ) ) time , where ( · ∧ · ) is short for min ( · , · ) . Remark 3.1 . When k n , the complexity of Algorithm 1 for finding the top k interactions is O ( n log ( mn ) log ( n ) + km log ( n ) ) under some natural assumptions on the distribution of ∆i , being superior to the naive algorithm ( every arm is pulled m times ) with complexity O ( nm ) . Remark 3.2 . For k = 1 , namely the strongest interaction pair identification , it requires at most∑n i=1 ( ( 8σ2 ( ∆ ( 1 ) i ) 2 log ( n3m ) ) ∧m ) pulls . We leave the complete proof in supplement C .

Summary: Identification of feature interactions from black-box models, on a global as well as on a local scale, can lead to a better understanding of the operating modes of the black-box models. In this paper, the authors presented a principled approach to identify global interactions by casting the problem as a multi-arm bandit problem and proposes a solution using UCB algorithm. The authors claim that their method interaction discovery method is free of ad-hoc assumptions with good detection accuracy and stability. Furthermore, the authors showcase the importance of the learned interactions by proposing a new deep learning model based on these interactions and showcase the improvements in model size (thereby competing against pruning methods) as well as in accuracy (thereby competing against generalization methods).