Fast Generic Interaction Detection for Model Interpretability and Compression

1 INTRODUCTION . Explainable machine learning is an active research field that focuses on providing interpretable models , transparent explanations , and confident decisions to practical AI systems . Investigating feature interaction is vital to model interpretability . Interaction detection should be able to reveal which subset of features influence the output jointly , and what the corresponding nonlinear transformation is . We aim to find the underlying interactions from data , such that we can interpret the model properly . To go one step further , we hope that new models can be built with the aid of the detected interaction knowledge economically to avoid heavy parameterization . In this paper , we first restrict ourselves to interaction detection . A novel method is derived directly from the most acknowledged definition of feature interaction ( Friedman et al. , 2008 ) . We further apply the obtained interaction knowledge to design a transparent and refined neural network ( NN ) . This approach fits well into the data science life cycle ( Yu & Kumbier , 2019 ) , which was applied , for instance in Tsang et al . ( 2020a ) , successfully for interpretable recommender system design . The main contributions of this paper include : 1 ) a fast and principled interaction detection method , 2 ) a lightweight and interpretable neural network model that can surpass its Teacher , 3 ) thorough theoretical analysis and performance evaluations with real datasets . More details are given below . 1 . We propose a generic interaction detection method based on a global statistical metric , namely the expected Hessian , Hij : = Ex [ ∂2F ( x ) ∂xi∂xj ] . Notably , our method is model-agnostic and applicable to any pre-trained learning model , F ( x ) , being for instance , a deep neural network model or a tree model . It is also flexible to use for multi-way interaction detection . 2 . To speed up the detection process , we evaluate the expected Hessian via adaptive sampling using the Upper Confidence Bound ( UCB ) algorithm ( Lai & Robbins , 1985 ) , which can significantly reduce the computational complexity . Besides , thorough analysis of the proposed interaction detection method are conducted . 3 . Using the detected interaction pairs , we further design a compressed but interpretable Student model which can surpass its Teacher by 26 % in terms of data fitting performance averaged over various datasets . The compressed model reduces its size over 100 times compared to the baseline fully-connected , over-parameterized neural network ( OverparaFC ) . 4 . We demonstrate the linkages between our compressed model and the classic alternating conditional expectation ( ACE ) model ( Breiman & Friedman , 1985 ) . 5 . We conduct large-scale performance evaluations and further explain the obtained model interpretability with some real datasets . The remainder of this paper is organized as follows . Section 2 introduces all related works . Our proposed interaction detection method is introduced in Section 3 . By exploiting the detection outcome , a new variant of lightweight and interpretable deep learning model is introduced in Section 4 . Experimental results are given in Section 5 . Finally , we conclude the paper in Section 6 . 2 RELATED WORKS . Interaction Detection : Early works adopt pure statistics for detecting feature interactions , and representatives include ANOVA and GUIDE ( Wonnacott & Wonnacott , 1990 ; Fisher , 1992 ; Loh , 2002 ) . These works have motivated a plethora of new methods with the aim to enhance the detection accuracy and/or efficiency . The first class of methods centered around the GA2M and tree models , see for instance Lou et al . ( 2013 ) ; Sorokina et al . ( 2008 ) ; Friedman et al . ( 2008 ) ; Lundberg et al . ( 2020 ) . The second class of methods were built on the so-called factorization machines ( Rendle , 2010 ) as well as its new variants ( Xiao et al. , 2017 ; Song et al. , 2019 ) with attention mechanism . The third class exploits the most recent advances in deep learning , including the Neural Interaction Detection ( NID ) and some new variants ( Tsang et al. , 2018a ; b ; Cui et al. , 2019 ) : Persistence Interaction Detection ( PID ) ( Liu et al. , 2020 ) , Integrated Hessians ( IH ) ( Janizek et al. , 2020 ) , Shapley interaction ( Zhang et al. , 2020 ; Sundararajan et al. , 2020 ) , etc . Although we have witnessed well improved interaction detection performance for many datasets over the decades , the above methods still lead to inconsistent detection results for some other datasets . The reasons are twofold . Firstly , the interaction strength is empirically defined , for instance , NID method computes the interaction strength via summarizing the neural network weights . Secondly , a specific deep learning model is required , for instance , an ` 1-regularized ReLU network is required by the latest NID and PID methods to maintain high accuracy . In contrast , our proposed method is derived directly from the definition of feature interaction ( Friedman et al. , 2008 ) and moreover is not confined to any specific learning model . Model Interpretability : There are two categories of approaches to address model interpretability , namely the transparency-based and post-hoc approaches ( Došilović et al. , 2018 ) . Transparency-based approaches require the model itself to be simple and interpretable , like linear models , decision trees , etc . One can directly read off the interpretations from their coefficients or decision rules . But often , they are less accurate due to limited representation power . In contrast , post-hoc approaches extract useful information from a pre-trained model , which is often complex and hard to interpret . Well-known methods such as LIME ( Ribeiro et al. , 2016 ) and SHAP ( Lundberg & Lee , 2017 ) fall in this category , but they did not take feature interaction into account . Recently proposed symbolic metamodel ( Alaa & van der Schaar , 2019 ) captures nonlinear interactions by approximating the black-box model with explicit symbolic expressions . There are also some other post-hoc approaches based on sensitivity analysis ( Cortez & Embrechts , 2013 ) , which return a quantification of feature importance νi = E [ ( ∂F ( x ) ∂xi ) 2 ] ( Kucherenko et al. , 2009 ) and interactions νij = E [ |∂ 2F ( x ) ∂xi∂xj |2 ] ( Roustant et al. , 2014 ) by input perturbation . Our work aims to combine the strengths of the two categories . Concretely , we first extract the interaction knowledge by a post-hoc method , and then build a transparent and interpretable learning model as illustrated in Figure 3 . Model Compression and Knowledge Distillation ( KD ) : Model compression ( Bucilua et al. , 2006 ) aims to learn a small compressed model ( Student ) from a large complex model ( Teacher ) with augmented training data produced by the Teacher . Compared to the Teacher , the Student can make similar or even better predictions . Knowledge distillation ( Hinton et al. , 2015 ) mainly deals with multi-class classification problems and extracts “ valuable information that defines a rich similarity structure over the data ” . Our work introduces a novel viewpoint of knowledge ( the interacted relationships ) and targets a lightweight but more accurate Student model . 3 PROPOSED INTERACTION DETECTION METHOD . Before diving into in-depth discussions of interaction detection , we need to formally define what feature interaction is . The textbook definition according to Friedman et al . ( 2008 ) is given below . Definition 3.1 ( Friedman & Popescu 2008 ) . A function F : Rp → R is said to exhibit an interaction between two of its variables xi and xj if the difference in the value of F ( x ) as a result of changing the value of xi depends on the value of xj . Equivalently , if Ex [ |∂ 2F ( x ) ∂xi∂xj |2 ] > 0 , namely the partial derivative w.r.t xj turns out to be dependent on xi , then we say xi and xj are interacted . Otherwise , xi and xj have no interaction , if F ( x ) can be expressed as the sum of two functions f\i and f\j ( Sorokina et al. , 2008 ) , namely , F ( x ) = f\i ( x1 , . . . , xi−1 , xi+1 , . . . , xp ) + f\j ( x1 , . . . , xj−1 , xj+1 , . . . , xp ) , where f\i ( j ) is irrespective to xi ( j ) . Similarly , higher-order ( multi-way ) interaction can be defined . In this paper , we mainly focus on pairwise interaction , while multi-way interaction is only briefly discussed due to space limitations . 3.1 INTERACTION STRENGTH MEASURE . Motivated by the above definition of interaction , it is natural to take advantage of the Hessian matrix H : = ∇2xxF ( x ) . The magnitude of its entry | ∂2F ( x ) ∂xi∂xj | contains rich information about the local curvature at a data point x . Note that F ( x ) is a regression function here , not a loss function . This idea was originally considered in the economics community ( Ai & Norton , 2003 ) and rediscovered for sensitivity analysis in Roustant et al . ( 2014 ) . The goodness of F ( x ) as an approximator can essentially influence the interaction detection performance , see our Theorem M.2 in the supplement . We define g ( x , i , j ) : = |∂ 2F ( x ) ∂xi∂xj |2 to measure the local interaction strength of the i-th and j-th features at point x . We then use f ( i , j ) : = Ex [ g ( x , i , j ) ] = Ex [ |∂ 2F ( x ) ∂xi∂xj |2 ] as a measure of the global interaction strength . If f ( i , j ) ≈ 0 , we say feature xi and xj have weak interaction ; otherwise , if f ( i , j ) is significantly larger than zero , then xi and xj have strong interaction . In this paper , we focus on the global interaction . 3.2 INTERACTION STRENGTH EVALUATION The above defined global interaction strength f ( i , j ) = Ex [ |∂ 2F ( x ) ∂xi∂xj |2 ] is mostly unavailable due to the unknown input distribution x ∼ P ( x ) . The Monte Carlo method can be used to approximate it by computing the sample mean over the training data ( Roustant et al. , 2014 ) . Analytical Evaluation : The Hessian matrix∇2xxF ( x ) for neural networks at a certain data point can be calculated analytically and efficiently , by using the automatic differentiation ( Paszke et al. , 2017 ) . However , using this analytical solution is problematic for some learning models , such as the ReLU network , Random Forest ( RF ) , etc. , see supplement A . For example , the landscape of a ReLU network is piece-wise linear as shown in Figure 1 , thus the exact Hessian is a zero matrix at almost every point . So , we turn to the following numerical evaluation for broader horizons . Numerical Evaluation : Finite difference method is a common way to approximate the Hessian on a given data point ( Campolongo & Braddock , 1999 ) , i.e. , ∂2F ( x ) ∂xi∂xj ≈ 1 4hihj [ F ( x + eihi + ejhj ) − F ( x + eihi − ejhj ) − F ( x + ejhj − eihi ) + F ( x− eihi − ejhj ) ] , ( 1 ) where ei is a one-hot vector with the i-th element being equal to one and the rest of elements being zeros . We also note that if the computation of gradient ∂F∂x is cheap ( e.g. , F is a neural network ) , then Hij can be approximated as 12h [ ∂F ( x+ejh ) ∂xi − ∂F ( x−ejh ) ∂xi ] to reduce computation , which is similar to the feature interaction score defined in Greenside et al . ( 2018 ) . The choice of perturbation size hi or hj ( abbr . hi ( j ) ) is critical . Generally , we do not want hi ( j ) to be too small ( incurring round-off error ) or too large ( incurring truncation error ) to get a good overall approximation of the derivative ( Jerrell , 1997 ; Baydin et al. , 2018 ) . For our problem , we particularly do not want hi ( j ) to be too small so that the four evaluated points ( shown in the numerator of Equation 1 ) lie on the same hyperplane ( e.g. , region A in Figure 1 ) , which makes the quantity in Equation 1 always zero . The following theorem reveals that choosing a sufficiently small hi ( j ) is not necessary . Theorem 3.1 . For any x and y , function F shows no interaction between them , i.e. , it can be decomposed as F ( x , y ) = a ( x ) + b ( y ) if and only if , for any h , k > 0 , F ( x + h , y + k ) − F ( x + h , y − k ) − F ( x− h , y + k ) + F ( x− h , y − k ) = 0 . The magnitude of the numerator in Equation 1 tells us whether F is locally separable for variables xi and xj at point x , see our proof in supplement B . The main issue of the finite difference method lies in the computational complexity for approximating the global interaction strength f ( i , j ) , especially when the function evaluation itself is expensive . Using all the training samples for finding just a few strongest interaction pairs can be a total waste of computation resources . In general , for a dataset with N samples in p-dimensional feature spaces , a total number of 4Np ( p− 1 ) /2 arithmetic evaluations of the surrogate regression function F are needed . If the Hessian happens to be sparse , that is , there are only a few interactions existing in the ground truth function , massive evaluations on those feature pairs with zero interaction strength should be avoided . 3.3 IDENTIFICATION OF THE k-STRONGEST PAIRWISE INTERACTIONS In practice , we do not have to obtain the interaction strengths for all interaction pairs , because many of them are simply too weak to have an impact on the output , and the top k strongest pairwise interactions are well sufficient for data modeling and prediction . Finding the top k strongest pairwise interactions fits perfectly into the best k-arms identification problem in the context of multi-armed bandits . Inspired by the recent work ( Bagaria et al. , 2018b ) , we can treat each entry of the Hessian Hij as an arm Ar ( see Figure 2 ) . Since Hessian is a symmetric matrix , we only need to consider n = p ( p− 1 ) /2 arms in the set { Ar : r ∈ [ n ] } . For example , if we choose to pull the arm for the i-th row and j-th column of the Hessian , we will evaluate the local interaction strength g ( xξ , i , j ) on a randomly selected training data point xξ . The strength g ( xξ , i , j ) can be regarded as a random reward , where xξ is uniformly drawn from the training data { x1 , . . . , xN } . For ease of notation , we let f ( r ; ξ ) : = g ( xξ , i , j ) as the random reward of the arm r , and define µr : = Eξ [ f ( r ; ξ ) ] as the true mean value . Our goal is to find the best k arms with the highest mean reward . This reformulation is reasonable since the following three essential assumptions for multi-armed bandits ( Slivkins et al. , 2019 ) are satisfied . a . Only the reward will be observed after each pull ; b . The reward for each arm is drawn independently and identically from its reward distribution ; c. The reward for each round is bounded . Since we only consider functions F ( x ) defined on a compact set , if F is a continuous function and twice differentiable , its first- and second-order derivatives are bounded accordingly . 3.4 DETECTING THE k-STRONGEST PAIRWISE INTERACTIONS WITH UCB ALGORITHM In this section , we briefly introduce the UCB algorithm ( Lai & Robbins , 1985 ) for finding the k-strongest interactions . Here , we design a sequence of estimators { f̂ ` ( r ) } for the mean reward of the arm Ar and construct confidence interval C ( ` ) . We let f̂ ` ( r ) denote the estimator after ` pulls of the arm Ar . Several assumptions need to be made before we give out the primary outcome . Assumption 1 . Finite m evaluations of one arm are sufficient to obtain an accurate reward . Assumption 2 . Estimators f̂ ` ( r ) are σr-subgaussian . Assumption 2 is satisfied because the reward is bounded ( Hoeffding ’ s lemma , 1963 ) . Here , σrs are pre-defined parameters . In the following analysis , we will choose σ = maxr { σr } , such that all f̂ ` ( r ) are σ-subgaussian . Let f̂ ` ( r ) : = 1 ` ∑ ` i=1 f ( r ; ξi ) , and we construct the 1− δ confidence intervals ( see supplement C ) of f̂ ` ( r ) as , C ( ` ) : = { √ 2σ2 log 2δ ` if ` ≤ m 0 if ` > m , ( 2 ) µr ∈ [ f̂ ` ( r ) − C ( ` ) , f̂ ` ( r ) + C ( ` ) ] , w.p . 1− δ , ( 3 ) where µr denotes the true expectation . Increasing number of pulls of one arm , the uncertainty of the reward for that arm will decrease . When the number of pulls reaches m , we can remove the uncertainty of the arm and set C ( ` ) = 0 . Let ` r ( t ) count the number of pulls of arm Ar till iteration t. To simplify the notations , we let µ̂r ( t ) = f̂ ` r ( t ) ( r ) and Cr ( t ) = C ( ` r ( t ) ) be the sample means and confidence intervals at iteration t. Now , we formally introduce the complete procedure in Algorithm 1 . The Algorithm 1 is initialized by evaluating all arms O ( log ( n ) ) times , then at each subsequent iteration , it picks the arm with the highest UCB , i.e. , maxj { µ̂j ( t ) +Cj ( t ) } , to evaluate ( see Figure 2 ) . If one arm has been evaluated for m times , we will its true mean value and set the uncertainty to zero . At the end of each iteration , if there exists one arm whose Lower Confidence Bound ( LCB ) is higher than all the other ’ s UCB , we will put it into the set of k best arms . In the following , we give a theoretical analysis on the complexity of this algorithm . We have to further define ∆ ( k ) i : = max ( 0 , µi∗k − µi ) , where i ∗ k is the index for the k-th best arm and ∆ ( k ) i indicates the gap of true values between the top-k arms and the rest . Theorem 3.2 . With probability 1−Θ ( 1n2 ) , given maximal pulling times m for each arm , Algorithm 1 returns the k-strongest interaction pairs in O ( ∑n i=1 log ( n ) ( σ2 log ( nm ) ( ∆ ( k ) i ) 2 ∧m ) ) time , where ( · ∧ · ) is short for min ( · , · ) . Remark 3.1 . When k n , the complexity of Algorithm 1 for finding the top k interactions is O ( n log ( mn ) log ( n ) + km log ( n ) ) under some natural assumptions on the distribution of ∆i , being superior to the naive algorithm ( every arm is pulled m times ) with complexity O ( nm ) . Remark 3.2 . For k = 1 , namely the strongest interaction pair identification , it requires at most∑n i=1 ( ( 8σ2 ( ∆ ( 1 ) i ) 2 log ( n3m ) ) ∧m ) pulls . We leave the complete proof in supplement C .

Summary: The paper handles the important area of model that can be interpreted using feature interactions - aligned to theme of explainable deep learning. The authors chose to use the problem of multi-arm bandit, solving it by UCB algorithm with good speed and accuracy. A lightweight and interpretable deep learning model (called ParaACE), built using alternating conditional expectation (ACE) method is the crux of the work. It is shown that the proposed method improves accuracy by 26% and reduces the model size by 100+ times as compared to its Teacher model over various datasets. The paper has extensive supplementary material, as well as shared code.