T HEORETICAL U NDERSTANDING OF D ROPOUT AND I TS A LTERNATIVE R EGULARIZATION
Anonymous authors
Paper under double-blind review
A BSTRACT
This paper aims to understand and improve the utility of the dropout operation from the perspective of game-theoretic interactions.
We prove that dropout can suppress the strength of interactions between input variables of deep neural networks (DNNs).
The theoretic proof is also veriﬁed by various experiments.
Furthermore, we ﬁnd that such interactions were strongly related to the over-ﬁtting problem in deep learning.
Thus, the utility of dropout can be regarded as decreasing interactions to alleviate the signiﬁcance of over-ﬁtting.
Based on this understanding, we propose an interaction loss to further improve the utility of dropout.
Experimental results have shown that the interaction loss can effectively improve the utility of dropout and boost the performance of DNNs.
We will release the code when the paper is accepted
I NTRODUCTION
Deep neural networks (DNNs) have exhibited signiﬁcant success in various tasks, but the overﬁtting problem is still a considerable challenge for deep learning.
Dropout is usually considered as an effective operation to alleviate the over-ﬁtting problem of DNNs.
Hinton et al.
; Srivastava et al.
(2014) thought that dropout could encourage each unit in an intermediate-layer feature to model useful information without much dependence on other units.
Konda et al.
(2016) considered dropout as a speciﬁc method of data augmentation.
Gal & Ghahramani (2016) proved that dropout was equivalent to the Bayesian approximation in a Gaussian process.
In this paper, we aim to explain, model, and improve the utility of dropout from the following perspectives.
First, we prove that the dropout operation suppresses interactions between input units encoded by DNNs.
This is also veriﬁed by various experiments.
To this end, the interaction is deﬁned in game theory, as follows.
Let x denote the input, and let f ( x ) denote the output of the
DNN.
For the i -th input variable, we can compute its importance value φ ( i ) , which measures the numerical contribution of the i -th variable to the output f ( x ) .
We notice that the importance value of the i -th variable would be different when we mask the j -th variable w.r.t.
the case when we do not mask the j -th variable.
Thus, the interaction between input variables i and j is measured as the difference φ w/ j ( i ) − φ w/o j ( i ) .
Second, we also discover a strong correlation between interactions of input variables and the overﬁtting problem of the DNN.
Speciﬁcally, the over-ﬁtted samples usually exhibit much stronger interactions than ordinary samples.
Therefore, we consider that the utility of dropout is to alleviate the signiﬁcance of over-ﬁtting by decreasing the strength of interactions encoded by the DNN.
Based on this understanding, we propose an interaction loss to further improve the utility of dropout.
The interaction loss directly penalizes the interaction strength, in order to improve the performance of DNNs.
The interaction loss exhibits the following two distinct advantages over the dropout operation.
(1) The interaction loss explicitly controls the penalty of the interaction strength, which enables people to trade off between over-ﬁtting and under-ﬁtting.
(2) Unlike dropout which is incompatible with the batch normalization operation (Li et al., 2019), the interaction loss can work in harmony with batch normalization.
Various experimental results show that the interaction loss can boost the performance of DNNs.
Furthermore, we analyze interactions encoded by DNNs from the following three perspectives.
First, we discover the consistency between the sampling process in dropout (when the dropout rate p = 0 . 5 ) and the sampling in the computation of the Banzhaf value.
The Banzhaf value (Banzhaf III, 1964) is another metric to measure the importance of each input variable in game theory.
Unlike the Shapley value, the Banzhaf value is computed under the assumption that each input variable independently participates in the game with the probability 0.5. We ﬁnd that the frequent inference patterns in Banzhaf interactions (Grabisch & Roubens, 1999) are also prone to be frequently sampled by dropout, thereby being stably learned.
This ensures the DNN to encode smooth Banzhaf interactions.
We also prove that the Banzhaf interaction is close to the aforementioned interaction, which also relates to the dropout operation with the interaction used in this paper.
(2) Besides, we ﬁnd that the interaction loss is better to be applied to low layers than being applied to high layers.
Furthermore, we decompose the overall interaction into interaction components of different orders.
We visualize the strongly interacted regions within each input sample.
We ﬁnd out that interaction components of high orders take the main part of interactions, and are more prone to be suppressed by the dropout operation and the interaction loss than other interaction components.
Contributions of this paper can be summarized as follows.
(1) We mathematically represent the dependence of feature variables using as the game-theoretic interactions, and prove that dropout can suppress the strength of interactions encoded by a DNN.
In comparison, previous studies (Hinton et al., 2012; Krizhevsky et al., 2012; Srivastava et al., 2014) did not mathematically model the the dependence of feature variables or theoretically proved its relationship with the dropout.
We ﬁnd that the over-ﬁtted samples usually contain stronger interactions than other samples.
(3) Based on this, we consider the utility of dropout is to alleviate over-ﬁtting by decreasing the interaction.
We design a novel loss function to penalize the strength of interactions, which improves the performance of DNNs.
(4) We analyze the properties of interactions encoded by DNNs, and conduct comparative studies to obtain new insights into interactions encoded by DNNs.
R ELATED W ORK
The dropout operation.
Dropout is an effective operation to alleviate the over-ﬁtting problem and improve the performance of DNNs (Hinton et al., 2012).
Several studies have been proposed to explain the inherent mechanism of dropout.
According to (Hinton et al., 2012; Krizhevsky et al., 2012; Srivastava et al., 2014), dropout could prevent complex co-adaptation between units in intermediate layers, and could encourage each unit to encode useful representations itself.
However, these studies only qualitatively analyzed the utility of dropout, instead of providing quantitative results.
Wager et al.
(2013) showed that dropout performed as an adaptive regularization, and established a connection to the algorithm AdaGrad.
Konda et al.
(2016) interpreted dropout as a kind of data augmentation in the input space, and Gal & Ghahramani (2016) proved that dropout was equivalent to a Bayesian approximation in the Gaussian process.
Gao et al.
(2019) disentangled the dropout operation into the forward dropout and the backward dropout, and improved the performance by setting different dropout rates for the forward dropout and the backward dropout, respectively.
Gomez et al.
(2018) proposed the targeted dropout, which only randomly dropped variables with low activation values, and kept variables with high activation values.
In comparison, we aim to explain the utility of dropout from the view of game theory.
Our interaction metric quantiﬁed the interaction between all pairs of variables considering all potential contexts, which were randomly sampled from all variables.
Furthermore, we propose a method to improve the utility of dropout.
Interaction.
Previous studies have explored interactions between input variables.
Bien et al.
(2013) developed an algorithm to learn hierarchical pairwise interactions inside an additive model.
Sorokina et al.
(2008) detected the statistical interaction using an additive model-based ensemble of regression trees.
Murdoch et al.
; Singh et al.
; Jin et al.
(2019) proposed and extended the contextual decomposition to measure the interaction encoded by DNNs in NLP tasks.
Tsang et al.
(2018) measured the pairwise interaction based on the learned weights of the DNN.
Tsang et al.
(2020) proposed a method, namely GLIDER, to detect the feature interaction modeled by a recommender system.
Janizek et al.
(2020) proposed the Integrated-Hessian value to measure interactions, based on Integral Gradient (Sundararajan et al., 2017).
Integral Gradient measures the importance value for each input
variable w.r.t.
the DNN.
Given an input vector x ∈ R n , Integrated Hessians measures the interaction between input variables (dimensions) x i and x j as the numerical impact of x j on the importance of x i .
To this end, Janizek et al.
(2020) used Integral Gradient to compute
Integrated Hessians.
Appendix L further compares Integral Hessians with the interaction proposed in this paper.
Besides interactions measured from the above views, game theory is also a typical perspective to analyze the interaction.
Several studies explored the interaction based on game theory.
Lundberg et al.
(2018) deﬁned the interaction between two variables based on the Shapley value for tree ensembles.
Because Shapley value was considered as the unique standard method to estimate contributions of input words to the prediction score with solid theoretic foundations (Weber, 1988), this deﬁnition of interaction can be regarded to objectively reﬂect the collaborative/adversarial effects between variables w.r.t the prediction score.
Furthermore, Grabisch & Roubens (1999) extended this deﬁnition to interactions among different numbers of input variables.
Grabisch & Roubens (1999) also proposed the interaction based on the Banzhaf value.
In comparison, the target interaction used in this paper is based on the Shapley value (Shapley, 1953).
Since the Shapley is the unique metric that satisﬁes the linearity property, the dummy property, the symmetry property, and the efﬁciency property (Ancona et al., 2019), the interaction based on the Shapley value is usually considered as a more standard metric than the interaction based on the Banzhaf value.
In this paper, we aim to explain the utility of dropout using the interaction deﬁned in game theory.
We reveal the close relationship between the strength of interactions and the over-ﬁtting of DNNs.
We also design an interaction loss to improve the performance of DNNs.
G AME - THEORETIC E XPLANATIONS OF D ROPOUT
Preliminaries: Shapley values.
The Shapley value was initially proposed by Shapley (1953) in game theory.
It is considered as a unique unbiased metric that fairly allocates the numerical contribution of each player to the total reward.
Given a set of players N = { 1 , 2 , · · · , n } , 2 N def = { S | S ⊆ N } denotes all possible subsets of N .
A game f : 2 N → R is a function that maps from a subset to a real number.
f ( S ) is the score obtained by the subset S ⊆ N .
Thus, f ( N ) − f ( ∅ ) denotes the reward obtained by all players in the game.
The Shapley value allocates the overall reward to each player, as its numerical contribution φ ( i | N ) , as shown in Equation (1).
The Shapley value of player i in the game f , φ ( i | N ) , is computed as follows.
[Equation]
where P Shapley ( S | M )
= ( | M |−| S | )!
| S | !
( | M | +1)!
is the likelihood of S being sampled, S ⊆ M .
The Shapley value is the unique metric that satisﬁes the linearity property, the dummy property, the symmetry property, and the efﬁciency property (Ancona et al., 2019).
We summarize these properties in Appendix A.
Understanding DNNs via game theory.
In game theory, some players may form a coalition to compete with other players, and win a reward (Grabisch & Roubens, 1999).
Accordingly, a DNN f can be considered as a game, and the output of the DNN corresponds to the score f ( · ) in Equation (1).
For example, if the DNN has a scalar output, we can take this output as the score.
If the DNN outputs a vector for multi-category classiﬁcation, we select the classiﬁcation score corresponding to the true class as the score f ( · ) .
Alternatively, f can also be set as the loss value of the DNN.
The set of players N corresponds to the set of input variables.
We can analyze the interaction and the element-wise contribution at two different levels.
(1) We can consider input variables (players) as the input of the entire DNN, e.g. pixels in images and words in sentences.
In this case, the game f is considered as the entire DNN.
(2) Alternatively, we can also consider input variables as a set of activation units before the dropout operation.
In this case, the game f is considered as consequent modules of the DNN.
S ⊆ N in Equation (1) denotes the context of the input variable i , which consists of a subset of input variables.
In order to compute the network output f ( S ) , we replace variables in N \ S with the baseline value ( e.g. mask them), and we do not change the variables in S .
In particular, when we consider neural activations before dropout as input variables, such activations are usually non-negative after
ReLU.
Thus, their baseline values are set to 0.
Both (Ancona et al., 2019) and Appendix I introduce details about the baseline value.
In this way, f ( ∅ ) measures the output score when all input variables are masked, and f ( S )
− f ( ∅ ) measures the entire reward obtained by all input variables in S .
Interactions encoded by DNNs.
In this section, we introduce how to use the interaction deﬁned in game theory to explain DNNs.
Two input variables may interact with each other to contribute to the output of a DNN.
Let us suppose input variables i and j have an interaction.
In other words, the contribution of i and j when they work jointly is different with the case when they work individually.
[Figure]
For example, in the sentence he is a green hand , the word green and the word hand have a strong interaction, because the words green and hand contribute to the person’s identity jointly, rather than independently.
In this case, we can consider these two input variables to form a certain inference pattern as a singleton player S ij = { i, j } .
Thus, this DNN can be considered to have only ( n − 1) input variables, N (cid:48)
=
N \{ i, j }∪ S ij , i.e. S ij is supposed to be always absent or present simultaneously as a constituent.
In this way, the interaction I ( i, j ) between input variables i and j is deﬁned by Grabisch & Roubens (1999), as the contribution increase of S ij when input variables i and j cooperate with each other w.r.t.
the case when i and j work individually, as follows.
[Equation]
where ∆ f ( S, i, j ) def = f ( S ∪ { i, j } ) − f ( S ∪ { j } ) − f ( S ∪ { i } ) + f ( S ) .
φ ( i | N \{ j } ) and φ ( j | N \{ i } ) correspond to the contribution to the DNN output when i and j work individually.
Theoretically,
I ( i, j ) is also equal to the change of the variable i ’s Shapley value when we mask another input variable j w.r.t.
the case when we do not mask j .
Please see Appendix B for the proof.
If I ( i, j ) > 0 , input variables i and j cooperate with each other for a higher output value.
Whereas, if
I ( i, j ) < 0 , i and j have a negative/adversarial effect.
The strength of the interaction can be computed as the absolute value of the interaction, i.e. | I ( i, j ) | .
We ﬁnd that the overall interaction I ( i, j ) can be decomposed into interaction components with different orders s , as follows.
[Equation]
where s denote the size of the context S for the interaction.
We use I ( s ) ( i, j ) to represent the s -order interaction between input variables i and j .
I ( s ) ( i, j ) reﬂects the average interaction between input variables i and j among all contexts S with s input variables.
For example, when s is small, I ( s ) ( i, j ) measures the interaction relying on inference patterns consisting of very few input variables, i.e. the interaction depends on a small context.
When s is large, I ( s ) ( i, j ) corresponds to the interaction relying on inference patterns consisting of a large number of input variables, i.e. the interaction depends on the context of a large scale.
Visualization of interactions in each sample, which are encoded by the DNN: There exists a speciﬁc interaction between each pair of pixels, which boosts the difﬁculty of visualization.
In order to simplify the visualization, we divide the original image into 16 × 16 grids, and we only visualize the strength of interactions between each grid g and its neighboring grids g (cid:48) as Color ( g ) =
E g (cid:48) ∈ neighbor ( g ) [ | I ( g, g (cid:48) ) | ] .
Figure 1 visualizes the interaction strength within the image in the CelebA dataset (Liu et al., 2015), which are normalized to the range [0 , 1] .
Grids on the face usually contain more signiﬁcant interactions with neighboring grids than grids in the background.
Proof of the relationship between dropout and the interaction.
In this section, we aim to mathematically prove that dropout is an effective method to suppress the interaction strength encoded by
DNNs.
Given the context S , let us consider its subset T ⊆ S , which forms a coalition to represent a speciﬁc inference pattern T ∪{ i, j } .
Note that for dropout, the context refers to activation units in the intermediate-layer feature without semantic meanings.
Nevertheless, we just consider i, j as pixels as a toy example to illustrate the basic idea, in order to simplify the introduction.
For example, let S represent the face, and let T ∪ { i, j } represent pixels of an eye in the face.
Let R T
( i, j ) quantify the marginal reward obtained from the inference pattern of an eye.
All interaction effects from smaller coalitions T (cid:48) (cid:36) T are removed from R T ( i, j ) .
According to the above example, Let T (cid:48) (cid:36) T correspond to the pupil inside the eye.
Then, R T (cid:48)
( i, j ) measures the marginal reward beneﬁted from the existence of the pupil T (cid:48) ∪ { i, j } , while R T ( i, j )
[Figure]
[Figure]
represents the marginal beneﬁt from the existence of the entire eye, in which the reward from the pupil has been removed.
I
[Figure]
e. the inference pattern T ∪ { i, j } can be exclusively triggered by the co-occurrence of all pixels in the eye, but cannot be triggered by a subset of pixels in the pupil
T (cid:48) ∪ { i, j } .
The beneﬁt from the pupil pattern has been removed from R T ( i, j ) .
Thus, the s -order interaction can be decomposed into components w.r.t.
all inference patterns T ∪ { i, j } , T ⊆ S . I ( s ) ( i, j ) = E S [Equation]
N \{ i, [Equation] } , | S [Equation]
s
[Equation]
i,
[Equation]
[Equation]
(4) where J q ( i, j ) def = E T ⊆ N \{ i,j } , | T | =
q [ R T ( i, j )] denotes the average interaction between i and j given all potential inference patterns T ∪ { i, j } with a ﬁxed inference pattern size | T | =
q ; Γ ( q ) ( i, j | s ) def = (cid:0) sq (cid:1)
J ( q ) ( i, j ) .
The computation of R T ( i, j ) and the proof of Equation (4) are provided in Appendices
E and C, respectively.
However, when input variables in N are randomly removed by the dropout operation, the computation of I ( s ) dropout ( i, j ) only involves a subset of inference patterns consisting of variables that are not dropped.
Let the dropout rate be (1 − p ) , p ∈ [0 , 1] , and let S (cid:48) ⊆
S denote the input variables that remain in the context S after the dropout operation.
Let us consider cases when r activation units remain after the dropout operation, i.e. | S (cid:48) | = r .
Then, the average interaction I ( s ) dropout ( i, j ) in these cases can be computed as follows.
[Equation]
The interaction only comprises the marginal reward from the inference patterns consisting of at most r ∼ B ( s, p ) variables, where B ( s, p ) is the binomial distribution with the sample number s and the sample rate p .
Since r = | S (cid:48) | ≤ s , we have
[Equation]
Equation (6, right) shows that the number of inference patterns usually signiﬁcantly decreases when we use dropout to remove s − r , r ∼ B ( s, p ) , activation units.
Please see Appendix D for the proof.
Experimental veriﬁcation 1: Inference patterns with more activation units are less likely to be sampled when we use dropout, thereby being more vulnerable to the dropout operation, according to Equation (6, left).
Inspired by this, we conducted experiments to explore the following two terms, (1) which component { I ( s ) ( i, j ) } ( s = 0 , ..., n − 2) took the main part of the overall interaction strength among all interaction components with different orders; (2) which interaction component was mainly penalized by dropout.
To this end, the strength of the s -order interaction component was averaged over images, i.e.
I ( s ) =
E image [ E ( i,j ) ∈ image ( | I ( s ) ( i, j ) | )] .
Speciﬁcally, when we analyzed the strength of interaction components, we selected the following orders, s = 0 .
1 n, 0 .
3 n, ..., 0 .
9 n .
Figure 2 shows curves of different interaction components 1 within VGG-11/19 (Simonyan & Zisserman, 2015) learned on the CIFAR10 (Krizhevsky & Hinton, 2009) dataset with and without the dropout operation.
We found that the interaction components with high orders took the main part of the interaction.
According to Equation (6, left), dropout mainly penalized the interaction component with high orders.
This phenomenon was also veriﬁed in Figure 2.
In this way, we proved that high-order interaction components were usually more suppressed than low-order interaction components.
In other words, inference patterns with more activation units (with higher orders) were more vulnerable to the dropout operation.
Experimental veriﬁcation 2: We also conducted experiments to illustrate how dropout suppressed the interaction modeled by DNNs, which was a veriﬁcation of Equation (6, right).
In experiments, we trained AlexNet
(Krizhevsky et al., 2012), and VGG-11/16/19 on CIFAR-10 with and without the dropout.
Figure 3(left) compares the strength of interactions 1 encoded by DNNs, which were learned with or without the dropout operation.
When we learned DNNs with dropout, we set the dropout rate as 0 . 5 .
Please see Appendix J.1 for experiments on different dropout rates.
We averaged the strength of interactions over images, i.e. I = E image
[ E ( i,j ) ∈ image ( | I ( i, j ) | )] , where I ( i, j ) was obtained according to Equation (2).
Note that accurately computing the interaction of two input variables was an NP-hard problem.
Thus, we applied a sampling-based method (Castro et al., 2009) to approximate the strength of interactions.
Furthermore, we conducted an experiment to explore the accuracy of the interactions approximated via the sampling-based method.
Please see Appendix M for details.
Appendix G shows technical details of (Castro et al., 2009).
We found that dropout could effectively suppress the strength of the interaction, which veriﬁed the above proof.
Property: The sampling process in dropout is the same as the computation in the Banzhaf value.
In this section, we aim to show that the sampling process in dropout (when the dropout rate is 0.5) is similar to the sampling in the computation of the Banzhaf value.
Just like the Shapley value, the Banzhaf value (Banzhaf III, 1964) is another typical metric to measure importance of each input variable in game theory.
Unlike the Shapley value, the Banzhaf value is computed under the assumption that each input variable independently participates in the game with the probability 0 .
The Banzhaf value is computed as ψ ( i | N )
= (cid:80) S ⊆ N \{ i } P Banzhaf ( S | N \{ i } ) (cid:2) f
( S ∪ { i } ) − f ( S ) (cid:3) , where
P Banzhaf ( S | N \{ i } ) = 0 .
5 n − 1 is the likelihood of S being sampled.
The form of the Banzhaf value is similar to that of the Shapley value in Equation (1), but the sampling weight of the Banzhaf value
P Banzhaf ( S | N \{ i } ) is different from P Shapley ( S | N \{ i } ) of the Shapley value.
For dropout with the dropout rate 0 . 5 , let n be the number of input variables, and let S be the units not dropped in N \{ i } .
Then, the likelihood of S not being dropped is given as P dropout ( S | N \{ i } ) =
5 | S | 0 .
5 n −| S |− 1 = P
Banzhaf ( S | N \{ i } ) .
In this way, dropout usually generates activation units S following P Banzhaf ( S | N \{ i } ) .
Therefore, the frequent inference patterns in the computation of the Banzhaf value are also frequently generated by dropout, thereby being reliably learned by the DNN.
This makes that f ( S ∪ { i } ) − f ( S ) in the computation of the Banzhaf value can be sophisticatedly modeled without lots of outlier values.
Thus, the dropout can be considered as a smooth factor in terms of the Banzhaf value.
Please see
Appendix F for the analysis of dropout with other dropout rates.
Grabisch & Roubens (1999) deﬁned the interaction based on the Banzhaf value as I Banzhaf ( i, j )
= (cid:80)
S ⊆ N \{ i,j } P Banzhaf (cid:0) S (cid:12)(cid:12) N \{ i, j } (cid:1)
∆ f ( S, i, j ) , just like Equation (2).
Thus, dropout can also be considered as a smooth factor in the computation of the Banzhaf interaction.
Figure 3(right) shows that the target interaction used in this study is closely related to the Banzhaf interaction, which potentially connects the target interaction based on the Shapley value to the dropout operation.
U TILITY OF D ROPOUT & I MPROVEMENT OF THE D ROPOUT U TILITY
Close relationship between the strength of interactions and over-ﬁtting.
In this section, we conducted various experiments to explore the relationship between the interaction strength and the over-ﬁtted samples.
We noticed that in the classiﬁcation task, the over-ﬁtted samples were usually
[Table]
[Table]
[Figure]
[Table]
[Figure]
outliers.
To this end, we assigned 5% training samples with random incorrect labels in MNIST (Lecun et al., 1998), CelebA(Liu et al., 2015), and Tiny ImageNet (Le & Yang, 2015).
When the DNN was well-trained on such training data with an almost zero training loss, then we took samples with incorrect labels as the over-ﬁtted samples.
We trained ResNet-34 (RN-34)
(He et al., 2016) for the classiﬁcation task using the Tiny ImageNet dataset and the CelebA dataset, and trained ResNet- (RN-44) for the classiﬁcation task using the MNIST dataset.
For each trained DNN, we divided input images into grids, and computed the average interaction strength between grids on the overﬁtted samples and ordinary samples, respectively.
As Table 1 shows, the over-ﬁtted samples usually contained stronger interactions 1 than ordinary samples.
Understanding of dropout.
We have proved that dropout can decrease the strength of interactions encoded in DNNs in Section 3.
Besides, previous paragraphs have shown that the over-ﬁtted samples usually encode more interactions by the DNN than other samples.
Therefore, we consider the utility of the dropout operation is to decrease interaction strength to reduce the signiﬁcance of over-ﬁtting.
Further improvement of the utility of dropout using the interaction loss.
Based on the above understanding, we develop the interaction loss as an improvement of the utility of dropout.
To this end, we apply this loss to learn DNNs, in order to boost the performance, as follows.
[Equation]
where λ > 0 is the weight of the interaction loss.
Let us focus on an intermediate-layer feature after the ReLU operation h ∈ R n as n activation units.
We aim to suppress interactions between any two units i, j ∈ N .
Thus, the interaction loss can be formulated as Loss interaction =
E i,j ∈ N,i (cid:54) =
j [ | I ( i, j ) | ] .
The baseline value in the computation of I ( i, j ) is set to 0, as is explained in the paragraph understanding
DNNs via game theory , Section 3.
Loss interaction =
[Equation]
However, it is extremely computational expensive to use the above interaction loss to train the DNN.
Therefore, we propose the following approximation of the interaction loss, which is implemented in a batch manner, instead of averaging all pairs of activation units.
Speciﬁcally, we sample disjoint subsets of units A, B (cid:36) N, A ∩ B = ∅ , instead of sampling two single units i, j , to compute for the interaction loss.
Here, we can regard A and B as batches of the sampled units { i } , { j } , respectively.
Accordingly, the context S is disjoint with A and B , i.e. S ⊆ N \
A \ B .
We can get the following approximation of Loss interaction .
Please see Appendix H for the proof.
[Equation]
where ∆ f ( S, A, B ) def = f ( S ∪ A ∪ B ) − f ( S ∪ A ) − f ( S ∪ B )+ f ( S ) , and α is a small positive constant.
In this paper, we set α =0 .
Figure 4(left) veriﬁes the proposed approximated interaction loss can successfully reduce the interaction deﬁned in Equation (2).
Advantages.
Compared with dropout, the interaction loss exhibits following advantages.
[Table] Advantage 1: The interaction loss enables people to explicitly control the penalty of the interaction strength by adjusting the weight λ in Equation (7).
The explicit control of the interaction strength is crucial, because it is important to make a careful balance between over-ﬁtting and under-ﬁtting during the learning of the DNN.
The weight λ needs to be carefully set, since it is usually difﬁcult to trade off between over-ﬁtting and under-ﬁtting in deep learning.
A large value of λ may lead to under-ﬁtting, while a small value of λ may increase the risk of over-ﬁtting.
In comparison, the dropout rate can control the strength of the dropout, when we consider the utility of the dropout as a penalty of interactions.
However, we ﬁnd that the interaction loss exhibits a stronger power to reduce the interaction than the dropout.
To this end, we have conducted experiments to compare the strength of the dropout with the strength of the interaction loss, which veriﬁed the conclusion on the strength of the dropout.
Please see Appendix J.1 for details.
Advantage 2: Due to the disharmony between dropout and the batch normalization (Li et al., 2019), people usually cannot apply dropout in the DNN with batch normalization.
In comparison, the interaction loss is compatible with batch normalization.
For example, we compute Loss interaction using a new track, which is different from the track of computing Loss classiﬁcation .
In other words, we do not update the parameter in batch normalization layers when we compute Loss interaction .
Please see
Appendix I for more discussions.
Furthermore, Appendix J.7 veriﬁes the disharmony in (Li et al., 2019), in which we added batch normalization layers to VGG-16/19, and compared the classiﬁcation accuracy with and without dropout.
Experiments: We trained DNNs for the classiﬁcation based on the CIFAR-10 dataset (Krizhevsky &
Hinton, 2009), the classiﬁcation based on the Tiny ImageNet dataset (Le & Yang, 2015) , and the face attribution estimation based on the CelebA dataset ( Liu et al., 2015).
We trained six types of DNNs, including AlexNet 2 (Krizhevsky et al., 2012), VGG-11/13/
Simonyan
Zisserman, 2015), and ResNet-18/34 (RN-18/34)
(He et al., 2016).
Note that we only randomly sampled 10% training data in the CIFAR-10 dataset and the Tiny ImageNet dataset, and randomly sampled training data form the CelebA dataset.
This was because a DNN usually showed a low signiﬁcance of over-ﬁtting, when it was learned on the complete dataset.
We only sampled a small proportion of training images to better show the over-ﬁtting problem.
In this experiment, when we trained DNNs with dropout, we set the dropout rate as 0 . 5 .
Please see Appendix J.1 for experiments on different dropout rates.
Furthermore, we trained the BERT (Devlin et al., 2018) using the SST-2 dataset (Socher et al., 2013) in order to verify the effectiveness of the interaction loss on linguistic data.
Please see Appendix J. for more details.
• Learning DNNs with the interaction loss.
We trained the DNNs with different weights of the interaction loss, and evaluated their testing accuracy.
Table 3 shows that the accuracy of DNNs trained with different values of λ and the accuracy of DNNs trained with dropout.
The accuracy of DNNs usually increased along with the λ at ﬁrst, and then decreased when the value of λ continued to increase.
This could be explained as the interaction loss alleviated the signiﬁcance of over-ﬁtting when the weight λ was small, and then caused under-ﬁtting when the weight λ was large.
Furthermore, the DNN trained using the interaction loss usually outperformed the DNN trained using the dropout, when the weight λ was properly selected.
This veriﬁed that the interaction loss would improve the utility of the dropout.
Figure 4 shows curves of the interaction and the accuracy of DNNs during the learning process with different weights λ of the interaction loss.
First, we found that the interaction decreased along with the increase of the weight λ , which veriﬁed the effectiveness of the interaction loss.
Second, when the weight λ was properly selected, the DNN trained with the interaction loss usually outperformed the DNN trained with dropout.
The interaction of the DNN trained using the interaction loss was much lower than the interaction of the DNN trained using the dropout.
In other words, the interaction loss was more effective to suppress the interaction and boost the performance than the dropout.
Besides, we found that the interaction loss mainly penalized interaction components with high orders, just like the dropout operation.
Please see Appendix J.6 for more results.
In particular, Table 3 shows that the performance of ResNets sometimes dropped when these DNNs were learned with dropout.
This phenomenon also veriﬁed the conclusion in (Li et al., 2019), i e. dropout was not compatible with batch normalization.
In comparison, learning with the interaction loss did not suffer from the use of batch normalization.
• Exploring the suitable position for the interaction loss and the dropout operation.
We learned the AlexNet and VGG-13/16 using the CIFAR-10 dataset, learned the VGG-19 using the Tiny ImageNet dataset, and learned the VGG-16 using the CelebA dataset.
For each DNN, we put the dropout operation and the interaction loss in the low convolutional layer (before the 3rd/5th convolutional layer of the AlexNet/VGGs) and the high fully-connected layer (before the 2nd fully-connected layer), respectively, and compared their performance.
As Table 2 shows, the interaction loss was better to be applied to the feature of a low convolutional layer.
In comparison, we did not ﬁnd out a clear principle about the suitable position for dropout.
Discussions about advantages of the interaction loss.
As is mentioned above, the interaction loss exhibits two advantages over dropout.
Table 3 shows that the interaction loss enabled people to explicitly control the interaction strength.
When the λ value increased, DNNs were controlled from over-ﬁtting to under-ﬁtting.
Both the over-ﬁtting problem and the under-ﬁtting problem would hurt the performance.
However, this explicit control could not be achieved by dropout.
Furthermore, unlike dropout, the interaction loss was compatible with batch normalization.
As Table 3 shows, the use of dropout decreased the classiﬁcation accuracy of ResNet-18 (RN-18) on the Tiny ImageNet dataset by 1.4%, and decreased the classiﬁcation accuracy of ResNet-18 (RN-18) on the CelebA dataset by 0.6%.
In comparison, the use of the interaction loss does not suffer from the batch normalization operation.
Appendix J.7 reports extended results beyond Table 3.
For fair comparison, the interaction loss and dropout were put in the same position of the DNN.
C ONCLUSION
In this paper we explain, model, and improve the utility of the dropout operation using game theory.
We prove that dropout can reduce the strength of interactions, thereby improving the performance of DNNs.
Experimental results have veriﬁed this conclusion.
Furthermore, based on the close relationship between the interaction and over-ﬁtting, we propose an interaction loss to directly penalize the strength of interactions, in order to improve the utility of dropout.
We ﬁnd that the interaction loss can reduce the interaction strength effectively and further boost the performance of DNNs
A F OUR D ESIRABLE P ROPERTIES OF THE S HAPLEY V ALUE
In this section, we mainly introduce the four desirable properties of the Shapley value mentioned in Section 3, including the linearity property, the dummy property, the symmetry property and the efﬁciency property (Ancona et al., 2019).
Linearity property: Given three games f , g , and h .
If the reward of the game f satisﬁes f ( S ) =
g ( S ) + h ( S ) , then the Shapley value of each player i ∈ N in the game f is the sum of Shapley values of the player i in the game g and h , i.e. φ f
( i | N ) =
φ g ( i | N ) + φ h ( i | N ) .
Dummy property: In a game f , If a player i satisﬁes ∀ S
⊆
N \ { i } , f
( S ∪ { i } ) =
f ( S ) + f ( { i } ) , then this player is deﬁned as the dummy player.
The dummy player i satisﬁes φ
( i | N ) =
f ( { i } ) − f ( ∅ ) , i.e. the dummy player has no interaction with other players in N .
Symmetry property:
Given two player i and j in a game f , if ∀ S ⊆ N \ { i, j } , f ( S ∪ { i } ) =
f ( S ∪ { j } ) , then φ ( i | N ) =
φ ( j | N ) .
Efﬁciency property: For a game f , the overall reward can be distributed to each player in the game.
I.e. (cid:80) i ∈ N φ ( i | N )
=
f ( N ) − f ( ∅ ) , where f ( ∅ ) is the obtained reward when no player participates in the game.
B T HE C ONSISTENCY OF T WO U NDERSTANDINGS TOWARDS THE I NTERACTION
This section aims to give a proof for the consistency of the following two understandings towards the interaction, as is mentioned in Section 3.
I ( i, j ) is deﬁned as the contribution increase of S ij when input variables i and j cooperate with each other w.r
t. the case when each input variable works individually (Grabisch & Roubens, 1999).
I ( i, j ) can also be understood as the difference between the Shapley value of input variable i when we mask another input variable
j w.r
t. the case when we do not mask j . •
According to the ﬁrst understanding, the interaction of input variables i and j can be computed as follows.
[Equation]
where ∆ f ( S, i, j ) =
f ( S ∪ { i, j } ) − f ( S ∪ { i } ) − f ( S ∪ { j } ) + f ( S )
According to the second understanding, the interaction of input variables i and j can be computed as follows.
[Equation]
where ∆ f ( S, i, j ) =
f ( S ∪ { i, j } ) − f ( S ∪ { i } ) − f ( S ∪ { j } ) + f ( S ) .
Equation (10) and (11) prove the consistency of the two understandings towards the interaction mentioned in Section 3.
C P ROOF OF E QUATION (4)
This section provides the detailed proof for Equation (4) in Section 3.
The s -order interaction can be decomposed as follows.
[Equation]
In Equation (19), J q ( i, j ) =
E T ⊆
N \{ i,j } , | T | =
q [ R T ( i, j )] denotes the average interaction between i and j given all potential inference patterns T ∪ { i, j } with a ﬁxed inference pattern size | T | = q .
D P ROOF OF E QUATION (5) AND E QUATION (6, LEFT )
This section gives the detailed proof for Equation (5) and Equation (6, left) in Section 3.
We ﬁrst prove Equation (5).
The interaction of i and j with dropout, I ( s ) dropout ( i, j ) , can be computed as
I
[Equation]
similar to Equation
[Equation]
15) ∼ Equation
[Equation] [Equation]
[Equation]
% similar to the proof of Equation
[Equation]
[Equation]
Thus, Equation (5) holds.
In order to prove Equation (6, left), we aim to prove ∀ r ≤ s and 1 ≤ q ≤ r − 1 , we have 1 ≥ Γ ( q ) ( i,j | r ) Γ ( q ) ( i,j | s ) ≥ Γ ( q +1)
( i,j | r ) Γ ( q +1)
( i,j | s ) ≥ 0 .
Next, we prove the three inequalities step by step.
First, Γ
[Equation]
Next, Γ ( q )
[Equation]
i
[Equation]
Then, Γ ( q +1) ( i,j | r ) Γ ( q +1) ( i,j | s ) =
r − q s − q ≥ 0 .
Thus, we prove Equation (6, left).
E T HE C OMPUTATION OF R T ( i, j )
This section gives a recursive formulation of R T ( i, j ) mentioned in Section 3.
R T ( i, j ) measures the marginal reward of the inference pattern T ∪ { i, j } , where all interaction effects from smaller inference patterns T (cid:48) (cid:36) T are removed from R T ( i, j ) .
In this way, R T ( i, j ) can be computed as follows.
[Equation]
Next, we prove the correctness of the formulation of R T ( i, j ) .
In other words, we aim to prove
I
[Equation]
S R T
[Equation]
i
[Equation]
j )
[Equation]
in Equation (4).
[Equation]
the inference
[Equation]
Thus, we prove the correctness of the recursive formulation of R T ( i, j ) .
F R ELATIONSHIP BETWEEN B ANZHAF V ALUES AND D ROPOUT
In this paper, we mainly focus on the dropout operation with the dropout rate 0 .
However, in practice, the dropout rate can be different from 0 .
In order to reveal a general relationship between dropout and the Banzhaf value, we extend the deﬁnition of the Banzhaf value to the Bernoulli- p Banzhaf value .
The Bernoulli- p Banzhaf value is computed under the assumption that each player independently participates in the game with a probability p following a ditribution of Bernoulli ( p ) .
Therefore, the likelihood of the subset S being sampled is computed as follows.
[Equation]
Thus, the Bernoulli- p Banzhaf value can be computed as follows.
[Equation]
The sampling process of dropout is exactly the same as the sampling in the computation of the Bernoulli- p Banzhaf value.
Let n be the number of units and S
be units that are not dropped in
N \ { i } .
If the dropout rate is (1 − p ) , then the probability distribution of S can be expressed as follows.
[Equation]
In this way, we reveal the relationship between dropout and the Bernoulli- p Banzhaf value, i.e. dropout usually generates a set of activation units S following the distribution of P p -Banzhaf ( S | p ) .
G T HE C OMPUTATION OF THE S TRENGTH OF I NTERACTIONS
In this section, we introduce implementation details to compute the strength of interactions between input variables modeled by the DNN mentioned in Section 3.
Let i and j denote two arbitrary input variables, and N represents the set of all input variables.
We use f to represent the DNN.
Note that the computation based on Equation (2) is NP-hard.
Thus, we apply a sampling-based method (Castro et al., 2009) to approximate the interaction between i and j , as follows.
[Equation]
=
I ( i, j
[Equation]
where ∆ f ( S, i, j ) =
f ( S ∪ { i, j } ) − f ( S ∪ { i } ) − f ( S ∪ { j } ) + f ( S ) , P ( S | N \ { i, j } ) is the likelihood of the subset S being sampled.
According to above equation, we ﬁrst uniformly sample an integer s from the range [0 , n − 2] , then sample the subset S , | S |
= s from N \ { i, j } .
Thus, we have P ( S | N \ { i, j } ) =
1 n −1 ( n − 2 s ) .
I (cid:48)
( i, j ) can be approximated by the sampling-based method, as follows.
[Equation]
According to Section 3, the strength of the interaction I (cid:48) ( i, j ) is measured by the absolute value | I (cid:48)
( i, j ) | .
In this way, the overall interaction encoded by the DNN can be approximated as follows.
[Equation]
In implementation, we randomly sampled 10 images for the computation.
We divided each image into 16 × 16 grids in experiments to further boost the computation efﬁciency.
We consider each grid as an input variable, and computed the interaction between two grids.
We randomly sampled 80 pairs of grids for computation of the interaction.
For each pair of grids, we sampled s in Equation (36) for 500 times.
For each value of s , we sampled one subset S of all input variables to approximate the interaction.
We averaged the strength of interaction over all sampled pairs of grids as the result of the interaction.
Besides, for fair comparison of interactions among different DNNs, we normalized the value of interaction using the range of output score of DNNs.
I.e. I norm =
I/Y , where Y was the normalization term that reﬂected the range of the output score of the DNN.
If the DNN was learned for multi-category classiﬁcation, then Y = | E x ∈ X
[ f ( x ) y ∗ − E y (cid:54) =
y ∗ [ f ( x ) y ]] | , where x was sampled from the training set X , y ∗ was the ground-truth label of x , and f ( x ) y denoted the value of the y -th dimension in the output score f ( x ) .
If the DNN was learned for binary classiﬁcation and the output score was a scalar, then Y = | E x ∈ X positive [ f ( x )] − E x ∈ X negative [ f ( x )] | , where X positive and X negative represent the set of positive samples and the set of negative samples, respectively.
H P ROOF OF THE A PPROXIMATION OF THE I NTERACTION L OSS
This section gives the proof for the trustworthiness of the approximated interaction loss Loss (cid:48) interaction in Equation (9).
The relationship between Equation (8) and Equation (9) is proved as follows.
According to Equation (3), Equation (8), and Equation (35), the interaction loss can be approximated by sampling-based method, i.e.
Loss
[Equation]
Let µ denote the strength of the interaction
I ( i, j ) , i.e. | I ( i, j ) | = µ ≥ 0 .
We assume that ∆ f ( S, i, j ) follows the Gaussian distribution, i.e. ∆ f ( S, i, j ) ∼ Gaussian
(ˆ µ, σ 2 ) .
ˆ µ =
I ( i, j ) .
In this way, given a speciﬁc pair of units i and j , we have
[Equation]
where C = 1 √ 2 πσ is a constant value.
Thus, given the value of ∆ f ( S, i, j ) , the likelihood of
I ( i, j ) =
+ µ is given as
[Equation]
To simplify the story, let us ﬁrst consider some easy cases.
During the learning process, if
P ( µ | ∆ f ( S, i, j ))
≈ 1 , then it indicates I ( i, j ) = µ > 0 .
In this case, we need to decrease ∆ f
( S, i, j ) to decrease | I ( i, j ) | .
To this end, we need to set the gradient of ∆ f ( S, i, j ) to be 1 .
In comparison, if P ( µ | ∆ f ( S, i, j )) ≈ 0 , then it indicates I ( i, j ) = − µ < 0 , and we need to increase ∆ f ( S, i, j ) to reduce | I ( i, j ) | .
To this end, we need to set the gradient of ∆ f ( S, i, j ) to be − 1 .
In this way, for other cases, we can consider the probability P ( µ | ∆ f ( S, i, j ) as the probability of setting the gradient of ∆ f ( S, i, j ) to 1, and consider 1 − P ( µ | ∆ f ( S, i, j ) as the probability of setting the gradient of ∆ f ( S, i, j ) to -1.
Therefore, the expectation of the gradient can be computed as P ( µ | ∆ f ( S, i, j ) · 1 + (1 − P ( µ | ∆ f ( S, i, j )) · ( − 1) =
2 P ( µ | ∆ f ( S, i, j ) − 1 .
Furthermore, the gradient of ∆ f ( S, i, j ) can be approximated as
[Equation]
% We assume that ∆ f ( S, i, j ) is close to zero
[Equation]
Thus, the interaction loss can be rewritten as follows.
[Equation] [Equation]
where µ 2 α 2 n 2 σ 2 is a positive constant.
Therefore, we can use Equation (9) to approximate the interaction loss.
I D ETAILS OF THE I NTERACTION AND THE I NTERACTION L OSS
Implementation of the interaction loss: In this section, we aim to introduce the implementation details of the interaction loss.
Since the computation of the interaction loss in Equation (8) is expensive, we propose Equation (9) as an approximation of the interaction loss.
As the Equation (9) shows, we need to sample subsets of units A , B , and S for the computation of the interaction loss.
For A and B , we randomly select 5% units from the intermediate-layer feature, respectively, and A ∩ B = ∅ .
As for the subset S , we sampled units from N \
A \ B .
Speciﬁcally, we ﬁrst uniformly select a sampling rate from the range [0 , 1] , and sample the subset
S with this sampling rate.
When we apply the interaction loss to DNNs with batch normalization layers (such as ResNets), we need to compute the interaction loss using a different track with the computation of the classiﬁcation loss.
I.e. when we compute the classiﬁcation loss, parameters in the batch normalization layers are normally updated.
In comparison, when we compute the interaction loss, we directly use the parameters in batch normalization layers without updating their values.
In this way, we can apply the interaction loss to DNNs without causing disharmony with batch normalization layers.
The selection of the output score: In this paper, we consider a DNN f as a game, and take the output score of the DNN as the reward score.
Since the reward score is usually a scalar value, for the DNN with a vectorized output, we usually take the output score before the softmax layer corresponding to the true class as the reward score.
Speciﬁcally, for DNNs with batch normalization layers, such as ResNets, we take the output score after the softmax layer corresponding to the true class as the reward score.
Details of the datasets and training settings in Section 4: In the ﬁrst experiment of Section 4, we used datasets with incorrect labels to reveal the relationship between the interaction and the overﬁtting of DNNs.
For the Tiny ImageNet dataset, we used all images from the ﬁrst ten categories for the classiﬁcation task, and each category contained 500 images.
For the MNIST dataset, we randomly selected 10% images for training the DNN.
In this case, each category contains 600 images.
For the CelebA dataset, we randomly selected 1% training samples for the estimation of the gender.
In order to build datasets with the over-ﬁtted samples, we randomly selected 5% samples of the training samples, and replaced their labels with a randomly selected incorrect labels.
In this case, when the DNN was well-learned and had a training loss close to zero, samples with incorrect labels could be considered as the over-ﬁtted samples.
In comparison, samples with correct labels were ordinary samples.
Besides, in this experiment, the strength of interaction was normalized by the normalization term of each individual image.
[Table]
In the following experiment of Section 4, we explored the utility of the interaction loss.
We trained
DNNs using the CIFAR-10 dataset, the Tiny ImageNet dataset and the CelebA dataset.
In order to make the DNN suffer from the over-ﬁtting problem, for the CIFAR-10 dataset and the Tiny ImageNet dataset, we randomly generated 10% training samples to train the DNN.
More speciﬁcally, we only used the ﬁrst 10 categories for the classiﬁcation task.
Besides, for the CelebA dataset, we used 1% training samples to train the DNN for gender estimation.
In these experiments, the strength of interactions was normalized by the normalization term over all training samples.
For fair comparison, the interaction loss and dropout were put in the same position of the DNN.
Speciﬁcally, for AlexNet, we put the interaction loss or dropout before the third convolutional layer.
For VGGs, we put the interaction loss or dropout behind the second block, i.e. behind the second
ReLU layer of VGG-11 and behind the forth ReLU layer of VGG-13/16/19.
We put the interaction loss or dropout behind the ﬁrst block for ResNet-18 (RN-18), and behind the second block for
ResNet-34 (RN-34).
When we put the interaction loss or dropout at a high layer, we always put the interaction loss or dropout behind the ﬁrst fully-connected layer.
The baseline value: In this section, we introduce details about the baseline value, which was proposed by (Ancona et al., 2019).
Ancona et al.
(2019) proposed a polynomial-time algorithm for
Shapley values approximation to explain deep neural networks.
When we estimate the attribution of input variables and the interaction between input variables, we usually use a baseline value to simulate the absence of variables.
Many methods of estimating attributions require the deﬁnition of a baseline value as the anchor value corresponding to the absence of the variable.
Setting an input variable to the baseline value indicates this input variable is toggled off.
Let us focus on the task of object classiﬁcation.
We follow the settings in Ancona et al.
If we aim to measure the interactions between input pixels, the baseline value is deﬁned as the average pixel value among all samples.
When we measure the interaction between activation units before the dropout operation, the baseline value is deﬁned as zero.
J M ORE R ESULTS J. T HE C OMPARISON OF DROPOUT USING DIFFERENT DROPOUT RATES .
In this section, we conducted new experiments to further study the performance of different dropout rates.
We trained AlexNet and VGG-11 using the CIFAR-10 dataset.
We set the dropout rate as 0 . 1 , 0 . 3 , 0 . 5 , 0 .
9 , respectively, and applied the dropout operation before the 3rd convolutional layers of the AlexNet/VGG-11, which was the same as in experiments corresponding to Table 3.
Table 4 shows the classiﬁcation accuracy of different DNNs, as well as the strength of interactions modeled by DNNs.
We found that the dropout rate of 0.9 usually penalized more interactions than the dropout operations with other dropout rates.
Nevertheless, DNNs trained using the interaction loss still decrease more interactions than DNNs trained with the dropout in most cases.
In addition, the DNN trained with a proper setting of λ outperformed all DNNs with the dropout.
J. E XPERIMENTS ON LINGUISTIC DATA
In this section, we applied our interaction loss to another type of data, i.e. the linguistic data.
We trained the BERT model (Devlin et al., 2018) on the SST-2 (Socher et al., 2013) dataset for the binary sentiment classiﬁcation task.
Just like experiments in the CIFAR-10 dataset and the Tiny
ImageNet dataset, we randomly sampled 10% training samples.
This was because the DNN showed a low over-ﬁtting level, when it was trained using the complete dataset.
We set the dropout rate as
[Table]
[Table] [Table]
[Figure]
0 . 5 , and applied the dropout/interaction loss after embedding layers of the BERT.
We designed two baselines.
The ﬁrst baseline was the DNN trained with the traditional dropout.
The second baseline was the DNN trained without the dropout or the interaction loss.
Table 5 shows that the interaction loss successfully decreased the interaction modeled by the BERT, and outperformed two baselines.
Therefore, the interaction loss was a more effective method to control the interaction, and boost the performance of DNNs than the dropout.
J. T HE COMPARISON BETWEEN DIFFERENT POSITIONS TO APPLY THE DROPOUT
In order to verify that the dropout is applicable to convolutional layers, in this section, we conducted an experiment to apply the dropout on various fully-connected layers and the convolutional layer.
Speciﬁcally, we applied the dropout before the 1st/2nd/3rd fully-connected layers and before the 3rd convolutional layer of the AlexNet and VGG-11.
In this experiment, we set the dropout rate as 0 . 5 .
Table 6 shows the accuracy of DNNs with different positions, to which the dropout was applied.
We found that DNNs trained with dropout on the convolutional layer had similar performance with DNNs trained with dropout on fully-connected layers.
Thus, dropout was applicable to convolutional layers
. J. T HE COMPARISON OF COMPUTATIONAL TIME BETWEEN DNN S TRAINED USING DROPOUT AND L OSS INTERACTION
In this section, we compared the time of learning DNNs with dropout and the time of learning
DNNs using the interaction loss.
We trained AlexNet and VGG-11 using the CIFAR-10 dataset on a GPU of GeForce GTX-1080Ti.
The experimental setting was the same as that in experiments corresponding to Table 3 and Figure 4.
As Table 7 shows, our interactions loss increased the time cost, but it was still of speciﬁc value considering its good performance and the harmony with the batch-normalization operation.
[Figure]
The interaction loss mainly penalized the interaction component with high order, which made the interaction component with low order became the main part of the interaction.
All interaction were computed between activation units before the layer that we used interaction loss.
∆ I ( s ) =
I ( s ) w/Loss interaction − I ( s ) w/oLoss interaction denotes the change of interaction strength.
[Figure]
[Table] [Figure] [Table] [Table]
[Table] [Table] [Table] VGG
[Table] [Table] [Table] accuracy due
J. T HE COMPARISON BETWEEN TAKING THE OUTPUT AS THE SCORE f , AND TAKING THE LOSS VALUE AS THE SCORE f
In this section, we aim to measure the interaction when we used the loss value of the DNN as the score f .
We trained the VGG-11 using the CIFAR-10 dataset.
Then, we compared the interaction when we set f = Loss with the interaction when we set f as the output score of the DNN.
As
Figure 5 shows, the interaction with f = Loss had similar behaviors with the interaction with the network output as f .
Therefore, the choice of the score f did not affect the conclusion that the interaction loss could effectively decrease interactions encoded in the DNN.
J. I
L
In this section, we aimed to analyze which interaction component was more vulnerable to the interaction loss.
We trained AlexNet and VGG-11 using the CIFAR-10 dataset.
The weight λ of the interaction loss was set as 50 .
We computed the interaction components with different orders by setting s = 0 .
1 n, 0 .
3 n, ..., 0 .
9 n .
As Figure 6 shows, the interaction component with higher order was more suppressed than interaction components with other orders by the interaction loss, which was similar to the utility of dropout.
J. T HE COMPARISON BETWEEN DROPOUT AND L OSS INTERACTION WHEN USING WITH BATCH NORMALIZATION
This section investigates the compatibility between dropout with batch normalization, and investigates the compatibility between the interaction loss with batch normalization.
Preliminary results have been shown in Table 3.
We added a batch normalization layer following each convolutional layer in VGG-16 and VGG-19, and compared the classiﬁcation accuracy after putting dropout or the interaction loss in the DNN.
For fair comparison, either the interaction loss or dropout was added following the fourth ReLU layer of VGG-16/19.
Table 8 shows that the disharmony between dropout and batch normalization would hurt the performance of VGG-16 and VGG-19 more seriously.
However, the use of the interaction loss would boost the classiﬁcation accuracy.
Speciﬁcally, the use of dropout decreased the classiﬁcation accuracy of VGG-16 and VGG-19 by 9.4% and 9.6%, respectively, on the Tiny ImageNet dataset.
Whereas, the use of the interaction loss boosted the classiﬁcation accuracy of VGG-16 and VGG-19 by 2.4% and 0.8%, respectively.
This phenomenon veriﬁed that dropout may hurt the performance of VGG-16/19 with batch normalization.
The interaction loss did not suffer from the use of batch normalization. [Figure] [Figure]
K D ETAILS FOR V ISUALIZING I NTERACTIONS
This section provides the implementation details to visualize interactions modeled by DNNs.
We used the CelebA dataset for visualization.
There were 224 × 224 pixels in images of CelebA, which made it expensive to compute and visualize the interaction between any pairs of pixels.
Thus, we divided the image into 16 × 16 grids, each grid contained 14 × 14 pixels.
We considered these grids as input variables, and computed the interaction between any two adjacent grids.
We sampled 100 times to approximate the interaction between two grids.
For a speciﬁc grid, we averaged the interactions between the grid with its adjacent grids, and used the strength of the averaged interaction as the attribution value of this grid.
L C OMPARISON BETWEEN OUR INTERACTION AND THE I NTEGRAL H ESSIANS
The Hessian matrix is another typical way to deﬁne interactions.
However, the standard way of using the Hessian matrix to deﬁne interactions is not to directly use the raw Hessian value ∇ 2 i,j L , but to use the Integrated-Hessian value that has been proposed by (Janizek et al., 2020).
Janizek et al.
(2020) proposed the Integrated-Hessian value to measure interactions, based on Integral Gradient (Sundararajan et al., 2017).
Integral Gradient is used to compute an importance value for each input
variable w.r.t.
the DNN.
Given an input vector x ∈ R n , Integrated Hessians measures the interaction between input variables (dimensions) x i and x j as the numerical impact of x j on the importance of x i .
To this end, Janizek et al.
(2020) used Integral Gradient to compute Integrated
Hessians.
More speciﬁcally, given a DNN f and two input variables x i and x j , the interaction between x i and x j is measured as follows.
[Equation]
where x (cid:48) is a baseline image.
Theoretically, the Integrated-Hessian value is not equal to our interaction metric, and has different theoretic foundations with our interaction metric.
Our interaction metric deﬁnes the interaction based on the Shapley value (a game-theoretic importance), In comparison, the importance value used to deﬁne the Integrated-Hessian value is computed using the Integral Gradient.
Strictly speaking, the
Integral-Gradient importance is highly related but not equal to the Shapley value (importance).
The
Shapley value is the unique importance that satisﬁes the linearity property, the dummy property, the symmetry property, and the efﬁciency property (Shapley, 1953; Weber, 1988; Ancona et al., 2019).
Compared to the raw Hessian value ∇ 2 i,j L , the Integrated-Hessian value integrates over multiple high-order gradients to generate the ﬁnal Integrated-Hessian value.
Unlike our interaction metric, the computation of Integrated-Hessian values requires the DNN to use the SoftPlus operation to replace the ReLU operation.
The computation of our interaction metric does not have such a requirement.
To this end, we conducted an experiment to show the difference between the raw Hessian value ∇ 2 i,j L , the Integrated-Hessian value (Janizek et al., 2020), and our interaction metric.
We implemented the Integrated Hessian using the code released by suinleelab (2020).
We directly used the code and the DNN architecture provided by suinleelab (2020) to train the DNN on the MNIST dataset, where all ReLU layers in the DNN were replaced with the SoftPlus operation (this was required by (Janizek et al., 2020) for the computation of the Integrated-Hessian value).
Given the trained DNN, we computed the strength of interactions based on the raw Hessian value, the
Integrated-Hessian value, and the interaction value deﬁned in this paper, respectively.
As Tableshows, these metrics generated fully different interaction values, because these interaction metrics deﬁned different types of interactions from fully different perspectives.
M E VALUATION OF THE ACCURACY OF THE APPROXIMATED INTERACTION
In this section, we conducted an experiment to explore the accuracy of the interactions approximated via the sampling-based method (Castro et al., 2009).
More speciﬁcally, given a certain image and certain sampling number, we measured the instability of the average interaction strength
I image =
E ( i,j ) ∈ image ( | I ( i, j ) | ) , when we repeatedly computed the interaction for multiple times considering the randomness in the sampling process.
This instability value indicated whether we obtained similar interactions with different sampling states.
The instability w.r.t.
the interaction
I image is computed as
[Equation]
, where I ( u ) image denotes the interaction result computed in the u -th time.
Then, we computed the average instability value over interactions of 100 pairs of input variables in 10 images to measure the accuracy of the approximated interaction.
A high instability indicated a low accuracy.
We used the trained AlexNet and VGG-11 to compute the instability of interactions.
We computed the instability with different sampling numbers.
Figure 7 shows that the instability of the approximated interaction decreased along with the increase of the sampling number.
We found that when the sampling number was large than 500 , the instability of the approximated interaction was less than 0 . 1 , which indicated that the approximated interaction was stable and accurate enough.
Thus, we set the sampling number as 500 in all other experiments in this paper.
Meanwhile, computing the interaction using such a sampling number was far more efﬁcient than the original NP-hard computation of the interaction.