On Dropout, Overfitting, and Interaction Effects in Deep Neural Networks
Authors: Ben Lengerich
Eric Xing
Rich Caruana
Venue:  (2021)
Keywords: Dropout
Interaction Effects
Neural Networks
Functional ANOVA

Abstract: We examine Dropout through the perspective of interactions. Given $N$ variables, there are $\mathcal{O}(N^2)$ possible pairwise interactions, $\mathcal{O}(N^3)$ possible 3-way interactions, i.e. $\mathcal{O}(N^k)$ possible interactions of $k$ variables. Conversely, the probability of an interaction of $k$ variables surviving Dropout at rate $p$ is $\mathcal{O}((1-p)^k)$. In this paper, we show that these rates cancel, and as a result, Dropout selectively regularizes against learning higher-order interactions. We prove this new perspective analytically for Input Dropout and empirically for Activation Dropout. This perspective on Dropout has several practical implications: (1) higher Dropout rates should be used when we need stronger regularization against spurious high-order interactions, (2) caution must be used when interpreting Dropout-based feature saliency measures, and (3) networks trained with Input Dropout are biased estimators, even with infinite data. We also compare Dropout to regularization via weight decay and early stopping and find that it is difficult to obtain the same regularization against high-order interactions with these methods.

Under review as a conference paper at ICLR 2021
ON DROPOUT, OVERFITTING, AND INTERACTION EF- FECTS IN DEEP NEURAL NETWORKS
Anonymous authors Paper under double-bling review
ABSTRACT
We examine Dropout through the perspective of interactions. Given variables, there O(N2) possible pairwise interactions, O(N3) possible 3- 3-way interac- tions, i.c ((N)) possible interactions of h variables. Conversely, the probability of interaction of l variables surviving Dropout at rate p is ((1 p)*). In this paper, we show that these rates cancel, and as a result, Dropout selectively regularizes against learning higher-order interactions. We prove this new perspec- tive analytically for Input Dropout and empirically for Activation Dropout. This perspective on Dropout has several practical implications: (1) higher Dropout rates should be used when we need stronger regularization against spurious high-order interactions, (2) caution must be used when interpreting Dropout- based feature saliency measures, and (3) networks trained with Input Dropout are biased esti- mators, even with infinite data. We also compare Dropout to regularization via weight decay and early stopping and find that it is difficult obtain the same regularization against high-order interactions with these methods.
INTRODUCTION
We examine Dropout through the perspective of interactions: learned effects that require multiple input variables. Given N variables, there are O(N2) possible pairwise interactions, O(N3) possible 3-way interactions, etc. We show that Dropout contributes regularization effect which helps neural networks (NNs) explore simpler functions of lower-order interactions before considering functions of higher-order interactions. Dropout imposes this regularization by reducing the effective learning rate of interaction effects according the number of variables in the interaction effect. As result, Dropout encourages models to learn simpler functions of lower-order additive components. This understanding of Dropout has implications for choosing Dropout rates: higher Dropout rates should be used when we need stronger regularization against spurious high-order interactions. This perspective also issues caution against using Dropout measure term saliency because Dropout regularizes against terms for high-order interactions. Finally, this view of Dropout as regularizer of interaction effects provides insight into the varying effectiveness of Dropout for different architectures and data sets. We also compare Dropout to regularization via weight decay and early stopping and find that it difficult obtaín the same regularization effect for high-order interactions with these methods.
Why Interaction Effects? When it was introduced, Dropout was motivated to prevent "complex co-adaptations which feature detector is only helpful in the context of several other specific feature detectors" (Hinton et al.) 2012) Srivastava et al., 2014).. Because most "complex "o-adaptations" are interaction effects, we examine Dropout under the lens of interaction. This perspective valuable because (1) modern NNs have so many weights that understanding networks by looking at their weights is infeasible, but interactions are far more tractable because interaction effects live in function space, not weight space, (2) the decomposition that we use calculate interaction effects has convenient properties such as identifiability, and (3) this perspective has practical implications on choosing Dropout rates for NN systems. To preview the experimental results, when NNs trained on data that has no interactions, the optimal Dropout rate high, but when NNs are trained on datasets which have important 2nd and 3rd order interactions, the optimal Dropout rate is 0.

---

Under review as a conference paper at ICLR 2021
2 RELATED WORK
Although Hinton et al proposed Dropout to prevent spurious co-adaptation (i.e., spurious interactions), many questions remain. For example: Is the expectation of the output of NN trained with Dropout the same as for NN trained without Dropout? Does Dropout change the trajectory of learning during optimization even in the asymptotic limit of infinite training data? Should Dropout be used at run-time when querying NN to see what it has learned? These questions are important because Dropout has been used as method for Bayesian uncertainty (Gal & Ghahramani] 2016 Gal et al.) 2017, Chang et al. 2017b a). which implicitly assume that Dropout does not bias the model's output. The use of Dropout as a tool for uncertainty quantification has been questioned due to its failure to separate aleotoric and epistemic sources of uncertainty (Osband (016) (i.e., the uncertainty does not decrease even as more data is gathered). In this paper we ask a separate yet related question: Does Dropout treat all parts of function space equivalently?
Significant work has focused on the effect of Dropout as a weight regularizer (Baldi & Sadowski 2013 Warde-Farley et al.) 2013 Cavazza et al. 2018; Mianjy al. 2018 Zumino et al. (2018).
including its properties of structured shrinkage (Nlismick et al.) 2018) or adaptive regulanization
(Wager al. 2013). However, weight regularization is of limited utility for modern-scale NNs, and
can produce counter-intuitive results such as negative regularization (Helmbold & Long 2017).
Instead of focusing on the influence of Dropout on parameters, take nonparametric view of NNs as function approximators. Thus, our work is similar in spirit to Wan et al. (2013), which showed linear relationship between keep probability and the Rademacher complexity of the model class. Our investigation finds that Dropout preferentially targets high-order interaction effects, resulting in models that generalize better by down-weighting high-order interaction effects that are typically spurious or difficult to learn correctly from limited training data.
3 PRELIMINARIES
Multiplicative terms like X2 are often used to encode "interaction effects". They are, however, only pure interaction effects if X, and X2 are uncorrelated and have mean zero. When the two variables are correlated, some portion of the variance in the outcome X2 can be explained by main effects of each individual variable. Note that correlation between two input variables does not imply an interaction effect on the outcome, and an interaction effect of two input variables on the outcome does not imply correlation between the variables.
In this paper, we use the concept of pure interaction effects from Lengerich et al. (2020): a pure interaction effect is variance explained by a group of variables that cannot explained by any subset of u. This definition is equivalent to the fANOVA decomposition of the overall function F;
Given a density wXX) and F C L (R" the family of allowable functions for variable set u, the
weighted FANOVA (Hooker 2004
2007
Cuevas et al. 2004) decomposition of F(X) is:
(fu( S [4]
arg min
/ Σ
Bx(X.) - -F(X) (X)dxx [CId]
(1a)
(gu (gue heet
where [d] indicates the power set of d features, such that
V u,
V go:
(Ib)
i.e., each member fu is orthogonal to the members which operate on any subset of u. An interaction effect fa is of order A if (w] = k. Given N variables in X, there are O(N) possible effects of individual variables, O(N2) possible pairwise interactions, O(N3) possible 3-way interactions, i.e. O(N+) possible interactions of order k.
The FANOVA decomposition provides a unique decomposition for a given data distribution; thus, pure interaction effects can only be defined by simultaneously defining a data distribution. An example of this interplay between the data distribution and the interaction definition is shown in Figure B.2 As Lengerich al. (2000) describe. the correct distribution to is the data tata-eenn distribution p(z). In studies on real data, estimating p(x) is one of the central challenges of machine learning; for this paper, use simulation data for which we know p(x))
2

---

Under review as a conference paper at ICLR 2021
4 ANALYSIS: DROPOUT REGULARIZES INTERACTION EFFECTS
Dropout operates by probabilistically setting values to zero (i.e. multiplying by Bernuulli mask). For clarity, we call this "Input Dropout" if the perturbed values are input variables, and "Activation Dropout" if the perturbed values are activations of hidden nodes.
First, we show that Input Dropout is equivalent to replacing the training dataset with samples drawn from a perturbed distribution:
Theorem 1. Let [ = Ene(d) (u)) with E
Then Input Dropout rate p produces
E|V
O MJ
Σ
(2)
wEld]
where M, vector of d. Bernoulli random variables, is the Dropout mask and @ is element-wise multiplication.
This theorem shows that Input Dropout shrinks the conditional expectation of Y|X @ M toward the expectation of Y. Furthermore, Input Dropout preferentially targets high-order interactions: the scaling factor shrinks exponentially with ju]. Implications of this theorem are:
1. The distribution of training data is different for different levels of Input Dropout, so even NNs trained for more epochs or with infinite sample size cannot overcome the bias introduced by Dropout and will converge to different optima based on the Input Dropout level. This is unlike or L2 regularization which can overcome by increasing the size of the training set.
2. Input Dropout affects higher-order interactions more than lower-order interactions, biasing the prediction of any model (regardless of whether not the model was originally trained with Input Dropout).
3. Input Dropout on the data distribution, not the model, so has the same effect on learning regardless of the downstream net architecture.
Next, we show that Input Dropout shrinks gradients by down-weighting the gradient scale, with shrinkage factor exponential in effect order:
Let vu(ss)) be the gradiens update for an interaction effect u. The expected concordance between the gradient with Input Dropout at rate p and the gradient without Input Dropout is:
Theorem 2.
Vu(X,,, Y) Va @ M,Y) (1- p) Va(X,, Vu,(,.,) Y)I
EM
(3)
This theorem shows that Input Dropout shrinks the gradient update corresponding each effect by an effective learning rate (k) = (1 - p)* which decays exponentially in the interaction order k. Implications of this theorem are:
1. The decreased learning rate persists throughout all training. Therefore, the disruption in the gradient will interplay with other mechanisms of optimizers (e.g. momentum).
2. The impact of training with Input Dropout could be undone by re-weighting gradients.
4.1 SYMMETRY BETWEEN DROPOUT STRENGTH AND NUMBER OF INTERACTION EFFECTS
From N input features, there are (N) distinct k-order interaction effects which could be estimated. Without any regularization, high-order interactions would dominate. However, as shown above, the effective learning rate of k-order interactions decays exponentially with A. This is symmetry with (7) (which is N for all k and ~ N for small k). As shown in Fig the exponential growth of the hypothesis space H4 with interaction order is balanced by the exponential decay the effective learning rate, providing strong regularization against high-order interaction effects.
3

---

Under review as a conference paper at ICLR 2021
5250
6375
(183944
0.25
200
0.50
Interaction Order
Interaction Order
(a) Full space.
(b) First four orders.
Figure 1: The growing hypothesis space of potential interaction effects is balanced against the effective learning rate imposed by Dropout. In this figure, we plot the product of the effective learning rate (ry(A) (k)) and the number of potential interaction effects of order k (H+)) for a variety of Dropout rates p. In a we plot these values on a log scale for the entire range of potential interaction orders for an input of 25 features. In b we plot up order 4 on linear scale.
EXPERIMENTS
As shown above, Dropout does not simply add unbiased noise to the gradient updates; instead, Dropout exerts an unceasing force throughout the optimization process. This means that Dropout changes the steady-state optima of the model. We examine this behavior empirically for both Input Dropout and Activation Dropout by decomposing the effect estimated by NN with the FANOVA. Anonymized code to reproduce all figures is available at
5.1 MEASURING INTERACTION EFFECTS IN TRAINED NEURAL NETWORKS
The function FX) estimated by a NN can be decomposed as: F(X) = fa by the
FANOVA (Eq. [b). We will use this decomposition to measure the interaction effects implicit in the NN. To approximate this decomposition, we repeatedly apply model distillation (Hinton et al.
2015|Bucilus Buciluá et al.) (006)) using the XGBoost software package (Chen & Guestrin 2016). First, we train boosted stumps (XGBoost with max depth of I) to approximate the output of the NN using only main effects of individual variables. We successively increase the maximum depth of trees (corresponding to an increase in the maximum order of interaction effect). By training on the residuals of the previous model, ensure that the estimated effects orthogonal. In the remainder of this paper, will refer to Var (X)) demonstration of the accuracy and robustness of this procedure, please see the experiments performed in Appendix A Given the extra space allowed for the camera-ready copy, we intend include these experiments in the main text here.
the effect size of estimated effect f. Fl For
5.2 DROPOUT REGULARIZES INTERACTIONS IN PURE NOISE DATA
In this experiment, are spurious). This gives us a testbench to easily see the regularization strength of different levels of Dropout. Specially, we generate 1500 samples of 25 input features where X; fni 1) and Y ~ (0, 1). We optimize NNs with 3 hidden layers and ReLU nonlincarities and measure effect sizes as described in Sec. [. In Fig. - we see the results for NNs with 32 units in each hidden layer. For this small network, both Activation and Input Dropout have strong regularizing effects on NN. Not only do they reduce the overall estimated effect size, both Activation and Input Dropout preferentially target higher-order interactions (e.g.. the proportion of variance explained by low-order interactions monotonically increases as the Dropout Rate increased for Figs. 2d 2e and 21 In
simulation setting in which there is signal (so any estimated effects
https: / /i thub. wwwp:/owwtoooooe.eeoeo / CIR2202
The FANOVA decomposition identifiahle for a given distribution of X; in experiments, we will mainly simulation data so that this decomposition be computed with respect to the correct distribution.
4

---

Under review as a conference paper at ICLR 2021
k=
14
12.
k=3
Effect Size
10-
2
Effect Size
k
Effect Size
10
6
0.0 0.1250:250.31 0.5 0.6250.75
0.0 0.1 0.375 0.6250.75
0.0 0.1250.250.31 0.5 0.625 0.75
Dropout Rate
Dropout Rate
Dropout Rate
(a) Total Effect: Act. Dropout
(b) Total Effect: Input Dropout
(c) Total Effect: Input Act.
1.0
=
Norma lized Effect Size
Normalized lized Effect Size
Norma lized Effect Size
k=2
0.8
0.8
k=
0.6
0.6
0.6
0.4
1
0.0
0.0 0.1350.250.375 0.5 0.6250.75
0.0
0.0
0.250.255.335 0.5 0.6250.75
0.0
0.0 2..0..55.250377 0.5 0.6250.75
Dropout Rate
Dropout Rate
Dropout Rate
(d) Norm. Effect: Act. Dropout
(c) Norm. Effect: Input Dropout
(f) Norm, Effect: Input Act.
In this experiment, we train fully-connected NNs on pure noise (details in Sec. 5.2). Displayed values are the (mean std. over initializations) of the trained model's variance explained by each order of interaction. Activation and Input Dropout both reduce the effect sizes of the learned high-order interactions. The top row (a-c) shows absolute effect sizes (which of course decrease Dropout increases), while the middle row (d-f) shows the relative effect sizes, making easier to see how the Dropout rate affects each order.
Figure 2:
Fig. we see results from the same experiment on NNs with 128 units in each hidden layer; as our analysis predicts, the effects of Input Dropout are just strong for this larger network (Fig. E.3e
5.3 OPTIMAL DROPOUT RATE DEPENDS ON TRUE INTERACTIONS
This understanding of Dropout as a regularizer against high-orde interaction effects suggests that Dropout should be used at higher rates where we would like to regularize against high-order interaction effects. To test this guideline, we perform two experiments.
Modified 20-NewsGroups Data We use the 20-NessGroups dataset which is classification task on documents from 20 organizations. We modify this dataset by adding & new features (each feature is IID Un 1)) and a 21st class which is the correct label if all of the ls new features take on a value greater than 0.5. This modified dataset then has strong k-way interaction effect, and as l grows, would expect the optimal Dropout rate to be lower. As predicted by our understanding of Dropout, indeed the optimal Dropout rate is lower for larger k; with optimal rates of 0.375 for k 1, 0.25 for k = 2, and 0. 0.1 for k = 3 (full results are shown in Table I.
The New York City BikeShare datase (preprocessing from F) is large dataset designed help predict the demand of Citi Bikes in New York City. Because individuals base their travel plans on hourly, daily, and weekly cycles, there are real 2nd- and 3rd-order interaction effects in
BikeShare
https: www. htttp://www.kaggge.e. http://www...gggeeoooooeeeeeeeee
shareesharr--eeeee dataset

---

Under review as a conference paper at ICLR 2021
Norraltzee Effact Size, km
6,22
Normallzzd Effect Size,
001. 005
05
Normalize Lffect
Mean Squared
Teu
Tram 0400
9.4
3.23
322
64
0.
Epoch
Epach
Epoch
Epach
(a) 1-way intxs, gen
(b) 2- way intxs, gen
(c) 3-w intxs, gen
(d) MSEs, gen 1
Normalized Effect Size, K-
Normalized Effect Size, k=3.
0.223
2333
141 9.0
To1. 135
0,255
3315
Normalizes Lifect
25
Tet 3.0
Tuam 0.00
-4
3223
6.
0.2
2329
Epoch
Epoch
Epach
Epoch
(c) 1-way intxs, gen
(f) 2-way intxs, gen 2
(g) 3-way intxs, gen
(h) MSEs, gen 2
Normglized Effect Size, km
3.0
1.3
Tan 000
Bormalized Effect Size,
0.32
kormalizes Lifect Size,
4325 225
Squared Error
1Mh 0325
Te 700
0.20
Test 2.50
210
6.0
353
203
Epoch
362
303
(i) 1-way intxs, gen 3
(i) 2-way intxs, gen 3 (k) 3-way intxs, gen 3
()) MSEs, gen
Figure 3: Learned interaction effects of order 1, 2 and (cols 1, 2, and respectively). and model error on train and test (col 4) vs. epochs. Each row corresponds to different generator as described in Sec. 1.4.1 the generator in the top row has only I-way interactions, the generator in the middle row has only 2-way interactions, and the bottom row has only true 3-way interactions. The figure is complex; key findings are described in Sec. 5.4.1
this dataset (Tan et al. Dropout for this dataset is actually O (full results in Fig. E.4)
2018). As predicted by the interaction view of Dropout, the optimal rate of
5.4 Do OTHER REGULARIZERS PENALIZE INTERACTION EFFECTS?
Seeing that Dropout regularizes against interaction effects, it is natural to ask whether other effective regularizers of NNs also achieve better generalization by penalizing high-order interaction effects. Here, we examine early stopping and weight decay as potential regularizers of interaction effects. We find that neither these regularization techniques specifically target interaction effects. However, because Dropout changes the effective learning rate of interaction effects, it can act in concert with early stopping to magnify the regularization against interaction effects.
5.4.1 EARLY STOPPING
has long been known that the effective capacity of NNs increases during training (Weigend 1994), and recent work supports the view that randomly-ine -inttomliiiiillizzeed NNs start as simple functions that are made more complex through training (De Palma al.) 2018 Nakkiran et al. 2019 Jacot et al. help select models that generalize well (Prechelt 1998 Caruana et al. (200T). To see how early stopping interplays with the Dropout-inde inndued effective learning rates, study the learned effects over the course of optimization.
2018). Thus, makes sense that early stopping
We generate 1500 samples of 25 input features where X, ~ Unif(-111) 1,, and the target gener- ated according to one of three settings: (1) only main effects: Y N(sin (X)) + cos (os(X1),00 o2).

---

Under review as a conference paper at ICLR 2021
1.0
0.8
Effect
Effect
k=7
K=3
Hoomalized
0.4
85
0.2
04
0.
09
5.55
91
5.00
0.10 0.02
0.04 0.06
0.58
Weight Decay
Weight Decay
Weight Decay
(a) Total Effect Sizes
(b) Normalized Effect Sizes
(c) Rate of Decay
Figure 4: Strong weight decay can have mild regularization effect against interaction effects; however, the regularization effect comparable to Dropout occurs at extremely strong weight decay for which training very unstable.
(2) only pair effects: Y (sin(Xo) cos(X)), o and (3) only three-way effects:
N (sin(Xo) cos( (X 2,, o²)). We optimize fully-connected NNs on these data and measure effect sizes as described in Sec. 5.1 Results are shown Fig. The key findings are: 1) the rightmost column shows that NNs with low rates of Dropout tend massively overfit due to a reliance high- order interactions; 2) the different levels of Dropout have different steady-state optima; 3) because Dropout slows the learning of high-order effects, early stopping is doubly effective in combination with Dropout. NNs tend to learn simple functions earlier (regardless of Dropout usage), and Dropout slows the learning of high-order interactions. As a result, early stopping reduces the complexity of the learned function and Dropout increases this effect by delaying learning high-order interactions so early stopping can halt training before they are learned.
5.4.2 WEIGHT DECAY
Another popular regularization mechanism weight decay: placing an &g penalty on the weights of the network. We study weight decay the same data generator as we studied Dropout in Sec. Sec. [.2] As the results Fig. [ show, strong weight decay (large values of X) has modest effect of regularizing against interaction effects. However, achieving the same practical benefit from weight decay as from Dropout is untenable due the training instability that strong weight decay introduces: when weight decay was set larger than about 0.2 the NNs learned simple constant functions.
6 DISCUSSION AND IMPLICATIONS
In this paper, we examined concrete mechanistic explanation of how Dropout works: by regularizing higher-order interactions. We see that Dropout does not introduce unbiased noise into learning training with higher levels of Dropout produces models that are less likely to learn strong interaction effects. This explanation of Dropout has several implications for its use and crystallizes some of the conventional wisdom regarding how and when to use Dropout.
6.1 DROPOUT FOR EXPLANATIONS
While Dropout has been used for measures of model confidence (Gl Ghahramani 2016 Gal et
2017) and to aid model interpretability (Chang et al. (017b]a)) a). it does not treat all effects equally. This must be taken into consideration both during training a NN and when querying a trained NN. when Dropout is used to train the NN, true statistical patterns that are in the training data may or may not be learned by the NN depending on the Dropout rate. And when a trained NN probed with Dropout enabled, false picture of the function learned by the NN can also emerge. For example, there are important 2nd and 3rd order interactions in the New York City BikeShare dataset (Fig. E.4 using Dropout examine NN trained this dataset will underweight these interaction effects. Thus one should be careful when using Dropout to interpret NNs, or interpreting what NNs trained with different Dropout rates tell about patterns in the data.

---

Under review as a conference paper at ICLR 2021
6.2 SETTING DROPOUT RATE
The Dropout rate should be set according the desired magnitude of the anti-interaction regularization effect. the dataset large or sufficient augmentation can be performed. lower rates Dropout can be used Dropout be omitted entirely Heweeelyeeeandez--enaeee City BikeShare dataset discussed in Section5.33
& König 2018 (e.g. the New York
In addition, it is often suggested to use larger Dropout rates in deeper layers than in initial layers (Ba & Frey, 2013). This conventional wisdom can be explained from the interaction point view: this regularization scheme encourages NNs to do representation learning in their initial layers this may require learning interactions between input features such as pixels or words, while encouraging deeper layers focus more on summing evidence from multiple sources.
In CNNs, Dropout is typically used at lower rates than in fully-connected networks (Park & Kwak 2016). The convolutional architecture creates constraints that prevent arbitrary high-order interactions by restricting N in (+) to be a carefully selected set of local input features hidden unit activations. Also, operators like max pooling further restrict the model's is ability to learn complex interactions. In
other words, convolutional nets create a strong bias for against different kinds of interaction effects via architecttre and thus depend less on a mechanism like Dropout to blindly regularize interactions.
6.3 EXPLICITLY MODELING INTERACTION EFFECTS
In this investigation, we have seen that the main challenge of estimating interaction effects is the hypothesis space which grows exponentially with the order of the interaction effect. If we were able to hone down the hypothesis space by specifying small number of interaction effects before looking at data, our models could efficiently learn the correct parameters for these few interactions from data. Several recent works have proposed to do this by explicitly specifying the interaction effects the NNs may consider: Of particular note is (Jayakumar et al.) 2020), which proposed to use multiplicative interactions to combine data modalities, and found that many common architectures can be seen in the lens of multiplicative interactions. These works make sense given the difficulty of picking interaction effects from the ahoonenialll-growing haystack of possible interactions: we know a priori which high-order interactions exist, it is better to explicitly model them rather than hope the NN learns them from data.
Another approach to explicitly model interaction effects is the Deep and Cross Network (Wang et al. 2017), which uses a two-part architecture consisting fully-connected network and a "cross" network in which each layer has its activation crossed with the vector of input variables before being transmitted to the next layer. This "cross" network increases the interaction order at every layer. Interestingly, the experiments of (Wang et al. 2017) (especially Fig. 3 within) show that the best-performing architecture has only a single cross layer- - this is exactly what we would expect based on the amount of spurious interaction effects which the model is otherwise capable of learning.
Finally, we can see these experiments as another view on the success CNNs: when interactions are important (such as in image recognition), it is important make the form of expected interactions explicit. High-order interactions in the data are not strong enough to cut through the hypothesis space of all potential interactions, so explicitly encoding the form can make a tremendous difference in model accuracy.
7 CONCLUSIONS
In this paper, we have examined concrete explanation of Dropout a regularization against interaction effects. We have shown that the effective learning rate of interaction effects decreases exponentially with the order of the interaction effect, crucial balance against the exponentially- growing number of potential interactions of & variables. Although Dropout can work concert with weight decay and early stopping, these do not naturally achieve Dropout's regularization against high-order interactions. By reducing the tendency of NNs to learn spurious high-order interaction effects, Dropout helps to train models which generalize more accurately to test sets.
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Jimmy Ba and Brendan Frey, Adaptive dropout for training deep neural networks. In C. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Wein- berger (eds.), Advances in Neural Information Processing Systems 26. pp. 3084 3092. Curran Associates, Inc., 2013. URL httpp:/pappers nips ca/paper/
Pierre Baldi and Peter J Sadowski. Understanding dropout. In Advances in neural information processing systems, pp. 2814-2822, 2013.
Cristian Buciluă, Rich Caruana, and Alexandru Niculescu-Mizil, Model compression. In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pp- 535-541, 2006.
Rich Caruana, Steve Lawrence, and C Lee Giles. Overfitting in neural nets: Backpropagation. conjugate gradient, and early stopping. In Advances in neural information processing systems, pp. 402-408, 2001.
Jacopo Cavazza, Pietro Morerio, Benjamin Haeffele, Connor Lane, Vittorio Murino, and Rene Vidal. Dropout as a low-rank regularizer for matrix factorization. In International Conference on Artificial Intelligence and Statistics, pp. 435 444,, 2018
Chun-Hao Chang, Elliot Creager, Anna Goldenberg, and David Duvenaud. Interpreting neural network classifications with variational dropout saliency maps. In NeurlPS Machine Learning in Computational Biology Workshop. 2017a.
Chun-Hao Chang, Ladislav Rampasek, and Anna Goldenberg. Dropout feature ranking for deep learning models. arXiv preprini 17X1::1712.086455 2017b.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. ArXiv, a 0ass/6003.02754 2016.
Xi Cheng, Bohdan Khomtchouk, Norman Matloff, and Pete Mohanty, Polynomial regression as an alternative to neural nets. CoRR, abs/1806 568500 2018. URL http::://axiv..og///////////
1806606850
Brent A Coull, David Ruppert, and MP Wand. Simple incorporation of interactions into additive models. Biometrics, 57(2)5539-545 2001.
Antonio Cuevas, Manuel Febrero, and Ricardo Fraiman. An anova test for functional data. Computa- tional statistics & data analysis, 47(1): 2004.
Giacomo De Palma, Bobak Toussi Kiani, and Seth Lloyd. Deep neural networks biased towards simple functions. arXiv preprint 18Xiv: 10156, 2018.
lan A Delbridge, David S Bindel, and Andrew Gordon Wilson. Randomly projected additive gaussian processes for regression. arXiv preprint 19Xiv:1912.12834, 2019.
Paul HC Eilers and Brian D Marx. Flexible smoothing with b-splines and penalties. Statistical science, pp. 89-102, 1996.
Yarin Gal and Zoubin Ghahramani. Dropout as bayesian approximation: Representing model uncertainty deep learning. In international conference on machine learning, pp. 1050-1059, 2016.
Yarin Gal, Jiri Hron, and Alex Kendall. Concrete dropout. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 3581-588880 13590. Curran Associates, Inc., 2017. URL TRL.ttp: htt.htttp/ppperr. paperss
nips cct
Andrew Gelman.
Statistical modeling. causal inference, and social science. Mar http://eee.eettttat....ooaaaaaaa
2018.
URL

---

Under review as a conference paper at ICLR 2021
Trevor Hastie and Rob Tibshirani. Generalized Additive Models. Chapman and Hall/CRC, 1990.
David P Helmbold and Philip M Long. Surprising properties of dropout in deep networks. The Journal of Machine Learning Research, 18(1):7284-7311 2017.
Alex Hemrnnee--aariia and Peter König. Do deep nets really need weight decay and dropout? arXiv preprint arXiv:1802.0704 2018.
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge neural network. arXiv preprint 1oXiv:/503.02::0 2015.
Geoffrey E Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.05800 2012.
Giles Hooker. Diagnostics and Extrapolation in Machine Learning. Stanford University, 2004.
Giles Hooker. Generalized functional anova diagnostics for high-dimensional functions of dependent variables. Journal of Computational and Graphical Statistics, 16(3):709-732, 2007.
Arthur Jacot, Franck Gabriel, and Clement Hongler. Neural tangent kernel: Convergence and generalization in neural networks. In S. Bengio, Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems 31, pp. 8571-8580, Curran Associates, Inc., 2018. URL http: httpttpp//ppppess..... http://pppers........ www.:iccp..........e...pepeeee
8 neural tangent kernel- convergence and generalizatior........mm neteo- pdf
Siddhant M. Jayakumar, Jacob Menick, Wojciech M. Czarnecki, Jonathan Schwarz, Jack Rae, Simon Osindero, Yee Whye Teh, Tim Harley, and Razvan Pascanu. Multiplicatiiee interactions and where to find them. In International Conference on Learning Representations, 2020. URL net
https://openreviln wttp://wwwwe.eeeeeooooooo/o/eeaa 2ote//oooomm111yyyyk6
Benjamin Lengerich, Sarah Tan, Chun-Hao Chang, Giles Hooker, and Rich Caruana. Purifying interaction effects with the functional anova: An efficient algorithm for recovering identifiable additive models. International Conference on Artificial Intelligence and Statistics, pp. 2402- 2412, 2020.
Andrew C Leon and Moonseong Hco. Sample sizes required to detect interactions between two binary fixed-effects in a mixed-effects linear regression model. Computational statistics data analysis, 53(3):603-608, 2009.
Yin Lou, Rich Caruana, and Johannes Gehrke. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 150-158. ACM, 2012.
Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD international conference Knowledge discovery and data mining, pp. 231.631,22 ACM, 2013.
Poorya Mianjy, Raman Arora, and Rene Vidal. On the implicit bias of dropout. arXiv preprint 180.091806.09777, 2018.
Preetum Nakkiran, Gal Kaplun, Dimitris Kalimeris, Tristan Yang, Benjamin L Edelman, Fred Zhang, and Boaz Barak. Sgd on neural networks learns functions of increasing complexity. arXiv preprini arXiv:1905.1 1604, 2019.
Eric Nalisnick, José Miguel Hernández-Lobato and Padhraic Smyth. Dropout as structured shrinkage prior. arXiv preprint arXiv: 2018.
Didrik Nielsen. Tree boosting with xgboost-why does xgboost win" every" machine learning competition? Master's thesis, NTNU, 2016.
Ian Osband. Risk versus uncertainty in deep learning: Bayes, bootstrap and the dangers of dropout. In NIPS Workshop Bayesian Deep Learning, volume 192, 2016.
10

---

Under review as a conference paper at ICLR 2021
Sungheon Park and Nojun Kwak. Analysis on the dropout effect in convolutional neural networks. In Asian Conference on Computer Vision, pp. 189-204. Springer, 2016.
Lutz Prechelt. Early stopping-but when? In Neural Networks: Tricks of the trade, pp. 55-69. Springer, 1998.
Daniel Selsam, Matthew Lamm, Benedikt Bünz, Percy Liang. Leonardo de Moura, and David Dill. Learning a sat solver from single-bit supervision. arXiv preprint arXiv: 1802.06683 2018.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdi- nov. Dropout: A simple way to prevent neural networks from overfitting. Jourmal of Ma- chine Learning Research, 15:1929- 2014. URL http::/tmmmmmm// prg.papers/v111
sr srivastaval4a html
Sarah Tan, Rich Caruana, Giles Hooker, Paul Koch, and Albert Gordo. Learning global additive explanations for neural nets using model distillation. arXiv preprint 1rXiv:1801.000 0864000 2018.
J Kenneth Tay and Robert Tibshirani. Reluctant additive modeling. arXiv preprint arXiv:1912.0144 1912.0188, 2019.
Yusuke Tsuzuku and Issei Sato. On the structural sensitivity of deep convolutional networks the directions fourier basis functions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 51 51-60, 2019.
Stefan Wager, Sida Wang, and Percy S Liang. Dropout training as adaptive regularization. In Advances in neural information processing systems, pp. 351-359, 2013.
Li Wan, Matthew Zeiler, Sixin Zhang, Yann LeCun, and Rob Fergus. Regularization of neural networks using dropconnect. In Sanjoy Dasgupta and David McAllester (eds.). Proceedings of the 30th International Conference Machine Learning, volume 28 of Proceedings of Machine Learning Research, pp. 1058-1066, Atlanta, Georgia, USA, 17-19 Jun 2013. PMLR. URL
http://proceedinng mlr. press v2/ w2t/wanl3..ttt.
MP Wand and John T Ormerod. Penalized wavelets: Embedding wavelets into semiparametric regression. Electronic Journal of Statistics, 5: 5:1654117 2011.
Haohan Wang, Xindi Wu, Zeyi Huang, and Eric P Xing. High-frequency component helps explain the generalization of convolutional neural networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8684-8694 2020.
Ruoxi Wang. Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. In Proceedings of the ADKDD'I1 17, pp. 12. ACM, 2017.
David Warde-Farley, Ian J Goodfellow, Aaron Courville, and Yoshua Bengio. An empirical analysis of dropout in piecewise linear networks. arXiv preprint arXiv: 13/2,6111 2013.
Andreas Weigend. On overfitting and the effective number of hidden units. Proceedings of the 1993 connectionist models summer school, volume 1, pp. 335- 335-342, 1994.
Bodgan M Wilamowski, David Hunter, and Aleksander Malinowski. Solving parity-n problems with feedforward neural networks. Proceedings of the International Joint Conference on Neural Networks, 2003., volume 4, pp.
IEEE, 2003.
Dong Yin, Raphael Gontijo Lopes, Jon Shlens, Ekin Dogus Cubuk, and Justin Gilmer. A fourier perspective on model robustness in computer vision. In Advances in Neural Information Processing Systems, pp. 13255-13265, 2019.
Andrea Zunino, Sarah Adel Bargal, Pietro Morerio, Jianming Zhang, Stan Sclaroff, and Vittorio Murino. Excitation dropout: Encouraging plasticity in deep neural networks. arXiv preprint arXiv:1805.09092, 2018.
11

---

Under review as a conference paper at ICLR 2021
Normalizes salizddd Effect Size
Normalized Effect Size
0.8
0.8
0.0
Normalized Effect
0.4
0.2
0.2
0.2
True Interaction Order
True Interaction Order
True Interaction Order
(a) 100 Samples
(b) 1000 Samples
(c) 10000 Samples
Figure A.1: Measuring the accuracy of the estimated interactions in NNs trained simulation data of pure interactions of order 1, 2, or 3. Each pane shows results for a given number of samples used for model distillation. For small numbers of samples, the distillation procedure can over-estimate the variance contained by high-order interaction effects. For large numbers of samples, the distillation procedure accurately recovers the true interaction order. Error bars represent the variance of the estimate over 10 experimental runs.
A ACCURACY OF FUNCTIONAL ANOVA DECOMPOSITION BY DISTILLATION
This section is the most important part of the Appendix and plan to include this material in Section of the main body of the final paper (which page longer).
critical step in experimental framework is accurately calculating the fANVOA decomposition of the function estimated by the NN. Because the FANOVA captures a high-dimensional function sequence lower-dimensional functions, each function is constrained approximation of the function represented by the NN. Thus, we do not recommend always using model compression as generae-purpose purpose explanation of NNs. However, in this paper we care about only single aspect of the compressed models: approximation error of the NNs. From the NN function F(X), we estimate an additive model (X) S(,, Fi) arg min seF, Ex [(f((), where JF the class of additive models and squared loss. The set of possible explanations S(F,F)) may have more than one member; however, all of these explanations must have the same compression loss Ex (X),,,, (())) Since the only metric we reporting about these models is the compression loss (how much the variance of the NN could be explained by a model in F1), in this paper it does not matter which explanation in S(F, Fi) is chosen.
To empirically measure this approximation error, test the distillation procedure using simulation data. Our goal this experiment is to accurately fit NN to a known function so that we can measure the approximation error of the distillation procedure against the known function encoded in the NN. For each run, we generate data according to X ~ Unif(-1, 1)5, and train the NN to fit function of pure k-order interactions multiplication of k uncorrelated features of X). this way, the NN represents a function of pure k-order interactions and a perfect distillation procedure would assign 100% of the variance of hyperparameters, are available at the main repository.
interactions of order Code for this simulation, with hardcoded values
In Fig. A.I we show the results for distillation with various numbers of samples. each pane, there are 4 bars which each represent pure interaction of a different order. The height of the bars (and the corresponding colors) represent the normalized effect size estimated by the distillation procedure for each of these underlying interaction effects. In Fig. A.la only 100 samples distilled models; as a result, the distilled models underfit the NN's behavior and the implicd effects of high-order interactions or to 10000 (Fig. I. the distillation procedure is increasingly accurate at recovering the true interaction order in the NN.
used to fit the
exaggerated. When the number of samples is increased 1000 (Fig. Ab
We have also run this experiment on data generated from a mixture of interaction effects and obtain similar recovery results.
12

---

Under review as a conference paper at ICLR 2021
(a) = 0.01
(h) 0.99
Figure B.2: A toy example of decomposing function into pure interaction and main effects. In each (a) and (b), there are four panes: (left) an overall function, (middle left) pure interaction effect of X1 and X2, (middle right) a pure effect X1, and (right) pure effect of X2. In both @ and b. the overall function is Y X2, but the decomposition changes based on the coefficient p of correlation between X1 and X2. For X, and X₂ uncorrelated, the multiplication a pure interaction effect; for X₃ and X₂ correlated, much of the variance can be moved into effects of the individual variables. The decomposition is unique given the joint distribution of the three variables.
B INTERACTION EFFECTS
An example of the distribution changing the meaning of pure interaction effect is shown in Fig. B.2
B. THE UNREASONABLE EFFECTIVENESS OF MODELS WITH FEW INTERACTION EFFECTS
Generalized additive models (GAMs) Hastie Tibshirani 1990) are a restrictive model class which estimate functions of individual features, i.e., functions of the form f(,,..... Xx)) = Σ/=1 (X;). There have been large number of methods for estimating these functions, including functional forms such as splines, trees, wavelets, etc. (Eilers & Marx 1996 Lou et al. 2012) Wand & Ormerod 2011).
While vanilla GAMs describe nonlinear relationships between each feature and the label, interactions sometimes added further capture relationships between multiple features and the label (Coull
ctt al. 2001 Lou al. 2013 Tay & Tibshirani 2019).
In the age of deep learning, it is surprising that GAMs with small number of added interaction effects could be state-of-the-we art on any dataset with a moderately large number samples. However, successful tree-based ensembles such XGBoost (Chen & Guestrin 2016) often require only few interaction effects to win competitions (Nielsen 2016). In certain cases, polynomial regression of order 2 be competitive with fully-connected deep NNs (Cheng al.) 2018), and even generalized additive models have a surprising capability to approximate deep NNs (TTnn er al., (018).. Similar phenomena have been observed for Gaussian Processes (Delbridge al. 2019) and computer vision models (Yin et al., 2019 Wang et al., 22020; Tsuzuku & 2019). How are these models, which ignore the majority of interaction effects, effective?
B.2 STATISTICAL (UN)RELIABBILTY OF INTERACTION EFFECTS
One reason why models which ignore high-order interaction effects can perform well is the tremendous difficulty that higher-order interaction effects present to learning algorithms. When trying learn high-order interaction effects, number of possible interaction effects grows exponentially (the number of k-order interaction effects possible from N input features is (N) while the the variance of an interaction effect grows with the interaction order (Leon & Heo 2009 This quandry is intensified when the effect strength decreases
stuck between rock and hard place: the
13

---

Under review as a conference paper at ICLR 2021
with interaction order, which is reasonable for real data (Gelman 2018 It is like searching for needle in a haystack, but as we increase k, the haystack gets larger and the needle gets smaller. For large k, are increasingly likely select spurious effects rather than the true effect is better to stop searching the haystack. Viewed this way, is less surprising that in the absence of prior knowledge of which interaction effects are true, simple models able to outperform large models.
some point
B.3 PARITY AND INTERACTION EFFECTS
Interaction effects are intricately linked to classically difficult function class: parity. In the case of two Boolean variables, a pure interaction effect is exactly a weighted XOR function and for continuous variables, pure interaction effects are continuous analog of parity (Lengerich al. 2020). Parity functions are notoriously difficult to learn with NNs (Wilamowski et al. | [003] Seelsam al. 2018). Does this suggest that NNs already robust against interaction effects, and if so, why is the extra regularization of Dropout against interaction effects necessary?
It is important for to distinguish between learning the correct interaction effect against learning a spurious interaction. Given N variables, there are O(N) possible main effects, O(N²) possible pairwise interactions, O(N3) possible 3-way interactions, O(N*) possible 4-way interactions, etc. This exponential growth in the hypothesis space of interaction terms simultancously increases the probability that universal approximator would estimate some interaction effect while decreasing the probability that the same universal approximator selects the correct interaction effect. For this reason, can be possible for model classes to struggle with accurate recovery of parity functions without being inherently biased against high-order interactions. As shown in Figure I the exponential growth in the number of potential interaction terms is balanced by the exponential decay learning rate induced by Dropout. In this way. large NNs trained with Dropout can have the convenient property that they are capable of learning high-order interactions but will put off the difficult task of learning these high-order interactions until simpler functions have been thoroughly explored.
C PROOF OF THEOREM
Proof. Let X = Enejd Sa(X,) and [V[ = 0. Then with Input Dropout,
E| M Σ P PXX---NAAC-----XXXM----- O
uEld
(4a)
Σ (1- (1 (1)) O
(4b)
wE|d]
Σ (1- p) (ul (1- (1) fu(Xa)o
for some u
wE|d
(4c)
Σ (a- ME(d [d]
(4d)
where M is drawn uniformly from the Dropout masks with at least zero value and the final equality holds by the orthogonality condition of the fANOVA decomposition (Eq. 1b in the main text).
14

---

Under review as a conference paper at ICLR 2021
D PROOF OF THEOREM
Proof.
Vu(Xu, 1). Vu(Xa M.Y),
EMI
(5a)
(V Y)
=(1- (1-p) ()-p))ll (Xu: Y) + (V Y)I (1- p)l IV @ OM Y) (5b)
(1 (1-p))"V,(X..:: Y)
(5c)
where the final equation holds the orthongonality of FANOVA.
E ADDITIONAL EXPERIMENTS
Pure Noise Data Figure F.3 shows the results of various Dropout rates NN with 128 hidden units in each layer. These results are analogous to the results shown Fig. Of the main text for NN with 32 hidden units in each layer.
Modified 20-NessGroups Table displays the results of various Dropout Rates on the Modified
20-NewsGroups datasets described Section 5.3
k=
Dropout Rate
0.0
0.125
0.25
0.375
0.5
0.625
1
0.52 + 0.01
0.54
0.54 + 0.03
0.57 + 0.02
0.55 + 0.02
0.47 + 0.02
2
0.39
+ 0.01
0.38 + 0.03
0.40 + 0.02
0.40 + 0.01
0.38 + 0.01
0.27 + 0.02
3
0.39
+ 0.01
0.41 + 0.01
0.41 0.01
0.40 0.02
0.40 0.02
0.27 0.04
Table 1: Test accuracies of the models trained on the modified 20-Newgroups datasets (Sec. 5.3) Reported values are (mean + std) of the test accuracies over 5 experiments, with the best setting in each row bolded. Each row indicates k. the order of the added interaction effect. As k is increased, lower levels Dropout tend to outperform. Diffcrent modifications of the dataset change the difficulty of the task, so the accuracy values not comparable across rows.
BikeShare Figure E.4 displays results of various Dropout rates NN trained the New York City Bikeshare dataset. Because this dataset contains real 2nd and 3rd-order interaction effects (Tan al. 2018), the optimal Dropout rate for generalizing the test set is actually 0.
15

---

Under review as a conference paper at ICLR 2021
12
12
k=1
8=
k=]
2
K=2
10
10
R=3
R
2
Effect Size
Effect Size
0.6
0.4
0.2
0.0 25..2.02550333 0.5 86250.15
0.0
0.0 0.120 250.375 0.5 0.6250.75
0.0 0.1250.250.11 175 0.5 0.655.775
Dropout Rate
Dropout Rate
Dropout Rate
(a) Total
Effect:
Activation
(h) Total Effect: Input Dropout
(c) Total Effect: Input Activa-
Dropout
tion Dropout
1.0
dorrmalized Effect Size
Normalized Effect Size
0.8
0.8
8.0
9.6
Normalized
0.4
0.2
0.2
0.2
0.0
0.0 0.1250250.375 0.5 0.6250.75
0.0
0.0 0. 1252
0.1200.250.373
0.5 0.625
0.0
0.0 0.1250.25..3
0.3375 0.5 0.6250.75
Dropout Rate
Dropout Rate
Dropout Rate
(d) Normalized Effect: Activation
(e) Normalized Effect:
Input
(f) Normalized Effect: Input Ac-
Dropout
Dropout
tivation
k
k=
Normalizer aizeaddeddd Effect Size
1.0
2=2
0.8
2
2
Normalized Effect
2=
0.6
0.6
Normalized
0.4
0.4
0.2
0.2
0.2
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7
0.0 0.1 0.2 0.3 04 0.5 0.6 0.7
0.0 0.1 0.2 0.3 0.4 0. 0.6 0.7 Dropout Rate
Dropout Rate
Dropout Rate
(g) Shrinkage: Dropout
Activation
(h) Shrinkage: Input Dropout
(i) Shrinkage: Input Activation Dropout
Figure E.3: In this experiment, we train fully-connecteo neural networks on a dataset of pure noise (details in Sec. [.2).. Displayed values are the (mean std. over 10 initializations) of the proportion of the trained model's variance explained by each order of interaction effect. All neural networks in this figure have 128 units in each hidden layer (compared to 32 units per layer in Figure 2 and we see that Activation Dropout has only small impact, while Input Dropout significantly reduces the estimated effect sizes the high-order interactions. As expected, increasing the size of the hidden layers from 32 in Figure 2 to 128 in this Figure decreases the impact of Activation Dropout on high-order interactions, but does not reduce the effectiveness of Input Dropout.
16

---

Under review as a conference paper at ICLR 2021
Normalized Effect Size, k=1
Tet 19
0.22
161 531
0.5
Mean Squared
Tetl 151
3.6
Normaltzed Effect
6.4
Tut C24
1.03
8.4
Normalizes
044
Tell C3u
Jut 630
6.
3.73
200 Epoch
320
412
100
250
312
Epoch
Epech
Epoch
1-way interactions
2-way interactions
(c) 3-way interactions
(d) MSES
Figure E.4: Learned interaction effects and model errors over epochs training on the BikeShare Dataset. In this dataset, there are true interaction effects of orders 2 and 3, so the models with high Dropout rates generalize worse than the models with low Dropout rates. This behavior is expected under our perspective of Dropout as an interaction regularizer, but unexpected under the perspective of Dropout as generic model regularizer.
17


Reviews:
[{"review_id": "x_JqHM1Szi", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper finds that Dropout preferentially targets high-order interaction effects, resulting in models that generalize better by down-weighting high-order interaction effects that are typically spurious or difficult to learn correctly from limited training data.\n\nAlthough the paper has theoretical proofs/intuition and interesting experimental results, I don’t think I find convincing and sufficient evidence to support the conclusions, as opposed to some other hypothesis. \n\n“To preview the experimental results, when NNs are trained on data that has no interactions, the optimal Dropout rate is high, but when NNs are trained on datasets which have important 2nd and 3rd order interactions, the optimal Dropout rate is 0.”\n\nEmpirical results on  Modified 20-NewsGroups Data and BikeShare support the conclusion. But other than empirical results, are there some theoretical support of optimal dropout rate? \nAdditionally In practice, we don’t know whether the data may have higher order interactions or not. How can we guess optimal dropout rate? \n\nThe way to implicitly measure “interaction effects”\nI am not fully convinced about distal procedure to measure different levels of the interaction effects. However, this procedure serves as key experimental foundations for the paper. \n\n\nFig 3’s  key findings: \n“the rightmost column shows that NNs with low rates of Dropout tend to massively overfit due to a reliance on high order interactions”, I see overfit, I don’t see why it is due to higher order interactions (row1, row2, row3 have similar trend)\n\n “because Dropout slows the learning of high-order effects, early stopping is doubly effective in combination with Dropout. NNs tend to learn simple functions earlier (regardless of Dropout usage), and Dropout slows the learning of high-order interactions.” \nHow does Fig3 reach conclusions about early stopping ? there seems no “early stopping procedure”\nHow does Fig3 show “Dropout slows the learning of high-order interactions.”? \n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "m8RirTNsAt9", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper analyzes Dropout through the lens of k-way interactions. The central claim of this paper is that Dropout reduces interaction effects. This is shown through both theory and experiment. The theory suggests that a higher dropout rate reduces the effective learning speed of higher-order interactions. Experiments suggest that increasing the dropout rate reduces the functional magnitude of higher-order interactions, even to some extent in real data.\n\n\nThis paper tackles an interesting problem, and I mostly want to agree with its message, but it lacks in several areas.\n\nFirst, I would argue that the authors are _probably correct_ in their conclusions. It seems reasonable that Dropout has the claimed effect. This would be coherent with what we know about Dropout. It's less clear if the speculation about larger scale architectures and problems is correct, as other more complex effects may make the interactions interpretation of hidden units less useful, but I think the authors make a good case to appeal to our intuition. More generally, there are several passages that are more speculation than result-backed extrapolation, I'd suggest the authors to modify the text to either clearly identify what is speculation and/or to align the claims with the results.\n\nThe theory portion of the paper eludes me. As I currently understand it, it is wrong. Although it seems to reach \"experimentally correct\" and intuitively appealing conclusions, it may be right for the wrong reasons. That being said, I may be totally off the mark and I'm happy to be corrected by the authors. Either way, this portion of the paper needs to be _much_ clearer.\n\nThe experimental section of the paper is interesting, but it has some serious flaws. Mainly, the toy experiments may be misleading, and the real-world experiments have suspicious results.\nIn particular, I see two experimental \"musts\" for this paper:\n- Redo the Figure 3 experiment with no or much less noise\n- Find reasonable hyperparameters for BikeShare such that learning doesn't diverge. \n\nPlease find detailed comments below.\n\nI'm inclined to reject the paper as it is. I think the authors are exploring some very important aspect of deep neural networks that goes beyond Dropout rates, but the exposition could be improved, the experiments could be much more robust, and I'd like to understand the Theorem proofs before accepting this paper.\n\n\n\n\n- \"higher Dropout rates should be used when we need stronger regularization against spurious high-order\ninteractions\", how do we know _when_ data requires high-order interactions or not? how do we know if they are spurious or not? This doesn't seem like a solved problem, as such recommending to tune Dropout rates based on this seems a bit impractical.\n- \"when NNs are trained on data that has no interactions, the optimal Dropout rate is high, but when NNs are trained on datasets which have important 2nd and 3rd order interactions, the optimal Dropout rate is 0\", arguably _any_ natural/sense-like data has 2nd+ order interactions, because data can only be understood through patterns and aggregate computations. Are the authors suggesting that we should just not use Dropout?\n- \"Hinton et al proposed Dropout to prevent spurious co-adaptation\", as far as I know Hinton claims that Dropout breaks co-adaptations period, spurious or not. Plus, the original paper never actually backs that statement with quantitative evidence. As far as I know, Dropout has e.g. been shown not to recover causal structures particularly well. Eliminating all co-adaptations will eliminate spurious ones as well; I know Dropout doesn't remove _all_ co-adaptation, I'm just skeptical it removes spurious ones _more_.\n- Theorem 1\n  - there seems to be a typo in the main text, should be $f_u(X_u)$? (as in the proof)\n  - in the proof, I don't understand 4b to 4c. $\\mathbb{E}[f_u(X_u M^+)]$ is an expectation over all masks, $M^+$. This turns into an integral over the output values obtained by changing one of the features, $X_v$, \"for some v\". This is like picking two masks which happen to have exactly $M_v$ change, but then instead of taking the expectation, kind of like $f(X_v * 0, X_u) + f(X_v * 1, X_u)$, the integral over the entire domain of $X_v$ is taken. I do not understand how they are equivalent.\n  - I don't understand what makes 4c to 4d valid. 1b for fANOVA's decomposition describes a constraint of the argmin such that the decomposition finds orthogonal functions. (a) I fail to pattern-match the integral of 1b with the integral in 4c, (b) I fail to see why the equality of 1b, the integral being 0, applies to 4c. Is the Bernoulli mask $g$? This should be made very clear.\n  - I'm skepical that the conclusion is even correct, here's a counterexample, let's take $f(X \\in \\mathbb{R})$ with $f(-1)=-1$, $f(0)=0.5$, $f(1)=0.5$, 0 otherwise. Assume we have $p(X)$ s.t. $E[Y]=0$, as in the proof, and note that even so, $f(0)\\neq 0$, i.e. a dropped out input isn't a 0 output. Let's say $p=0.5$, for $X=-1$, $\\mathbb{E}[Y|XM] = -1 * 0.5 + 0.5 * 0.5 = -0.25$ , whereas Theorem 1 states that this value should be $(1-p)^1 f(-1) = (1-0.5) * -1 = -0.5$. Am I misunderstanding something? Is there an error in my reasoning?\n\n- \"The distribution of training data is different for different levels of Input Dropout\", this seems fairly uncontroversial, by (somewhat) arbitrarily setting some input features to 0 (why not 1.42?), Dropout without a doubt changes the data distribution. A more interesting question is whether it changes the information content of the inputs, or if for some data distributions such as natural images the redundancy leaves information unaffected and Dropout simply forces the network to pick up on such redundancies.\n- Theorem 2\n  - I'm not sure what is meant by \"the gradient update for an interaction effect $u$\", so I don't really understand what is claimed here.\n  - As for theorem 1, it's not clear how to go from 5b to 5c. How is 1b related?\n\n- Appendix A, I'm skeptical of the authors adding an extra page in the appendix in expectation of being accepted being in accordance with the spirit of the ICLR submission instructions. This is fair game I guess, but a bit odd. For the record, Appendix A provides evidence for a base assumption of the paper, i.e. that a DNN **can** be decomposed into a number of additive low-order low-complexity interaction effects. This is done on a fairly toyish task, a small 5-d task with k=3, but shows this is possible and consistent.\n\n\n- Figure 3\n  - it's hard to see what is train and test in the 4th column, unless zooming in a lot. The authors could improve this by scaling the figures appropriately such that gaps in the dashed lines are visible. It would be useful for the font size to be scaled up as well for similar readability reasons.\n  - it's very suspicious that the Test MSE virtually doesn't change or gets worse depending on the dropout rate. It suggests that the network used is massively underfitting the problem as soon as some regularization is applied. I looked at the code and plotted the generated data, it seems to me that Y is dominated by noise rather than either additive or multiplicative cos(x)/sin(x) effects. This makes the task analogous if not identical to that of Figure 2. Either way, this is not really representative of any real world setup or of an interesting k-way interaction setup.\n\n- The \"real-world\" experiments aren't very revealing\n  - 20-NewsGroups, Table 1. Why add extra interactions? Why not repeat the analysis of Figures 2 and 3 for this setup? It's also not clear if the reported accuracy includes the extra class or not. It's also not clear that $k=2$ is significantly different from $k=3$, how are \"best\" columns chosen?\n  - Bikeshare, Figure E.4, it would be interesting to see a repeat of Figure 2 here. It does seem like there is a small effect of dropout for $k=3$ early on, but by then models start _diverging_, even the train loss is going up. This, to me, suggests that the learning rate is too large or that something else is wrong. It's hard to draw strong conclusions from this.\n  - The Bikeshare results also suggest, as I pointed out earlier, that Dropout may indiscriminately be damaging to both spurious and real co-adaptations/interactions. This is at odds with statements in the paper.\n\n- What is the expected effect size of a randomly initialized DNN? This seems like a useful value to track and compare against in these experiments.\n\n[Rebuttal update]\nThank you for your response.\n\nThis alleviates some of my concerns about Theorem 1, although I feel like I'd need to see a revised version of the entire proof to make sure I understand it.\n\nOn Figure 3, I'm not sure you've understood my concerns; perhaps I did not explain them clearly enough. Regularized models do no better than chance, and less-regularized models do worse than chance on test points. This is presumably because of what I mention in my review, which is that the synthetic data is basically noise. Thus the \"improvement in test accuracy\" isn't really an improvement, but rather that the model is no longer free to extremely overfit.\n\nOn the interpretation of Dropout you provide, this differs somewhat than the message of your paper. I agree more with this interpretation, although not fully. Either way, the paper doesn't really contain strong evidence for that interpretation, which I think would be great to have.\n\nI encourage you to rethink the experimental setup somewhat and to have clear experimental support for the proposed intuitions/insights. I think this is a valuable research direction but I think that a more mature paper would have a much higher impact.", "extra_scores": {}, "extra_text": {}}, {"review_id": "3vcvT6pvDkp", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "*Summary*\nThe authors are analyzing to which extent dropout is regularizing the training stage of deep networks, showing that high-order interactions are discouraged, this being a proxy for a better generalization capability once spurious co-adaptations are removed. In an extended mathematical analysis, the authors carry out their arguments taking advantage of the weighted analysis of variance, showing results on both the expected dropout rate and the impact on gradients while back-propagating. Experimental results are paired to the paper to demonstrate that changes the steady-state optima of the model. \n\n*Pros*\n* A solid mathematical dissertation is provided to discuss the presence of spurious co-adaptations. Despite dropout was originally proposed to accommodate for that, and to generalize better, the explanation behind was quite elusive and the authors greatly contributed in shedding light on that. \n\n*Cons*\n* One crucial aspect of the paper is to measure how good is the ANOVA decomposition in terms of approximation. If the approximation is not good, then it means that there are interactions in the model that are not captured by boosted decision trees, making them poor instruments for investigating interaction effects. Although authors have partially analyzed this trend in the appendix, I believe that not only this needs to be added to the main paper (as the authors seem to promise for the camera-ready), but a deeper quantitative evaluation is, in my opinion, necessary.\n\n*Preliminary Evaluation [Pre-Rebuttal]*\nI was totally convinced by the arguments provided by the authors and I do believe that the analysis submitted within the manuscript will be valuable and interesting for the ICLR audience. The paper can be still however improved by giving additional insights on the approximation of the ANOVA decomposition and this is something that I would kindly like to ask authors for the rebuttal stage.\n\n*Final Evaluation [Post-Rebuttal]*\nI am thankful to the authors for their clarification regarding ANOVA decomposition and I am inclined in confirming my initial score.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper analyzes dropout and shows it selectively regularizes against learning higher-order interactions. The paper received mixed reviews, with two in favor of rejection and one in favor of acceptance. Specifically, while all reviewers find the intuitions and ideas in the paper adequate/plausible, two reviewers didn't find sufficient evident that supports the conclusions. The reviewers provided very detail feedback, which the authors responded to, but it is apparent that some of the analysis needs to be reviewed again before the paper can be published.
Decision: Reject