Information Plane Analysis for Dropout Neural Networks
Authors: Linara Adilova
Bernhard C Geiger
Asja Fischer
Venue: ICLR 2023 poster (2023)
Keywords: information plane
deep learning
mutual information
dropout
continuous distributions

Abstract: The information-theoretic framework promises to explain the predictive power of neural networks. In particular, the information plane analysis, which measures mutual information (MI) between input and representation as well as representation and output, should give rich insights into the training process. This approach, however, was shown to strongly depend on the choice of estimator of the MI. The problem is amplified for deterministic networks if the MI between input and representation is infinite. Thus, the estimated values are defined by the different approaches for estimation, but do not adequately represent the training process from an information-theoretic perspective. In this work, we show that dropout with continuously distributed noise ensures that MI is finite. We demonstrate in a range of experiments that this enables a meaningful information plane analysis for a class of dropout neural networks that is widely used in practice.

Published as a conference paper at ICLR 2023
INFORMATION PLANE ANALYSIS FOR DROPOUT NEURAL NETWORKS
Linara Adilova
Bernhard C. Geiger
Ruhr University Bochum,
Know-Center GmbH ge beigers r eeee org
Faculty of Computer Science
linara. adi aeva@rrrrruus -ww.hiibbbhuum.... de
Asja Fischer
Ruhr University Bochum,
Faculty of Computer Science
as asja. . fi @acher@ruh--ur -bochum.de
ABSTRACT
The inforoition-theorriiic framework promises to explain the predictive power of neural networks. In particular, the information plane analysis, which measures mutual information (MI) between input and representation as well as representa- tion and output, should give rich insights into the training process. This approach, however, was shown to strongly depend on the choice of estimator of the MI. The problem amplified for deterministic networks if the MI between input and representation is infinite. Thus, the estimated values are defined by the different approaches for estimation, but do not adequately represent the training process from an information -heromtttonn-hhoreeiie perspective. In this work, we show that dropout with continuously distributed noise ensures that MI is finite. We demonstrate range of experiments that this enables meaningful information plane analysis for a class of dropout neural networks that is widely used in practice.
INTRODUCTION
The information bottleneck hypothesis for deep learning conjectures two phases of training feed- forward neural networks (Shwartz-Ziv and Tishby, 2017): the fitting phase and the compression phase. The former corresponds to extracting information from the input into the learned represen- tations, and characterized by an increase mutual information (MI) between inputs and hidden representations. The latter corresponds to forgetting information that is not needed to predict the target, which is reflected in a decrease of the MI between learned representations and inputs, while MI between representations and targets stays the same grows. The phases be observed via an information plane (IP) analysis, i.e., by analyzing the development representations and between representations and targets during training (see Fig. for example). For an overview of information plane analysis we refer the reader to (Geiger, 2022).
MI between inputs and
While being elegant and plausible, the information bottleneck hypothesis is challenging inves- tigate empirically. As shown by Amjad and Geiger (2020, Th. 1), the MI between inputs and the representations learned by deterministic neural network is infinite if the input distribution contin- uous. The standard approach is therefore to assume the input distribution to be discrete (c.g., equiv- alent to the empirical distribution of the dataset S at hand) and to discretize the real-valued hidden representations by binning to allow for non-trivial measurements, i.e., to avoid that the MI always takes the maximum value log([S]) (Shwartz-Zir and Tishby, 2017). In this discrete and deter- ministic setting the MI theoretically gets equivalent to the Shannon entropy of the representation. Considering the effect binning, however, the decrease of MI is essentially equivalent to geomet- rical compression (Basirat et al., 2021). Moreover, the binning- based estimate highly depends on the chosen bin size (Ross, 2014). To instead work with continuous input distributions, Goldfeld
Code for the experiments public https: http:://ggtttuuuueeeee com/ iinker/iin PP_dropuu..

---

Published as a conference paper at ICLR 2023
Information LeNet (training)
100
Gaussian drapout. LeNiet (training)
Gaursian dropout binning. LeNee (traiinn)
2.2
2.1
1.5
0.30
1.0
-
1.0
40
20
20
20
100
150
200
250
1000.0 1200.0
1400.0
1,00
DX.2
1022)
10.22
(a) Information dropout
(b) Gaussian dropout (our)
(c) Gaussian dropout (binning)
Figure 1: IPs w.r.t. the activations of one layer with information dropout Gaussian dropout in LeNet network. In contrast to the IP based on estimating MI using binning, our estimates (both for Gaussian and information dropout) clearly show compression. This suggests that even if MI is finite, the binning estimator fails to converge to the true MI (see also Section 4).
al. (2019) suggest to replace deterministic neural networks by stochastic ones via adding Gaus- sian noise to each of the hidden representations. This kind of stochastic networks is rarely used in practice, which limits the insights brought by the analysis.
In contrast, dropout, being a source of stochasticity, is heavily used in practice due to its effective regularizing properties. The core questions investigated in this work therefore are: i) Can we obtain accurate and meaningful MI estimates in neural networks with dropout noise? ii) And if so, do IPs built for dropout networks confirm the information bottleneck hypothesis? Our main contributions answer these questions and can be summarized as follows: We present a theoretical analysis showing that binary dropout does not prevent the MI from being infinite due to the discrete nature of the noise. In contrast, we prove that dropout noise with any continuous distribution not only results in finite MI, but also provides an elegant way to estimate it. This in particular holds for Gaussian dropout, which is known benefit generalization even more than binary dropout (Srivastava et al., 2014), and for information dropout (Achille and Soatto, 2018). We empirically analyze the quality of the MI estimation in the setup with Gaussian and information dropout in range of experiments on benchmark neural networks and datasets. While our results do not conclusively confirm or refute the information bottleneck hypothesis, they show that the IPs obtained using our estimator exhibit qualitatively different behavior than the IPs obtained using binning estimators and strongly indicate that a compression phase is indeed happening.
2 MUTUAL INFORMATION ESTIMATION FOR NEURAL NETWORKS
We use the following notation: Lower-case letters denote realizations of random variables (RVs), e.g.. b denotes a realization of the RV B: H(A) denotes the Shannon entropy of a discrete RV A whose distribution is denoted pa: M(B) is the differential entropy of continuous RV B whose distribution is described by the probability density function pb: [AA; B) is the MI between RVs A and B: X E X S R and Y E y are the RVs describing inputs a neural network and corresponding targets; f(XX is the result of the forward pass of the input through the network to the hidden layer of interest; Z is an N-dimensional RV deseribing the hidden representations.
The caveats of different approaches to measure the MI between input X and hidden representation Z of a neural network e.g., the MI being infinite for deterministic neural networks and continuous input distributions, the dependence of the MI estimate on the parameterization of the estimator, etc. were discussed widely in the literature (Saxe et al., 2019; Geiger, 2022) and are briefly reviewed in this section. These caveats do not appear for the MI measured between representations Z and targets Y. since the target is in most cases a discrete RV (class), for which MI is always finite.
One option for estimating [(X; 2) is to assume the input to be drawn from discrete distribution. This view is supported by the finiteness of the accuracy of the used computational resources (Loren- zen et al., 2021) and makes it easy to use finite dataset S to describe the distribution. In such setup, the distribution of (X, Y) is assumed uniform on the dataset S, and the discretization of Z is performed at fixed bin size (e.g., corresponding to the computer precision). The MI between
2

---

Published as a conference paper at ICLR 2023
X and the discretized 2 is computed as I(x; H(2) - -H(2) H 0 0 H()),
where ( 0 since f(-) and the discretization of Z are deterministic. Thus, the estimated MI between input and representation corresponds to the entropy of the discretized representation, which for small bin sizes equal to the entropy H (X) log |S| of the empirical distribution on the dataset, unless f(-) maps different points from the dataset to the same point in latent space.
A different option that is more aligned to the common description of worldw data is to assume X to be drawn from continuous distribution. If the network transformation f(-) results in a discrete distribution of the representations Z, one can use the decomposition T(X, 2) = ((z))H((]))) (Z) to estimate MI based on Shannon entropy, provided that the sample size is sufficiently large (note that the dimensionality N of Z may be large, and therefore the estimation of (2) may suffer from the curse of dimensionalityy. However, as shown in Theorem I of (Amjad and Geiger, 2020) for neural networks with commonly used activation functions the distribution of the latent representation is not discrete. In this case (i.e.. f(-) is deterministic, X is continuous, and Z is not purely discrete) the MI between X and Z is infinite?. By binning, i.e., by quantizing Z to a discrete RV 2, the MI [X; 2)= = H H() remains finite, but the qualitative behavior of this entropy will be defined by properties of activation functions and selected bin size (Saxe et al., 2019).
From the discussion above it follows that estimating (X;;2 2) in deterministic neural networks is an ill-posed problem, and that the estimates reveal not an rrfoommaiion-hhoreeee picture, but often rather geometric one that is determined by the properties of the chosen estimators. As solution to the aforementioned challenges, several authors have suggested to investigate the information planes of stochastic neural networks instead (Amjad and Geiger, 2020; Goldfeld et al., 2019). Goldfeld et al. (2019) proposed to add zero-mean Gaussian noise D to the representations during training- This transforms a deterministic neural network into a stochastic one that was shown to yield similar training results and predictive abilities of the model. The addition of Gaussian noise in 2 = f(X) + D guarantees finite MI¹ and therefore allows for estimating MI using Monte Carlo sampling with bounds on the estimation error. Futhermore, it links the courmationnthhorete perspective of the IP to geometric effects taking place in latent space. Indeed, when the MI between input and representation is decreasing, it means that noise-induced Gaussians centered at the representations of different data points overlap more strongly. Thus, it is becoming harder distinguish between inputs of the same class based on their representations, which translates into lower MI between representation and input while leaving MI between representation and target unchanged.
As discussed above, for continuous input distributions both the IPs of deterministic neural networks as well as of stochastic neural networks with additive noise show geometric picture (and in the former case the geometric interpretation is the only valid one, since MI is infinite in this case). Therefore, in this work we study the estimation of MI in networks with dropout layers, i.c., in settings where the stochasticity is introduced by multiplicative, rather than additive noise. In what follows we will investigate the requirements on the multiplicative noise for MI to remain finite, and whether the resulting IPs confirm the information bottleneck hypothesis.
3 MUTUAL INFORMATION IN DROPOUT NETWORKS
As discussed in the previous section, the MI between inputs and hidden representations of deter- ministic networks is infinite, if we assume the input distribution to be continuous. To overcome this problem, some form of stochasticity has to be introduced. While adding noise to activations (Gold- feld et al., 2019) indeed allows to compute the MI, this is not used in most contemporary neural networks. In contrast, neural networks with dropout are one of the most popular classes of neural networks used in practice and are stochastic in nature as well: Adding a dropout layer to a neu- ral network corresponds to multiplying the hidden representation with some form of random noise. Formally, denoting the random noise by a RV D of the same dimension as f(X), the hidden repre- sentation becomes Z f(X)( D, where denotes dement-wise multiplication. In the most basic form, D follows a Bernoulli distribution (Srivastava et al., 2014). Such binary dropout is widely used and can intuitively been understood as "turning off" fraction of neurons during training. There is a
2 "There are multiple mathematical derivations explaining why MI is infinite, one for example is discussed in (Saxe et al., 2019, Appendix C).
't least when the Ps and f() are such that f(X) has finite variance, then the finiteness of MI follows from the result about the capacity of the additive Gaussian noise channel, cf. (Cover and Thomas, 1991, eq. (10.17).
3

---

Published as a conference paper at ICLR 2023
variety of other dropout schemes, including multiplicative Gaussian noise, fast dropout (Wang and Mamning, 2013), or variational dropout (Kingma et al., 2015). Information dropout (Achille and Soatto, 2018) is variant that uses closed-form expression of MI as regularization term. In order to obtain such closed form, dropout noise is sampled from a log-normal distribution, and the prior distribution on representations is chosen depending the activation function (ReLU Softplus). We provide details on the derivation in Appendix A.I.
In this section, we investigate whether neural networks with dropout have indeed finite MI between input X and representation 2. While we first show a negative result by proving that binary dropout still leads to [X; 2) = oo, our Theorem 3.3 shows that dropout with continuous distribution keeps MI finite. This fact allows us to estimate MI for such dropout neural networks in Sections 4 and 5.
3.1 BINARY DROPOUT
We start by analyzing binary dropout, which forces individual neurons to be "turned off"" with some probability. More formally, the output of each neuron is multiplied with an independent Bernoulli RV that is equal to I with a predefined probability p. The following theorem shows that this kind of (combinatorial) stochasticity is insufficient to prevent T [(;; B) from becoming infinite.
Theorem 3.1. In the setting of (Amjad and Geiger, 2020, Th. 1), let the oulput f()) of a hidden
layer be parameterized as a deterministic neural network with Ñ neurons, let B E {0,1}² 1) be the set of independent Bernoulli RVs characterizing the dropout pattern, and let Z = fB (X) denote the output of the hidden layer after applying the random pattern B. Then it holds that I (X;; 2) = 00.
In the proof (provided in Appendix A.2) we use the fact that dropout mask (1, .,.., 1) leads to an infinite MI. While the Bernoulli distribution guarantees that b = (1, 1) always has non- zero probability, other distributions over {0, 1 might not have this property. Theorem 3.1 can however be generalized to arbitrary distributions over {0, 1
Theorem 3.2. In the setting of (Amjad and Geiger, 2020, Th. 1), let the output f() of hidden layer be parameterized as a deterministic neural network with Ñ neurons, let B E {0, 1]. be the binary random vector characterizing the dropout pattern, and let 7 =fB(X fx(X) denote the output of the hidden layer after applying the random pattern B. Then, it either holds that T(X: Z) = Do or that [X;; 2) if the dropout patterns almost surely disrupt information flow through the network.
The proof for the theorem is provided in Appendix A.3.
Both Theorem 3.1 and Theorem 3.2 cover special case the setting where dropout is applied to only a subset of layers, by simply setting those elements of B to 1 that correspond to a neuron output without dropout. If dropout is applied to only single layer, then fX) = fXX B', where B' is the dropout pattern of the considered layer and o denotes the element-wise product.
As consequence of Theorem 3.2, for neural networks with binary dropout any finite estimate of MI is "infinitely wrong", and the resulting IP does not permit an information-theerrtic interpre- tation. Essentially, the stochasticity added by binary dropout is combinatorial, and hence cannot compensate the "continuous" stochasticity available in the input X.
3.2 DROPOUT WITH CONTINUOUS NOISE
As proposed by Srivastava et al. (2014), dropout can also be implemented using continuous Gaus- sian noise with mean vector f 1 and diagonal covariance matrix To² with fixed variance d². Achille and Soatto (2018), in contrast, proposed log-normally distributed dropout noise, the vari- ance of which depends on the input sample x (this is termed information dropout). Generalizing both Gaussian and information dropout, in this section we consider continuously distributed mul- tiplicative noise D. In contrast to binary noise sampled from discrete distribution, continuously distributed noise turns the joint distribution of (Z, X) to be absolutely continuous with respect to the marginals of Z and X allowing for finite values of MI between the input X and the hidden repre- sentation Z. The following theorem states that the MI between input and the hidden representation of the dropout layer is indeed finite even if the variance of the noise depends on the input.
Theorem 3.3. Let X be bounded in all dimensions, f(-) be parameterized by a deterministic neural network with Lipschitz, activation functions, and let 7. f(X)= D(X), where the components of
4

---

Published as a conference paper at ICLR 2023
(X()) are conditionally independent given X and have essentially bounded differential entropy and second moments, i.e., [X), < M < 00 X almmsst surely, for some M and all i =....., N. Then, the conditional expectation E E[log(I [f(X)) > 0] is finite in each of its elements, we have [X;; 7) < cc.
noise D(X)
Theorem 3.3 (proof in Appendix A.4) can be instantiated for Gaussian dropout, where Dz(x) D; ~ (², and for information dropout, where Di(x)) ~ log.N a0, Note that for information dropout we have to ensure that the (learned) variance a² (z) stays bounded from above and below; e.g., in the experiments of Achille and Soatto (2018), a² (xx) is restricted to be below 0.7.
The requirement that the conditional expectation ([)))((((X)) I [ > is finite in each of its elements is critical for the proof. Indeed, one can construct a synthetic (albeit unrealistic) example for which this condition is violated:
Example 3.4. Let X' have the following probability density function
Pe (xx) (2 " if E [2" 2" + 1), n 1,
0, else
Evidently. E[X'== = co. Then, X = is bounded, since its alphabet is subset of (0, e0,e-7).
Now consider a neural network with a single hidden layer with one neuron. Let the weight from X to the single neuron be 1, and assume that the neuron uses a ReLU activation function.. Then,
E[log [ = E[log X|] = Ellog le
x' -- E[-X' -00
can be shown that in this example the probability density function of X (as well as of f(X)) is not bounded. Under the assumption that the probability density function pj of f(X) is bounded, the conditional expectation in the assertion of the theorem is finite: Assuming that pf S C < co, by the law of unconscious statistician we have
E., (x[log(I((X)a)) [(())
Tog())r((Dd)
11(x))l log(s)p; (g)());(s)df
Log Log(Ae:(nddf+
It is obvious that 12 is positive and finite. Due to the boundedness of pf we also have I1 2 C' Jo log(f)df = (fog((f 1)ld
However, the boundedness of py of is hard to guarantee for an arbitrary neural network. In contrast, the boundedness of Ps is more realistic and easier check. For bounded Pa we can prove (in Appendix A.5) the finiteness of the expectation E E[log(If(X)) [If(X) > 0] for ReLU networks:
Proposition 3.5. Consider a deterministic neural network function f(-) constructed with finitely many layers, finite number of neurons per layer, and ReLU activation functions. Let X be continuously distributed RV with probubility density function Pa that is bounded (pa < P < oo)
and has bounded support X. Then, the conditional expectation [E]og((](()) [S(X)) > 0] is fanite in each of its elements.
Finally, note that Theorem 3.3 assumes that the network is deterministic up the considered dropout layer. This does not come with loss of generality for feed-forward networks (c.g.. with no residual connections): Indeed, one can apply Theorem 3.3 to the first hidden layer representation 2 with dropout, where this assumption always holds. Then, for the l-th hidden layer and irrespective of whether this layer also has dropout, the MI I(X; Z is finite due to the data processing inequal- ity (Cover and Thomas, 1991, Th. 2.8.1). Therefore, Theorem 3.3 ensures that MI is finite for all hidden layers after the first continuous dropout layer.
4 ESTIMATION OF MI UNDER CONTINUOUS DROPOUT
We now consider estimating [(X; Z) in networks with continuously distributed dropout, starting with information dropout. As discussed by Achille and Soatto (2018), networks with information
5

---

Published as a conference paper at ICLR 2023
dropout are trained with the cross- cross-entropy loss tee (which is involved in the known variational lower bound I ((Z 2 H(Y) lee) and regularized using a variational upper bound on I(X; Z). There- fore, estimates of the quantities displayed in the information plane are directly used in the training loss and, thus, easy to track, at least for softplus activation functions*.
In the case of Gaussian dropout, to estimate I(X; Z) we approximate ((Z) and ((( separately (pseudocode is given in Algorithm in Appendix A.6).
For estimating h(Z) we employ Monte Carlo (MC) estimate, simi- lar to the one proposed by Gold- feld et al. (2019). That is, we approximate the distribution of Z as a Gaussian mixture, where we draw samples f 1,.... and place Gaussians with a diago- nal covariance matrix with variances a² (==(()))))) ,i,, = I,...,N N on each samplef(x()) For sanity check, we also compute an upper bound of h(Z) given by the entropy of a Gaus- sian with the same covariance matrix as Z. Note that the estimation of the upper bound requires a sufficiently large number of samples to guarantee that the sample covariance matrix is not singular and that the resulting entropy estimate is finite.
MC estimate hZ))K)
estimate MZ)KK
-38.5
-385
-1.8
-40.0
59000
300300
150030
52020
300030
150000
(a) 0.1, n. 2
(b) 0 0.1, 50
Figure 2: Independent of the dimensionality, MC estimation
of (x) stabilizes with increasing amount of samples.
For each fixed x the conditional dis- tribution P=|z is a Gaussian distribu- tion (f(((x), (f
2.2
Moreover, when the input is fixed, the components of Z|X x are in- dependent, since components of the noise are independent. This allows to compute h(ZlX) as sum of h(Z, X) where Z; is the i-th compo- nent of the representation vector. The computation of X)) requires in- tegration over the input space for computing the mathematical expec- tation E, [x,, = x²). This can be approximated via MC sampling. That is, we approximate h (x) by 1s|
100
8D
Gaussian estimate
Mixture estimate
Gaussian estimate
Mixture estimate
10"
10
10'
107
10
Numbor samples
Namber samples
(a) 0.1, n
(b) 0 = 0.1, n = 50
Figure 3: Estimates of the differential entropy h(Z) of the hidden representation Z. With growing dimensionality of X, the Gaussian upper bound becomes very loose, com- pared to the Gaussian mixture-based MC estimation.
1/|8| 1=1 b = 20) where M(Z, [ = x0) = (og(ll(()))))))))))))))))))))))) (ooV2TT)
We consider simple toy problem for validating our approach to estimating MI: the input X is generated from an n-dimensional standard normal distribution, modified with a function f(X) = 2X 0.5, and then subjected to Gaussian dropout distributed according to (11, o²). We investigate the convergence of our estimator for h (Z for increasing number of samples. For each input data point, we generate 10 noise masks, thus obtaining 10 samples of for each 200), The results in Fig. 2 show that the estimation stabilizes with larger amount of samples for different dimensionality of the data. We also compare the estimate to the upper bound for (Z)) in Fig 3.
We finally compare estimation of MI to binning, the EDGE estimator (Noshad et al., 2019), and the lower bounds analyzed by McAllester and Stratos (2020). The results are shown in Fig. 4. In the plot, doe stands for the fffferenceeoo-enrropiie (DoE) estimator and doe_ stands for DoE with logistic parametrization (McAllester and Stratos, 2020). The binning estimator underestimates the
'Indeed, for softplus activation functions. the variational approximation of I (X; 2) is available in closed form, while for ReL.U. activation functions, the available expression is only useful for minimizing, rather than for computing, T(X; Z) (see Appendix A.

---

Published as a conference paper at ICLR 2023
Size bins
0
0
0.4
0.6
0.6
0.0
0.2
0.4
0.6
0.8
lootsize)
logistbll
our (upper)
our (upper)
(mixturet
our (mixtture)
EDGE
100
EDGE
doa
doe doe.
2
binning by num
50
binning
binning by
binning by sizo
0
20
40
60
90
100
0
20
40
60
8D
100
Nu moer pins
Number of bins
(a) 0.1, n
(b) 0 = 0.1, n 50
Figure 4: Comparison of various approaches to MI estimation for the toy example with multiplica- tive Gaussian noise. For low-dimensional X and Z, different bin sizes lead to different MI estimates of the binning estimator. For higher dimensions, the binning-basse estimate collapsing. Our es- timation is very close to the lower bound estimation proposed by McAllester and Stratos (2020), while still being larger as expected.
MI when the bin size is large and overestimates it with small bin size (Ross, 2014). which can be clearly seen in the plots where bins organized both by size (upper axis) and by number (lower axis). Moreover, with the high-dimensional data, binning hits the maximal possible value of log(( very fast, not being able to reach larger MI values. According McAllester and Stratos (2020), lower bound-based MI estimators (e.g., MINE (Belghazi et al., 2018)) also need exponentially (in the true value of MI) many data points for good value approximation, otherwise they will always heavily underestimate the MI.
Further plots for different dropout variances and inputs dimensionality are given in Appendix A.6.
5 INFORMATION PLANE ANALYSIS OF DROPOUT NETWORKS
We use the estimators described in the previous section for an IP anal- ysis of networks with Gaussian and
Gaussian dropout, FC. (trining)
Gaussian dropout birvingg EC (traiing))
information dropout. We always con-
2.1
2,00
sider only the representation corre- sponding to the first dropout layer and measure the MI in nats, e.g., the natural logarithm. For estimating (Y; 2). we employ the EDGE esti- mator (Noshad et al., 2019) for Gaus- sian dropout and variational estimate for information dropout. IPs created using the binning estimator use bin- ning for both T(X; Z) and T(Y; Z).
300
1.40
3.8
30
50
40.0
50.0
6.00
8.00
10.00
(002)
(X:Z)
Our estimator
(b) Binning estimator
Figure 5: IPs for a FC network with Gaussian dropout trained on MNIST. Compared to the binning estimation of MI our approach shows compression.
In the first set of experiments we in- vestigate the difference between IPs obtained via our proposed estimator and via binning. The analysis on the MNIST dataset was performed for LeNet network (LeCun et al., 1998) that achieves 99% accuracy and a simple fully-connected (FC) network with three hid- den layers (28 x 28 2221-12- - 128 - 32 - 10) and softplus activation functions achieving 97% accuracy. We analyze both information dropout and Gaussian dropout in the LeNet network and only Gaussian dropout in the FC network. In both cases dropout is applied penultimate layers. We compare IPs based on binning estimators to IPs based on our estimators in Fig. and Fig. 5.
$ 'This makes the MI estimation more efficient, since the previous part of the network is deterministic which allows for an analytical expression of /1( b/2|X- = x). Note however, that the estimation could be extended to higher layers well since for those MI also remains finite. However, an estimator different from ours should he used for those layers.

---

Published as a conference paper at ICLR 2023
We also analyze the IPs for ResNet18 trained on CIFARIO (see Fig. 6), where we added addi- tional bottleneck layer with 128 neu- rons and Gaussian dropout before the output layer, and which achieves an accuracy of 94%.
Gaussian drapout, ResNie11 (trininn)
Gaussian
binning. RecNietl (train.)
2.00
1.350
1.75
210
1.525
150
1:30
1,500
1:25
1.275
20
1250
50
1.225
Interestingly, for all networks and datasets we observe sigmificant com- pression for our estimator and lack of compression for binning estima- tors (also for different bin size, see Appendix A. This indicates that either the MI compression measured in dropout networks is different from purely geometrical compression, or that the number of samples |S| is insufficient to reliably estimate [X; Z) by binning.
0.50
1,200
1X:23
(a) Our estimator
(b) Binning estimator
Figure 6: IPs for a ResNet18 network with Gaussian dropout trained on CIFARIO. In contrast to the binning-based esti- mator of MI our approach clearly shows compression.
In the second set of experiments, analyze IPs in information dropout networks, with MI estimations as de- scribed before. To this end, we trained fully convolutional neural network (fullCNN) on CIFARIO us- ing code provided by Achille and Soatto (2018). Training proceeded for 200 epochs using SGD with mo- mentum and, different from the origi- nal setup, with only one dropout layer after the third convolutional layer. The batch size was set to 100, the learning rate was initially set to 0.05 and was reduced by multiplying it with 0.1 after the 40, 80, and 120 epoch. The network was trained with different values of the regularization weight 3 and different amounts of fil- ters in the convolutional layers. That is, the full-size fulICNN has 3 layers with 96 filters succeeded by 4 lay- ers with 192 filters, while only 25% of these filters are constituting the small network. Also different from the original setup, we allowed the noise variance grow up to 0.95 in order to see the effect of the limited
mg'ymaation dropout. fullCNN (trainino)
Information cropaut, CulCNN (training)
2.
255
150
330
100
3.5
300
0.5
0.5
00.
18.7)
(a) fullCNN with 3
(b) fullCNN with 3 = 20
iggomatiion dropout, fullCNN (trainina)
1ofg Information cropout, NulCNN (traininol 300
25
2.0
3..
100
1.5
300
1.0
50
50
0.5
00.
500
1000
1500
2000
2500
0.0
2005
1500
2000
2500
(052)
(X:2)
(c) 0.25fulICNN with | =
(d) 0.25fulICNN with / 20
Figure 7: IPs demonstrate more (a), (c) and less (b), (d) compression of MI between input and representation de- pending on B. The values of I(X; Z) are smaller for the smaller network (c) and (d).
information between representation and input more pronounced. Results are shown in Fig. 7. It can be seen that regularizing 1(X; 2) is effective (i.e., larger values of 3 lead to smaller [(X; Z)), and that regularizing too strongly (3 = 20) leads to worse performance: the test error is 5% higher and train error is 10% higher. We can further see stronger compression for smaller B and almost no compression for larger B. We conjecture that compression can only become visible if sufficient information is permitted to flow through the network (which happens only for small (8). Fig. 7 (c) and (d) show the IPs for the small fulICNNN. It can seen that the smaller network appears not to compress at all (see Fig. 7 (c)), but that [(X; Z) rather increases throughout training until it is at the same level as in Fig. 7 (a). This indicates that 3 determines to which point in the IP information compresses, and that the IP curve that is traversed during training depends the overall capacity of the neural network.
Plots for the additional experiments can be found in Appendix A.8.
8

---

Published as a conference paper at ICLR 2023
6 DISCUSSION
Whether not cntommation-thorretiie compression is correlated with improved generalization is the main question connected to and the most prominent justification for information plane analysis of deep neural networks. Such connection, however, can only be tested for neural networks for which MI is finite and therefore measurable. In our theoretical analysis, we investigate if different variants of dropout noise allow for finite values of MI under assumption of continuous input distribution. We answered this question positively by showing that in networks with certain constraints on the induced distribution of the representations, continuous dropout noise with finite differential entropy prevents TX;; Z) from becoming infinite. We have further shown that these constraints on the distribution of the representation are satisfied in ReLU networks if the probability density function of the input is bounded.
Following this conclusion we propose an MC-based estimate of MI Gaussian dropout networks and perform an IP analysis for different networks with Gaussian and information dropout on dif- ferent datasets. The experiments show that the binning estimator behaves very differently from our estimator: While our estimator mostly exhibits compression in the IP, the binning estimator does not. Further, the values of T(X; Z) for our estimator are often orders of magnitude larger than the values of [(; Z), especially when compared to the binning estimator. Assuming that the proposed estimators are reasonably accurate, this makes a connection between iiffrmmtiiontthooree pression and generalization questionable. While these preliminary experiments do conclusively answer the question such a connection exists, they show a practically relevant setting in which this correlation can be studied.
com-
The discrepancy between the binning estimator and our estimator further suggests that either the informatiionthhoreee theoretic compression we observe using our estimator not geometric, or that there are insufficient samples to obtain reliable estimates from the binning estimator. This is in contrast with the work of Goldfeld et al. (2019), which showed that information-theoretic and geometric compres- sion were linked in their networks with additive noise. We thus believe that closer investigation of whether multiplicative noise induces geometric compression, and whether the induced compression improves generalization performance, are interesting questions for future research.
ACKNOWLEDGEMENT
The authors want to thank Michael Kamp, Simon Damm, Ziv Goldfeld, and Jihao Andreas Lin for valuable discussions about the work.
Asja Fischer acknowledges support by the Deutsche Forschungsgemeinschaft (DFG, German Re- search Foundation) under Germany's Excellence Strategy EXC 2092 CASA 390781972.
The Know-Center is funded within the Austrian COMET Program Competence Centers for Ex- cellent Technologies under the auspices the Austrian Federal Ministry of Climate Action, En- vironment, Energy, Mobility, Innovation and Technology, the Austrian Federal Ministry of Digital and Economic Affairs, and by the State of Styria. COMET is managed by the Austrian Research Promotion Agency FFG.
REFERENCES
Alessandro Achille and Stefano Soatto. Information dropout: Learning optimal representations through noisy computation. JEEE Transactions on Pattern Analysis and Machine Intelligence, 40 (12):2897-2905, 2018.
4, 5, 8,
Rana Ali Amjad and Bernhard C. Geiger. Learning representations for neural network-based clas- sification using the information bottleneck principle. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(9))2225-2239, September 2020. doi: 10.100/7PAM1221192990031
open-access:
arXiv: 1802. 09766 [es LG]. 1, 4. 12
Mina Basirat, Bernhard C. Geiger, and Peter M. Roth. A geometric perspective on information plane analysis. Entropy, 23(6):711, 2021.

---

Published as a conference paper at ICLR 2023
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation. Proc. ICML, pages 531- 540. PMLR, 2018.
Thomas M. Cover and Joy A. Thomas. Elements information theory. New York: Wiley, 1991. 3, 5, 13
Bernhard C. Geiger. On information plane analyses of neural network classifiers - review, Decem- ber 2022. open-access: arXiv:2003: 09671 [cs LG]. 1,2
Ziv Goldfeld, Ewout van den Berg, Kristjan Greenewald, Igor Melnyk, Nam Nguyen, Brian Kingsbury, and Yury Polyanskiy. Estimating information flow in deep neural networks. In Proc. ICML, 2019. 6,3,6,9
Durk P Kingma, Tim Salimans, and Max Welling. Variational dropout and the local reparameteri- zation trick. Advances neural information processing systems, 28, 2015. 4
Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. wraddeen-bbasd learning applied to document recognition. Proceedings of the IEEF, (6(11):227--233 1998. 7
Stephan Sloth Lorenzen, Christian Igel, and Mads Nielsen. Information bottleneck: Exact analysis of (quantized) neural networks. arXiv preprint wwXiv:2166.12912 2021.
David McAllester and Karl Stratos. Formal limitations on the measurement of mutual information. In Proc. AISTATS, pages 875-884. PMLR, 2020. 6,
Morteza Noshad, Yu Zeng, and Alfred O Hero. Scalable mutual information estimation using de- pendence graphs. In Proc. ICASSP, pages 2962-2966. IEEE, 2019. 6,
Brian C. Ross. Mutual information between discrete and continuous data sets. PloS one, 9(2): c87357, 2014. 7,7
Andrew M Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan D Tracey, and David D Cox. On the information bottleneck theory of deep learning. Journal of Statistical Mechanics: Theory and Experiment, 2019(12):124020 2019. 3,
Ravid Shwartz-Ziv Ziv and Naftali Tishby. Opening the black box of deep neural networks via informa- tion. arXiv preprint arXiv: /703 0000000001,,... 2017.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, llya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, (5(1):1929-195) 1958, 2014. 2, 3, 15
Sida Wang and Christopher Manning, Fast dropout training. In Proc. ICML, pages 118-126. PMLR, 2013.
10

---

Published as a conference paper at ICLR 2023
A APPENDIX
A.1 INFORMATION DROPOUT
One type of dropout with continuous noise is termed information dropout (Achille and Soatto, 2018). It is a technique that combines dropout noise sampled from a log-normal distribution learnable parameter dependent on the parameters
Pc log.N (0,))))))) where ag 0 of network, and the introduction of regularization term (T(P= lay, I P=,). This regular- ization term based on an information bottleneck objective for training neural networks: Rewriting the information bottleneck Lagrangian and adding a disentanglement term (i.e., we want each el- ement of representation to be independent of the others) results in the aforementioned formula. Additionally, it is proposed to use as prior ps, defined by the choice of activation function (ReLU or Softplus), a particular distribution whose validity is empirically verified. Such priors and selected dropout noise allow for deriving closed form of KL-divergence, which makes it easy to directly track IP values while training.
In the following, provide the closed form for computation of T(X; Z) proposed by Achille and Soatto (2018):
(e. log log (2, drdz
I(X; Z)
(x(
/ Ps (x())) log
(a(())ata(2))
drds f Pa Dolt))KL(DAtllllddddd
(=(
LK
Empirically we can approximate this as T(X; = Σ}= dataset of size |S| of samples of X.
lp=), where we sum over the
First, we discuss ReLU neural networks. The prior distribution ps in this case consists is a mixture of two parts: and improper log-uniform distribution and a point mass at O. Such prior is empirically valid for ReLU activations. First we restrict the derivation to the case when f f(X) * (which in
turn means that Z * 0, since noise is log-normal and cannot be 0). In the following we will omit the subscript of probability density functions, when is clear from its argument.
K (P3)a0))) ll.))
K (P (PPooog))))))))))
(1)
((log(z(a(0)) log p(log(z|x(0)) dz p(lox(=))
/ p(log(e) + (o log p(log(e) + log(
de (2)
/ p(log(e)) log(())g)))()(((( - / p(log(e)) log(c)de
(3)
/ p(log(e) log(e)))))((((((()))) -lo(()) log(c) = - hlog((e)) - log(c)
(4)
-(log(a(< + log(2 10g((1m log(c)
(5)
where equation 1 holds due to the invariance of the divdeveeee under parameter transformation with a strictly monotone function (log(-)); equation 2 holds since log( log(c) + (X)( and (log(=) for the improper log-uniform distribution; equation 3 is taking into account that
Patconst Pa. that (z)( ( is constant, and that Piog(e)]a()> Piog(e) because is indepen- dent of X; equation uses that f Pl = 1; finally equation holds because log(e) is normally distributed and its entropy can be computed in closed form.
Now we put fX) = O, and also get = Then Pz|x = du (point mass or Dirac delta) and MI becomes:
K (T(psjo0 (0Pp))))ppp>>>>>>>>>>))))>))))) f (Palo()) log
(P)lx(2) Pz (2)
dz / 80 log
0o dz log(g) (6)
900
11

---

Published as a conference paper at ICLR 2023
where 9 is the weight of the point mass in the prior P:-
Combination equation and equation results computable [[x; 2). As it can be seen, one has to conrectly combine non-zero and zero values of f(X) and also know the parameters of the prior Pz: constant с and weight q. This makes it not practical for IP analysis.
If instead of ReLU the network has softplus activations, then the prior the representations distri- bution standard log-normal instead of log-uniform with delta Dirac. this case the computation very simple, since KL divergence between two log- lor-normal distributions is computed as KL diver- gence between corresponding normal distributions:
(a (x!t)) + +1²) log(a(x()))
K [ lpps)
20
where x² = and μ = 0 are known parameters of the prior. Thus, softplus activations (equation 7) allows for direct computations of [[X; Z).
A.2 PROOF THEOREM 3.1
Proof. Using the chain rule of MI, we have
T(X; Z) [X;;
= + - I(B;
21x;; ZIB) H(B)
where the inequality follows from dropping /(B;X) since B and X independent and the fact that T(B; Z) < (B). Having B E 10, as a discrete RV, it immediately follows that H (B) < Ñ log 2. Now note that
T(X; B) =
Σ = b)(x; = b).
6E10.13.
Since the Bernoulli RVs independent, positive probability mass is assigned to b 11, 1), iee, to the case where all neurons are active. Evidently, when h = 1,,,, 1) it follows that Z = f(X). Thus, with (Amjad and Geiger, 2020, Th. 1)
I(X; > P(b 1, f(x()
and 1(x;2) = 00.
A.3 PROOF OF THEOREM 3.2
Proof. If the binary dropout is such that nonzero probability is assigned to the dropout mask b = (1,1, 1), then the statement of the theorem follows as the proof of the theorem 3.1.
Assume now that B is such that zero mass is assigned to b = (1, 1,... 1). suppose that the distribution of X has a portion with continuous probability density function on compact set and that the neural network has activation functions that are either bi-Lipschitz or continuously differentiable with strictly positive derivative (following the requirements of Amjad and Geiger (2020, Th. 1)). Then, obtain I [Xx; f(X)) for almost all parameterizations of the neural network. Under this setting, fx(X) is again a neural network with activation functions that either bi-Lipschitz
To treat this case, we
from (Amjad and Geiger, 2020, Th. I)
continuously differentiable with strictly positive derivative. Assuming that is such that the input of the network is not completely disconnected from the considered layer, for this pattern have I(X; Z| b)
Otherwise, we obviously have / [(X; 2 b) = 0. The statement of the theorem follows from taking the expectation over all patterns b.
A.4 PROOF OF THEOREM 3.3
Proof. W.l.ogg we first restrict our attention to the dimensions of representations Z that different from zero. Specifically, suppose that Z = (Z,...., Zn) and that B = (B)..., Bx) with B, if 7, and B; otherwise. Clearly, B is function of Z, hence T(X; T(X; ; 7, B)
12

---

Published as a conference paper at ICLR 2023
I[B; + (IB; Since B is binary, we have that [( < H(B) S n nlg Let ZB (Z,li: B; = 1) denote the sub-vector of zero elements of Z, then
[(X; & nog + Σ P ()I
where, if = b. ((Z; to MI. Therefore, I(X; Z) is finite iff I [Z2;: = IZ; X B = b) is finite B-almost surely. We thus now fix arbitrary B b and continue the proof for Z Za-
= holds because constant (i.e., 0) RVs do not contribute
We decompose MI into differential entropies as / (Xx 2) = h(Z)-h(Z)x) The differential entropy of the representations (2)) is upper-bounded by the entropy of a Gaussian RV with the same covari- ance matrix as the distribution of = (Z1-
Zx), i.e., by N/2 log(2 + 1/2 log det (E) +
N/2. From Hadamard's inequality and since E is positive semidefinite it follows that det(E) S
Th o.. where of are diagonal elements of the covariance matrix, i.e., of = V Var[Za [Z:] This vari-
can be bounded from above. Specifically, since X; is bounded and [(-) is a composition of
Lipschitz functions, f(X); is bounded as well. Recalling that E(D:(z)²) I M holds X-almost surely, this yields
Var [2]
X]]
S ME, S (
remains to show that the h(2(X) > -o.. Due to the conditional independence D; and D given X, for all i + j. the conditional differential entropy of Z factorises in the sum of conditional differential entropy of its components, i.e., X) Σ ((Z,(X). We write this conditional entropy as an expectation over X and obtain using (Cover and Thomas, 1991, Th. 9.6.4)
xZ = E. E.. = = E. (<)))))))))))))))))))))))))))))) x)]
E. [ED((()))) = =)] EE, [loo((((((((())))
by the formula of change of variables for differential entropy. Both terms are finite as per the assertion of the theorem. The first term is finite since we assumed that the differential entropy of D2(X) is essentially bounded, i.e., there exists number < oo such that h (x)) S C X-almost surely. The second term is finite since we assumed that the conditional expectation IE ((log((f(X))) [f(x)] > 0] is finite in each of its elements, and since Z; + 0 implies [X()), > This completes the proof.
A.5 PROOF OF PROPOSITION 3.5
Proof. We assume w.l.o.g. that f() has range with dimension D 1, i.,. f: - R, where X' C R" the function domain. The proof can straightforwardly extended to the several dimensions of f()).
Since f(-) is constructed using a finitely-sized neural network with ReLU activation functions, is piecewise affinely linear finite partition of the function domain. The fact that E[lg([f(X))) [f(X] > 0j < co follows then immediately from the fact that X, and thus [f('),, is bounded.
To investigate whether E[log(I [f(X) o] -oo, split domain X in the following partitions:
1. X = f- ({0}) denotes the element of the partition on which f(X) vanishes;
2. {X,)) {X.)i=l.... Ji=l....! denotes elements of the partition of X which f(X) Cii i.e. on which
f()) is constant;
3. X = - X:n denotes the union of the all other sets {2 J of the partition, where
f(.) is not constant.
For the last subset, define the function f x - RD via f(x) (1f())), I3, ...,, .. TT)). Note that f(-) is piecewise bijective, hence W f(x) has probability density function that obtained
13

---

Published as a conference paper at ICLR 2023
from the change of variables formula:
Σ (x)
Pai (10)
[J (x(])
26 xEf-1(0)
where (1(x)= = [ae; (x)] is the Jacobian matrix of f(),, with fi(z) - If())) and fz(2)) Ij for all
of j 2. It follows that Jacobian matrix is diagonal and has determinant āe. (2:)]- The density xr of the conditional random variable W lf(X)) X E X² can be then obtained by marginalization from Pa:
Pooxx = / Pui (w(w, x )xx =
Σ
Ps (a)
dx
(8)
Ø1 Das.
(x))
ØEF feff-(ww,a) (w.az)
where æz =
and where we perform an (n - 1o-f integral.
Thus, by the Lebesgue decomposition, the distribution of W = |s(X) can be split into an absolutely continuous component with a probability density function Pw a and a discrete component with
finitely many mass points, for which we have P(W = c;) = -xxe P (r)dd =: px(2))) By the law of unconscious statistician, we then obtain
E| (()g(()((X) [[(x)] > 0]
[loo(W) W > 0]
(X() ) log cil + Px / log(w)Pwlxe (w)dw
{ log(w)pwix= (w)) (Xx
Po(2))) log cal + po(2))
log ( Purlx (w)dw
where in the last line we split the integral at a fixed < 1. Clearly, the first sum is finite since > 0 for all i. For the remaining summands involving integrals, suppose for now that Puo (w) I C 00. Then,
[ Pu log(w)dw 2
L/ C' C'log(w)dww = C(e log(t) -e) > -00
I
J Pio log(w)dw
poc
Pw 1 (1-1)
000
Pu (1- dw 2
1 E
I
>
dw 2
-c.
w
We thus remain to show that Xo (w) & C for w E [0.c). To this end, we revisit equation 8 and note that the integral finite i) Pr bounded, ii) the integration over a bounded set, and iii) de, (x) 2 él O. Conditions i) and ii) are ensured by the assertion of the lemma. It remains to show that condition iii) holds.
Note that in contrast using f(x) (2, ... T.), the same Pwi (x)xx(w) can also be obtained by using the piecewise bijective function f(x) = (f()), 23 I....., Rn)) etc. Hence, Pujp (w) < C if the partial derivative of f is bounded from below for at least dimension, i.e., there exists an i such that
0f (x)) 2 61- Since we have Seg
2
0f
|V,((>)|
Σ
Fr: (x)
this equivalent to requiring that the L, norm of the gradient bounded from below. Indeed, remember that f is piecewise affinely linear with finitely many pieces, and its restriction to X is non-constant. On its restriction to X we thus have Vzf(z) = gi > for all æ X and some i E {l...., m). Hence, we can find an e such that min; 9i 2 61 0. which implies that there exists an i for which
Øf 8z (=)) é1 for all 2 This completes the proof.
14

---

Published as a conference paper at ICLR 2023
A.6 ESTIMATION OF MI UNDER GAUSSIAN DROPOUT
In the Algorithm we describe how the estimation of T(X; Z) with Z being representation under Gaussian dropout can be done. This is the way we estimated MI for our experiments, but any other estimator can be used in this setup.
Algorithm Estimation of MI under Gaussian dropout
Require: GMM-MEANS, a, nonoise-reprs
Amount of Gaussians in GM for approximation;
noise variance; no noise representations reprs <- []
Generate noisy samples with corresponding variance
for all nr in nonoise-reprss do
for i <- 1, do
6 - noisep
reprs reprs
end for end for points <- nonoise-reprs:: GMM-MEANS)
Create a GMM on restricted amount of points for
faster computation d +-]
for all p points do
d x- d + Gaussian(p, lpl)
end for
gmm MixtureModel(c)
Ip <
> Get estimates of log-probabilities from GMM for noisy samples
for all in reprs do
Ip + lp gmm comm.log.proiie probablogrrobbbitty((()
end for
h(2) t mean((p)
h(2(2)) +-
>> Compute conditional entropy using closed form formula > For each dimension of the representation
for i + 1, (reprssss do
h(z(z) <- h(2/z) + mean(In( (Tre :
D Use no noise representations
here, each dimension separately end for
I( 2) t h22) - h(2lz)
> Obtain final estimate for the MI
A.7 EVALUATION ESTIMATOR
Fig. shows upper bounds and estimation of h(2) with a higher noise than the Fig. 3. Larger noise increases the gap between the Gaussian entropy based upper bound and the mixture based estimation as expected.
In Fig. 9 we convergence of the MC estimate for X(Z under larger noise.
As expected larger noise variance results in smaller MI values (Fig. 10), while the trend observed when changing dimensionality stays the same.
A.8 INFORMATION PLANE ANALYSIS
Note, that in the experiments we analyze IPs on the training samples and test samples separately. In order obtain a valid sample of hidden representations for the MI estimation during inference, apply MC-Dropout, as opposed to the usual way of performing inference with dropout being turned off. According to Srivastava al. (2014) this is the theoretically sound way to obtain predictions, while turning off dropout and re-scaling weights results in an approximationthat allows for faster computation.
In Fig. 11, Fig. 12, and Fig. 13 we provide IPs built on the test set of the corresponding datasets (MNIST, MNIST, and CIFAR10).
15

---

Published as a conference paper at ICLR 2023
Gaussian estimate
110
Mixture estimate
2.4
100
h(Z)
Gaussian estimate
2.3
Mixture estimate
80
2.2
70
10a
102
10a
10
10
10a
10a
10
Number of samples
Number of samples
(a) Noise variance 0.4, dimensionality 1
(b) Noise variance 0.4, dimensionality 50
Figure 8: Entropy of the hidden representation. It can be seen that with growing dimensionality the Gaussian upper bound becomes very loose, compared to the Gaussian mixture estimation.
MC estimate h(Z
1.3
29.5
h(Z)
1.2
hiz(x]
29.0
28.5
MC estimate h(Z[X)
28.0
50000
100000
150000
50000
100000
150000
Number of samples
Number of samples
(a) Noise variance 0.4, dimensionality 2
(b) Noise variance 0.4, dimensionality 50
Figure 9: Conditional entropy of the hidden representation. Independent of the dimensionality the MC estimation of f(Z[X) stabilizes with increasing amount samples.
Size of bins
0.3
0.4
0.6
0.8
0.0
0.6
logisize)
logísize)
(upper)
our (upper)
our (mixture)
(mixtura)
4
EDGE
doe
doe
doe.
binning num
binning by num binning
binning size
20
80
100
80
100
(a) Noise variance 0.4, dimensionality 1
(b) Noise variance 0.4, dimensionality 50
Figure 10: Comparison various approaches to MI estimation for the setup the multiplicative Gaussian noise.
16

---

Published as a conference paper at ICLR 2023
Informetion dropout. Lefett (validation
Goussian Lefet (validetion)
Gouszian
binning, Leklet (validation)
222
0.40
0.39
1.0
40
20
30
200
10000
1,50
2,00
2,50
DX.2
1002)
(a) Information dropout
(b) Gaussian dropout (our)
(c) Gaussian dropout (binning)
Figure 11: Compared to the IP analysis based binning (discrete) estimation of MI, the IP based on our approach shows compression as well for Gaussian as for information dropout.
Gaussian dropout, FC (validation)
Gaussian dropout binning, FC (validation)
2.00
1.80
[0Y:2)
1.9
Epoch
[X:ZZ)
1.60
1.8
50
50
1.40
1.7
0
0
250
500
750
1000
1250
4.00
6.00
8.00
((:2)
KX:Z)
(a) Our estimator
Binning estimator
Figure 12: Compared to the IP analysis based on binning (discrete) estimation of MI, the IP based on our approach shows compression.
In the Fig. 14 provide additional IPs for the binning estimator with varying amount of bins used for MI estimation. We report the results for the fully-connected network trained on MNIST with Gaussian dropout variance 0.2.
In the Fig. 15 we show the IPs obtained for the same fully-connected network trained on MNIST with the variance of the Gaussian dropout set 0.4.
17

---

Published as a conference paper at ICLR 2023
Gaussian dropout, ResNet18 (validation)
Gaussian dropout binning, ResNet18 (val.)
1.4
1.75
150
1.2
1.50
1.0
1.25
1100
Ipoch
100
Epoch
1.00
0.8
:50
500
0.75
0.6
0
0.4
0
120
140
10
12
[(::Z)
([:Z)
(a) Our estimator
(b) Binning estimator
Figure 13: Comparing to the discrete estimation of MI for IP analysis approach clearly shows compression.
Gaussian dropout binning, FC (training)
Gaussian dropout binning, FC (training)
2.00
150
150
1.80
I0::2)
Epech
100
100
1.60
0.02
1.40
0.50
1.00
6.00
8,00
10.00
(X:Z)
(X:Z)
(a) Amount of bins 3
(b) Amount of bins
Gaussian dropout binning, FC (training)
Gausslan dropout binning, FC (training)
2.30
2.25
150
2.30
150
2.20
100
2.30
100
2.10
2.30
2.05
9.50
10.00
10.50
11.00
10.99
10.99
11.00 (X:Z)
11,00
11.00
(X:2)
(c) Amount of bins 15
(d) Amount of bins 30
Figure 14: Larger amount of bins used for estimation of MI leads collapse of the IP into point corresponding to the amount of samples available. All the smaller amount of bins demonstrates no compression terms of 2(X;
18

---

Published as a conference paper at ICLR 2023
Gaussian dropout, FC (training)
Gaussian dropout binning, FC (training)
1.40
1.80
1.20
NY:-Z)
100
100
1.00
1.40
50
1.20
50
0.80
1.00
0
30.00
35.00
40.00
6.00
8.00
10.00
((::Z)
1(:7))
(a) Our estimator (training)
(b) Binning estimator (training)
Gausslan dropout, FC (valldation)
Gausslan dropout binning, FC (valldation)
2.00
1.40
1.20
1.50
(0Y:2)
(0::2)
1.00
1.00
50
0.80
0.50
1000.00
2000.00
3000.00
2.00
4.00
6.00
8.00
((:Z)
KX:Z)
(c) estimator (testing)
(d) Binning estimator (testing)
Figure 15: Same as for the variance of the dropout 0.2 we observe compression when measured with our estimator compared to no compression with binning.
19


Reviews:
[{"review_id": "fmzgGCBO5ND", "reviewer": "Reviewer_DSDh", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper analyses the computation of mutual information in DNN with dropout.  It shows that mutual information for discrete dropout is infinite, but for continuous drop it is a well defined finite quantity.  The paper shows how this can be estimated using Monte Carlo techniques and empirically shows that this quantity converges.  The paper finishes with an analysis of training a neural network with dropout in the information plane.", "summary_of_the_review": "This is a well written paper addressing a problem that has received considerable attention in the research community.  It provides a sensible way of making sense of the Information Plane.  It is mathematically sophisticated, but leads to a practical application.  In my view this is of sufficient interest to warrant publication.", "strength_and_weaknesses": "The paper resolves a long standing problem with using information theory to study learning in deep neural networks and particularly computing mutual informations.  This is a problem that has received an enormous amount of interest.  The proposed solution is both technically sophisticated (in terms of proofs) and elegant (in terms of implementation).\n\nMy very personal question as a naturally sceptical person is whether all the effort put into studying the information plane is worth all the effort. I'm prepared to accept that there are enough people interested that this is worth publishing, but the results section did not convince me that there is a lot of understanding to be gained.  I accept though that the authors have said that a full analysis will end up elsewhere so I guess I need to wait.  I also have a rather practical question that these measure are intrinsically super unstable when the mutual information becomes unbounded in a deterministic network.  This gives me an uncomfortable feeling as this does not reflect the generalisation performance of networks with and without dropout (or with continuous versus discrete dropout).  It seems to me that the true solution is to find a more meaningful picture than the information measure where networks that perform similarly don't have such different behaviour.  However, I realise this is just me being picky and I'm not really expecting the authors to address my scepticism,", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "WO5Oi-ofFf7", "reviewer": "Reviewer_e4St", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The goal of the paper is to obtain a sufficiently accurate estimate of MI (between input and representation) for dropout neural networks  and to use it to confirm the information bottleneck hypothesis for this NN model.\n\nContributions:\n- The authors propose a monte-carlo based estimate of MI in Gaussian dropout networks that is relatively accurate and well-supported by the performed theoretical analysis.\n- The authors use their MI estimate to provide information plane analyses for several NN models with Gaussian and information dropout (LeNet and MLP on MNIST, ResNet on CIFAR10)\n\n", "summary_of_the_review": "A solid contribution without greater flaws except limited area of application. ", "strength_and_weaknesses": "Plus\n\n A solid contribution to the area of information-theoretic analysis of neural networks. \n-  The idea of utilizing stochasticity induced by dropout layers to estimate MI (between input and internal model representation) is novel and interesting. \n-  The authors provide a thorough theoretical analysis of MI estimation in NNs and its limitations for both binary dropout and dropout with continuous noise.\n-  The proposed monte-carlo MI estimate seems promising (based on the experiments).   \n\n\nMinus\n\nI see the main limitation of the paper in relatively narrow area of ​​application. The method is restricted to dropout networks with continuous noise. \nThe authors provide a proof that the principle cannot be extended to (the widely used) binary dropout (or to NNs without dropout). \nThe concurrent technique of (Goldfeld et al. 2019) seems to offer wider possibilities of use despite its other disadvantages (namely the need to alter the internal representation of the model by noise).  \n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "56M9YywIUz", "reviewer": "Reviewer_A9Ut", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper puts forward a method of mutual information estimation within neural networks where the stochasticity of observations is ensured by Gaussian dropout. The authors then use MC methods to estimate entropies and conditional entropies. The proposed approach is compared to MI estimation based on binning and the authors claim to observe the compression phase postulated by Tishby et al., which was not observed with the binning estimator.\n", "summary_of_the_review": "This paper takes an interesting approach to the analysis of NN with the information plane, but the presented results seem neither significant nor novel enough to warrant publication.", "strength_and_weaknesses": "STRENGTHS\n\nThe interpretation of neural networks with the information plane approach is a relevant problem which has garnered some attention since it was proposed in 2017.\n\nThe authors' idea of using dropout for MI estimation is valid, although not necessarily novel.\n\n\nWEAKNESSES\n\nThe main weakness of the paper is its limited novelty. The negative results presented in Section 3 are straightforward and the only new results of this paper follow from the restriction to Gaussian dropout and estimating entropies with MC methods. The observed results concerning the compression phase in the last section are not convincing (the curves resemble straight lines). The authors also only compare this result to the binning estimator which is known to result is spurious effects (see e.g. the works of Saxe or Gabrie).\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "-jwrnlffLB-", "reviewer": "Reviewer_iYGf", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper address the ill-posed problem of IB of deterministic neural networks. Motivated by the success of stochastic neural networks, the authors proposed using the dropout technique to define and estimate the mutual information in neural networks. The authors showed that MI is bounded when the dropout uses continuous noise. Then they conducted numerical experiments of MI and studied the IP of neural network models.", "summary_of_the_review": "This paper proposed elegant way of defining and estimating MI and IP for networks that uses dropout. However, as far as I understood, the technique is not so novel and numerical experiments are limited to show the usefulness and understand the IP in dropout models.", "strength_and_weaknesses": "# Strength\n* Compared to stochastic neural networks, using the dropout is more practical and realistic. \n* The estimation procedure of MI using the dropout seems easy and worked well compared to binning approaches.\n\n# Weakness\n* Compared to binning approaches, this method only applies to the dropout models with continuous noise. There are many networks that do not use dropout and continuous noise.\n* When using information dropout, its noise is learned simultaneously. This means the estimator of MI changes during the training. Thus, we use different MI estimators for different epochs when plotting the IP. So it is hard to understand the obtained IP curves or meaningless. Is my understanding correct?\n* I wonder how the dropout hyperparameters affect the estimation of MI and the IP. For example. The number of dropout layers or the magnitude of the noise should have a large impact on them, but there is no discussion and no numerical experiments about them.\n* Why I(Y; Z) are so different between dropout and binning methods, as shown in Figures 1, 5, 11, 12? As far as I understand, I(Y; Z) is estimated by subtracting H(Y) from the cross entropy as mentioned in Sec 4. This is the same procedure for the dropout and binning approaches.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster