Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression
Authors: Ivan Butakov
Alexander Tolmachev
Sofia Malanchuk
Anna Neopryatnaya
Alexey Frolov
Kirill Andreev
Venue: ICLR 2024 poster (2024)
Keywords: deep learning
information bottleneck principle
stochastic neural networks
lossy compression
Primary area: learning theory
TL;DR: We propose and justify a lossy compression step to overcome the obstacles associated with high dimensionality in the information-theoretic approah to DNNs

Abstract: The Information Bottleneck (IB) principle offers an information-theoretic framework for analyzing the training process of deep neural networks (DNNs). Its essence lies in tracking the dynamics of two mutual information (MI) values: between the hidden layer output and the DNN input/target. According to the hypothesis put forth by Shwartz-Ziv & Tishby (2017), the training process consists of two distinct phases: fitting and compression. The latter phase is believed to account for the good generalization performance exhibited by DNNs. Due to the challenging nature of estimating MI between high-dimensional random vectors, this hypothesis was only partially verified for NNs of tiny sizes or specific types, such as quantized NNs. In this paper, we introduce a framework for conducting IB analysis of general NNs. Our approach leverages the stochastic NN method proposed by Goldfeld et al. (2019) and incorporates a compression step to overcome the obstacles associated with high dimensionality. In other words, we estimate the MI between the compressed representations of high-dimensional random vectors. The proposed method is supported by both theoretical and practical justifications. Notably, we demonstrate the accuracy of our estimator through synthetic experiments featuring predefined MI values and comparison with MINE (Belghazi et al., 2018). Finally, we perform IB analysis on a close-to-real-scale convolutional DNN, which reveals new features of the MI dynamics.

INFORMATION BOTTLENECK ANALYSIS OF DEEP NEU- RAL NETWORKS VIA LOSSY COMPRESSION
Ivan Butakov+.1.20 Alexander Tolmachev! Sofia Malanchuk 1,2 Anna Neopryatnaya Alexey Frolov', Kirill Andreev!
1 Skolkovo Institute of Science and Technology; Moscoow Institute of Physics and Technology; Sirius University of Science and Technology;
(butakov. id, tolmachev.ade malanchuk. sv, neopryatnaya am @phystech.edu, (al. frolov, k. andreev) 8skoltech.
ABSTRACT
The Information Bottleneck (IB) principle offers information-thcoretic frame- work for analyzing the training process of deep neural networks (DNNs). Its essence lies in tracking the dynamics of two mutual information (MI) values: between the hidden layer output and the DNN input/target. According to the hy- pothesis put forth by Shwartz-Ziv & Tishby (2017), the training process consists of two distinct phases: fitting and compression. The latter phase is believed to account for the good generalization performance exhibited by DNNs. Due to the challenging nature of estimating MI between nigh-dimensional random vectors, this hypothesis was only partially verified for NNs of tiny sizes or specific types, such as quantized NNs. In this paper, we introduce a framework for conducting IB analysis of general NNs. Our approach leverages the stochastic NN method pro- posed by Goldield et al. (2019) and incorporates compression step to overcome the obstacles associated with high dimensionality. In other words, we estimate the MI between the compressed representations of dimenssonn random vectors. The proposed method is supported by both theoretical and practical justifications. Notably, we demonstrate the accuracy of estimator through synthetic experi- ments featuring predefined MI values and comparison with MINE (Belghazi et al.) 2018). Finally, we perform IB analysis on close-to-real-scale convolutional DNN, which reveals new features of the MI dynamics.
INTRODUCTION
The information -hoomation-thhoretiic analysis of deep neural networks (DNNs) is developing branch of the deep learning theory, which may provide a robust and interpretable way to measure the performance of deep models during training and inference. This type of analysis might complement current non-transparent neta-optimization algorithms for architecture search, like ENAS (Pham et al. (0188). DARTS (Liu et al., 2019: Wu al.) 2019 He al., 2020), evolutionary algorithms (Fan al. 2020), and others. This method may also provide new approaches to explainable Al via estimation of
information flows in NNs (Tishby & Zaslavsky 2015 Xu & Raginsky 2017 Goldfeld al. 2019
Steinke & Zakynthinou 2020; Amjad et al. (222) or via independence testing (Berrett & Samworth
2017 Sen al. 2017). ). as opposed to existing methods of local analysis of model (Ribeiro et al.
2016 Springenberg et al. [2155) Rs et al.) 2020 Ivanovs et al., 2021) or methods based on complex
manipulations with data (Lipovetsky & Conklin 2001 Strumbelj & Kononenko, 2013 Datta et al.
2016). Information -thoorrticc quantities can also be considered as regularization terms or training
objectives (Tishby & Zaslavsky 2015 Chen et al. 2016 Belghazi et al. 2018).
The enformatiin-ttheorede proposed in Tishby et al. (1999). This concept was later developed in Tishby & Zaslavsky (2015) and applied to DNNs in (Shwartz-Ziv & Tishby (2017). The major idea of the IB approach is to track the dynamics of two mutual information (MI) values: I I(X; L) between the hidden layer output (L) and the DNN input (X) and / [Y; L) between the hidden layer output and the target of the model (Y). As
analysis of DNNs relies on the Information Bottleneck (IB) principle
Coorreponndence meeakovidd@pysseeche

---

result of the IB analysis, the authors of the latter article put forth the so-called fitring- crrtinggoomrreessoon hypothesis, which states that the training process consists of two phases: a ceaturee-etrrctionnn "fitting" phase (both MI values grow) and a representation compression phase (I(Y; L) grows while I [X;; L) decreases). The authors conjectured the compression phase to account for the good generalization performance exhibited by DNNs. However, it is still debated whether empirical confirmations of the compression phase are related to improper mutual information estimators, activation function choice, or other implementation details. For a more complete overview of current IB-related problems, we refer the reader to Geiger (2022).
In the original work by (Shwartz-Ziv Ziv & Tishby (2017). quantization (or binning) approach was proposed to estimate MI. However, this approach encountered two primary challenges. Firstly, the MI estimate was highly sensitive to the bin size selection. Secondly, for a fixed training epoch, when the training weights are held constant, T. becomes a deterministic function of X, resulting in the MI being independent of the DNN parameters (and infinite for practically all regimes of interest if we speak about continuous case and reasonable activation functions, see e.g., Amjad & Geiger (2018) ) The subsequent papers addressed the aforementioned problems. To tackle the infinite MI problem it was proposed to consider (a) stochastic NNs (Goldfeld al. 2019 Tang Nguyen & Choil 2019
Adilova et al.] 2023), (b) quantized NNs (Lorenzen et al. (022) or (c) a mixture of them (Cerrato
al. 2023). Simple and inconsistent binning entropy estimators have been replaced with estimators
more appropriate for contimuous random variables (Gabrié et al. 2018 Goldfeld et al. 2019: Goldfeld
al. 2020 Adilova et al. 2023).
However, the high-dimensional problem still holds, as the sample complexity (the least number of samples required for an estimation within a fixed additive gap) of any entropy estimator proven to depend on the dimension exponentially (Goldfeld et al. 2020 McAllester & Stratos 2020). Due to the challenging nature of estimating MI between high-dimensional random vectors, the fitting- compression hypothesis has only been verified for tiny NNs or special classes of models with tractable information-theorrs quantities (e.g.. Gabrié et al. (2018); Lorenzen et (2022)). ). Some existing works on IB-analysis of large networks also exhibit signs of the curse of dimensionality (Goldfeld
ctt al. 2019 Adilova et al. (023).. We mention papers that suggest using lower bounds or other surrogate objectives (Belghazi et al. 2018 Elad et al. 2019 Poole et al.) 2019 Darlow & Storkey 2020 Jónsson et al. 2020 Kirsch et al. 2021 McAllester & Stratos 2020). advanced binning (Noshad et al. 2019) or even other definitions of entropy (Wickstrøm et al. 2019 Yu et al.) (221)) in order to perform Bnaallysss of large networks. It may be assumed that these methods can partially overcome the curse of dimensionality via utilizing the internal data structure implicitly, or simply from the fact that non-conventional information theory might be less prone to the curse of dimensionality.
In contrast to the approaches mentioned above, propose a solution to the curse of dimensionality problem by explicitly compressing the data. Since most datasets exhibit internal structure (according to the manifold hypothesis (Fefferman et al., 2013)) it is usually sufficient to estimate information- theoretic quantities using compressed or latent representations of the data. This enables the application of conventional and well-established information-theoreticc theoretic approaches to real-world machine learning problems. In the recent work of Butakov et al. (2021), the compression was used to obtain the upper bound of the random vector entropy. However, it is necessary to precisely estimate or at least bound from both sides the entropy alternation under compression in order to derive the MI estimate. In the work of Greenewald et al. (2023), two-sided bounds are obtained, but only in the special case of linear compression and smoothed distributions. Our work heavily extends these ideas by providing new theoretical statements and experimental results for MI estimation via compression- based entropy estimation. We stress out the limitations of the previous approaches more thoroughly in the Appendix.
Our contribution is as follows. We introduce a comprehensive framework for conducting IB analysis of general NNs. Our approach leverages the stochastic NN method proposed in Goldfeld al. (2019) and incorporates a compression step to overcome the obstacles associated with high dimensionality. In other words, we estimate the MI between the compressed representations of nigh-diimnsional random vectors. We provide a theoretical justification of MI estimation under lossless and lossy compression. The accuracy of our estimator is demonstrated through synthetic experiments featuring predefined MI values and comparison with MINE (Belghazi et al. 2018). Finally, the experiment with convolutional DNN classifier of the MNIST handwritten digits dataset (LeCun et al. (010) is performed. The experiment shows that there may be several compression/fiting phases during the training process. It may be concluded that phases revealed by information plane plots are connected to different regimes of learning (i.c. accelerated, stationary, or decelerated drop of loss function).
2

---

It is important to note that stochastic NNs serve proxies for analyzing real NNs. This is because injecting small amounts of noise have negligible effects on outputs of layers, and the introduced randomness allows for reasonable estimation of information -itrmatioo--hhortiic quantities that depend on NN parameters. We also mention that injecting noise during training is proven to enhance performance and generalization capabilities (Hinton al. 2012 Srivastava al. 2014).
The paper is organized as follows. In Section 2 we provide the necessary background and introduce the key concepts used throughout the paper. Section describes our proposed approach for estimating mutual information under compression, along with theoretical justifications and bounds. In Section 4 we develop a general framework for testing mutual information estimators on synthetic datasets. This framework is utilized in Section to evaluate MINE and four selected mutual information estimators, complemented by the proposed compression step. The best-performin method is then applied in Section o to perform information plane analysis on convolutional NN classifier trained on the MNIST dataset. Finally, the results are discussed in Section / We provide all the proofs in the Appendix, as well as discussion of state-of-the-at methods other than MINE.
2
PRELIMINARIES
Consider random vectors, denoted as X Ω + R" and Y S + R"", where Q represents the sample space. Let us assume that these random vectors have absolutely continuous probability density functions (PDF) denoted as p(r)), p(y). and p(ar, y). respectively, where the latter refers to the joint PDF. The differential entropy of X defined as follows
h(X)= = E Elg = / p(x)) log p(x) dr,
X
where supp X Rn represents the support of X, and log(-) denotes the natural logarithm. Similarly, we define the joint differential entropy as (xX, Y) = - E log p(x, y) and conditional differential entropy as 1(X XX] Y)= = log (X = - Ey (Exxy== log p(X X Y = v)) Finally, the mutual
information (MI) is given by I(X;Y) = f(()) - h(X Y). and the following equivalences hold
I(X;Y) h(X) h(X Y) h(Y) A(Y X),
I(X;Y) = h(x)= + A(Y) - h(X,Y).
(2)
Note that supp X supp Y may have measure zero and be uncountable, indicating a singular distribution. In such cases, the supports are manifolds, PDFs can be treated as induced probability densities, and dr and dy can be seen as area elements of the corresponding manifolds. Hence, all the previous definitions remain valid.
In the following discussion, we make use of an important property of MI, which is its invariance under non-singular mappings between smooth manifolds. In the next statement we show that the MI can be measured between compressed representations of random vectors.
Statement 1. Ler:: Ω + R be an absolutely continuous random vector, and let f: Rn' + R" be an injective piecewise- smooth mapping with Jacobian J, satisfying 2 n' and det (JJ + almost everywhere. Let either 1) be a discrete random variable, or (E.1)) be an absolutely continuous random vector. Then
I(E:n) = 1 (f(E::))
Remark 1. In what follows by E: Ω RRa we denote the compressed representation of X. n' S n.
Recall that we utilize the stochastic neural network (NN) approach to address the problem of infinite mutual information ((X) for a deterministic mapping f. As demonstrated in Goldfeld et al. (2019), introducing stochasticity enables proper MI estimation between layers of the network. The stochastic modification of a network serves as a proxy to determine the information inforattion-theortiic thhooretc properties of the original model.
conventional feedforward NN can be defined as an acyclic computational graph that can be topologically sorted:
Lo X, Ln == (T(To), L2 = f2(To, In),
A In Sn([oa,... -....... Im-1);
3

---

where Lu...., Ln denote the outputs of the network's layers. The stochastic modification is defined similarly, but using the Markov chain stochastic model:
Definition 1. The sequence of random vectors Lo.... In is said to form a stochastic neural network with input X and output Y, if Lo - X. Y. La, and
Lo + (Lo, L1) -
(Lo...., La)
is a Markov chain; L represends outputs of the k-th layer of the network.
Our primary objective is to track (L:; L2) during the training process. In the subsequent sections, we assume the manifold hypothesis hold for X. In such case, under certain additional circumstances (continuity of JR. small magnitude of injected stochasticity) this hypothesis can also be assumed for T.k. thereby justifying the proposed method.
MUTUAL INFORMATION ESTIMATION VIA COMPRESSION
In this section, we explore the application of lossless and lossy compression to estimation of MI between high-dimensional random vectors. We mention the limitations of conventional MI estimators, propose and theoretically justify complementary lossy compression step to address the curse of dimensionality,, and derive theoretical bounds on the MI estimate under lossy compression.
3.1 MUTUAL INFORMATION ESTIMATION
Let ((IR, Mb) be a sequence of i.i.d. samples from the joint distribution of random vectors X and
Y. Our goal is to estimate the mutual information between X and Y. denoted as I [X; Y). based on these samples. The most straightforward way to achieve this is to estimate all the components in (T) or () via entropy estimators. More advanced methods of MI estimation, like MINE (Belghazi et al.) 2018), also applicable. However, according to Theorem 1 in Goldfeld al. (2020) and Theorem 4.1 in McAllester & Stratos (2020), sample complexity of entropy (and MI) estimation is exponential in dimension. We show that this obstacle can be overcome if data possesses low-dimensional internal structure.
In our work, we make the assumption of the manifold hypothesis (Fefferman et al. 2013). which posits that data lie along or close to some manifold in multidimensional space. This hypothesis is believed hold for wide range of structured data, and there are datasets known to satisfy this assumption precisely (e.g., photogrammetry datasets, as all images are parametrized by camera position and orientation). In our study, we adopt a simplified definition of the manifold hypothesis:
Definition 2. A random vector X: Ω RD strictly satisfies the manifold hypothesis iff there exist g: + Rnn and f: R + Rn satisfying the conditions of Statement | such that X = f(E). A random vector X': Ω + R² loosely satisfies the manifold hypothesis iff X' X Z, where X strictly satisfies the manifold hypothesis, and Z is insignificant in terms of some metric.
To overcome the curse of dimensionality, coders (Kramer 1991; Hinton & Salakhutdinov (2066) and applying conventional estimators to the compressed representations. To address the issue of measure-zero support, consider the probability measure induced on the manifold.
propose learning the manifold with autoen-
Let us consider absolutely continuous X, compressible via autoencoder A D E.
Corollary 1. Let E-1; RR 2 E(supp X) - R² and E(X); Ω + R= exist, let (E o E) = X, let Y, E(X) and E-1 satisfy conditions of the Statement [ Then
I(X;Y) I(E());Y),
In case of absolutely continuous (X, Y), the mutual information estimate can be defined as follows:
f(x;Y) & & ?(E(X) k(Y) ((()))-((((E))))) i(E(X),
(4)
In case of absolutely continuous X and discrete Y, is impractical to use 2 as the (induced) joined probability distribution is neither absolutely continuous nor discrete. However, ( is still valid:
[-
h(X |Y) Σ (1) ESsupp Y
px ( Y = log (xx Y = 3)) dx
x(X
4

---

Probabilities py be estimated using empirical frequencies: py (v) N |f yk y Conditional entropy h(X Y y) be estimated using corresponding subsets of
{mx) )
i(X] y) ((fx yk = - The mutual information estimate this case can be defined as
follows:
I():)) & M(E()) EEuupy
Σ (v( ((E(X) v)
(5)
According to the strong law of large numbers, p - p. That is why the convergence of the proposed MI estimation methods solely relies on the convergence of the entropy estimator used in (4) and (5)
Note that this method can be obviously generalized account for compressible Y.
3.2 BOUNDS FOR MUTUAL INFORMATION ESTIMATE
It can be shown that it is not possible to derive non-trivial bounds for I(E(X); in general case if the conditions of Corollary do not hold. Let us consider a simple linear autoencoder that is optimal in terms of mean squared error, which is principal component analysis-based autoencoder. The following statement demonstrates cases where the proposed method of estimating mutual information through lossy compression fails.
For any given x 2 there exist random vectors X Ω - IR", Y: Ω + I and non-trivial linear autoencoder A = Do E with latent space dimension n' < that is optimal in terms
Statement 2.
of minimizing mean squared error E ||X A(X)II, such that [(X; Y) = x and (EE(); )); Y) = 0.
This statement demonstrates that an arbitrary amount of information can be lost through compression of the data. It arises from the fact that "less significant" in terms of metric spaces does not align with "less significant" in terms of information theory. However, with additional assumptions, a more useful theoretical result can be obtained.
[ = ={(X,))
X = g(L))
Z
E
X
E
X
(nolay)
Statement 3. Let X, Y, and Z be random variables such that I(X;Y) and I ((X,Z);)) Z): are defined. Let I be function of two arguments such that T(J(X, Z):Y is defined. If there exists a function such that X =
1((X22)))
2 I(EY) 2 ((X:))
Figure 1: Conceptual scheme of State- ment 3 application to lossy compres- sion with autoencoder A = Do a E. gX, 2)), then the following chain of inequalities holds:
I(X;Y) S. (If(X,2;;)) S I((X, Z);Y) & I(X:)) + h(2)
ZZ X,Y)
In this context, f(X, Z) can be interpreted as compressed noisy data, X as denoised data, and y as a perfect denoising decoder. The term h( can be upper-bounded via entropy of Gaussian distribution of the same variance, h(Z X, Y) can independent random vectors, at least one of which is of finite entropy); see Section of the Appendix for details. We also note the special case where the data lost by compression can be considered as independent random noise.
hower-hounded in special cases (e.g., when Z is a sum of
Corollary 2. Let X, Y, Z, S, and g satisfy the conditions of the Statement Let random variables (X, Y) and Z be independent. Then I(X;Y) = (f(X,Z);))
We note that (a) the presented bounds cannot be further improved unless additional assumptions are made (e.g., linearity of in (Greenewald et al., 2023)); (b) additional knowledge about the connection between X, Y, and is required to properly utilize the bounds. Other bounds can also be derived (Sayyareh) 2011 Belghazi al. 2018 Poole et al. 2019) but they do not take advantage of the compression aspect.
The provided theoretical analysis and additional results from Section B of the Appendix show that the proposed method allows for tracking the true value of MI within the errors of third-party estimator ran on compressed data and the derived bounds imposed by the compression itself.
4 SYNTHETIC DATASET GENERATION
In order to test the proposed mutual information estimator, we developed a universal method for synthetic dataset generation with defined information-theorrtiic properties. This method yields two
5

---

random vectors, X and Y, with predefined value of mutual information (() The method requires X and Y to be images of normally distributed vectors under known nonsingular smooth mappings. The generation consists of two steps. First, a normal vector (5. ~ _N M) is considered, where & g~ (O, ), y ~ (O, Jmr), and n', m' are dimensions of & and v/, respectively. The covariance matrix M is chosen to satisfy n():n) x, where x is arbitrary non-negative constant.
Statement 4. For every x 2 0 and every n', m' E N exists a matrix M E "(n'tt")xxx'' +m²) such that (E, (N M). E NO0, ) ) (0, I and I(E: n) x.
After generating the correlated normal random vectors (f. 11) with the desired mutual information, we apply smooth non-singular mappings to obtain = =(( and Y g()). According to Statement this step preserves the mutual information, [(:: = (X;
5 COMPARISON OF THE ENTROPY ESTIMATORS
The MI estimate is acquired according to Subsection 3.1 To estimate the entropy terms in 1 or 2 we leverage conventional entropy estimators, such as kernel density-based (Turlach) 1999, Sayyareh 2011 Sain 1994) and Dozachenko-Leonende estimators (original Kozachenko & Leonenko (987) and weighted Berrett et al. (2019) versions). To test the accuracy of these approaches, we use datasets sampled from synthetic random vectors with known MI. We generate these datasets in accordance with Section A
To examine the impact of the compression step proposed in Subsection 3.1 utilize special type of synthetic datasets. Synthetic data lies on a manifold of small dimension. This is achieved by generating low-dimensional dataset and then embedding it into high-dimensional space by smooth mapping (so Statement can be applied). Then, the acquired datasets are compressed via autoencoders. Finally, the obtained results are fed into a mutual information estimator.
Algorithm D and Figure@deecrrbe the proposed mutual information estimation quality measurement. We run several experiments with and g mapping normal distributions to rasterized images of geometric shapes (e.g., rectangles) or 2D plots of smooth functions (e.g., Gaussian functions) The results are presented Figures and and 4 The blue and green curves correspond to the estimates MI marked by the corresponding colors in Figure Thus, we see that the compression step does not lead to poor estimation accuracy, especially for the weighted Kozachenko-Leonene which demonstrates the best performance. Note that we do not plot estimates for uncompressed data, as all the four tested classical estimators completely fail to correctly estimate MI for such high-dimensional data; for more information, we refer to Section B.3 in the Appendix. We also conduct experiments with MINE (without compression), for which we train a critic network of the same complexity, as we use for the autoencoder.
(WKL.) estimator,
Algorithm 1 Mcasure mutual information estimation quality on high-dimensionas synthetic datasets
1: Generate two datasets of samples from normal vectors and with given mutual information as described in section 4 [(rx) y1s) )
2: Choose functions/ f and g satisfying conditions of Statement I (so T(E:n) I(f(S): and obtain datasets for f({) and 9(7) - (f (g(w)) k=
3: Train autoencoders Ax Dx o Ex, Ay = Dy Ey on (f(z))) {o(w)) respectively.
4: Obtain datasets for (Ex o (E) and (Ey (n). We assume that Ex, Ey satisfy conditions of the Corollary I
expect
T({:n) T(f(S):(()) = I((Ex o f)(: (Ey o (n))
5: Estimate 7 7(EEx ()(S): g))) and compare the estimated value with the exact one.
'Due to the high complexity of the used and g. we do not define these functions in the main text; instead, we refer the source code published along with the paper (Butakov al.).

---

f2
Ex
(5.))-W(o.n))
((Em))
(Eh( (optional)
92
Ey
Figure 2: Conceptual scheme of Algorithm 1 In order to observe and quantify the loss of information caused by the compression step, split f: R R" into two functions: fi: R= + R² maps structured latent representation of X (e.g., parameters of geometric shapes), and f2: Rd' R
maps latent representations to corresponding high-dimensional vectors (e.g., rasterized images of geometric shapes). The same goes for g = 92 91- Colors correspond to the Figures and 4 For proper experimental setup, we require f1, f2. 91 92 to satisfy the conditions of Statement
10
10
10
10
10
8
entmmation
EINE
extmmatin
FTI INCImALTio
estimsation
6
2
4
10
0
6
10
10
0
3366666 910
(a) KDE, MI.
(b)
KDE, LSE
(c) KL.
(d) WKL
MINE
10
cortmatinng Domppe esL.
tenaatnng cocppe esl.
estiitstion
cumpee
compr. est.
0
0
0
(1) KDE, MI.
(g) KDE, LSE
(h) KI.
(i) WKL
(i) MINE
Figure 3: Maximmu--likclhhol and Least Squares Error KDE, Non-weighted and Weighted Kozachenko-Leonenke MINE for 16 16 (first row) and 32 x 32 (second row) images of 2D
Gaussians (n' m' 2), 5 10a samples. Along axes is [[X; Y), along y axes is I(X:Y).
true value Doclig CSL.
ETLE VaLUE ConpO
ITUS value DOUREE
2
INLD value
4
8
10
0
6
10
10 0
10 00 2 1033422 5 8910
(a) KDE, ML
(b) KDE, LSE
KL
(d) WKL
(C) MINE
10
10
12
10
frue CXtImEii
Irac value
Due Vallo
8
6
2
0
4
8
10
0.
6
10
10
4
6
10
0
7 910 MINE
(i) KDE, MI.
(g) KDE, LSE
(h) KL.
(i)
WKL
Figure 4: Maximum-likelihood and Least Squares Error KDE, Non-weighted and Weighted Kozachenko-Lecn Lernneknk--Leonenko, MINE for 16 16 (first row) and 32 32 (second row) images of rectangles (w' m' 4), 10 samples. Along x axes [(X; Y), along y axes is ((;Y).
7

---

6 INFORMATION FLOW IN DEEP NEURAL NETWORKS
This section is dedicated to the information flow estimation in DNNs via the proposed method. We estimate the information flow in convolutional classifier of the MNIST handwritten digits dataset. This neural network is simple enough to be quickly trained and tested, but at the same time, is complex enough to suffer from the curse of dimensionality. The dataset consists of images of size 28 x 28 = 784 pixels. It was shown in Hein & Audibert (2005) that these images have a relatively low latent space dimension, approximately 12-13. If the preservation of only the main features is desired, the latent space can even be narrowed down to 3 10. Although the proposed experimental setup is nowadays considered to be toy and small, it still problematic for the IBaanllssss, as it was shown in Goldfeld et al. (2019).
can be concluded from the previous section that the weighted Kozachenko-Leenent estimator is superior to the other methods tested in this paper. That is why it is used in experiments with the DNN classifier described in the current section. The analyzed network is designed to return the output of every layer. To avoid the problem of deterministic relationship between input and output, we apply Gaussian dropout with a small variance after each layer. This allows for the better generalization during the training (Srivastava et al. 2014) and finite values of MI during the IB- IB-analysis (Adiloval cet al. 2023). Lossy compression of input images X is performed via a convolutional autoencoder with latent dimension of d'stent, Lossy compression of layer outputs L; is performed via principal component analysis with dgpicnt as the number of principal components, it showed to be faster and not significantly worse than general AE approach this particular case. The general algorithm is described in Algorithm 2
Algorithm 2 Estimate information flow in the neural network during training
1: Compress the input dataset { ) via the input encoder Ex:: ox Ex (xx).
2: for epoch 1...., number of epochs do
3: for L, layers do
4: Collect outputs of the layer L;: vf= = Li(xx). Each layer must be noisy/stochastic.
5: Compress the outputs via the layer encoder EL, ch. EL, (vk").
6: Estimate I (Ex(X); Ez, (L,)) and 1(E, (L:);
( (X)), where Y maps inputs to true targets.
7: end for
8: Perform one-epoch training step of the network.
9: end for
We use the architecture of the classification net- work provided in Table We train our network
L1:
Conv2d(1, 8, ks=3), LeakyReLU
with learning rate of 10 using the Nvidia Titan
L2: Conv2d(8, 16. ks=3), Leaoo))LUr0.00)
RTX. We use alatemt datent 4. For other hyper-
La: Conv2d(16, 32, ks=3), LeakyReL.U00.01)
parameters, we refer to Sf the Appendix
La: Dense(32, 32), LeakyReLU(0.01)
and to the source code (Butakov et al.).
Ls: Dense(32, 10), LogSoftMax
The acquired information plane plots are provided in Figure [5] As the direction of the plots with respect to the epoch can be deduced implicitly (the lower left corner of the plot corresponds to the first epochs), we color the lines according to the dynamics of the loss function per epoch. We do this to emphasize one of the key observations: the first transition from fitting to the compression phase coincides with an acceleration of the loss function decrease. It is also evident that there is clear large-scale compression phase. Moreover, it seems that the number of fitting and compression phases can vary from layer to layer.
Table 1: The architecture of the MNIST
convolution- DNN classifier used in this paper.
DISCUSSION
An information thhooreticc approach to explainable artificial intelligence and deep neural network analysis seems promising, as it is interpretable, robust, and relies on the well-developed information theory. However, the direct application of information-theonetiic analysis still poses some problems.
8

---

2.5
Inss spadd x100
1.7
NL NLLLoss
1.5
166
loss pC
0.5
100
((X,L)
(a) Negative log likelihood loss (train data)
(b) Lg (convolutional, LeakyReLU)
2
22
2
1ob gwod X1P
1.6
3.40 pM 4000
1.8
1.6
11E.Y1
JiL.Y
1.2
0.8
0.5
2.5
3.5
1.5
2.5
3.5
L(X.L)
((.
(c) La (fully-commal Colly-connected, LeakyReLU)
(d) La (fully-connected) LogSoftMax)
Figure 5: Information plane plots for the MNIST classifier. The lower left parts of the plots (b)-(d) correspond to the first epochs. We use 95% asymptotic CIs for the MI estimates acquired from the compressed data. The colormap represents the difference losses between two consecutive epochs.
We have shown that it is possible apply information analysis to compressed representations of datasets or models' outputs. To justify our approach, we have acquired several theoretical results regarding mutual information estimation under lossless and lossy compression. These results suggest that this approach is applicable to real datasets. Although it has been shown that an arbitrary amount of information can be lost due compression, the information required for optimal decompression is still preserved.
We have also developed framework to conventional mutual information estimators comple- mented with the proposed lossy compression step. This framework allows the generation of pairs of nigh-dimensional datasets with small internal (latent) dimensions and predefined quantity of mutual information. The conducted numerical experiments have shown that the proposed method per- forms well, especially when the entropy estimation is done via the weighted Coraachnkk-LLeoneee estimator. Other methods tend to underestimate or overestimate mutual information.
Finally, an information plane experiment with the MNIST dataset classifier has been carried out. This experiment has shown that the dynamics of information-theoreeie quantities during the training of DNNs are indeed non-trivial. However, is not clear whether the original ciminnocompressioo hypothesis holds, there is clear large-scale compression phase after the fitting. We suggest that there may several compression/ittingg phases during the training of real-scale neural networks.
An interesting observation has also been made: the first compression phase coincides with the rapid decrease the loss function. It may be concluded that the phases revealed by information plane plots connected to different regimes of learning (i.e., accelerated, stationary, the loss function). However, we note that this observation is not the main contribution of our work, and further investigation has be carried out in order support this seeming connection with more evidence and theoretical basis.
decelerated drop of
Future work. As further research, consider using normalizing flows (Rezende & Mohamed 2015) to improve our approach. Normalizing flows are invertible smooth mappings that provide means of lossless and information-pen preservvng compression. They can be used transform the joint distribution to Gaussian, thus facilitating mutual information estimation. Besides, we will apply our method to various large neural networks and perform corresponding information plane analysis.

---

REFERENCES
Linara Adilova, Bernhard C. Geiger, and Asja Fischer. Information plane analysis for dropout neural networks, 2023.
Rana Ali Amjad and Bernhard Geiger. Learning representations for neural network- based classifi- cation using the information bottleneck principle. (submitted to) IEEE Transactions Pattern Analysis and Machine Intelligence, PP, 02 2.18. doi: 10. 10.1109/TPAMI.2019 299901....
Rana Ali Amjad, Kairen Liu, and Bernhard C. Geiger. Understanding neural networks and individual neuron importance via information-ordere cumulative ablation. IEEE Transactions on Neural Networks and Learning Systems, 33(12):7842- 2022. doi: 10.1 109/TNNI S.....2.......8...
Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeshwar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and Devon Hjelm. Mutual information neural estimation, In Jennifer Dy and Andreas Krause (eds.), Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 531-540. PMLR, 07 2018. URL https //proceedings ml prwss http///00eeeegaaaieeeeee
Thomas Berrett and Richard Samworth. Nonparametric independence testing via mutual information. Biometrika, 106, 11 2017. doi: 10. 10010///oome//ss/2244..
Thomas B. Berrett, Richard Samworth, and Ming Yuan. Efficient multivariate entropy estimation via k-nearest neighbour distances, Ann. Statist, 47(1):288-318, 02 2019. doi: 10. 10.1214/1-AASS1888
URL https: //doi www.doooorooo.. www.oorggooo... 1214
688
I. D. Butakov, S. V. Malanchuk, A. M. Neopryatnaya, A. D. Tolmachev, K. V. Andreev, S. A. Kruglik, R. A. Marshakov, and A. A. Frolov. High-dimensional dataset entropy estimation via lossy compression. Journal of Communications Technology and Electronics, 66(6))764-768, 7 2021. ISSN 1555-6557. doi: 10. 10.1134/1106422222100000
URL /RL.
1134/81042269911000
Ivan Butakov, Aleksander Tolmachev, Sofia Malanchuk, Anna Neopryatnaya, Alexey Frolov, and Kirill Andreev. Package for information-thcore data analysis. URL
VanessB/ Information-
Mattia Cerrato, Marius Köppel, Roberto Esposito, and Stefan Kramer. Invariant representations with stochastically quantized neural networks. Proceedings of the AAAI Conference on Artificial Intelligence, 37:6962-693 6970, 06 2023. doi: 1...609/aai..v376..55555 23756.25811.
Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Picter Abbeel. Info- gan: Interpretable representation learning by information maximizing generative adversarial nets. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pp. 2172-2180, 2016. URL
www.riiiiiinnnnnnnneeeeemo neurips CC/ wtcpppape///016666 6/ html
Thomas M. Cover and Joy A. Thomas. Elements of Information Theory (Wiley Series in Telecommu- nications and Signal Processing). Wiley- Intey-Interscceece, USA, 2006.
Luke Nicholas Darlow and Amos Storkey. What information does a resnet compress?, 2020.
Anupam Datta, Shayak Sen, and Yair Zick. Algorithmic transparency via quantitative input influence: Theory and experiments with learning systems. In 2016 IEEE Symposium on Security and Privacy (SP), pp. 598-617, 2016. doi: 10. 10.11/9....1...2...
Adar Elad, Doron Haviv, Yochai Blau, and Tomer Michaeli. Direct validation of the information bottleneck principle for deep nets. In 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pp. 758-762, 2019. doi: 10.1009/CCC22200 .00099.
Zhun Fan, Jiahong Wei, Guijie Zhu, Jiajie Mo, and Wenji Li. Evolutionary neural architecture search for retinal vessel segmentation, 2020.
10

---

Charles Fefferman, Sanjoy Mitter, and Hariharan Narayanan. Testing the manifold hypothesis. Journal of the American Mathematical Society, 29, 10 2013. doi: 10.1090fjams/852.
Marylou Gabrié, Andre Manoel, Clément Luneau, jean barbier, Nicolas Macris, Florent Krza- kala, and Lenka Zdeborová. Entropy and mutual information in models of deep neural net- works. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Gar- nett (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Asso- ciates, Inc., 2018. URL htttps::/pppteeeeeetteeee ngs neurips cctpapercpppppppppppe files/paper/ 
Bernhard C. Geiger. On information plane analyses of neural network classifiers review. IEEE Transactions on Neural Networks and Learning Systems,, 331113303331 12 2022. doi: 10.1109/ 2onls.0221.388033 /ttp://///////////oo////////////
2021. 2021.3099037
Z. Goldfeld, K. Greenewald, J. Niles- Weed, and Y. Polyanskiy. Convergence of smoothed empirical measures with applications to entropy estimation. IEEE Transactions Information Theory, 66 (7):4368-4391, 2020, doi: 10. 10901/97
Ziv Goldfeld, Ewout van den Berg, Kristjan H. Greenewald, Igor V. Melnyk, Nam H. Nguyen, Brian Kingsbury, and Yury Polyanskiy. Estimating information flow in deep neural networks. In ICML, 2019.
Kristjan H. Greenewald, Brian Kingsbury, and Yuancheng Yu. High-dimensional smoothed entropy estimation via dimensionality reduction. In IEEE International Symposium on Information Theory, ISIT 2023, Taipei, Taiwan, June 25-30, 2023, pp. 2613-2618. IEEE, 2023, doi: 10. 1099/S/7777/ 2023.10206641 URL TRL wttpt: doii org. 1tt.//1011111//911 ITI.1199SII1111333 2023. 00..2002066...... 10206641
Wa He, Yuting Wu, Peng Liang, and Gang Hao. Using darts to improve mold id recognition model based mask r-cnn. Journal of Physics: Conference Series, 1518:012042, 04 2020. doi: 14118///7722-699/11111111111
Matthias Hein and Jean-Y Yves Audibert. Intrinsic dimensionality estimation of submanifolds in rd. In Proceedings of the 22nd international conference on Machine learning, pp. 289-296, 2005.
G. E. Hinton and R. R. Salakhutdinov. Reducing the dimensionality of data with neural net- works. Science, 31365786):504-507, 2006. doi: 10.1126/sciencc 1277447. URL https www.sccience..r//doooooooo
wtt.//ciic.......icnn........... 1127647
Geoffrey Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint, arXiv, 07 2012.
Maksims Ivanovs, Roberts Kadikis, and Kaspars Ozols. Perturbation methods for explaining deep neural networks: A survey. Puttern Recognition Letters, 150:228-234 2021. ISSN 0167-8655. doi: http:://hoi.oogg/looooooo/opgrpp com/ http://cccooeeereeee.e//////////
URL https: www. www.w.wwwwnnncr.ecttttttttcc....
Hlynur Jónsson, Giovanni Cherubini, and Evangelos Eleftheriou. Convergence behavior of dnns
with inntttaa-iiffmmaiioo---aaaa regularization. Entropy, 22(7), 2020. ISSN 1099-4300, doi:
10.3390/e22070727. URL www. wwppi com 1099--09999000 4300/22/7/727
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization, 2017.
Andreas Kirsch, Clare Lyle, and Yarin Gal. Unpacking information bottlenecks: Unifying information- theoretic objectives deep learning, 2021.
L. F. Kozachenko and N. N. Leonenko. Sample estimate of the entropy of a random vector. Problems Inform. Transmission, 23:95-101, 1987.
Mark A. Kramer. Nonlinear principal component analysis using autoassociative neural networks.
AIChE Journal, 37 2433, 1991. doi: http:://bbi.org//o...0//a...... 7/a che. on 1 ne ibbrar..r.. wi 1
htthts?
com/ http://yww..c/////////////////// 69037700999999999

---

Yann LeCun, Corinna Cortes, and CJ Burges. Mnist handwritten digit database. ATT Labs [Online]. Available: http://yamn.. www.llecccoo....ee..... www.ex/b///iiiitt 2, 2010.
Stan Lipovetsky and Michael Conklin. Analysis of regression in game theory approach. Applied Stochastic Models in Business and Industry, 17:319 330, 10 2001. doi: 10.1002/asmb.444
Hanxiao Liu, Karen Simonyan, and Yiming Yang. Darts: Differentiable architecture search. ArXiv, 0bbs/880.09000 2019.
Stephan Sloth Lorenzen, Christian Igel, and Mads Nielsen. Information bottleneck: Exact analysis of (quantized) neural networks. In International Conference Learning Representations, 2022.
URL
net forum? forum?id=kF id=kF 9D20QrUOw
David McAllester and Karl Stratos. Formal limitations on the measurement of mutual information. In Silvia Chiappa and Roberto Calandra (eds.). Proceedings of the Twenty Third International Conference on Artificial Intelligence and Staristics, volume 108 of Proceedings of Machine Learning Research, pp. 875-884. PMLR, 08 2020. URL httptss://ppooeeediigieeoe/ ml mlr
press/ vv 0 mca lestaalleterrraaaem
Youssef Mroueh, Igor Melnyk, Pierre Dognin, Jarret Ross, and Tom Sercu. Improved mutual information estimation. Proceedings of the AAAI Conference on Artificial Intelligence, 35(10): 9009-9017, May 2021. doi: 10. wta116609aaa.i551111 w3w.10..1111110. URL wtw.h:tt///t/ //op:://ojs..oo aaai. aaaaaarorrg /rg/
index. https/ppp////attttttiieee///////
Morteza Noshad, Yu Zeng, and Alfred O. Hero. Scalable mutual information estimation using dependence graphs. In ICASSP 2019 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2902-29622222 2966, 2019. doi: 10.1 1091CASSPP210 1683351..
Hieu Pham, Melody Y. Guan, Barret Zoph, Quoc Le, and Jeff Dean. Efficient neural architecture search via parameter sharing. In ICML, 2018.
Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On varia- tional bounds of mutual information. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Pro- ceedings of Machine Learning Research, pp. 5171-5180. PMLR, 06 2019. URL https?
//proceedingscoo mir press /eeeess/vvv// / e 9a. html
Danilo Jimenez Rezende and Shakir Mohamed. Variational inference with normalizing flows
In Proceedings of the 32nd International Conference on International Conference on Machine Learning Volume 37, (CML'15,, pp. 1530-1538. JMLR.org. 2015.
Marco Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?": Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135-1144. ACM, 08 2016. doi: 10. 10.1145/2939672. 2939778.
Ramprasaath Rs, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. International Journal of Computer Vision, 128, 02 2020. doi: 10.10077/11133-01--000
Stephan R. Sain. Adaptive kernel density estimation. PhD thesis, Rice University, 1994.
Abdolreza Sayyareh. new upper bound for kullback-leibler divergence. Applied Mathernticaa Sciences, 5:3303-3317, 01 2011.
Rajat Sen, Ananda Theertha Suresh, Karthikeyan Shanmugam, Alexandros G Dimakis, and Sanjay Shakkottai. Model-powered conditional independence test. In Guyon, U. Von Luxburg, S. Bengio. H. Wallach, R. Fergus, S. Vishwanathan, and R. Gamett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https: http::::/prrooceiinn.......e.ree neurips.see.......ee..... cttppps....ppppppeeeeeeeee cwwpaper....p.......p... www.fiees//e wwt.fffasaapppre/1111// ppf
Ravid Shwartz- Ziv and Naftali Tishby. Opening the black box deep neural networks via information, 2017.
12

---

Jost Tobias Springenberg. Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net, 2015.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: A simple way prevent neural networks from overfitting. Journal of Machine Learning Research, (5(56):1929-195 2014. URL http://mmm..oro/pppeeess///
srivastaval4a.ntt
Thomas Steinke and Lydia Zakynthinou. Reasoning About Generalization via Conditional Mu- tual Information. In Jacob Abernethy and Shivani Agarwal (eds.), Proceedings of Thirty Third Conference on Learning Theory, volume 125 of Proceedings of Machine Learning Research, pp.
3437-3452. PMLR, 09-12 Jul 2020. URL http::t/pprccceinnnggggso/////et .lr. wwt.ss.www.....sseeeeeeeee ptwss//1sssv. steinke20a. html
Thanh Tang Nguyen and Jaesik Choi. Markov information bottleneck to improve information flow in stochastic neural networks. Entropy, 21(1)), 2019. ISSN 1099-4300. doi: 10.3390/e2110090
URL httph: www 2ttpa//co//0099-00////20000000
Naftali Tishby and Noga Zaslavsky. Deep learning and the information bottleneck principle. 2015 IEEE Information Theory Workshop (ITW), pp. 1-5, 2015.
Naftali Tishby, Fernando C. Pereira, and William Bialek. The information bottleneck method. In Proc. of the 37-th Annual Allerton Conference Communication, Control and Computing, pp. 368 367-377, 1999. URL /RRl//rokeurrl wwnckkennrlciitseeeee #citeseer. .nw..nnn........................ cww.t.t.ooo.oo...m hiimmttssbyyaa winnoorattinnnnnee......
Berwin Turlach. Bandwidth selection in kernel density estimation: A review. Technical Report, 02 1999.
Stanislaw Weglarezyk. Kernel density estimation and its application. ITM Web of Conferences, 23: 00037, 01 2018. doi: 10.105 00.10///mmcoom/221230003333
Kristoffer Wickstrøm, Sigurd Løkse, Michael Kampffinyee, Shujian Yu, Jose Principe, and Robert Jenssen. Information plane analysis deep neural networks via matrix-based renyi's entropy and tensor kemelss, 2019.
Bichen Wu, Xiaoliang Dai, Peizhao Zhang, Yanghan Wang, Fei Sun, Yiming Wu, Yuandong Tian, Péter Vajda, Yangging Jia, and Kurt Keutzer. Fbnet: Hardware-awend efficient convnet design via differentiable neural architecture search. 2019 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pp. 10726-10734, 2019.
Aolin Xu and Maxim Raginsky. Information- analysis of generalization capability of learning algorithms.
Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Asso- ciates, Inc., 2017. URL ht. htwplhtttppsspppporeeenn...io.. papp fi Files// eellesppl les/paper/ 201 2017/file ee/ 70777
Shujian Yu, Kristoffer Wickstrøm, Robert Jenssen, and José Príncipe. Understanding convolutional neural networks with information theory: An initial exploration. IEEE Transactions on Neural Networks and Learning Systems, 32(1):435-442. 2021. doi: 10. 10.1100//NNLSS
Erik Štrumbelj and Igor Kononenko. Explaining prediction models and individual predictions with feature contributions. Knowledge and Information Systems, 41:647-665, 12 2013. doi: 10.1007/s 101 15-013-067-x
13

---

A COMPLETE PROOFS
Statement 1. Let 92 + R2 an absolutely continuous random vector, and let f: Rnn + R
be an injective diecewisessmooth mapping with Jacobian J, satisfying 2 n' and det (J + almost everywhere. Let either 1) be a discrete random variable, or (5.7) be an absolutely continuous random vector. Then
n(: = ( ((((s))))
Proof of Statement For any function f. let denote / (Jz (z))()) (area transformation coefficient) by a(x) where it exists.
Foremost, let us note that in both cases, pe 17) and Prag( n) pę(æ nn/a(z) exist. Hereinafter, we integrate S { by the assumption, the values of the integrals are not altered.
(x)) + 0) instead of supp {; as # 0 almost everywhere
According to the definition of the differential entropy,
/
(x)
h(f(E))
log
a(x) dr
a(x) a(x)
=- / (x(x) log (pe(x) dr + (og()))))(() dx
h(E) Eloga(E).
A(((S) (1) 11) -Bn (-/ pe( (xr v) log
pe(x m) a(x) de
a(x)
a(x)
E, (- (-fpe(e pe(2) m) log (pe(2) m)) dr + / pc(z m) log (a(x)) dm)=
= ALE 11) + log
Finally, by the MI definition,
I(f(E):n) h(f(E)) h(f(E) m) = h(E) - A(E n) = I(E:n).
Corollary Let E-1; 18² (supp X) R and EXX) : - Rd X, let Y, E(X) and E-1 satisfy conditions of the Statement ] Then
exist, let (E- Ex(
= (X;Y)=I(E(X)))
Proof of Corollary 1
I(X:Y) 1 ((E ' E) = (((()))
from the Statement
For any given 2 0 there exist random vectors X: Ω R" Y: + R and non-trivial linear autoencoder Do with latent space dimension n' < n that is optimal in terms
Statement 2.
of minimizing mean squared error E. || A(X)] such that T(X:Y) and ((F(();;) 0.
Proof of Statement 2 Let us consider the following three-dimensional
Gaussian vector
(X1,X2, Y)
(X,Y):
X X~N N 0.
1 0).
[
0
Y Y~ N(011 (X,, (2, ~ N 0,
where cov(X2, Y) A 2 2x cov(Xi, intrinsic dimension be n' = 1, and < 1. According to the principal component analysis, the optimal linear encoder is defined up to a scalar factor by the equality E(X) X2. However. I(X;Y) y In (1 a²) x (see Statement but T(F(X);)) 0, X1 and Y are independent.
(so X, and Y are independent). Let the
n
14

---

Statement 3. Let X, Y. and Z be random variables such that I(X;Y) and I ((X, Z); Y) defined. Let f be a function of two arguments such that [[f(X, Z);Y) ) is defined. If there exists a function such that X ((,(X, Z)), then the following chain of inequalities holds:
(1xYY) S Z);Y) I((X, ((X;)) M(Z) h2 XX
Proof of Statement 3 According to data processing inequality (Cover & Thomas, 2006). ( 2):Y) T Y,Y). As - (o([(X, Z));
=
Y):
Note that DPI is optimal, additional assumptions on f, X, Y and Z are required to tighten the bounds.
The last inequality is derived via the following equations from Cover & Thomas (2006):
[X, 2; (XX X) I(X,Y: = I(X; I(; X)
As I[;; 2 20
T(X;Y) + T(X,Y: I(X:Y) ((; x) - 2(X;))>
21(X;Y) + I(Y XX: 1(X,
Finally, recall that T(X,Y: 2) = -h(Z) h(2 X,Y).
Corollary 2. Let X, Y, Z. f. and g satisfy the conditions of the Statement 3 Let random variables (X,Y) and Z be independent. Then I(X;Y) I (f((X Z):
Proof of Corollary 2 Since (X,Y) and Z are independent, 7 (XY) Z;Y) I(X;), which implies T(X;Y) TJ(X, (U(X,2);)) according to the Statement
Statement 4. For every Z 0 and every n', m' N exists matrix M E "in'tt") such that (E, ~ (O, M). & ~ NO,,, I,), ~ NO, Im)) and [(E: m)) x.
Proof of Statement 4 We divide the proof into the following statements:
Statement 5. Let (E. m) ~ N(0 M) be Gaussian pair of (scalar) random variables with unit
variance such that [E::) 2.. Then
al a= v1 V1-6-22
M
(6)
a 1
Proof. Differential entropy of multivariate normal distribution NW,, E) is h In (det (2iTe E)).
This and (2) leads to the following:
x = [(E:)) = In ln(e + (n(T))) ((2Te)² (1 (1-a?)) = = In (1 - a²)
a == V1 - 2x
Statement 6. Let & and be independent random variables. Then [(:: = 0, (((. m) = h(E)+/((n).
Proof. We consider only the case of absolutely continuous Ç. As and r7 are independent, (5 | , v) (x)).. That is why I(E;n) h(E) 1(E n) h(E) h((E) 0, according to the definition of MI. The second equality is derived from 2
Corollary Let E1, E2 and mh. 72 be random variables. independent in the following tuples: (E1, E2), (m, 12) and ((E,, m) (Ea, 12)). Then T ((E1, (m, m2)) T(E1; m) T(E2; 12)
15

---

Proof. From and Statement following chain of equalities is derived:
(() (2)) h((1,(2) + h(m, 72) - h(Gi, m, 7,712) = h(()) + (((2) + h(m) h((1)) h((),m) Io(E2,712)
[((mm)) + (22)
The Statement and Corollary S provide with trivial way of generating dependent normal random vectors with a defined mutual information. Firstly, we consider E NO, M³), where M' is (n' +mn') (n' + +mm)) block-diagonal matrix with blocks from (6). The number of blocks is = min(n', m²) (other diagonal elements are units). The parameter x for each block equals n)E:,)/ where (g; is the desired mutual information of the resulting vectors. The components of E are then rearranged to get (5.n) ~ (O, where & ~ ,N and (0, Ime). The final structure of M is as follows:
a
M
n
B ENTROPY BOUNDS
In this section, we provide several theoretical results that complement the bounds proposed in Section 3.2 The following inequalities can be used bound the entropy terms in Statement 3
Statement 7 (Cover & Thomas (2066), Theorem 8.6.5). Let X be a random vector with covariance matrix R. Then h(X) < hON(O, R)).
Statement 8. Let X, 2: Ω + R" be independent random vectors. Then h(X + Z) 2 h(Z).
Proof. Recall that
1(X, + Z) 1(X + 2) + +|X X x(X) + 4(X + +2 +Z|X).
from which the following is derived:
h(X + 2) ((X) h(X + - xx M(X X +
Note that )(x + Z X) = Ex hix + Z X = x) = |(( X). As X and Z are independent,
I X) = h(Z). Thus, we derive the following:
1(X + Z) h(X) + h(Z) h(X
X Z) MZ)) + T(X; X Z) > M(Z)
Statement 9. Let X: Ω R and Z: Ω + Rn be a random matrix and vector, correspondingly. Let X and Z be independent. Then h(X 7) h(Z) E (In /det XI).
Proof. Note that h(X Z x) = Ex h(x Z X = 2) = h( x) + E (In |et XI). The rest of the proof is the same as for Statement
h(X- 2) 1(X) + (()) + E (In Iee XI) - h(x X 2)
1((2) + I(X; X Z) (In /det XI) 2 ((2) + (In ddet X().
20
16

---

Corollary 4. Let X, Z: : Ω - IR" be independent random vectors. Then h(X Z) 2 h(Z)
E (In Xi),, where @ is an element-wise product.
Proof. Note that X @ = diag(X) Z, and log det diag(X)) Σ
[ We then apply
Statement 9
Note that entropy terms in Statements and [ can be conditioned. The independence requirement should then be replaced by independence under corresponding conditions.
We also note that Statement can utilize autoencoder reconstruction error (via error covariance matrix), and Statements use, as this information is easily accessible in typical experimental setup.
magnitude of random vector and injected noise, which is of particular
Practical use cases include using Statement when stochasticity is introduced via additive noise (e.g.. Goldfeld et al. (2019) and Corollary 4 when stochasticity is introduced via multiplicative noise
(e.g., Adilova et al. (2023)).
C LIMITATIONS OF PREVIOUS WORKS
In this section, we stress the novelty our contribution to the problem of high-dimensional MI estimation.
In the Introduction, we mention the works of Butakov et al. (22T)) and Greenewald al. (2023). The first article focuses entropy estimation via lossy compression. The main theoretical result of the paper in question the following upper bound of random vector entropy:
Statement 10 (Butakov al. (2021)). Let X be random vector of dimension n. let Z N(O, o² I. Let A = Do be an autoencoder of input dimension and latent dimension n'. Then
h(X)= S ME(X) + 2) - n' (c+ log(o²) + n c+
log(E2)
2
2
where
log(2
Σ -ーー
D(E(X) +
2
This bound takes advantage of the compression aspect, it incorporates the reconstruction mean squared error Σ². However, it is important to note several limitations of the proposed bound. Firstly, this Statement is insufficient acquire any bound of MI, MI is computed via difference of entropy terms (see (I) and (2)). so a two-sided bound of entropy is required. Secondly, this bound is derived in case of additive Gaussian noise being injected into the latent representation of the vector. It is inapplicable to other cases of stochasticity injection (e.g., noise added to the vector itself) and, moreover, deteriorates when o + That why we consider this result inapplicable to the task of MI estimation in the current form.
Now, consider the following two-sided bound derived the work of Greenewald al. (023)):
Statement 11 (Greenewald al. (023)). ) Let X be random vector of dimension n, let 7 (0, 6² In). ). Let E. be a PCA-projector to a linear manifold of dimension n' with explained variances denoted by >i in the descending order. Them?
n2 - n
log (2mea?) S h((x + Z) M(E(X + Z)) < 12-1 n1
log (2me(Anm+1 + +o?))
This bound also takes advantage of the compression aspect, as incorporates the reconstruction mean squared error via Aç. We also note that, as the bound is two-sided, corresponding bounds of MI estimate under Gaussian convolution and linear compression can be derived:
Corollary 5. Under the conditions of Statement 77
μI( Z; I(E(X Z): n n' log 1 An'+1
We helieve "+7" in "h(P(X + =))" be missing in the original article; counterexample: X const.
17

---

10
10
4
true valuee compe: est. compr, (PCA)
Irue valiue compl: est. (AE) compe
Irue value compr. (AE)
3.5
compr. A
2.5
2
4
1.5
1
0.5
0.3
2.5 3.5
(a) 32 32 images of 2D Gaus- sians (1l' m²
(b) 32 x 32 images of rectangles
(c)
Hiighly-nonlineee manifold in
2)
(n' ml
R (n
2)
Figure 6: Comparison of nonlinear AE and linear PCA performance in task of MI estimation via lossy compression, 5 100 samples. Along axes is (X) along axes is [(X; WKL entropy estimator is used
Proof. From the we acquire
T(X Z: [(E(X X Y);
[h(X + 2) - M(E(X + 2)) [M(X + -(E) + Y),
Bounds from Statement [] can be applied to the joint entropy, as Y can be viewed as additional components of the vector X, unaffected by smoothing and compression.
To acquire the upper bound of difference of MI terms, we apply the upper bound from Statement to the first difference of entropy terms and the lower bound to the second, and vice versa in case of the lower bound. After simplification, we acquire the desired formula.
Corollary provides a useful result, as (a) the difference between the true MI and MI under compres- sion is bounded, (b) the bound converges to zero as An++1 - 0 (which corresponds to the lossless compression). However, as the authors of the original paper mention, this bound deteriorates when + O, which coincides with Statement 2]
The linearity of the encoder E is another limitation we have to mention. Although the possibility of extension nonlinear dimensionality reduction approaches is mentioned in the paper, it is unclear if the derived bounds could be directly transferred to the nonlinear case. To accomplish this, one has propose nonlinear generalization of explained variance and provide a more general analysis of entropy alternation via discarding nonlinear components. We also perform tests with synthetic data to show that autoencoder-based approach outperforms PCA-based in case of nonlinear manifolds (see Figure 6b. Although the gap is relatively small for the datasets we used in Section 4 it is possible to provide an example of highly nonlinear manifold, in which case the linear compression is very the synthetic data generator is provided in the source code).
lossy (see Figure 6c
Finally, we note that similar, or even tighter bounds can be derived from the theoretical results of our work.
Corollary 6.
Under the conditions of Statement 17 the following inequalities hold:
0 -xY((x22Y + Y): < n n
Aw σ²
log 1
2
Proof. To avoid notation conflicts, we denote and Z used in Statement as X' and Z' corre- spondingly. In order to simplify the following analysis, we consider E: R R (ran E S R" is embedded in RT) and E E E, FT E (PCA is used in the form of orthogonal projector). We then choose f = F,, Id, X' E(X + Z), 7' X 7 F(X + Z) (so 2' 1 X'), which yields the following inequalities:
I(X:;Y) < (X', Z': Y) S I(X: +(Z)) h(Z') X',Y)
We then utilize Statements and to bound the entropy terms:
h(Z') < n n'
h(Z' X(Y) > n n 2
log (2 + +0²)),
log (2mea²) 2)
2
18

---

Thus, the following result is acquired:
T(X"; Y) - (X', ((x))) n n- n'
log (1 1
An'+1 o²
2
Now recall that X" = E(X + Z). (X', (2) X' + 2 = X + Z. This yields the final result.
D LIMITATIONS OF OTHER ESTIMATORS
In this section, we provide brief overview of modern entropy and MI estimators that achieve relative practical success in dealing with the curse of dimensionality. We provide reasoning why we choose MINE (Belghazi et al., 2018) as the only modern MI estimator among the mentioned in the Introduction to compare our results with.
MINE is widely considered
good benchmark estimator and featured in several recent works (Poole et al. 2019; Jónsson et al., [2020; McAllester & Stratos, 2020 MMrochh et al. 202Tp. As MINE is a neural estimator, it theoretically able to grasp latent structure of data, thus performing compression implicitly.
Other lower/upper bounds and surrogate objectives.
According to McAllester Stratos (2020), not many methods in question outperform MINE. In fact, among the other methods mentioned in McAllester & Stratos (2020), only the difference of entropies (DoE) estimator achieves good results during a stan- dard correlated Gaussians test. Unfortunately, DoE requires good parametrized and differentiable (by parameters) estimates two PDFs, which is difficult to achieve in the case of complex multidimensional distributions.
According another overview (Poole al. 2019). the methods in question have various significant trade-offs. Some of them require parts the original distribution (like py]x) or even some unfformation-theorettt quantities (like f(X)) )) to be tractable, which is not achievable without utilizing special kinds of stochastic NNs. The others heavily rely on fitting a critic function to partially reproduce the original distribution, which leads to poor bias-variance trade-off off (it is illustrated by the results of these estimators in standard correlated Gaussians test, see Figure 2 in |Poole al. (2019). )
Compared to autoencoders, critic networks in methods in question are usually unstable and hard to train, see experiments in Poole et al. (2019); McAllester & Stratos (2020).
We also have witnessed this instability while conducting experiments with MINE, see the attached source code.
We, however, note that all these methods are of great use for building information-theoredl training objectives (as they are differentiable and usually represent upper lower bounds). In addition to the limitations mentioned above, we would like to note that the relative practical success of the modern NN-based MI estimators might be attributed to the data compression being performed implicitly.
In the work of Poole et al. (2019) it has been shown that other complex parametric NN-based estimators (NJW, JS, InfoNCE, etc.) exhibit poor performance during the estimation of MI between a pair of 20-dimensional incompressible (i.c.. not lying along manifold) synthetic vectors. These vectors, however, are of much simpler structure than the synthetic datasets used in our work (Gaussian vectors and + 2 mapping applied to Gaussian vectors in Poole et al.) (2019) versus high-dimensional images of geometric shapes and functions in our work). We interpret this phenomenon problem of MI estimation, which also affects the performance of modern NN-based MI estimators in the case of hard-to-compress data.
practical manifestation of the universal
EDGE Noshad et al. 2019) is a generalized version of the original binning estimator proposed in Shwartz-Ziv & Tishby (2017): the binning operation is replaced by a more general hashing. We suppose that this method suffers from the same problems revealed in Goldfeld et al. (2019), unless special hashing function admitting manifold-iie or cluster-like structure of complex high-dimensional data is used.
Other definitions of entropy. We are interested in aiting-compressionnn hypothesis (Tishby Shwartz-Ziv & Tishby 2017) which is formulated for the classical
& Zaslavsky 2015
19

---

mutual information, other definitions not appropriate for this particular task. We also note that the classical theory information is well-developed and provides rigorous theoretical results (c.g., data processing inequality, which used to prove Statement 3
We also mention the approach proposed in Adilova al. (2023), where h(L X) is computed via a closed-fore formula for Gaussian distribution and Monte-Cario sampling. However, we note the following drawbacks of this method: (a) closed-form formula applicable to the entropy estimation only for the first stochastic NN layer, (b) a general-case estimator still has to utilized to estimate L(()) (in the work of Adilova et al. (2023), the plug-in estimator from Goldfeld et al. (2019) is used; this estimator also suffers from the curse of dimensionality).
E CLASSICAL ENTROPY ESTIMATORS
In this section, provide definitions of conventional entropy estimators used to conduct the experiments, as well as provide proofs that these estimators fail in case of high-dimensional data.
E. KERNEL DENSITY ESTIMATION
The estimation of the probability density function for codes in the latent space plays important role in the proposed method of mutual information estimation. There are many methods for probability density function estimation (e.g.. Weglarczyk] (2018); Kozachenko & Leonenko (987); Berrett et al. (2019)) One of the most popular methods for solving this problem is kernel density estimation (KDE). In this section, study this method in application entropy estimation.
Let fx,x(z) (x) be a density estimate at point æ, which is obtained from the sampling { without the k-th element by KDE with the kernel K. We get the following expression for the density:
N
E
21
(b.- = b" (N
(8)
||
Here, K(x)
(2T) exp
is standard Gaussian kernel
2
The entropy estimate is obtained via Leave-One-Out method. Densities at each sample oy are calculated according to the formula 8
1 = N
A(X)
Σ
log pb. - (xrk
E. MAXIMUN-LIKELIHOOD
The optimal bandwidth can be selected in accordance with the minimization of the Kullback-Leibler divergence between the estimated distributions and the empirical one (Pemp (x)) N Σ 8(x xx))). This is equivalent to selecting the bandwidth as a maximum likelihood estimate:
i arg max (X) arg mIxX
log Po. - ((FR)
(10)
The experiments have shown that this method tends to underestimate mutual information, and the difference increases with an increasing true value of mutual information.
Asymptotic: the entropy estimation and bandwidth selection take O (n log n.), compression takes O (n), resulting in total time complexity of O (n log n)
'Hereinafter, it is possible to use any kernel with infinite support K, but the Gaussian one is preferable hecause of its light tails and infinite differentiability.
20

---

E.1.2 LEAST SQUARES ERROR
Now let consider the Least Square Cross Validation method (see Turlach (1999); Sain (994)). In this method, bandwidth selection is based on the minimization of the mean squared error between the exact density and the corresponding kernel density estimate. We minimize the following expression:
ISE(b) = (Po(#) - p(x))² de
R
Here, p is the true probability density function, and pn(x)) the estimate with the bandwidth b
po(x)
K
JR
baN
Since the true distribution is unknown, we substitute with Pemp- This leads to the following objective function to be minimized:
N
N Jb - =zz))=
2
Σ
N
N 1=1
1=
j=
where
E
Jo(E)
K
dr,
6
b
which can be computed via the Fourier transform.
Asymptotic: The entropy estimation takes O (I no) compression takes O (n), same as KDE ML. However, the optimal bandwidth selection takes O (n²) due to the quadratic complexity of the minimized objective. Therefore, this algorithm has total time complexity of O (n²). making KDE LSE asymptotically the slowest algorithm implemented within this research.
F.2
KOTACHENKO-LEONNNCE
There is another method of entropy estimation, which was proposed by Kozachenko and Leonenko in Kozachenko Leonenko (1987). The main feature of this method is that it utilizes k-nearest neighbor density estimation instead of KDE.
E.2.1 NON-WEIGHTED KOZACHENKKOLLEONNVOO
Let {æx ]== C I be the sampling of random vector X. Let us denote f(x) min r(x, n.)) the 15k$N
distance to the nearest neighbour using the metric (by default, Euclidean metric).
According to Kozachenko & Leonenko 1987), the density estimation at æ is given by:
p(x) Y ((()) -)-
where ei (n) = / is a unit n-dimensional ball volume and 7 is a constant which makes the entropy estimate unbiased (In = 02 ze 0.5772 is the Euler constant).
Asymptotic: the entropy estimation takes O (n log n), compression takes O (n), resulting in total time complexity of O (n log n).
E.2.2 WEIGHTED KOZACHENKO-
The main drawback of the conventional Kozachenko-Leonee estimator is the bias that occurs in dimensions higher than 3. This issue can be addressed by using weighted nearest neighbors estimation. modified estimator is proposed in Berrett al. (2019):
A
wa log E(),
21

---

where is the weight vector, &(), function. We choose the weight vector w (WI,..., wx) as follows. For let
9() c1 (n)) (N 1) P v denotes the digamma
r (j Tj++ - 0 for =I,...
W E
R
Wy
I(j)
j=1
2k
Σ
w/ and wy jf $
k
n
n
and let the w be vector from Wik) with the least (2-norm.
Asymptotic: the entropy estimation takes O (n log r.), weight selection -0 (1*) (1), compres-
sion O (n), resulting in total time complexity of O (n logn).
E.3 LIMITATIONS CLASSICAL ENTROPY ESTIMATORS
Although the entropy estimation is an example of a classical problem, it is still difficult to acquire estimates for high-dimensional data, as the estimation requires an exponentially (in dimension) large number of samples (see Goldfeld et al. (2020); McAllester & Stratos (2020)). As the mutual information estimation is tightly connected to the entropy estimation, this problem also manifests itself in task. Although this difficulty affects every MI estimator, classical estimators may be assumed to be more prone to the curse of dimensionality, as they are usually too basic to grasp a manifold-like low-dimensional structure of high-dimensional data.
In this section, we provide experimental proofs classical estimators' inability to yield correct MI estimates in the high-dimensional case. We utilize the same tests with images of 2D Gaussians used in Section 5 Figure However, due to computational reasons, the size of the images is reduced to 4 x 4 and presented int Table For a comparison we also provide the results for WKL estimator fed with the PCA-compressed data.
(so the data is of even smaller dimension compared Section 5). The results
Table 2: MSE (in nats) classical MI estimation methods ran on 5 10s images of 2D Gaussians. The character "_" denotes cases, in which the method failed to work due computational reasons (numerical overflows, ill-conditioned matrices, etc.).
Images size
KDE ML
KDE LSE
KL
WKL
WKL, PCA-compressed
4 x 4
4.1 103
1,95 10
9,7
2,9.
10a
1,87 1,
8 x 8
2.8 10
6, 6,6.10
10
7,5
10
16 16
0, 67
Note that although WKL estimator performs better in Section 3 due to lower bias, it is outperformed by the original KL estimator in the case of uncompressed data due to lower variance. However, this observation is not of great importance, as all the four methods perform poorly in the case of 8 x 8 images and bigger.
F TECHNICAL DETAILS
In this section, we describe the technical details of our experimental setup: architecture of the neural networks, hyperparameters, etc.
For the tests described Section 5 we use architectures listed in Table 3 The autoencoders are
trained via Adam Kingma & Ba 2017) optimizer on 5 10 images with batch size 5 10 learning rate 10-4 and MAE loss for 2 10s epochs. The MINE critic network is trained via Adam optimizer on 10 images with batch size 512, learning rate 10-3 for 5 103 epochs.
For the experiments described in Section 6 we use architectures listed in Table The input data autoencoder is trained via Adam optimizer 5 10a images with batch size 1024, learning rate
22

---

Table 3: The NN architectures used to conduct the synthetic tests in Section
NN
Architecture
Conv2d(1, 4, ks=3), BatchNorm22,, LeakyReLU(002),, MaxPool2d(2)
x1:
Conv2d(4, 8, ks=3), BatchNorm22, LcakkRel.UU(0.2), MaxPool2d(2))
AEs,
x2(3):
Conv2d(8, 8, ks=3), BatchNorm2d, L.cakyReL.U00.), MaxPool2d(2))
16 x 16 (32 x 32)
x1:
Dense(8, dim), Tanh, Dense(dim, 8), LeakyRel
images
x2(3):
Upsample(2). Conv2d(8, 8, ks=3), BatchNormm2,, Leaky
x1:
Upsample(2), Conv2d(8, 4, ks=3), BatchNorm22,, L
x1:
Conv2d(4, 1, ks=3), BatchNorm2d, LeakyReL.U(00))
x1:
[Conv2d(1, 8, ks=3), Maxxooll2d()) Leaky
paralll
MINE, critic NN, 16 x 16 (32 x 32)
x1(2):
[Conv2d(8, 8, ks=3), MaxPool2d((), LeakyRe x2
x1:
Dense(128, 100), LeakyReLU((000))
images
x1:
Dense 100), LeakyReLLUO0001)
x1:
Dense(100, 1)
10-3 and MAE loss for 2 102 epochs; the latent dimension equals 4. The convolutional classifier is trained via Adam optimizer on 5 10 images with a batch size 1024, a learning rate 10-5 and NLL loss for 300 epochs. The noise-to-signal ratio used for the Gaussian dropout is 10 10-3 Outputs of the layers are compressed via PCA into 4-dimensional vectors. Mutual information is estimated via WKL estimator with nearest neighbours.
Table 4: The NN architectures used conduct the information plane experiments in Section 6
NN
Architecture
x1:
Dropout(O.1] I), Conv2d(1, 8, ks=3), MaxPool2d((2,, LeakyRel.U(00000
Input data AE, 24 24 images
x1:
Dropout(O.1) 1), Conv2d(8, 16, ks=3), MaxPool2d((),, Leak)Rel.U(0.00)
x1:
Dropout(O. 1), Conv2d(16, 32, ks=3), MaxPool2d((). LeakyReLU(0.00)
x1:
(288, 128), LeakyReLU(0.01)
x1:
Dense(128, dim), Sigmoid
L1: Conv2d(1, 8, ks=3), (ea0)ReLU(0.00)
Lz:
Conv2d(8, 16, ks=3), Lca.)ReLU(0.00)
CNN classifier
La:
Conv2d(16, 32, ks=3), L0000))
L4: Dense(32, 32), LeakyReLU(0.00)
Lg: Dense(32, 10), LogSoftMax
Here we do not define fi and gi used in the tests with synthetic data, as these functions smoothly map low-dimensional vectors to high-dimensional images and, thus, are very complex. A Python implementation of the functions in question available in the supplementary material, the file source/ wwwpce/e/scceeeeeeeeeeeeeeeeeeee
www.rc/ppttooo...////// http//yyttoo/mmeee htttttoottmttt/eee mut info/uttiii utils//a wew.ssssynttti synthetic.py..
23


Reviews:
[{"review_id": "RjgFSDr3mP", "reviewer": "Reviewer_Fd1K", "summary": "This paper proposed a framework for Information Bottleneck analysis of neural networks. To estimate the mutual information(MI) of high dimensional vectors, the paper proposes to use an auto-encoder to compress the data into a low dimensional space and provide a theoretical analysis of the error bounds. To avoid the hidden representation of a trained network $L$ becoming a deterministic function of $X$, the paper incorporates the Stochastic Neural Network. Numerical experiments on synthetic data show that the MI estimator of the compressed data can be close to the true value using the weighted Kozacheko-Leonenko estimator. Then the estimator is used on compressed data of several hidden layers $L_i$ of a convolutional network trained on MNIST to demonstrate the evolution of $I(L_i, X)$ and $I(L_i,Y)$.", "questions": "Please see the weakness part above, some minor questions\n\n1. In section 3.1, it says \"This hypothesis is believed to hold for a wide range of structured data, and there are datasets known to satisfy this assumption precisely.\" Any references to the hypothesis or some examples that the datasets satisfying this assumption precisely? I think some details would help understand the hypothesis.\n\n2. The stochastic NN is introduced to help compute the mutual information, does It also affect model training? if not, what is the relationship of a stochastic NN and a regular trained NN?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "Using information-theoretic analysis of DNNs to understand the training evolutions is an interesting topic. The proposed method provides a practical method to conduct information bottleneck analysis. Theoretical analysis and experiments on synthetic data demonstrate that an appropriate estimator on the compressed data can approximate the mutual information well.", "weaknesses": "I think a lot of details of the proposed method are missing. \n\n1. What are the structures of the auto-encoder used to compress the data, and how are they trained? Is the result sensitive to the choice of the auto-encoder structure or some hyper-parameters? In algorithm 1, $E_X$, $E_Y$ of the auto-encoder is assumed to satisfy conditions of Corollary 1, so I think the auto-encoder is an important detail of the proposed method and should be discussed more in the paper or appendix. If the performance is sensitive to the details of auto-encoder choice and requires a lot of tuning, it will be a major drawback of the proposed method.\n\n2. How are the stochastic NNs used in the proposed method, and what are the noises introduced in each layer? Incorporating stochastic NN is emphasized several times in the paper as a main component of the proposed method, but the details are missing.\n\n3. Another weakness is that the proposed method is only applied to one real data example, the MNIST data. More experiments and analysis would improve the significance of the paper.\n\nReaders are directed to the source code about these details, I think at least some more details of the proposed method should be provided in the paper or appendix.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "plDP1Lj1Yd", "reviewer": "Reviewer_MJhB", "summary": "The paper proposes to measure mutual information (MI) in neural networks via compressed representations, utilizing auto-encoder architectures that are assumed to be bijective mappings to a lower-dimensional space. With this rationale, the authors propose performing information plane analyses of stochastic neural networks using these compression-based estimators.", "questions": "- In Fig. 3 and 4, why do the curves for most estimators decrease when compared to the true MI?\n- In the same figures, why does MINE require a critic with the same architecture as the autoencoder, if its input are low-dimensional ($n'=2$ or $4$) signals? (Note that the MINE curves are blue.)\n- In the same figures, there is good agreement between the blue and green curves. However, it can be assumed that the encoders $E_X$ and $E_Y$ act in a way to retrieve $f_1(\\xi)$ and $g_1(\\eta)$, respectively (or some other bijection thereof). With this in mind, it is not surprising that the curves agree so well. Has this been tested? For example, it could be a good idea to use a higher/smaller dimension for the output of the encoders as for the dimension of the generated signals.\n- Why does the KL estimator have such a high offset?\n- In Section 6, is noise also added during training or only during computing MI estimates? In any case, since noise is added one must not ignore the potential geometric affects that are induced by it (effects both during training and during estimation).\n- How where the confidence intervals in Fig. 5 computed? How many networks have been trained?\n- In Fig. 6, why is there so much variation/so large CIs at the end of training?\n- In the same figure, why is there no decrease of \"loss delta\" at the end of training?\n- In the same figure, does \"loss delta\" refer to the difference of losses between two consecutive epochs?\n- In the same figure, the data processing inequality seems to be violated for $I(L;Y)$, see $L_2$ vs. $L_5$. Do you have an explanation for this?\n\n_EDIT:_ After discussion with the authors, I have improved my score.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is quite accessibly written and deals with an interesting topic. The results seem interesting, as they are not fully in line with what has commonly been observed in information plane analyses.", "weaknesses": "In addition to the questions below, I see a few weaknesses that prevent me from giving a better score:\n- While it is clear that MI does not change under bijective mappings, the same may not hold for estimators of MI. In other words, I question the validity of the first equality in (4). To be more concrete, I think one can create counterexamples where a KDE of MI differs depending on whether it is obtained from the original data (dimension $n$) or the latent representation (dimension Sn'$). Since MI can only be estimated and not measured, this leads me to questioning the validity of the study: If compression affects the results of the estimator, then how can we be sure that the estimate represents the \"true\" (i.e., uncompressed) behavior of MI?\n- Along that line, it would be important to see in Figs. 3 and 4 how the MI estimate obtained directly from the high-dimensional images behaves. This would more clearly show that compression is useful in comparison with estimators for high-dimensional data.\n\nSummarizing, the paper does not show that the compressed representations are \"better\" for MI estimation than the original layer outputs/data, or that fundamentally different qualitative results can be obtained. While the approach seems reasonable and valid, I would like to see more evidence that it outperforms MI estimation from high-dimensional random variables.\n\n\nMinor:\n- In (5) and below, it should be made clear that $E_X$ is the encoder for $X$, while $E_Y$ is the encoder for $Y$.\n- In the third paragraph of Section 5, the abbreviation WKL is not introduced.\n- The colormap in Fig. 5 is not ideal, there is too little variation in the color.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yguRR5Cw08", "reviewer": "Reviewer_d5Zt", "summary": "In this paper, authors have proposed a method for estimating the Shannon Mutual Information (MI) between the compressed representations of high-dimensional random vectors. Moreover, a discussion of information flow estimation based on the Information Bottleneck (IB) principle is presented for the MNIST classification problem using a simple LeNet architecture.", "questions": "1 - Does the function $f$ in statement need to to be bijective (both injective and surjective) ? I think only being  injective is not sufficient.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "A systematic comparison between different entropy estimation methods in the MI estimation problem has been conducted in section 5.", "weaknesses": "While this paper tries to solve the well-known challenge in the IB principle, i.e., MI estimation for high dimensional vectors, I have some issues with the approach in this paper:\n\n1 - I don't follow the claimed MI estimation approach. The proposed method has considered some of existing entropy estimation methods, (which they all have their own limitations) to estimate MI using the empirical frequencies in the MI expression (equation 1). What is the novelty of this approach ? The entropy estimation is a canonical problem in statistics. Both parametric and non-parametric have their own pros and cons. While non-parametric methods (e.g., KDE) do not consider any underlying assumption about the distribution of the data, they are typically not scalable in very high-dimensional regime (and they need a lot more training data), and they might be less sensitive than their parametric counterparts when the assumptions of the parametric methods are met.\n\n2 - Another issues is the one that authors have also mentioned in the paper. Just providing a toy example of MNIST for a small classification model is not yet convincing to consider the scalability of the MI estimation approach, its accuracy on the high-dimensional data (not just 784), and its applicability in other non-classification problems (regression, unsupervised, generative, etc.).\n\n3 - Some details about the functions $f_1$, $f_2$, $f$, $g$, etc do not exist in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pcDfF2tIxF", "reviewer": "Reviewer_Lyy3", "summary": "The Information Bottleneck (IB) principle provides an information-theoretic framework to analyze the training process of deep neural networks (DNNs) by tracking mutual information values. The training process involves two phases: fitting and compression, with compression believed to enhance generalization. The paper introduces an approach for IB analysis in general DNNs, overcoming challenges in estimating mutual information between high-dimensional vectors. The method involves compressing representations to estimate mutual information accurately. The approach is supported by theory and demonstrated through experiments, including analysis of a real-scale convolutional DNN, uncovering new insights into mutual information dynamics.", "questions": "see weaknesses", "limitations": "", "rating": 6, "confidence": 1, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The theoretical analysis is sound and the experiments conducted in the paper are interesting and showing insides into the information flow of DNN.", "weaknesses": "Here are some comments / weaknesses / things to improve:\n\nAbstract: „has only been verified for NNs of tiny sizes or specific types, such as quantized NNs“: I would not say that this is verified see summary table of [1]. The SNN paper did not saw a compression.\n\nIntroduction: „(a) stochastic NNs (Goldfeld et al., 2019; Tang Nguyen & Choi, 2019; Adilova et al., 2023) or (b) quantized NNs (Lorenzen et al., 2022).“: there is also the combination of both which was a paper at AAAI‘23 [2].\n\nStatement 2: „This statement demonstrates that an arbitrary amount of information can be lost through compression. It arises from the fact that “less significant” in terms of metric spaces does not align with “less significant” in terms of information theory. However, with additional assumptions, a more useful theoretical result can be obtained.“: I am not sure but following the arguments from Goldfeld [3] MI is ill defined in deterministic DNN. So with this in mind no information can be removed in the compression phase. Can you maybe discuss this in the paper?\n\n\n[1] Bernhard C. Geiger. On information plane analyses of neural network classifiers—a review. IEEE Transactions on Neural Networks and Learning Systems, 33(12):7039–7051, 12 2022. doi: 10.1109/ tnnls.2021.3089037.\n[2] https://ojs.aaai.org/index.php/AAAI/article/download/25851/25623\n[3] Ziv Goldfeld, Ewout van den Berg, Kristjan H. Greenewald, Igor V. Melnyk, Nam H. Nguyen, Brian Kingsbury, and Yury Polyanskiy. Estimating information flow in deep neural networks. In ICML, 2019.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)