Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper puts forward a method of mutual information estimation within neural networks where the stochasticity of observations is ensured by Gaussian dropout. The authors then use MC methods to estimate entropies and conditional entropies. The proposed approach is compared to MI estimation based on binning and the authors claim to observe the compression phase postulated by Tishby et al., which was not observed with the binning estimator.

strength_and_weaknesses: STRENGTHS

The interpretation of neural networks with the information plane approach is a relevant problem which has garnered some attention since it was proposed in 2017.

The authors' idea of using dropout for MI estimation is valid, although not necessarily novel.


WEAKNESSES

The main weakness of the paper is its limited novelty. The negative results presented in Section 3 are straightforward and the only new results of this paper follow from the restriction to Gaussian dropout and estimating entropies with MC methods. The observed results concerning the compression phase in the last section are not convincing (the curves resemble straight lines). The authors also only compare this result to the binning estimator which is known to result is spurious effects (see e.g. the works of Saxe or Gabrie).

clarity,_quality,_novelty_and_reproducibility: The paper is relatively easy to read, but the originality is limited.
summary_of_the_review: This paper takes an interesting approach to the analysis of NN with the information plane, but the presented results seem neither significant nor novel enough to warrant publication.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper analyses the computation of mutual information in DNN with dropout.  It shows that mutual information for discrete dropout is infinite, but for continuous drop it is a well defined finite quantity.  The paper shows how this can be estimated using Monte Carlo techniques and empirically shows that this quantity converges.  The paper finishes with an analysis of training a neural network with dropout in the information plane.
strength_and_weaknesses: The paper resolves a long standing problem with using information theory to study learning in deep neural networks and particularly computing mutual informations.  This is a problem that has received an enormous amount of interest.  The proposed solution is both technically sophisticated (in terms of proofs) and elegant (in terms of implementation).

My very personal question as a naturally sceptical person is whether all the effort put into studying the information plane is worth all the effort. I'm prepared to accept that there are enough people interested that this is worth publishing, but the results section did not convince me that there is a lot of understanding to be gained.  I accept though that the authors have said that a full analysis will end up elsewhere so I guess I need to wait.  I also have a rather practical question that these measure are intrinsically super unstable when the mutual information becomes unbounded in a deterministic network.  This gives me an uncomfortable feeling as this does not reflect the generalisation performance of networks with and without dropout (or with continuous versus discrete dropout).  It seems to me that the true solution is to find a more meaningful picture than the information measure where networks that perform similarly don't have such different behaviour.  However, I realise this is just me being picky and I'm not really expecting the authors to address my scepticism,
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written.  There are a few small grammatical errors and if the the authors have the energy they might consider rereading the text once more to eliminate these.  The paper is a high quality paper (the proofs although in a way straightforward requires a sophisticated understanding of probability which is rare).  The work is novel and the proofs and empirical results reproducible.
summary_of_the_review: This is a well written paper addressing a problem that has received considerable attention in the research community.  It provides a sensible way of making sense of the Information Plane.  It is mathematically sophisticated, but leads to a practical application.  In my view this is of sufficient interest to warrant publication.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The goal of the paper is to obtain a sufficiently accurate estimate of MI (between input and representation) for dropout neural networks  and to use it to confirm the information bottleneck hypothesis for this NN model.

Contributions:
- The authors propose a monte-carlo based estimate of MI in Gaussian dropout networks that is relatively accurate and well-supported by the performed theoretical analysis.
- The authors use their MI estimate to provide information plane analyses for several NN models with Gaussian and information dropout (LeNet and MLP on MNIST, ResNet on CIFAR10)


strength_and_weaknesses: Plus

 A solid contribution to the area of information-theoretic analysis of neural networks. 
-  The idea of utilizing stochasticity induced by dropout layers to estimate MI (between input and internal model representation) is novel and interesting. 
-  The authors provide a thorough theoretical analysis of MI estimation in NNs and its limitations for both binary dropout and dropout with continuous noise.
-  The proposed monte-carlo MI estimate seems promising (based on the experiments).   


Minus

I see the main limitation of the paper in relatively narrow area of ​​application. The method is restricted to dropout networks with continuous noise. 
The authors provide a proof that the principle cannot be extended to (the widely used) binary dropout (or to NNs without dropout). 
The concurrent technique of (Goldfeld et al. 2019) seems to offer wider possibilities of use despite its other disadvantages (namely the need to alter the internal representation of the model by noise).  


clarity,_quality,_novelty_and_reproducibility: Novelty
- Both the idea and realization of using stochasticity hidden in dropout layers to estimate MI are novel. The MI estimates seem to be accurate enough to provide a meaningful information plane analysis of the model.
- Related work is cited and analyzed adequately.

Quality 
- The technical and experimental results seem to be well-executed to the best of my assessment. I appreciate namely the detailed theoretical analysis. 

Clarity And Reproducibility

- The paper is written comprehensibly and its structure is good. 
1. Nevertheless, the description and computation of the proposed MI estimate is "hidden" in the paragraphs.
 I recommend the authors to summarize the computation into one comprehensible expression (or in a box with pseudocode). This will help readers to quickly understand and reproduce it.
2. The colors in Figure 4 and Fig. 10 are not well chosen. It impossible to distinguish the individual shades of orange/red. That makes the graphs incomprehensible. 
Please, change the colors and support the comprehensibility by dashed lines.  
3. Figures and their labels are too small (compared to other text) and therefore hardly comprehensible without high magnification on the screen (e.g., Figure 1, 5, 6, 7).
4. It is not clear what "doe" and "doe_l" stand for in Fig. 4 and Fig. 10. Please add an explanation of this notation. 


summary_of_the_review: A solid contribution without greater flaws except limited area of application.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper address the ill-posed problem of IB of deterministic neural networks. Motivated by the success of stochastic neural networks, the authors proposed using the dropout technique to define and estimate the mutual information in neural networks. The authors showed that MI is bounded when the dropout uses continuous noise. Then they conducted numerical experiments of MI and studied the IP of neural network models.
strength_and_weaknesses: # Strength
* Compared to stochastic neural networks, using the dropout is more practical and realistic. 
* The estimation procedure of MI using the dropout seems easy and worked well compared to binning approaches.

# Weakness
* Compared to binning approaches, this method only applies to the dropout models with continuous noise. There are many networks that do not use dropout and continuous noise.
* When using information dropout, its noise is learned simultaneously. This means the estimator of MI changes during the training. Thus, we use different MI estimators for different epochs when plotting the IP. So it is hard to understand the obtained IP curves or meaningless. Is my understanding correct?
* I wonder how the dropout hyperparameters affect the estimation of MI and the IP. For example. The number of dropout layers or the magnitude of the noise should have a large impact on them, but there is no discussion and no numerical experiments about them.
* Why I(Y; Z) are so different between dropout and binning methods, as shown in Figures 1, 5, 11, 12? As far as I understand, I(Y; Z) is estimated by subtracting H(Y) from the cross entropy as mentioned in Sec 4. This is the same procedure for the dropout and binning approaches.
clarity,_quality,_novelty_and_reproducibility: # Clarity
* This is a minor point. The colors of lines in Figure 4 are similar and hard to recognize. 

# Novelty
* The idea of using dropout is interesting, but it is the straightforward extension of the idea of stochastic neural networks. As far as I understood the technique is not quite novel.
summary_of_the_review: This paper proposed elegant way of defining and estimating MI and IP for networks that uses dropout. However, as far as I understood, the technique is not so novel and numerical experiments are limited to show the usefulness and understand the IP in dropout models.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper has received mixed reviews, with two reviewers strongly recommending acceptance (8-8) and two reviewers recommending rejection (3-5). The author-reviewer discussion was productive and the authors addressed many of the reviewers' concerns, although they have not expressed whether they are satisfied with the authors' responses. 

The paper presents a theoretical result that, in my opinion, constitutes a valuable stepping stone towards the understanding of the training dynamics of neural networks. The paper studies mutual information estimates in neural networks with dropout noise and finds that binary dropout does not prevent MI from being infinite. The analysis follows with an information plane analysis of neural networks with continuous dropout, leading to the (supposed) observation of compression during training, as postulated by Tishby et al. (2017). 

As an area chair, I believe that the contribution of the paper is interesting and novel. It is technically well done and the experimental results reveal interesting insights. Concerns about the limited novelty are not strongly justified. I recommend accepting the paper for publication.
justification_for_why_not_higher_score: Given the initial concerns of two reviewers, I do not recommend a higher score.
justification_for_why_not_lower_score: Valuable theoretical contribution. Interesting experimental obervations. Technically solid.
Decision: Accept: poster