Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces the PAC-Bayes Information Bottleneck (PIB). Starting from the generalization bound Eq. 4 which shows that the generalization gap is upper bounded by a function of I(w;S), the authors proposes PIB which has an additional regularization term of \beta I(w;S). Since the computation of I(w;S) is intractable, the authors then make several assumptions to simplify its computation, arriving at an estimate of I(w;S) by \tilde{I}(w;S) (eq. 15), and use SGLD to compute it in practice. Experiments show that (1) there is a two-phase transition in SGD training as indicated by \tilde{I}(w;S); (2) \tilde{I}(w;S) seems to correlate with the generalization gap, under different variations of experiment hyperparameters: number of hidden layers, noise ratio, and number of random label data; (3) it improves performance compared to l2 and dropout regularization. 
main_review: Strength:

This paper addresses an interesting and important problem. The proposed PIB is novel. The experiments show that the proposed \tilde{I}(w;S) correlates with the generalization gap, and helps improving the performance.

Weakness:
In order to make the computation of I(w;S) tractable, the authors make several important assumptions. It would strengthen the paper a lot if the paper discuss and perform experiments to show if the assumptions are valid, in the experiments the authors run.

Furthermore, in section 5.5, can the authors show the generalization gap (together with the train and test acc), with different regularization? Ideally we should see that with PIB as the objective, the generalization gap is much smaller than the other methods. With this, we can then be confident that the improvement is due to reduced generalization gap instead of better training.
summary_of_the_review: In summary, this paper is novel, but the experiment should be strengthened as detailed in the main review.
review_title: null

final_score: rating: 10: strong accept, should be highlighted at the conference
review_content: summary_of_the_paper: This paper proposes a new version of the Information Bottleneck objective for training neural networks. This is in part motivated by previously-derived PAC-Bayes bounds on the generalization error that are proportional to the square root of the mutual information of the weights and the training dataset: I(w;S). Thus this new information bottleneck objective attempts to minimize both the empirical risk and this mutual information.The paper derives a computationally-tractable algorithm for estimating I(w;S), then this algorithm is used to show that this quantity is inversely correlated with generalization loss on a variety of neural network architectures. 
main_review: Strengths:
- The paper proposes an exciting general principle of deep learning. As far as I know, the contributions here are novel and will be of high interest to the community. 
- The authors build on previous work by showing how their IB objective addresses the shortcomings of previous work in this area.
- It is very well written. This is a highly-technical paper, and the details are presented in a careful and thoughtful way.
- The experiments are well done and the results support the conclusions. Specifically, this objective is motivated by a PAC-bound (the tightness of which is not clear) and various approximations are used to estimate I(w;S) (the accuracy of these are not immediately clear). The experiments address these issues by showing that the motivation and the approximations are reasonable. 

Weaknesses:
- The limitations of this method are not discussed clearly. For example, the paper provides an algorithm for sampling from the weight posterior p(w|S), but how does this compare computationally to standard training of a neural network, or a estimating the posterior in a Bayesian Neural Network? 
- There are some minor grammatical and spelling typos throughout, e.g. "infection point".

summary_of_the_review: An excellent paper with exciting ideas, clear presentation, and technical depth.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The authors propose new interpretation of the Information Bottleneck (IB), dubbed PAC-Bayes Information Bottleneck (PIB).

Where the IB is defined wrt the mutual information between feature representations $T$ and either inputs $X$ or targets $Y$, PIB is defined wrt the empirical risk over the dataset $S = \\{X_i, Y_i\\}_{i=1}^n$ and the mutual information between model parameters and $S$, $I(\mathbf{w}; S)$, or information stored in weights.

The authors show that PIB is a bound on generalization error.

The authors derive a tractable estimator for $I(\mathbf{w}; S)$.

The authors present an approximate inference method for p(\mathbf{w} \mid S)  that utilizes the proposed PIB.

The authors show that PIB reflects the hypothesized two-phase fitting and compression modes of neural networks across different activation functions, network depth, and network width.

They show that $I(\mathbf{w}; S)$ yields a good estimator of the generalization error that is robust to label noise.

They show that their inference method improves generalization across several benchmark datasets.
main_review: **Strengths**
To my knowledge this paper has significant technical and empirical novelty.
The authors do a good job of summarizing previous work and differentiating their contributions.
This is not my area of expertise, but the derivation of PIB, the estimator for $I(\mathbf{w}; S)$, the proposed optimal posterior all look novel and correct.
The experiments are well done, thorough, and support the main claims of the paper.

**Weaknesses**
The main weaknesses of this paper are language and clarity.
I would recommend a thorough Grammarly or perhaps external advice.
The graphs should report means and standard error intervals over multiple random seeds.

**Some specifics**
- Some technical terms are used before definition (e.g., phase transition in abstract)
- In the abstract IB cannot explain, maybe IB theory can, as you use in the intro.
- IIW and $I(\mathbf{w}; S)$ are redundant, would recommend just using $I(\mathbf{w}; S)$
- "Third, mutual information becomes trivial in deterministic cases." Please elaborate / cite.
- "(2) we derive a solution to the intractable...," can something intractable have a solution? maybe approximation is better.
- "optimal posterior of PIB," does PIB have a posterior or is the posterior over the weights?
- Figure 2. IIW only shows compression phase, can the loss also be included in these plots?
summary_of_the_review: To my knowledge this paper demonstrates significant technical and empirical novelty.
I believe the main weaknesses can be addressed prior to publication.
Therefore I recommend acceptance.
However, I am not an expert on this topic, so my confidence is only a 2.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors propose a formulation of the information bottleneck problem, replacing the mutual information between input X and latent representation Z via the mutual information between the sample S and the weights W obtained from the sample. They derive closed-form solutions for this mutual information in the Gaussian setting and propose an SGLD scheme to optimize the objective. Using this objective and optimization algorithm, the authors investigate several interesting scenarios, including different activation functions and noisy labels.
main_review: The paper is generally well written and treats an interesting and timely topic. The idea to limit the information about the sample that is contained in the weights is not new (the authors cite several works that bound the generalization error via this information), but this is the first time that I have seen a corresponding cost function implemented in practice. There are, however, a few issues that are not perfectly clear to me:

- The authors cite the literature stating that the generalization gap is limited by I(S;W) if the loss is sigma-sub-Gaussian. Does this hold for the negative log-likelihood in (6)? Also, in (6) is S a random variable or not? (4) requires that I(S;W) is computed as an expectation over p(S), while the log-likelihood in (6) is an expectation over P(w|S), i.e., not over p(S) but over a concrete S. How can this be understood? 
- Connected to this, is it safe to call the resulting cost function an information bottleneck cost function? I assume that this is better called an IIW-regularization rather than an IB cost. The IB cost is a very specific formulation that combines a mutual information cost with a mutual information utility, whereas here we have a general cost with an additional mutual information cost as regularization term.
- The authors correctly claim that I(X;T) becomes trivial if the network is deterministic. More precisely, this mutual information becomes infinite in many of these cases (see "Learning Representations for Neural Network-Based Classification Using the Information Bottleneck Principle" by Amjad and Geiger). I believe that this result carries over to I(S;W) being infinite for deterministic learning algorithms. This may not hold for all learning algorithms, but certainly for some. My own gut feeling suggests that I(S;W) is infinite for SGD with finitely many epochs (e.g., by the fact that there are only combinatorially many options to shuffle the batches), but that it is finite for SGLD, where noise is added to the weights. It is therefore not clear to me in which settings the analysis in Section 3 is a valid approximation. In other words, in which settings is the assumption that p(w|S) is Gaussian valid? Does it only hold for SGLD?
- Connected to the point above: In which cases is the assumption that p(w) is Gaussian a valid approximation? 
- Can this Gaussian assumption about p(w) be used to bound I(S;W) from above? (E.g., for a Gaussian learning algorithm, can it be shown that the term I(S;W) is maximized if W becomes Gaussian as well? This would be at least intuitive from a channel coding perspective, where a Gaussian channel input is known to maximize the mutual information through a Gaussian channel, and which is then known to produce a Gaussian channel output.)
- In Algorithms 1, pls. compare line 9 with your equation (15). In (15), you sum over squared inner products. In line 9 and 11, you square over the resulting sum of inner products. Is this difference intended, and if so, how can it be explained? Also, do we have $T_0 \ge T_1$ in Algorithm 1?
- In Fig. 1, why is the mutual information I(W;S) evaluated for different layers? What is the exact meaning of splitting the IIW between layers in terms of the generalization bound? I was assuming that the generalization bounds all consider the entire set of weights, and that the proposed PIB should do so as well. 
- Also in Fig. 1, the discussion of the inflection point is not fully clear. 
- In Section 5.1, it is claimed that the variance of the information explodes. Can this be made more precise (e.g., by writing down the mathematical symbol for this variance)? Furthermore, this is not shown in the figures, if I remember correctly.
- In all figures, why is the mutual information I(S;W) so small? These numbers do not seem right. I would assume that it is necessary to "learn" more than 10⁻2 bits/nats to successfully solve a classification problem. In other words, while the general trend of IIW seems to be correct, I am not convinced of the correctness of the absolute numbers. Can you provide some intuition about these small numbers? Is this connected with the proportionality symbol in (14)? (But going from (8) to (9) it seems to be that additive constants are dropped, not multiplicative constants.)

For the sake of clarity, I would prefer that footnote 3 is in the main text. Also, in some instances the notation and terminology is not clear. E.g., is S sampled iid in (4)? Why is the "oracle prior" called an oracle? How exactly is the bootstrapping resampling weight \zeta_k defined? Why is the temperature $\beta$ called the annealing temperature just before (18)? At the end of Section 5.2 you write that the l2-norm keeps increasing -- the norm of what?
summary_of_the_review: A very interesting paper, dealing with an interesting and timely topic. Unfortunately, the paper is not perfectly clear throughout all sections.
review_title: null

Meta-review: This paper revisits the information bottleneck principle, but in terms of the compression inherent in the weights of a neural network, rather than the representation. This gives the resulting IB principle a PAC-Bayes flavor. The key contribution is a generalization bound based on optimizing the objective dictated by this principle, which is then tractably approximated and experimentally verified. Reviews raise concerns about assumptions made to achieve the tractable version and a public discussion debates whether this is truly a PAC-Bayes bound. The authors address these adequately. Another concern is whether improvements in experiments can be ascribed to the new objective. Authors add new experiments in support of this. Additional concerns about the clarity of certain aspects of the paper were or were promised to be addressed by the authors. Overall, the perspective of this paper, its technical contributions, and experimental evaluations appear to be worthwhile to share with the community, as they advance the applicability of the information bottleneck principle.
Decision: Accept (Spotlight)