Latent Point Collapse Induces an Information Bottleneck in Deep Neural Network Classifiers

Authors: Luigi Sbailò, Luca M Ghiringhelli
Primary area: optimization
Keywords: classification, regularization, information bottleneck, latent representations.

Abstract:
The information-bottleneck principle suggests that the foundation of learning lies in the ability to create compact representations. In machine learning, this goal can be formulated as a Lagrangian optimization problem, where the mutual information between the input and latent representations must be minimized without compromising the correctness of the model's predictions.
Unfortunately, mutual information is difficult to compute in deterministic deep neural network classifiers, which greatly limits the application of this approach to challenging scenarios. In this paper, we tackle this problem from a different perspective that does not involve direct computation of the mutual information. We develop a method that induces the collapse of latent representations belonging to the same class into a single point. 
This point collapse not only significantly reduces the entropy of the latent distribution, thereby creating an information bottleneck that correlates with improved generalization, but also makes the network Lipschitz, offering guarantees for enhanced robustness.
Our method is straightforward to implement. We demonstrate that it substantially improves the network's robustness, provides a small yet statistically significant increase in generalization, and enhances the network's ability to detect misclassifications.

Decision: reject
Reviewer scores: 3, 6, 3, 5

Reviews and discussion:
<thread>
creation date:
1730268498233

thread_type:
official_review

<thread_object>
creator: reviewer_gevm

<content>
summary:
The paper presents an innovative approach to enhancing robustness and generalization in deep neural network classifiers. The authors introduce an additional linear layer as the latent feature layer, along with a regularization term for the learned latent features. They claim that this method creates compact representations of input data by inducing a latent point collapse, where same-class representations converge into single points in the latent space, effectively reducing entropy and creating a bottleneck effect without the need to directly compute mutual information.

strengths:
The proposed method is easy to implement and slightly increases classification accuracy.

weaknesses:
1. **Lacking of rigorous analysis**. Although the authors claim that their method leads to the collapse of all same-class representations into a single point, there is no theoretical proof to substantiate this claim.
2. **Lacking of experiment proof**. The experiments show that the entropy and covariance of latent features decrease, but these metrics could naturally decline due to the regularization term applied to z . Clear evidence of latent feature collapse is lacking, such as metrics comparing the distance between class centers or between the origin and class centers.

questions:
Currently, the paper is purely hypothetical, lacking both theoretical and empirical evidence of latent point collapse.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730392122427

thread_type:
official_review

<thread_object>
creator: reviewer_vlmu

<content>
summary:
The paper introduces a method that induces the collapse of latent representations belonging to the same class into a single point, creating an information bottleneck in machine learning models. By focusing on reducing the entropy associated with the latent distribution, the method enhances the network's robustness, generalizability, and reliability without the need for direct computation of mutual information.

strengths:
>The paper introduces a novel method to avoid calculating mutual information in IB settings.

>Some improvements are achieved on 3 standard benchmarks.

weaknesses:
>The improvements seem to be marginal, for e.g. in Table 1.

>Some more experiments are expected on larger datasets like ImageNet.

>Some theoretical benefits of applying the proposed method in IB are needed.

>There seem to be discussions of neural collapse and information theory in the literature [1] [2] that are closely related but are not discussed.

[1]: Matrix Information Theory for Self-Supervised Learning. ICML 2024

[2]: Unveiling the Dynamics of Information Interplay in Supervised Learning, ICML 2024

questions:
See weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730685132286

thread_type:
official_review

<thread_object>
creator: reviewer_mpu4

<content>
summary:
The paper investigates a phenomenon termed latent point collapse by introducing information bottleneck (IB) layers and modifying the loss function in neural networks. Through an information-theoretic lens, the paper derives the loss function and demonstrates its effects. Experimental results indicate that incorporating IB layers enhances the network's robustness to input perturbations and improves reliability in predictions.

strengths:
* The paper is overall clearly written.
* The experiments on the improved robustness when adopting the IB layer are persuasive and interesting.

weaknesses:
The paper would benefit from a more detailed comparison with the existing Neural Collapse (NC) literature. Based on the reviewer's understanding, the main distinctions between this work and NC research lie in two areas:
* Architectural Addition Between Penultimate Layer and Classifier: This paper introduces an extra architectural component between the penultimate layer and the classifier. However, if we interpret the output of this new component as the features used in NC, the framework here still aligns with the NC paradigm. Additionally, the loss function presented in Equation (3) closely resembles the unconstrained feature models commonly employed in NC studies.
* Class Number Exceeding Feature Dimension: This study explores scenarios where the number of classes can exceed the feature dimension, a setup less frequently examined in NC literature. However, recent works have addressed this case, as in [1] and [2]. The authors may want to discuss these works more closely. 

[1] Jiang et al; Generalized Neural Collapse for a Large Number of Classes, 2023

[2] Liu et al; Generalizing and decoupling neural collapse via hyperspherical uniformity gap, 2023.

questions:
* The binary encoding structure shown in Figure 1 appears to represent a single class. Could the authors clarify the relationship between the features of different classes within this structure?

* The paper employs the Swish activation function for all experiments. How would the results differ with the standard ReLU activation? The binary encoding won't be possible in this scenario. 

* In Tables 1 and 2, the NOPEN architecture seems to represent the default ResNet model without architectural modifications. How would applying the loss function from Equation (3) directly to the true penultimate layer affect the results? Would it still exhibit the benefits of the Information Bottleneck (IB) method?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730724558403

thread_type:
official_review

<thread_object>
creator: reviewer_qafb

<content>
summary:
The paper proposes a new approach to compress feature vectors by integrating a bottleneck layer and incorporating L2 norm regularization for the newly added layer. This method is grounded as an surrogate for the Lagrangian optimization framework, aiming to minimize mutual information loss.

strengths:
- The proposed approach is straightforward and easy to implement, requiring the addition of only a single bottleneck layer and minimal modification of the loss function
- The empirical results effectively demonstrate the method’s ability to reduce entropy, aligning well with the theoretical intuition presented in the paper.

weaknesses:
- The empirical improvements are unconvincing, as the study lacks comparisons with other established regularization techniques. 
- The experimental setup is also limited, with no evaluations conducted on large-scale datasets, such as ImageNet for image classification or Wikitext for language modeling.
- The paper does not provide a theoretical guarantee that the proposed method reduces mutual information loss.

questions:
- Could you include additional baselines to compare your method with other regularization techniques for a more comprehensive evaluation?
- Since your approach appears to induce neural collapse, could you provide a comparison with methods that explicitly leverage neural collapse to enhance model performance?
- Could you offer a theoretical proof or rationale to substantiate that minimizing the proposed loss function effectively reduces mutual information loss?
- Could you consider adding experiments on large-scale datasets to enhance the empirical evaluation of the paper?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732035953053

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Rebuttal

comment:
Dear Reviewers,

We sincerely thank you for thoroughly reading our manuscript and providing valuable feedback.
We have uploaded a revised version of the manuscript, where we believe the main topic and achievements of this paper are more clearly articulated.
In this paper, we develop a method to induce the collapse of same-class latent representations into a single point. This specific condition renders the network Lipschitz, providing theoretical guarantees for improved robustness. We empirically validate this condition, demonstrating an order-of-magnitude improvement in robustness, which we consider the primary achievement of this work.
The latent point collapse also leads to a reduction in the entropy of the distribution that generates the latent representations. This creates an information bottleneck, which, as noted in the literature, is associated with improved generalization. Consistent with this theory, we observe a small but statistically significant improvement in accuracy. In this revised version, we emphasize in both the abstract and the conclusion that this improvement in generalization is small.
However, in addition to discussing a previously unknown phenomenon occurring in the latent space of deep neural network classifiers—namely latent point collapse—the primary utility of this method lies in its significant improvement in robustness.

In addition, we have included another baseline, as suggested by one of the reviewers. Specifically, we compare the effect of adding L1 regularization to the penultimate layer, demonstrating that the enhancements achieved by our method are distinct from those obtained through other regularization techniques. We notice that we had already compared our method against dropout in the original manuscript.
In the revised version, we compare against only one architecture implementing dropout instead of two, as in the initial submission. We made this change to avoid redundancy and to maintain the clarity and readability of the tables and figures, ensuring that the ablation study does not include an excessive number of baselines.

We believe we have addressed all the points raised by the reviewers, with the exception of the request to conduct tests on larger datasets such as ImageNet.
In our study, we performed an extensive ablation analysis involving 8 different architectures, tested across 5 learning rates, and using 5 different initializations. These experiments were designed to provide an accurate and thorough description of the phenomenon we discuss in the paper.
We estimate that replicating these experiments on a large-scale dataset like ImageNet would require between 300,000 and 500,000 GPU hours, a computational cost we currently cannot accommodate.
We respectfully note that requiring such results may disproportionately disadvantage smaller research groups that lack access to such substantial computational resources. 
We would also like to point out that numerous exceptional works published at top conferences such as ICLR have employed the same datasets as we do in this study.
Therefore, we appeal to the reviewers’ understanding to accept our experimental settings and to evaluate our work based on the results we provide, which are supported by sound and principled explanations.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces latent point collapse, a method that encourages within-class latent representations to converge into single points in the latent space of deep neural network classifiers. This collapse creates an information bottleneck that enhances the robustness, generalization, and reliability of the network. 

Reviewers generally agree that the method is easy to implement and brings incremental accuracy gains. However, there are major concerns along the following lines:

- Lack of a clear comparison with Neural Collapse. The proposed method conceptually overlaps with Neural Collapse (NC). While the authors provided clarifications, they did not fully resolve reviewers' skepticism. I share the view that the distinction between the within-class collapse induced here and the natural behavior of NC remains unclear.

- Lack of Large-Scale Experiments on datasets like ImageNet. The authors defended this with computational resource constraints, but given that the improvement on smaller datasets CIFAR / SVHN is quite minor, I’d agree with the reviewers that a large scale experiment is necessary to justify the approach.

- Lack of Theoretical Rigor, both for 1) the claim that class points converge to single points or vertices of a hypercube, and 2) the claim that collapsing leads to benefits in robustness / generalization. Regarding the latter, the only argument I found is based on the Lipschitzness as in Eq. (4), but that is still quite a weak argument as it only applies to within-class points but not to between-class points.

Finally, the topic of within-class feature distribution has been extensively discussed in the literature such as Center Loss (https://ydwen.github.io/papers/WenECCV16.pdf) and many followups. There are also works such as Maximal coding rate reduction (https://arxiv.org/abs/2006.08558) that argue otherwise, i.e., encouraging diversity brings robustness benefits. The authors are encouraged to thoroughly review the literature and clarify how their method compares and contributes beyond these existing approaches.