Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors prove an upper bound on the estimation error that depends on pairwise distances between hidden unit values per layer. The bound itself is quite likely very loose in standard deep learning settings due to the dependence on various norms of the weights. Loosely inspired by the bound, the authors then propose an optimization algorithm with a particular regularizer that encourages within-layer activation value diversity. The empirical results suggest that the proposed algorithm boosts the performance in most regimes studied (CIFAR 10, CIFAR100 trained on ResNets, Dense nets). The boost is minimal for standard datasets, but it increases when the labels are randomized. Compared to standard empirical risk minimization, a small boost in test accuracy on transfer learning tasks is also reported.
main_review: *The purpose of the bound*

The authors derive an upper bound on Rademacher complexity. However, as a reader I am not convinced of the importance/utility of these bounds:
The bounds depend on upper bounds on norms of the weights. The norms are distribution dependent. Further, such norms (in unregularized SGD and SGD with weight decay) have been shown to grow with the amount of data (Nagarajan and Kolter ‘19). Thus it is not even clear if the bound decreases with the amount of data when considering classifiers that are obtained with the proposed algorithm. Do the norms grow with the data? If so, how fast relative to the inter-layer activation diversity term?
Demonstrating that the bound on Rademacher is tight for some distributions would be another way to demonstrate that the bounds are interesting.

The authors talk about studying generalization starting in the abstract and throughout the text, e.g.: “..we analyse the effect of the within-layer activation diversity on the generalization error bound of neural network.”.

I see multiple inaccuracies with this and related statements: the authors provide a bound, rather than study an effect of within-layer activation diversity -- the effect would be totally different on a different upper bound. And these are just upper bounds, after all.

I also wanted to note that what is actually provided in the paper is an estimation error bound, not a generalization error bound. I am guessing the authors arrived at the estimation error by starting from a Rademacher-based risk bound. But there is no mention of that. Why not state a generalization bound instead?

Here is another statement (out of several) that is very misleading and incorrect (Section 3, first sentence):

“As shown in the previous section, promoting diversity of activations within a layer can lead to tighter generalization bound and can theoretically decrease the gap between the empirical and the true risks.”

Where are the results showing that promoting diversity of activations decreases the gap between the risk and empirical risk? Also, the bounds are tighter than what? There is no comparison to any other bounds.

Just after Eq (3), f* is defined as argmin_f L(f). What class is the minimization performed over?


*How does activation diversity connect to the rest of the literature*

The authors discuss other approaches to increasing weight, activation, etc., diversity. However, since they argue that increased diversity improves generalization, it would be interesting to see a comparison to other properties that are linked to improvements in generalization. Probably one of the most recently discussed properties is the flatness of the minima. Are these complementary properties? Do they relate in any way? (see, e.g., work on Sharpness Aware minimization by Foret et al, Computing Nonvacuous Generalization Bounds by Dziugaite et al, Sharp minima work by Keskar et al., and many others)

What exactly is “Standard Training” in Section 5? Is it regularized in any way? How would the performance of the proposed algorithm compare to various other regularization methods? 

*The efficiency*

In my opinion the real contribution of the paper is the algorithm, not the bound on estimation error. I do not see any discussion around the computational costs or training times of the algorithm.

*Hyperparameter settings*

The algorithm requires tuning regularization-related hyperparameters. How are those chosen exactly? Do the authors have a held-out set? Based on Fig 1, it seems that the wrong values of these hyperparameters may wipe out all the improvements in generalization presented in Table 1.

Also, the discussion of Figure 1 and more generally section 5 once again suggest that one may expect to see a smaller generalization error gap when optimizing for the inter-layer activation diversity. There is no such theoretical connection made in the paper.

How about other hyperparameters? How are those set? Are they all standard or have they been changed at all?

*Ensembling* 

The authors several times relate activation diversity to functional diversity (introduction, section 4). I do not quite understand the connection. I can easily construct cases how improved activation diversity in individual classifiers would result in a poor ensemble. Can the authors explain the connection?


Minor: The loss set  in Lemma 1 is undefined.

summary_of_the_review: To reconsider my score, I would like to see the language around implications of the bound clarified throughout the text (in response to my comments under “the purpose of the bound”). Currently there is a huge disconnect between what has been shown and what the claims are.

Also, since the main contribution is the algorithm, I would like to see more evidence towards the algorithm potentially being adopted in practice. What are the computational costs? How do they compare? How does the algorithm compare to other recently proposed algorithms that were derived to improve generalization (like sharpness aware minimization)? How does the algorithm compare to other regularization techniques?
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a theoretically-grounded technique to improve neural network generalization. The loss is augmented with a term that encourages units to have a more diversity in its activations. More specifically, the authors use a radial basis function to measure pairwise similarity between activations and proposed loss terms based on either the sum or the determinant of the similarity matrix.

Based on experiments in several classification models on CIFAR10, CIFAR100, and ImageNet, models trained with this additional loss term achieve higher test accuracy, and also reduce the generalization gap between train and test. 
main_review: Strengths:
* Theoretically-motivated and simple (in a good way) technique to encourage activation diversity.
* Empirically evaluated a wide set of benchmarks and tasks.
* The significant generalization gap reduction in Table 2 (from 2.77 -> 1.07) is intruiging and worth additional study.

Weaknesses

The mathematical section was difficult to follow for this reviewer, because the theorems and lemmas are presented with minimal description or intuition of how they were derived and connected. The paper could be improved by only describing one of the cases (e.g. single hidden-layer network) in the main text, but in greater detail, and leaving the rest to the appendix. As an example in Theorem 3, its difficult to reason about the effect of d_min on the bounds, as J appears in two terms. The explanation would benefit from a figure showing how the bounds change with d_min.

To provide a more direct link between the theory, loss term, and empirical results, the paper could be improved by visualizing the activation diversity across layers -- given that the diversity term is only applied in one layer, what is its broader effect?

Empirically, the results are mixed. On CIFAR10/CIFAR100, the apporach improves over DeCov. For ImageNet, the approach does not yield better results than previous work (DeCov) on one model, but improves on a different model. With MLP-mixer, some methods improve over DeConv, some do not. It should be noted that the DeCov results are not cited from the original paper, but the authors own implementations (because DeCov was only evaluated on AlexNet at the time). 

Methodological questions:
* how is the author applying ResNet-50 on CIFAR10 and CIFAR100? The original paper (He et al, 2016) uses a slightly modified architecture in ResNet-56.
* given that DeCov results are not from the original paper, how were its hyperparameters tuned? e.g. in Table 7 in the Appendix, I see tuning of the hyperparameters for the author's method, but not for DeCov.
summary_of_the_review: The proposed method is appealingly simple, but yields mixed results. The paper's presentation and analysis does not present a strong link between the theory and the empirical section, which leaves me to recommend not accepting the paper in its current form.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors propose a regularization technique encourage the "activation diversity". Specifically, they design a within-layer loss that add penalty to the similar neurons with the same activation pattern. They also showed that encouraging the within-layer diversity can be used to control the Rademacher complexity of model. 
main_review: Definition:

- In the definition of the diversity, there is a $\tau$. What does that mean? What is the probability space? Does it mean when you sample a point from the input distribution, with probability $\tau$ you have the lower bound? But you should also quantify based on the weights. Am I missing something? For instance when you have a multilayer, the events are not independent so the statement of Theorem 4 is difficult to understand for me. 

- It would be nice if the authors can provide some sufficient conditions under which we can have the desired lower bound on the diversity. 

Motivation behind the diversity method:
- In the definition of the diversity we want that the output of activations are not close to each other. However, is it sufficient? what if the weight vectors of the next layer basically "kill" this diversity? In general, what is the impact of the weights on the diversity?

Comparison with Dropout method:
- Empirical comparison: My understanding of the dropout is that it also has a very similar impact as your method on learning. I appreciate it if you could compare the performance of your method with the dropout method.
-Computation Comparison: 
What is the difference of the dropout and your method in terms of computational cost?

Theorem 4: 

What is the intuition behind the Tau^p? what is the dependence of your bound on depth and width and how does it compare with Golowich et al paper?

A Note regarding the references:

In many places in the manuscript, the citations do not match with the content. 
For instance on Page 2, before definition 1, almost all of the citations are wrong. PAC learning is not from Hanneke'16. VC dimension is not from those papers you cited, and the same for Rademcher complexity.


summary_of_the_review: Very Interesting idea but the presentation can be improved.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper proposes to use a regularizer to encourage the diversity of activations within each layer of deep neural networks. The contribution of this paper is two-fold: theoretically, the authors proved that more diverse activations correspond to a lower generalization error bound due to a lower Rademacher complexity; practically, this work showed that the proposed diversity regularizer led to better performance on several image classification tasks.
main_review: Strengths:
1. This work has both a theoretical analysis of the proposed diversity regularizer and an empirical study of the effectiveness of the regularizer on existing models.

Weaknesses:
1. This work appears similar to existing diversity promoting regularizers, especially Xie et al 2015b (https://arxiv.org/pdf/1511.07110.pdf). While the diversity regularizer proposed here is applied to the neural activations instead of the weights, there seems to be no intuition/justification for why a diversity regularizer over activations is preferred over a diversity regularizer over weights, other than this sentence "... Decov, yielded superior empirical performance; however, it lacks theoretical proof. In this paper, we closed this gap ..." (page 7, first paragraph). I think this work would be stronger if it can either make a theoretical argument of why diversity regularization over activations is better, or make an empirical argument by comparing to Xie et al 2015b as well as other diversity-promoting works.

2. This work aims at regularizing neural networks, so more broadly it would be nice to compare to other regularization methods such as dropout and MC dropout.

3. Theoretically, I am not convinced that a better generalization bound always leads to a better performance, for several reasons: first, while the generalization error decreases with regularization, the approximation ability gets worse (i.e., the model cannot fit the training data as well), and this part is not analyzed in this paper. Second, the generalization bound seems to be very loose upon inspecting the constants: for example, $C_5$ in Eq. (8) is on the order of $C_1C_3$, with $C_1$ being the max norm of the input and $C_3$ being the max norm of the weight of the hidden layer. Lastly, a lower generalization error doesn't justify a regularizer. For example, if we flip the direction of the regularizer, i.e., regularize the activations to be less diverse instead of more diverse, the Rademacher complexity also decreases since the function family gets more constrained, but intuitively we don't want to use such a regularizer.

4. Empirically, this work led to "0.36%−0.54% improvement on CIFAR10 and1.86%−1.94% on CIFAR100" (page 7, under table 1), which does not seem very significant. Besides, this work proposed multiple versions of diversity regularizer and presented results for all of them, while a more reasonable thing to do is to select a version on the validation set and then present the test result of that one only. Besides, from Fig. 1, the accuracy seems to be sensitive to hyperparameters.

5. Computationally, the proposed regularizer seems expensive. For both the determinant and log determinant variants, the complexity of $O(C^3)$ with $C$ being the hidden size. I wonder if it's practical to apply this regularizer to larger networks with more hidden units.

Minor Issues:
1. "was empirically been proven to" (page 2, second paragraph)
2. Eq. (14), missing sum w.r.t. j?
3. Fig. 1, the legend appears small.

Suggestions:
1. It would be interesting to see the effectiveness of the proposed regularizer under different amounts of training data to better understand its behavior.
summary_of_the_review: Overall, this is well-structured work with both theoretical analyses and empirical results. However, this work did not provide a justification of why diversifying activations is better than diversifying weights, and it did not compare to commonly used regularization techniques such as dropout and other diversity-promoting works. Therefore, I am not recommending the acceptance of this paper.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an interesting angle on the generalization of neural networks. Through proving a new generalization bound that has an internal distance term, the authors argue the neural networks’ generalization performance is related to the internal diversity of the neurons within each layer. As such, some regularization terms are introduced to encourage the internal ``diversity” within each layer. The effectiveness is verified empirically through experiments on a series of data sets including CIFAR and ImageNet. 

main_review: My review on this paper is mixed. The main concern is on the theory side. 

The Assumptions 1 seem way too strong so that the conclusion of Theorem 1 is not interesting any more. In particular there may be some issues with the last line of assumption in assumptions 1. 


First that line of assumption is not stated clearly. 

a) There is a minor notation issue on the indexing i and j. They should be m and n instead?


b) Is this an argument for all w or for a fixed w or for a randomized w? This needs to be explicitly stated.

c) From the proof I turn to believe the authors are assuming that with probability \tau the inequality holds for all ws in the last line of assumption. I would say, for a fixed w, such assumptions may make some sense. Now to prove the theorem 1 we need that assumption to hold for all ws with probability \tau. And we have an infinite number of ws. That assumption is almost as strong as the original Rademacher bound on the model generalization. As a consequence the theorem 1 is not so informative to me.


On the other hand, the empirical side does show that diversifying the within-layer output may help improve models’ performance. This is an interesting empirical observation even though I do not quite buy the authors' claim on the connection to the Rademacher bound.

summary_of_the_review: Overall I feel this is a borderline paper with a mixed feedback from me. It has some interesting thoughts and empirical results with come concerns on the theoretical justifications.


#######Review after reading the authors' feedback########


I would like to thank the authors for their explanation and the revised draft. Still I feel the assumption in the theoretical justification is kind of strong even though similar assumptions have been used in some previous work. I would keep the ratings unchanged.
review_title: null

Meta-review: This work proposes an approach to encourage within-layer diversity in neuron activations, and derive a generalization bound meant to motivate their approach.

None of the reviewers support the acceptance of this work, despite the authors' detailed rebuttals, with the majority of reviewers confirming their preference for rejection following the author response. Many raised concerns regarding the value of the accompanying theory. The empirical results demonstrated by the proposed regularizer were also not judged to be sufficiently compelling to compensate for the shortfall on the theory side.

I unfortunately could not find a good reason to dissent from the reviewers majority opinion, and therefore also recommend rejection at this time.
Decision: Reject