Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper studies the "class sensitivity" of a ResNet-50 backbone during its training on Imagenet. "Class sensitivity" (defined in previous work) characterizes whether a given neuron fires disproportionally for a particular class. They also study the effect of ablating (pruning) class sensitive neurons on final accuracy an observe they are more prunable than a randomly selected neuron. Main results show that (1) Class sensitivity increases during the first few epochs of training and reduces (2) During this initial phase pruning highly class sensitive neurons incur higher damage compare to later epochs (still less than random). (3) higher similarity between certain layers during the early phase. 
strength_and_weaknesses: # Strengths
- It's nice to see the specialization of neurons in earlier layers being investigated. Though initial results doesn't lead to better algorithms, it could help us understand training and eventually help us improve optimization of NNs.

# Weakness
- It would help to check whether findings transfer to a different architecture / dataset. And obviously one doesn't need to repeat all experiments. A single plot combining different metrics could suffice (see suggestions). Maybe a ViT?

[During rebuttal] Authors said no-time (which is understandable)

- I couldn't see a motivation for the importance of studying neuron selectivity. Do we want/expect neuron selectivity to be 0? Why? I think this is a great exploratory study but I don't it is well motivated why this is important for the community. 

[During rebuttal], authors pointed out previous work who uses this metric in different settings (most of which already cited in paper). I don't think this is enough for motivating a research work. Why do they use it? Why not another metric (we can come up with many metric like this)? Why we should look at this phenomenon and not others? Addressing these questions would improve the work (instead of saying this metric is being used in previous work) 

- I'm not sure adding an additive regularizer for reducing neuron selectivity makes the results/relation causal. This could be solely about optimization. One way to get a very low neuron selectivity is to push all weights to zero and thus all zero activations and this could prevent/effect later learning. However, it doesn't mean that reducing neuron sensitivity always lead to worse generalization. You also probably want to have different coefficients for different layers as we want/expect class sensitivity in later layers.

[During rebuttal] Thanks for sharing the difference between previous work and regularization used. It might worth mentioning this in the paper (if it is not there already) I still believe using `causal role` and `causal experiment` can be miss-interpreted.
 
# Suggestions
- I think it would help to story to have a plot where first 10 epoch of the training is highlighted with more frequent data points. It would be also nice to look at the Class sensitivity and CKA score AOC in the same plot to see the correlation more clearly.
- I think it would be nice to track the linear-probe performance of different layers over the course of the training. It would be nice to see whether that correlates with the Class-sensitivity metric. I think previous work on early-exit of NNs had some experiments on this, but only looked at things at the end of the training. 

# Minor
- [First sentence of the paper] uses `neuron selectivity`. It would be nice to explain what neuron selectivity is or use a high level description ("understanding the role of neurons"). 
- [Section 3.2] It would be nice to define the index using a more precise notation. For example: would the stem convolutional layer with kernel shape (3x7x7x64) have 64 neurons in your notation? Are the activations assumed to be non-negative (after relu)?
- Figure 2 appears after 3 and 4.

# After Rebuttal
- I thank the authors for their response. I keep my score as it is, as my concerns for "why not higher?" stays the same. I think this is an interesting work and I support acceptance. Regardless I encourage authors to improve their work in camera ready or for next submission using the feedback given. 

clarity,_quality,_novelty_and_reproducibility: Quality, clarity and originality of this work is I think appropriate (good). 
summary_of_the_review: Overall this work is a potentially a useful study for explaining the evolution of neural networks during the early phase of training. The observation that early layers learn to class-sensitive neurons together with the high similarity between layers suggest a kind of shortcut learning which to my knowledge a novel observation and would be beneficial to the community. My concerns lie around the (a) experimental evaluation (limited to a single setting) (b) motivation (c) 4th claim/contribution. If addressed, would be willing to change my evaluation.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors conducted a study on the importance of class-selective neurons in early and intermediate neurons during the training process. For their study, the authors trained a ResNet-50 on ImageNet, and recorded the class selectivity index of neurons in modules 4 through 7 of the ResNet-50 during training. They found that class selectivity of early and intermediate neurons generally increases during the first few epochs of training, and rapidly decreases after the first few epochs. The authors also regularized for/against class selectivity of early and intermediate neurons. They found that regularizing against class selectivity during early epochs of training significantly hampers network training.
strength_and_weaknesses: Strengths:

- Writing: The paper is clearly written and easy to follow.
- Insight: The paper does provide some new insight regarding the usefulness of class selectivity in early epochs of training.

Weaknesses:

- Novelty: There is little technical novelty in this paper.
- Significance: While the paper does provide some insight regarding class selectivity, I fail to how this insight may benefit us when we train neural networks.
clarity,_quality,_novelty_and_reproducibility: This work is strong in terms of clarity, quality, and reproducibility. However, there is little novelty in terms of technical innovations.
summary_of_the_review: Based on the strengths and weaknesses discussed above, I believe that this paper has some value, but is lacking in terms of technical novelty and significance.

************************************
After reading the authors' response, I decided to increase my rating to 6 ("marginally above the acceptance threshold"), because the authors did explain how their study could be potentially used to improve neural network training. I do encourage authors to incorporate their response into the main paper.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper studies in a systematic way the effects of "class selectivity" in early epochs of training.  
1) The main backbone architecture is ResNet-50 on ImageNet and their early / intermediate / latter modules. Units (or neurons) are defined as individual channels of bottleneck layers in each module. 
2) For each unit, the author used "class selectivity index" defined in previous literature to study how "special" the unit reacts to each ImageNet class.
3) The first part of the main paper is an observation that class selectivity appears strongly in all modules in early epochs. After that, the early and intermediate layers saw decrease in the selectivity while latter layers there does not seem to be a decrease. The author also gives a tentative explanation that the network is approximately linear so all modules behave similarly. 
4) The second part of the main paper discusses the importance of class selectivity, by applying regularization to either suppress or encourage selectivity, from the first epoch or from later epoch. The author concludes that the selectivity in early epochs is very important for quality of training.
strength_and_weaknesses: The strength of the paper lies mainly in the thoroughness of the analysis. The author separates the research into two aspects: empirical observations (and the tentative explanation), and their role in model training. 

There are several weakness of the paper:
1) The limitation of the model backbone. They only consider ResNet-50s, but not other ResNet (such as ResNet-152), or other architecture (ViT). It will be much more interesting to see how different model contributes to the effect of class selectivity.
2) For image classification, the paper restricts to ImageNet only. ImageNet is a relatively small sized data and there is a good notion of "epochs", but in real practice once can encounter data that takes a long time to train to one epoch. How does class selectivity behave for those large dataset within one epoch?
3) How about other tasks beyond vision? Such as NLP, speech, etc?
4) Currently, the paper is mostly about empirical observations; have the author thought about how to utilize the observation on class selectivity to improve the model training? The improvement can be in terms of accuracy, robustness, etc.
clarity,_quality,_novelty_and_reproducibility: The paper is of very good clarity and quality. In particular, the paper clearly states the objective (ResNet-50s, class selectivity in the temporal dimension) to study, and the model component (bottleneck layers) for analysis. During analysis, the authors proposed concrete method and metric to quantify the class selectivity change and their explanations.

The analysis is also original, which not only considered selectivity across layers, but also considered change across training epochs. Using regularization starting at various epochs is also new.
summary_of_the_review: I recommend the paper with marginally acceptance for two reasons: 1) The analysis on temporal change is new and comprehensive. 2) The study itself is, however, too narrow. ResNet-50 is no longer the most up-to-date architecture, and ImageNet is a well-curated dataset where class distribution is relatively balanced. So there is still questions regarding how general this phenomenon could be. On the other hand, the authors didn't proceed with how to use the observation to motivate a better training procedure.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this paper, the authors research the effects of class-selective neurons in the early learning phase. And the authors design different experiments for the claims, class-selective neurons are important for networks during the early training phase, the early and later layers are similar in the early training phase, and class-selective neurons in the early and intermediate layers are essential to the successful training of the network in the first few epochs.
strength_and_weaknesses: Strength: The research topic is important. What happens in the early learning phase is still known little. 

Weakness:

1. As the authors discussed, the main limitation of this paper only provides experiments on ResNet-50 and ImageNet. More experiments with more kinds of CNNs and datasets are necessary. Otherwise, many questions may occur, e.g., if using a larger CNN with more redundant parameters, does the phenomenon exist? 
2. The authors only use one kind of learning rate to conduct experiments. Different optimizations should be adopted to check whether the phenomenon is from the optimization.  
3. More explanations about experiments and metrics for Figure 3 and 4 are needed.
4. In Figure 6 (b) and 7, the authors only provide results of 20 epochs, without standard deviation. With more punishment from regularization, the learning speed should be slower than others at the beginning. After 20 epochs, will the experiment with regularized from epoch 0 catch up or be close to others? 

clarity,_quality,_novelty_and_reproducibility: The clarity should be improved. More experimental details should be introduced

The quality, novelty, and reproducibility are good for this paper.

summary_of_the_review: Although the research topic is interesting, experiments are not sufficient for supporting claims. I do not recommend this work for the current version.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper investigates the importance of class selectivity during training.


Strength:
- Writing: The paper is clearly written and easy to follow.
- mainly in the thoroughness of the analysis for case of ResNet50 and imageNet dataset


Weakness:
- Limited novelty
- Limitation of model backbone and dataset, it needs to be checked whether findings transfer to a different architecture / dataset. 
- The considered problem is not well motivated and reduces the significance of the work.

I encourage the authors to consider the received feedback for improving the paper and its future iteration.
justification_for_why_not_higher_score: Please see meta review
justification_for_why_not_lower_score: Reject
Decision: Reject