Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: In this paper, the authors present a mechanistic phenomenon linked to plasticity problems incurred in sequential task learning. They call this phenomenon "mutually frozen weights", describing a situation where triples (or higher order tuples) of weights that multiply each other in a system, all end up with zero value, which prevents error gradients from influencing their value when learning new tasks. They then argue that such a situation arises in real world setting, and explain how weight decay can promote such a situation. They also demonstrate that such frozen weights play a role in input invariance, and thus should be considered in a tradeoff perspective. Finally, the present a "resetting" method that helps mitigate the problems brought by mutually frozen weights, and perform some numerical experiments to demonstrate the efficacy of their method.
main_review: Originality: To my knowledge, the concept surrounding mutually frozen weights, and their link to learning trajectory deficiencies in sequential task learning, is novel. The concept is sound and the identification of its impact is new.

Quality & Clarity: The paper is thorough and takes the necessary steps to formally define mutually frozen weights, and test for their impact on training. Experiments are well-documented and follow quality guidelines. However, a number of questions remain unanswered with respect to how extended mutually frozen weights might arise.

1- It is not clear to me why the concept of mutually frozen weights needs 3-tuples of weights to be zero, and not 2-tuples. In Definition 3, couldn't the multple \theta_j*\theta_k be replace by a single zero weight ? Maybe I am missing something here, but it seems a 2-tuple is enough to get the phenomenon. In any case, the reason behind why 3-tuples are needed is not explicitly explained, which would greatly improve clarity.

2- In Definition 3, it is stated that mutually frozen weights are 3-tuples that "only appear multiplied together" in the network. In practice, this seems like a very special situation in deep networks, such as in filters, etc. (as treated in experiments). Indeed, typical layer weights will multiply several other weights and thus, gradients will be able to reach them in principle. Is the phenomenon of mutually frozen weight restricted to this situation? (maybe I am missing something here) Also, is the mechanism of hindered sequential task learning observed in the literature restricted to architectures that allow mutually frozen weights? If not, a discussion about the limitation of  mutually frozen weights should be present.

3- Table 1 describes an experiment with frozen weights AND perturbation meant to test invariance (horizontal flip and random shift of magnitude). Why combine both these effects ? What happens to training with different sparsity scenarios when no perturbations are introduced ?

4- Ablation experiments that measure gradients to mutually frozen weights before and after resetting would strengthen the conclusion that the phenomenon, and the solution, are mechanistically related to improvements.

Significance:
If clarity issues described above are resolved, I believe this contribution is significant in understanding mechanisms that influence training trajectories for sequential task learning.
limitations_and_societal_impact: I did not notice any discussion of potential societal impact.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: The paper studies the "optimal task tracking" problem under the training setting where we use the "weight decay" regularization.
More specifically, the paper studies the relationship between plasticity-stability tradeoff and shows that a consequence of training with (large enough) weight decay is the "mutually frozen" weights that can damage the plasticity of the network. Moreover, the paper introduces an intervention to reset the mutually frozen weights and improve the performance.
main_review: ### Overall
**Originality: Are the tasks or methods new?** Yes    
**Quality: Is the submission technically sound?** Partially  
**Clarity: Is the submission clearly written?** Yes   
**Significance: Are the results important?** Partially

------
### Strengths
**(1)** In general, I appreciate the works that aim to understand the behavior of neural networks rather than jumping into problem-solving and beating the benchmarks. This paper belongs to this category, and I appreciate the novelty and the approach of the paper.

**(2)** The overall theoretical reasoning seems sound to me.  

**(3)** The authors provide the code and details for reproducibility.

-----

### Weaknesses

**(1)** While the introduced concept (mutually frozen weights) seems very interesting, I believe the study could benefit if other contributing factors to this concept were studied. For instance, the authors study the impact of weight decay and (to some degree) learning rate (Sec. 5.3) but since the aim is to study the training dynamics. But, several other common factors may or may not contribute to the impact of mutually frozen weights. While I understand it is infeasible to study every contributing factor, the paper could at very least study the prominent factors in training dynamics such as BatchNorm, Dropout, and Optimizer that is shown to have an impact on training dynamics and stability-plasticity in continual learning [1]. More specifically, [2] shows that L2 regularization with Batch Normalization leads to an Exponential Learning Rate Schedule that depends on the momentum and weight decay rate. According to the code that the authors provided, most of the models use Batch Normalization layers. Hence, I believe an additional study on these factors is needed to improve the paper's insights.

**(2)** In the continual learning literature, weight decay is known to generally hurt the performance regarding the stability and catastrophic forgetting [1,3], which contradicts the paper's claim. While the benchmarks in those studies are different, the problem setting seems similar (i.e., distribution shift). Could the authors discuss the relationship of those findings to the findings of their paper? 

**(3)** Regarding Table 4, I have three comments/questions:
   *(3.1)* If I understood correctly, the number of frozen weights is very small compared to the total number of parameters in the network.  How this small number of frozen weights can make such a gap (e.g., ~6.6% on CIFAR-10 -> CIFAR-100)? According to the appendix, the reported numbers are calculated for only one specific learning rate. Could it be because of this?
   *(3.2)* It is difficult to find the real impact of the intervention on performance, given that only one hyper-parameter is used. Could the authors provide the results for at least a few of the rows (e.g., one from each category) where a grid (even small) is used to find the best parameters? The rationale is some reported numbers are lower than the typical performances reported in the other studies. For instance, training a ResNet18 on CIFAR-100 from scratch should give above 70% accuracy. While one can argue that they might use different techniques for improving performance, I suppose given this sub-optimal training setting, it is clear how much real gain the proposed intervention provides.

**(4)** [Misc]: In addition to some grammatical errors (e.g., L38), I believe some figures (2, 3, 4)  are misplaced and very far from where they have been referenced. Also, the y-axis titles of Fig. 4 are not entirely readable. It could improve the readability of the paper if the authors could improve the figures. However, I should note that this did not contribute to my decision and is only a suggestion.

[References]   
*[1] Mirzadeh, Seyed Iman, et al. "Understanding the Role of Training Regimes in Continual Learning." Advances in Neural Information Processing Systems, 2020.*   
*[2] Li, Zhiyuan, and Sanjeev Arora. "An Exponential Learning Rate Schedule for Deep Learning." Eighth International Conference on Learning Representations, 2020.*   
*[3] Lange, Matthias De, et al. Continual Learning: A Comparative Study on How to Defy Forgetting in Classification Tasks. 2019.*


----- Update (After Rebuttal) ----   
I believe the authors have addressed most of my questions and comments. Hence, I increase my initial score. 
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary: The authors of 'On Plasticity, Invariance, and Mutually Frozen Weights in Sequential Task Learning' introduce 'mutually frozen weights', a new metric for evaluating how well neural networks trained on one task can learn a novel task. They provide preliminary theoretical analysis relating their new metric to weight decay and show how it can be manipulated to facilitate sequential task learning.
main_review: The authors define the phenomena of 'mutually frozen weights' as a triplet of parameters that appear on the neural function only as their product and when they are all equal to zero, they will stay zero for the rest of training. They authors also claim that with the MSE loss and weight decay, all of the parameters being zero is a stable attractor of the training dynamics. 

It is very unclear to me how often does it happen when three (or more) parameters only appear as their product. This is a critical point of their work as the 'mutually frozen weights' only happens in that scenario. Interestingly enough, fully connected neural network (i.e. Weights multiplication and bias addition followed by non linearity) do not have this feature, and neither do standard Convolutional neural network composed (they don't even have two parameters only appearing as their product), and even with Batch Norm, I believe we are still left with only two parameters sequentially multiplied. I may be missing something, but still, the authors should provide a clear example when this feature occurs in a common neural architecture rather. 

The theoretical analysis in parts 4.1 seems sound aside from the reliance on the MSE loss and weight decay. While I do understand why  all thetai/j/k = 0 is a fixed point in the non weight-decay case, it is not clear why it would occur if it is not an attractor (which it isn't since the eigenvalues of the fixed point would be zero if lambda = 0). An extension to more common loss functions (like, softmax cross entropy) and an explanation of what happens without weight decay (as weight decay is not a very popular regularization measure anymore) would enhance the applicability of this analysis.

The relationship to sequential task learning is also not so clear. In their analysis in 4.2, the authors show that weight gradient and gradients to temporal change in input have common factors and conclude that they are thus interlinked. Due to the familiarity of the community with vanishing gradients (and why we all now use relu instead of tanh and sigmoid) I believe this could simply be stated in a couple of sentences. Devoting an entire section and introducing it as a lemma seems odd, especially considering that the authors definitely lacked space judging from their 2nd and 3rd figures which are hard to read and should be at least twice their current size.

Due to lack of obvious applicability of this work, I recommend it be rejected from Neurips 2021

limitations_and_societal_impact: The authors provide a theoretical analysis of neural networks and I do not see any negative societal impact from their work.
review_title: null

Meta-review: This paper studies the learning dynamics in deep networks by making a novel observation regarding weight decay as mutually frozen weights and their role in generalization. The paper initially received reviews that tended towards rejection. The reviewers had difficulty understanding some details and were concerned whether the results will hold true in real-world settings. The authors provided a thoughtful rebuttal that addressed the reviewers' concerns. The paper was discussed and all the reviewers updated their reviews in the post-rebuttal phase. All reviewers switched their score to weak acceptance (note one reviewer still has a score of 4 in the main review, but they have switched to 6 in comments). AC agrees with the reviewers and suggests acceptance. However, the authors are requested to look at reviewers' feedback and incorporate their comments in the camera-ready.
Decision: Accept (Poster)