On Plasticity, Invariance, and Mutually Frozen Weights in Sequential Task Learning

1 Introduction . Lifelong learning [ 1 , 2 ] holds both the promise of benefitting from past experiences and the challenge of having to continually adapt to new problem settings . The characteristics of this intriguing sequential learning problem make it a balancing act between retaining knowledge and adapting to new experiences referred to as the stability-plasticity dilemma [ 3 ] . Similar to a coach or teacher , we want to understand when a sequence of tasks helps or hinders further learning . Clearly , some pre-training tasks are highly beneficial [ 4–7 ] while others can hurt performance [ 8 , 9 ] . The existing literature on pre-training and curriculum learning sheds some light on the desired characteristics of sequential learning . In some cases , following a sequence of learning tasks can lead to better results than simply training on the target task from scratch [ 10 ] . Work by Achille and Soatto [ 8 ] contrasts this picture by highlighting that neural networks can suffer from pre-training on some tasks . In an extreme case , when learning data is corrupted for a sufficiently long period , even switching back to clean data does not allow the neural network to recover its original performance . Contribution : We formalize the basic ingredients of sequential task learning , introduce mutually frozen weights and establish a connection between plasticity and invariance of deep networks . To understand mutually frozen weights , we investigate their weight update dynamics , show that such frozen weights are different from regular sparse weights that are not connected to other sparse weights and that frozen weights can lead to a decrease in performance when retraining on a new task . Based on this analysis , we show that applying an intervention which “ resets ” frozen weights can improve retraining performance on a number of tasks as long as sufficiently many samples are available for retraining and even achieve state-of-the-art results on FashionMNIST image classification . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . 2 Related work . We call the problem setting in this work optimal task tracking which involves training a model on a sequence of tasks to improve performance on a final target task . This strongly resembles the objectives of lifelong learning [ 1 ] , continual learning [ 11 ] and meta-learning [ 12–14 ] and connects to many time-dependent learning aspects in relation to neural networks . Time-dependent learning : We are aware of several time-dependent phenomena in training neural networks such as catastrophic forgetting [ 15 ] , critical learning periods [ 8 ] , and time-sensitivity of regularization and data augmentation [ 16 ] . Achille et al . [ 8 ] show that by dropping out certain frequencies during the beginning of training , networks are unable to recover to full performance , even when those frequencies are reintroduced . Similarly , Liu et al . [ 17 ] show that first optimizing on random labels ruins performance on clean data shown indefinitely to the same network afterwards . Recent information-theoretic analysis [ 18 ] suggests that neural network have distinct learning phases : First a phase where network parameters grow in information content , before , in a second phase , self-regularizing and pruning away irrelevant information . More commonly known time-dependent or sequential heuristics are learning rate schedules [ 19 ] which may even be cyclical [ 20 ] , regularization annealing such as dropout annealing [ 21 ] , student-teacher model transfer [ 22 ] , and pre-training [ 23 ] . Finally , our introduced concept of mutually frozen weights also touches upon the lottery ticket hypothesis [ 24 ] and may help explain some of its underlying gradient dynamics . Curriculum learning and generation : Curriculum learning [ 25 , 10 ] is the technique of training agents by showing them a progression of tasks from easier variants to harder examples to attempt to solve the final target task . Both the utility of transferring between similar or dissimilar tasks is debated , with some results pointing towards similarity [ 26–28 ] and others towards dissimilarity [ 29 , 30 ] . Metalearning [ 13 , 31 ] is similarly related by focusing on speeding up or enhancing performance on a target task by pre-training on other tasks to first learn to learn . Unlike curriculum learning , curriculum generation seeks to find a path through task space , generating an agent that can solve some final , held-out task [ 32–36 ] . Invariance-plasticity-generalization : Empirically , Mehta et al . [ 37 ] showed that ReLU networks with batch normalization trained with gradient descent lead to filter-level sparsity . Evci et al . [ 38 ] demonstrate that training sparse networks usually leads to worse outcomes than without sparsity . On the other hand , Bartoldson et al . [ 39 ] showed that increased sparsity through pruning can lead to better generalization . Bringing sparsity emergence and difficulty of training together , we will demonstrate how certain types of sparsity can lead to problems in retraining a model after pre-training in the context of mutually frozen weights . From a theoretical viewpoint , Neyshabur et al . [ 40 , 41 ] connect the path norm of a network to its generalization via Rademacher bounds , where the path norm is based on the sum across all paths of the product of weights associated to individual paths . 3 Optimal task tracking . We are interested in better understanding what influences the outcome of an optimal task tracking problem , i.e . given a fixed task sequence how do we train a model to optimize the performance on the final target task . To clarify our further writing , we summarize essential elements involved in sequential learning and visualize the overall setting in Fig . 1 . A task ⌧ consists of a probability distribution p ( x , y ) of data points and desired outputs as well as a loss function L : ( p , f ) 7 ! R+ that maps an input-output behavior of a model to a cost score . The learning mapping ! : ( f , ⌧ ) 7 ! fnew , e.g . stochastic gradient descent , uses the information in task ⌧ and initial model finit to train a model flearn optimized for the task until convergence . The tasks and learning mappings ( ⌧i , ! i ) 2 T are applied sequentially starting from an initial model finit to produce the final target model ftarget to be evaluated on the target task ⌧target . 4 Plasticity and frozen weights . With the basic building blocks in place , we ask in which cases , even with an infinite number of data points , previous tasks can hinder learning on new tasks . We are interested in understanding what causes a model to be more difficult to adapt to a new task than another . This is evidently the case when some of the model weights ✓ can not be updated when training on the new task even when this would yield a better model . Based on this reasoning , we introduce the concept of frozen weights , i.e . weights that are unchanged when training on a learning task s.t . plasticity is clearly lacking . Definition 1 ( Frozen weights ) . We define a frozen weight ✓frozen of model f as a parameter that is not changed through learning ! on task ⌧ . ✓frozen ( f ) = ✓frozen ( ! ( ⌧ , f ) ) Associated to frozen weights , plasticity captures how well a model can adapt to a new task . Definition 2 ( Plasticity ) . We define plasticity as the rate of change of the model parameters subject to gradient descent updates P : = r✓L ( f ( ✓ , x ) , y ) . The definitions of frozen weights and plasticity beg the question in which scenarios frozen weights can appear . A promising candidate for such a constellation are mutually frozen weights . Definition 3 ( Mutually frozen weights ) . For a model f ( ✓ ; x ) with parameters ✓ and input x , we consider n non-overlapping sets of parameters { ✓i } as mutually frozen if , for each set , all parameters are zero and if each parameter in each set is exclusively multiplied with all other sets of parameters . We call such parameters n-sparse frozen weights . As the leading example , weights ✓i , ✓j , ✓k 2 R are triple-sparse mutually frozen if , first , the weights ✓i , ✓j , ✓k only appear multiplied together within the model as ✓ijk : = ✓i · ✓j · ✓k s.t . f ( ✓ ; x ) = f ( ✓ , ✓ijk ; x ) , where ✓ refers to all parameters except ✓i , ✓j , ✓k and , secondly if ✓i = ✓j = ✓k = 0 . Given a loss L ( ✓ ; x ) ! R+ we see that the gradient of ✓i is proportional to the remaining weights ✓j · ✓k = 0 : @ L ( ✓ ; x ) @ ✓i = @ L ( ✓ ; x ) @ f ( ✓ ; x ) @ f ( ✓ ; x ) @ ✓ijk @ ✓ijk @ ✓i = @ L ( ✓ ; x ) @ f ( ✓ ; x ) @ f ( ✓ ; x ) @ ✓ijk ✓j✓k Therefore with condition ✓i = ✓j = ✓k = 0 and assuming that the partial gradients L ( ✓ ; x ) @ f ( ✓ ; x ) and @ f ( ✓ ; x ) @ ✓ijk are bounded , the gradient of the mutually frozen weights ✓i , ✓j , ✓k weights are zero . We note that mutually frozen weights describe n-tuples ( n > 1 ) that “ freeze together ” . As depicted in Fig . 2 , we will focus on double-sparse and triple-sparse frozen weights in the remainder of the text . Of course also higher-order freezing involving n-tuples are possible yet do not necessarily contribute further insights . Likewise we deem this less likely to occur in neural network architectures since the influence of individual parameters spreads much further when involving more than three layers . Example 1 ( Linear regression ) . Linear regression does not have “ mutually frozen weights ” . Given f ( ✓ ; x ) = ✓Tx + b , y , b 2 R , x , ✓ 2 Rn , we have @ f ( ✓ ; x ) @ ✓ = x . Hence , unsurprisingly , no weights are multiplied with other weights in a way that could create mutually frozen weights . Thus we would not expect linear regression to incur a loss in plasticity in a sequential task learning setup . Example 2 ( Neural networks ) . A simple example of triple-sparse mutually frozen weights appears for the model f ( ✓i , ✓j , ✓k ; xi ) = ✓i✓j✓kx with x , ✓i , ✓j , ✓k 2 R. In this case , one can readily see that if ✓i = ✓j = ✓k = 0 then the gradients for all weights will also be zero . This can be regarded as a sub-part of a larger piece-wise linear neural network which will not be further trained due to mutually frozen weights . Note that with 3 or more weights multiplied as shown , the zero gradient is locally stable ( see Sec . 4.1 ) . The mutually frozen weights condition is particularly relevant for deep neural networks which consist of many pairwise multiplications of weights . In practice paired weights need not be zero exactly but simply have very small absolute values compared to the remaining weights . Other situations of training problems due to insufficient gradients exist such as the vanishing gradient problem [ 42 ] , wherein the magnitude of the gradient is reduced as it progresses through the network due to small weight norms . This problem however appears to be largely solved by methods such as identity connections such as in residual networks [ 43 ] and normalizations such as batch normalization [ 44 ] . From afar it might appear that the answer to the frozen weights problem is simply to avoid them , i.e . to ensure through training a model to not end up with frozen weights . However as we will document in the following , weight decay , which can cause frozen weights , is often connected to better generalization and thus makes frozen weights non-trivial to avoid . This naturally causes a curious trade-off between how we first train a model on a pre-training and subsequently on a target task .

Summary: In this paper, the authors present a mechanistic phenomenon linked to plasticity problems incurred in sequential task learning. They call this phenomenon "mutually frozen weights", describing a situation where triples (or higher order tuples) of weights that multiply each other in a system, all end up with zero value, which prevents error gradients from influencing their value when learning new tasks. They then argue that such a situation arises in real world setting, and explain how weight decay can promote such a situation. They also demonstrate that such frozen weights play a role in input invariance, and thus should be considered in a tradeoff perspective. Finally, the present a "resetting" method that helps mitigate the problems brought by mutually frozen weights, and perform some numerical experiments to demonstrate the efficacy of their method.