Model Zoo: A Growing Brain That Learns Continually

1 Introduction . A continual learner seeks to leverage data from past tasks to learn new tasks shown to it in the future , and in turn , leverage data from these new tasks to improve its accuracy on past tasks . It stands to reason that the performance of such a learner would depend upon the relatedness of these tasks . If the two sets of tasks are dissimilar , learning on past tasks is unlikely to benefit future tasks—it may even be detrimental . And similarly , new tasks may cause the learner to “ forget ” and result in deterioration of accuracy on past tasks . Our goal in this paper is to model the relatedness between tasks and develop new methods for continual learning that result in good forward-backward transfer by accounting for similarities and dissimilarities between tasks . Our contributions are as follows . 1 . Theoretical analysis.We characterize when multiple tasks can be learned using a single model and , likewise , when doing so is detrimental to the accuracy of a particular task . The key technical idea here is to define a notion of relatedness between tasks . We first show how if the inputs of different tasks are “ simple ” transformations of each other ( and likewise for the outputs ) , then one can learn a shared feature generator that generalizes better on every task , compared to training that task in isolation . Such tasks are strongly related to each other and therefore it is beneficial to fit a single model on all of them . We show that if tasks are not so strongly related , in particular if the optimal model for one task predicts poorly on another task , then fitting a single model on such tasks may be worse than training each task in isolation . Such tasks compete with each other for the fixed capacity in the single model . We also empirically study this competition using the CIFAR-100 dataset . 2 . Algorithm development . The above analysis suggests that a continual learner could benefit from splitting its learning capacity across sets of synergistic tasks . We develop such a continual learner called Model Zoo . At each episode , a small multi-task model that is fitted to the current task and some of the past tasks is added to Model Zoo . This method is loosely inspired from AdaBoost in that it selects tasks that performed poorly in the past rounds and could therefore benefit the most from being trained with the current task . At inference time , given the task , we average predictions from all models in the ensemble that were trained on that task . 3 . Empirical results.We comprehensively evaluate Model Zoo on existing task-incremental continual learning benchmark problems and show comparisons with existing methods . There is a wide variety in the problem settings used by existing methods , e.g. , some replay data from past tasks ( like Model Zoo is designed to do ) , some replay only a subset of data , some train only for one epoch in each episode , some use extremely small architectures , etc . We compare Model Zoo with existing methods in a number of these settings . Model Zoo obtains better accuracy than existing methods on the evaluated benchmarks . Improvement in average per-task accuracy is quite large in some cases , e.g. , 30 % for Split-miniImagenet . We also show that Model Zoo demonstrates strong forward and backward transfer . 4 . A critical look at continual learning . We find that even an Isolated learner , i.e. , one which trains a ( small ) model on tasks from each episode and does not perform any continual learning , significantly outperforms most existing continual learning methods on the evaluated benchmark problems , e.g. , by Right : Does the single-epoch setting show forward-backward transfer ? The evolution of individual task accuracy of Model Zoo ( the multi-epoch setting in bold and single-epoch setting in dotted ) , on the SplitminiImagenet dataset ( only 5 tasks are plotted here , see Fig . A6 for the full version ) . The X markers denote the accuracy of Isolated . Accuracy of tasks improves with each episode which indicates backward transfer . Also , the X markers are often below the initial accuracy of the task during continual learning , which indicates forward transfer . While both single-epoch and multi-epoch Model Zoo show good forward-backward transfer , the accuracy of tasks for the former is about 25 % worse than the latter ; corresponding plots for other methods are in Appendix B.7 . This indicates that we should also pay attention to under-training and per-task accuracy in continual learning . more than 8 % in Fig . 1 and Tables 1 and A1 . This strong performance is surprising because it is a very simple learner that has better training/inference time , no data replay , and a comparable number of weights as that of existing methods . This is surprising and points to a large intellectual gap in the current literature : while a number of existing methods seek to mitigate catastrophic forgetting , they often do so at the cost of forward or backward transfer . We advocate taking a step back and rethinking whether stylistic formulations are holding us back from building good continual learning methods . We advocate that per-task accuracy and forward-backward transfer should be the focus of future research . 2 A theoretical analysis of how to learn from multiple tasks . In this section , we ( i ) formulate the problem of learning from multiple tasks , ( ii ) discuss a simple model that highlights when training one model on multiple tasks is beneficial , and ( iii ) show new results on how the fixed capacity of the model causes competition between tasks . 2.1 Problem Formulation . A supervised learning task is defined as a joint probability distribution P ( x , y ) of inputs x ∈ X and labels y ∈ Y . The learner has access to m i.i.d samples S = { xi , yi } i=1 , ... , m from the task . A hypothesis is a function h : X → Y with h ∈ H being the hypothesis space . The learner may select a hypothesis that minimizes the empirical risk êS ( h ) = 1m ∑m i=1 1 { h ( xi ) 6=yi } with the hope of achieving a small population risk eP ( h ) = P ( h ( x ) 6= y ) . Classical PAC-learning results ( Vapnik , 1998 ) suggest that with probability at least 1− δ over draws of the data S , uniformly for any h ∈ H , we have eP ( h ) ≤ êS ( h ) + if m = O ( ( D − log δ ) / 2 ) ( 1 ) where D = VC ( H ) is the VC-dimension of the hypothesis space H . We define the “ excess risk ” of a hypothesis as EP ( h ) = eP ( h ) − infh∈H eP ( h ) . In the continual learning setting , a new task is shown to the learner at each episode ( or round ) . Hence after n episodes , the learner is presented with n tasks P̄ : = ( P1 , . . . , Pn ) , with the corresponding training sets S̄ : = ( S1 , . . . , Sn ) , each with m samples , and the learner selects n hypotheses h̄ = ( h1 , . . . , hn ) ∈ Hn , each hi ∈ H . If it seeks a small average population risk eP̄ ( h̄ ) = 1n ∑n i=1 ePi ( hi ) , it may do so by minimizing the average empirical risk êS̄ ( h̄ ) = 1n ∑n i=1 êSi ( hi ) . As Baxter ( 2000 ) shows , under very general conditions , if m = O ( −2 ( dH ( n ) − 1/n log δ ) ) , ( 2 ) then we have eP̄ ( h̄ ) ≤ eS̄ ( h̄ ) + for any h̄ ∈ Hn . The quantity dH ( n ) here is a generalized VC-dimension for the family of hypothesis spaces Hn , which depends on the joint distribution of tasks . Larger the number of tasks n , smaller the dH ( n ) ( Ben-David and Borbely , 2008 ) . Whether ( 2 ) is an improvement upon training the task in isolation as in ( 1 ) depends upon the hypothesis class H and the relatedness of tasks P1 , . . . , Pn through the quantity dH ( n ) . The most important thing to note here is that according to these calculations , if one wishes to obtain a small average population risk across tasks , training multiple tasks together can not be worse : dH ( n ) ≤ VC ( H ) . 2.2 Controlling the excess risk of a specific task for synergistic tasks . An important goal of continual learning is to have low risk on all tasks . This is a stronger requirement than for ( 2 ) which bounds the average population risk on all tasks . Suppose there exists a family F of functions fi : X → X that map the inputs of one task to those of another , i.e. , any task can be written as Pj ( A ) = f [ Pi ] ( A ) = Pi ( { ( f ( x ) , y ) : ( x , y ) ∈ A } ) for some function f ∈ F for any set A . We can assume without loss of generality that F acts as a group over the hypothesis space and H is closed under its action . In simple words , this entails that given h ∈ H suitable for task P , we can obtain a new hypothesis h ◦ f that is suitable for another task f [ P ] . Instead of searching over the entire space Hn like in §2.1 , we now only need to find a hypothesis h ∈ H such that its orbit [ h ] F = { h′ : ∃f ∈ F with h′ = h ◦ f } contains hypotheses that have low empirical risk on each of the n tasks . Conceptually , this step learns the inductive bias ( Baxter , 2000 ; Thrun and Pratt , 2012 ) . The sample complexity of doing so is exactly ( 2 ) . From within this orbit , we can select a hypothesis that has low empirical risk for a chosen task P1 . The sample complexity of this second step is |S1| = O ( −2 ( dmax − log δ ) ) ( 3 ) where dmax = suph∈H VC ( [ h ] F ) . By uniform convergence , as Ben-David and Schuller ( 2003 ) show , this two-step procedure assures low excess risk for every task P1 , . . . , Pn . We have suph∈H VC ( [ h ] F ) = dmax ≤ dH ( n+ 1 ) ≤ dH ( n ) ≤ D = VC ( H ) . ( 4 ) The total sample complexity is favorable to that of learning the task in isolation if both dH ( n ) and dmax are small . For instance , if F is finite and n/ log n ≥ D , we have dH ( n ) ≤ 2 log |F | which indicates that we get a statistical benefit of learning with multiple tasks if D log |F | . Remark 1 ( Data from other tasks may not improve accuracy even if they are synergistic ) . Let us make a few observations using the above analysis . ( i ) From ( 4 ) , number of samples per taskm decreases with n ; this is the benefit of the strong relatedness among tasks and as we see next , this is not the case in general . ( ii ) The number of tasks scales essentially linearly with D , which indicates that one should use a small model if we have few tasks . ( iii ) But we can not always use a small model . If tasks are diverse and related by complex transformations with a large |F | , we need a large hypothesis space to learn them together . If |F | is large and H is not appropriately so , the VC-dimension dmax is as large as D itself ; in this case there is again no statistical benefit of training with multiple tasks together , but there is no deterioration either .

Summary: This paper begins with a clear exposition of continual learning and an analysis from a statistical learning theory perspective.  It then introduces the Model Zoo: A sequence of small models that are each trained on a subset of the available tasks at that point (with a separate linear classifier for each task). The tasks selected for training are those that have high loss under the models trained so far (similar to boosting).  The following section contains an exhaustive set of experiments that compares the Model Zoo with a variety of baselines (with and without replay, with and without limited training, a single model with multiple heads, etc.). Surprisingly, the "Isolated" model (i.e., a separate model is trained on each task) does better than most baselines that train on multiple tasks. The Model Zoo outperforms all other baselines and even outperforms the multi-head baseline on ImageNet.  The paper ends with a critical discussion on how the problem of continual learning should be approached.