text: 212,9,1499207309083361285,3061733236,Dong Gong,"Our new Continual Learning work got accepted by #CVPR2022. We introduced a new Bayesian sparse regularization on the network neurons, with novel memory-based full experience reply. Lucky to work with a great team making this done! An early-version paper: <LINK> <LINK> The sparse regularization encourages to use less model capacity for each task and thus reserves capacity for the subsequent tasks in the continual learning streams. The hierarchical Bayesian modeling optimizes the sparsity. The full experience reply involves the intermediate features for more effective and flexible knowledge sharing and transferring. It also directly guide sparsity modeling at the intermediate layers. The technical details of the sparse regularization are related to our previous work about Bayesian variational dropout: <LINK>",http://arxiv.org/abs/2202.10203,"Continual Learning (CL) methods aim to enable machine learning models to learn new tasks without catastrophic forgetting of those that have been previously mastered. Existing CL approaches often keep a buffer of previously-seen samples, perform knowledge distillation, or use regularization techniques towards this goal. Despite their performance, they still suffer from interference across tasks which leads to catastrophic forgetting. To ameliorate this problem, we propose to only activate and select sparse neurons for learning current and past tasks at any stage. More parameters space and model capacity can thus be reserved for the future tasks. This minimizes the interference between parameters for different tasks. To do so, we propose a Sparse neural Network for Continual Learning (SNCL), which employs variational Bayesian sparsity priors on the activations of the neurons in all layers. Full Experience Replay (FER) provides effective supervision in learning the sparse activations of the neurons in different layers. A loss-aware reservoir-sampling strategy is developed to maintain the memory buffer. The proposed method is agnostic as to the network structures and the task boundaries. Experiments on different datasets show that our approach achieves state-of-the-art performance for mitigating forgetting. ","Learning Bayesian Sparse Networks with Full Experience Replay for