Despite huge success, deep networks are unable to learn effectively in sequential multitask learning settings as they forget the past learned tasks after learning new tasks. Inspired from complementary learning systems theory, we address this challenge by learning a generative model that couples the current task to the past learned tasks through a discriminative embedding space. We learn an abstract level generative distribution in the embedding that allows the generation of data points to represent the experience. We sample from this distribution and utilize experience replay to avoid forgetting and simultaneously accumulate new knowledge to the abstract distribution in order to couple the current task with past experience. We demonstrate theoretically and empirically that our framework learns a distribution in the embedding that is shared across all task and as a result tackles catastrophic forgetting. Past works have addressed catastrophic forgetting using two main approaches: model consolidation [Kirkpatrick et al., arXiv:1903.04566v1 [cs.LG] Proof: We consider Z (t) with empirical distributionq (t) and the pseudo-task with the distribution ψ(p (T −1) J ) in the network input space, in Theorem 1. Using the triangular inequality on the term W (q (t) , ψ(p ) for all t ≤ s < T , Lemma 1 can be derived. Lemma 1 explains why our algorithm can tackle catastrophic forgetting. When future tasks are learned, our algorithms updates the model parameters conditioned on minimizing the upper bound of e t in Eq. 7. Given suitable network structure and in the presence of enough labeled data points, the terms e J t−1 and e C (θ * ) are minimized using ERM, and the last constant term would be small. The term W (q (t) , ψ(p (t) J )) is minimal because we deliberately fit the distributionp (t) J to the distribution φ(q (t) ) in the embedding space and ideally learn φ and ψ such that ψ ≈ φ −1 . This term demonstrates that minimizing the discrimination loss is critical as only then, we can fit a GMM distribution on φ(p (t) ) with high accuracy. Similarly, the sum terms in Eq 7 are minimized because at t = s we draw samples fromp (s−1) J and enforce indirectlyp (s−1) J ≈ φ(ψ(p (s−1) J ) ). Since the upper bound of e t in Eq 7 is minimized and conditioned on tightness of the upper bound, the task Z (t) will not be forgotten.
Building upon our prior work REF , we develop a new method for generative experience replay to tackle catastrophic forgetting.
Complementary Learning for Overcoming Catastrophic Forgetting Using Experience Replay
ArXiv
ArXiv
Computer Science Mathematics