  Replay is the reactivation of one or more neural patterns, which are similar
to the activation patterns experienced during past waking experiences. Replay
was first observed in biological neural networks during sleep, and it is now
thought to play a critical role in memory formation, retrieval, and
consolidation. Replay-like mechanisms have been incorporated into deep
artificial neural networks that learn over time to avoid catastrophic
forgetting of previous knowledge. Replay algorithms have been successfully used
in a wide range of deep learning methods within supervised, unsupervised, and
reinforcement learning paradigms. In this paper, we provide the first
comprehensive comparison between replay in the mammalian brain and replay in
artificial neural networks. We identify multiple aspects of biological replay
that are missing in deep learning systems and hypothesize how they could be
utilized to improve artificial neural networks.
