Contributions by metaplasticity to solving the Catastrophic Forgetting Problem

Catastrophic forgetting (CF) is the sudden and severe loss of memory for previously stored information due to the learning of new information. Standard artificial neural networks have traditionally suffered from CF when undertaking sequential learning tasks. A wide variety of strategies have been used to minimise forgetting, such as generative replay, constructive algorithms, complementary systems, sparse storage, and meta-learning. Metaplasticity is the activity-dependent plasticity of future plasticity or learning. Various network models use metaplasticity-type strategies to reduce forgetting during continual learning. Combining metaplasticity-type rules with other strategies often improves network performance even further. Models may benefit even more by drawing on other properties of biological metaplasticity phenomena. Catastrophic forgetting (CF) refers to the sudden and severe loss of prior information in learning systems when acquiring new information. CF has been an Achilles heel of standard artificial neural networks (ANNs) when learning multiple tasks sequentially. The brain, by contrast, has solved this problem during evolution. Modellers now use a variety of strategies to overcome CF, many of which have parallels to cellular and circuit functions in the brain. One common strategy, based on metaplasticity phenomena, controls the future rate of change at key connections to help retain previously learned information. However, the metaplasticity properties so far used are only a subset of those existing in neurobiology. We propose that as models become more sophisticated, there could be value in drawing on a richer set of metaplasticity rules, especially when promoting continual learning in agents moving about the environment. Catastrophic forgetting (CF) refers to the sudden and severe loss of prior information in learning systems when acquiring new information. CF has been an Achilles heel of standard artificial neural networks (ANNs) when learning multiple tasks sequentially. The brain, by contrast, has solved this problem during evolution. Modellers now use a variety of strategies to overcome CF, many of which have parallels to cellular and circuit functions in the brain. One common strategy, based on metaplasticity phenomena, controls the future rate of change at key connections to help retain previously learned information. However, the metaplasticity properties so far used are only a subset of those existing in neurobiology. We propose that as models become more sophisticated, there could be value in drawing on a richer set of metaplasticity rules, especially when promoting continual learning in agents moving about the environment. mathematical model of artificial ‘neurons’ (as computational nodes/units) organised in layers and coupled via ‘synapses’ (as connections) with modifiable weights. a computational model of visual cortex plasticity designed to explain experience-dependent modification due to altered visual experience. It incorporates a dynamic activity-plasticity function based on the history of cell firing in the network (i.e., metaplasticity). severe loss of memory for previously learned information during the sequential learning of new information due to alterations of connection weights that are key for retaining the old information. the ability to learn tasks in an ongoing and sequential way across time without undue forgetting of previous tasks. Modelling is driven by the understanding that the obvious solution of continuous replay of old tasks concurrently with new learning is unrealistic and too energy intensive. an ANN consisting of many layers, where layers often have specialised functionality. Deep neural networks are typically more powerful than ‘shallow’ networks, and are currently the dominant paradigm in machine learning. a dual-memory architecture consisting of two structural components, each designed to use different learning strategies and which can have different functionalities. values that control the learning process, for example, number of layers or nodes, learning rates, number of training epochs/iterations, and many more. the rate at which synaptic or connection weights can be modified during learning. Metaplasticity-inspired models use rate parameters that are themselves modifiable based on experience or the needs of the network. the experimentally observed phenomenon of slowly shifting combinations of cells representing information in a neural circuit. the plasticity of synaptic plasticity involving activity-dependent and lasting changes of synaptic plasticity capability or learning rate, without necessarily changing the current synaptic strength. in the context of learning systems, parameters are values that are adjusted by the learning process; in ANNs, these are typically the weights and biases of nodes. the state by which a system meets multiple competing objectives in an optimal way without necessarily optimising performance on any one of them. modifies learning algorithms in ANNs to avoid overfitting and improve generalisation. Some regularisation techniques (e.g., elastic weight configuration and synaptic intelligence) have parallels to metaplasticity as they impose constraints on changes of synapses that are relevant for old knowledge. Thereby, they protect old knowledge against interference by new knowledge. memory of one task interferes with memory/learning of a different task. In proactive interference, old memory traces disturb learning of new memory traces. In retroactive interference, newly learned memory traces disturb old memory traces.