Related work
\diamond Meta-Reinforcement learning has recently flourished into several different approaches aiming at learning high-level strategies for capturing task rules and structures. A direct line of work consists in automatically meta-learning components or parameters of the RL arsenal to improve over heuristic settings [ 50 , 51 , 52 ] . Orthogonally, work building on the Turing-completeness of recurrent neural networks has shown that simple recurrent neural networks can be trained to store past information in their persistent activity state to inform current decision, in such a way that the network implements a form of reinforcement learning over each episode [ 53 , 48 , 54 ] . It is believed that vanilla recurrent networks alone are not sufficient to meta-learn the efficient forms of episodic control found in biological agents [ 55 , 13 ] . Hence additional work has tried to enhance the system with a better episodic memory model [ 56 , 57 , 58 ] or by modeling a policy as an attention module over an explicitely stored set of past events [ 49 ] . Optimization based approaches have tried to cast episodic adaptation as an explicit optimization procedure either by treating the optimizer as a black-box system [ 59 , 60 ] or by learning a synaptic configuration such that one or a few gradient steps are sufficient to adapt the input/output mapping to a specific task [ 61 ] .

\diamond Artificial fast plasticity: Networks with dynamic weights that can adapt as a function of neural activation have shown promising results over regular recurrent neural networks to handle sequential data [ 62 , 29 , 31 , 63 , 32 , 34 ] . However, contrary to our work, these models postulate a persistent neural activity orchestrating weights evolution. On the contrary, we show that synaptic states are the sole persistent components needed to perform fast adaptation. Additionally, the possibility of optimizing synaptic dynamics with evolutionary strategies in randomly initialized networks [ 64 ] or through gradient descent [ 65 ] has been demonstrated, as well as in a time-continuous setting [ 66 ] . Recent results have shown that plasticity rules differentially tuned at the synapse level allow to dynamically edit and query networks memory [ 31 , 67 ] . However another specificity of this work is that our model synaptic rule is a function of reward and synaptic state, allowing to drive weight dynamics conditionally on both an external feedback signal and the current model belief.

\diamond Associative memory: As discussed above, efficient memory storage and manipulation is a crucial feature for building rapidly learning agents. To improve over vanilla recurrent neural network policies [ 48 ] , some models have augmented recurrent agents with content-addressable dictionaries able to reinstate previously encoded patterns given the current state [ 68 , 69 , 70 , 13 ] . However these slot-based memory systems are subject to interference with incoming inputs and their memory cost grows linearly with experience. Contrastingly, attractor networks can be learnt to produce fast compression of sensory information into a fixed size tensorial representation [ 71 , 72 ] . One class of such network are Hopfield networks [ 73 , 74 , 75 , 76 ] which benefit from a large storage capacity [ 76 ] , can possibly perform hetero-associative concept binding [ 77 , 34 ] and produce fast and flexible information retrieval [ 47 ] .