Rethinking the hippocampal cognitive map as a meta-learning computational module
Authors: Luca Ambrogioni, H. Freyja Ólafsdóttir
Journal: Trends in Cognitive Sciences (2023)

The ability to draw on past experience to guide learning in novel and uncertain environments is ubiquitous across the animal kingdom. However, the neural underpinnings of such meta-learning remain relatively unexplored.The hippocampus is thought to lie at the centre of neuronal networks underlying learning. A wealth of experimental evidence suggests hippocampal neurons encode cognitive maps of familiar environments. Yet how the hippocampus could support learning under uncertainty and novelty remains an open question.To address this caveat, we draw on recently developed deep learning techniques known as meta-reinforcement learning. We propose the hippocampus can be thought of as a meta-learning computational module that not only represents familiar environments but also information states and information sources which allow animals to draw on past experience under uncertainty and thereby support dynamic meta-learning. A hallmark of biological intelligence is the ability to adaptively draw on past experience to guide behaviour under novel situations. Yet, the neurobiological principles that underlie this form of meta-learning remain relatively unexplored. In this Opinion, we review the existing literature on hippocampal spatial representations and reinforcement learning theory and describe a novel theoretical framework that aims to account for biological meta-learning. We conjecture that so-called hippocampal cognitive maps of familiar environments are part of a larger meta-representation (meta-map) that encodes information states and sources, which support exploration and provides a foundation for learning. We also introduce concrete hypotheses on how these generic states can be encoded using a principle of superposition. A hallmark of biological intelligence is the ability to adaptively draw on past experience to guide behaviour under novel situations. Yet, the neurobiological principles that underlie this form of meta-learning remain relatively unexplored. In this Opinion, we review the existing literature on hippocampal spatial representations and reinforcement learning theory and describe a novel theoretical framework that aims to account for biological meta-learning. We conjecture that so-called hippocampal cognitive maps of familiar environments are part of a larger meta-representation (meta-map) that encodes information states and sources, which support exploration and provides a foundation for learning. We also introduce concrete hypotheses on how these generic states can be encoded using a principle of superposition. A fundamental characteristic of biological intelligence is the ability to draw on past experience to guide and expedite learning; an ability often referred to as meta-learning (see Glossary). To give an example, if lost in a foreign European city, one would still be able to locate the main square of a city with ease. Rather than using an exhaustive, trial-and-error-like approach – weaving through different streets – one may for example look out for tall spires, a standard feature of cathedrals that are commonplace in the squares of European cities, or in modern times simply search for it on Google maps. In other words, humans and other animals are able to draw on their learning histories to adopt adaptive information-seeking strategies that facilitate learning in unfamiliar situations. Importantly, these strategies rely on an assessment of the currently available information and a determination of relevant information sources. The past few decades have seen a concerted research effort devoted to elucidating the neuronal basis of biological learning. A large share of this research has focused on the hippocampal formation – a brain region thought to lie at the centre of neural systems supporting learning and memory [1.Scoville W.B. Milner B. Loss of recent memory after bilateral hippocampal lesions.J. Neurol. Neurosurg. Psychiatry. 1957; 20: 11-21Crossref PubMed Google Scholar,2.O'Keefe J. Nadel L. The Hippocampus as a Cognitive Map. Clarendon, 1978Google Scholar]. Seminal findings in this field have described some of the key functional neuronal representations and mechanisms that are thought to underpin memory encoding, recall, and storage [3.O'Keefe J. Dostrovsky J. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat.Brain Res. 1971; 34: 171-175Crossref PubMed Scopus (3906) Google Scholar, 4.Wilson M.A. McNaughton B.L. Reactivation of hippocampal ensemble memories during sleep.Science. 1994; 265: 676-679Crossref PubMed Google Scholar, 5.Carr M.F. et al.Hippocampal replay in the awake state: a potential substrate for memory consolidation and retrieval.Nat. Neurosci. 2011; 14: 147-153Crossref PubMed Scopus (537) Google Scholar, 6.Taube J.S. et al.Head-direction cells recorded from the postsubiculum in freely moving rats. II. Effects of environmental manipulations.J. Neurosci. 1990; 10: 436-447Crossref PubMed Google Scholar, 7.Hafting T. et al.Microstructure of a spatial map in the entorhinal cortex.Nature. 2005; 436: 801-806Crossref PubMed Scopus (2495) Google Scholar]. This research has focused to a large extent on learning in the spatial domain [8.Morris R.G. et al.Place navigation impaired in rats with hippocampal lesions.Nature. 1982; 297: 681-683Crossref PubMed Scopus (5118) Google Scholar,9.Olton D.S. et al.Hippocampal connections and spatial discrimination.Brain Res. 1978; 139: 295-308Crossref PubMed Scopus (646) Google Scholar] given the robust spatial correlate observed in hippocampal single-unit activity [3.O'Keefe J. Dostrovsky J. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat.Brain Res. 1971; 34: 171-175Crossref PubMed Scopus (3906) Google Scholar]. Influential theories suggest the hippocampal neural representations can be thought of as cognitive maps [10.Tolman E.C. Cognitive maps in rats and men.Psychol. Rev. 1948; 55: 189-208Crossref PubMed Scopus (3756) Google Scholar] of explored environments, which guide goal-directed navigation [2.O'Keefe J. Nadel L. The Hippocampus as a Cognitive Map. Clarendon, 1978Google Scholar]. However, while navigation in familiar territories is an essential part of daily life, being able to navigate under uncertainty is of tremendous evolutionary importance and may even make a difference between life or death. By examining the existing literature in neuroscience and machine learning, we will attempt to answer the following, seemingly paradoxical, question: can a cognitive map be used to navigate in uncharted or uncertain environments? Given the ubiquitous use of selective information-seeking strategies across the animal kingdom [11.Kirk C.R. et al.Rats respond for information: metacognition in a rodent?.J. Exp. Psychol. Anim. Learn. Cogn. 2014; 40: 249-259Crossref PubMed Scopus (20) Google Scholar, 12.Templer V.L. Tools of engagement: information seeking in chimpanzees.Learn. Behav. 2019; 47: 107-108Crossref PubMed Scopus (0) Google Scholar, 13.Wilson T.D. Human information behaviour.Inf. Sci. 2000; 3: 49-55Crossref Google Scholar], answering this question is essential to effectively account for animal learning. To answer the question, we will review experimental evidence and outline a theoretical framework that takes lessons from a field of modern artificial intelligence research, known as meta-reinforcement learning (meta-RL). The central theoretical concept we introduce is that of the cognitive meta-map (Figure 1). Using this concept, we conjecture that the hippocampus may be thought of as a meta-learning computational module, where the representations of known environments are part of a unified meta-representation of accumulated learning, where both known environments and pre-existing templates of potential future environments are encoded. Fundamentally, we suggest that the hippocampal meta-network can be described as encoding information states and information sources. Information states (also known as belief states in the machine learning literature) encode the state of knowledge of the animal, such as how familiar a foreign European city is. Information sources are objects that lead to transitions in the information state that reduce uncertainty. In our ongoing urban navigation example, an information source would for example be tall spires as they signal where main square of a city is. Although our focus is on explaining learning within the spatial domain, this framework is by no means meant to be limited to a single domain. It can easily be extended to account for episodic memory encoding as well (Box 1). As such, we believe this framework may be a stepping stone in the development of a comprehensive computational, and neurophysiologically plausible, model for explicit learning.Box 1Meta-learning networks capture spatial and episodic memoryHistorically there have been, broadly speaking, two accounts of hippocampal function. One assumes the hippocampus supports spatial navigation abilities via the spatially correlated activity of its principal cells [2.O'Keefe J. Nadel L. The Hippocampus as a Cognitive Map. Clarendon, 1978Google Scholar], which is discussed here. While the other, which derives its empirical support largely from human neuropsychology and neuroimaging, emphasises its role in episodic memory [1.Scoville W.B. Milner B. Loss of recent memory after bilateral hippocampal lesions.J. Neurol. Neurosurg. Psychiatry. 1957; 20: 11-21Crossref PubMed Google Scholar,74.Squire L.R. Alvarez P. Retrograde amnesia and memory consolidation: a neurobiological perspective.Curr. Opin. Neurobiol. 1995; 5: 169-177Crossref PubMed Scopus (1020) Google Scholar]. Several attempts have been made to bring these seemingly disparate schools of thought together. Namely, some speculate that, rather than assuming the hippocampus supports episodic or spatial memory, its functional contribution may instead be captured by an account that assumes it supports relational encoding in general [73.Schiller D. et al.Memory and space: towards an understanding of the cognitive map.J. Neurosci. 2015; 35: 13904-13911Crossref PubMed Scopus (185) Google Scholar,75.Eichenbaum H. Cohen N.J. Can we reconcile the declarative memory and spatial navigation views on hippocampal function?.Neuron. 2014; 83: 764-770Abstract Full Text Full Text PDF PubMed Google Scholar,76.Milivojevic B. Doeller C.F. Mnemonic networks in the hippocampal formation: from spatial maps to temporal and conceptual codes.J. Exp. Psychol. Gen. 2013; 142: 1231-1241Crossref PubMed Scopus (37) Google Scholar]. Both episodic and spatial memory are forms of relational memory. Phenomenologically, the only aspect that differentiates the two is that episodic memory also includes a temporal component – that is, encoding the order in which events occur.This relational perspective fits well within our meta-map framework of the hippocampus. Namely, modules that make up the meta-map – reflecting, for example, similar places explored in the past or templates for similar but unexplored places – are formed through encoding of statistical regularities in the world. For example, what kind of topology many European cities have, or, if you are rodent, the layout of typical burrow systems. This framework can then easily be extended to accommodate temporal coding that underpins episodic memory formation. For example, if you go to the park in the summer, this event is likely preceded by a trip to the grocery store where you buy drinks and snacks to enjoy in the park. These regularities will be represented in the meta-learning network which then facilitate encoding of such an episodic memory. Our superposition hypothesis can also be directly translated to episodic memory encoding and recall. During encoding, novel sensory stimulations activates a superposition of hippocampal templates, corresponding to potential memories that gradually refine to a single representation that are with to the new event. Similarly, during recall a superposition of encoded memories will be elicited, with the spread of activated memories reflecting the imperfect, and therefore probabilistic, nature of recall.As such, although we present our meta-learning framework in the context of spatial navigation, we do not intend for it to be specific or limited to the spatial domain. Indeed, it has the capacity to incorporate any relational structure given it occurs with enough statistical regularity in the world. Historically there have been, broadly speaking, two accounts of hippocampal function. One assumes the hippocampus supports spatial navigation abilities via the spatially correlated activity of its principal cells [2.O'Keefe J. Nadel L. The Hippocampus as a Cognitive Map. Clarendon, 1978Google Scholar], which is discussed here. While the other, which derives its empirical support largely from human neuropsychology and neuroimaging, emphasises its role in episodic memory [1.Scoville W.B. Milner B. Loss of recent memory after bilateral hippocampal lesions.J. Neurol. Neurosurg. Psychiatry. 1957; 20: 11-21Crossref PubMed Google Scholar,74.Squire L.R. Alvarez P. Retrograde amnesia and memory consolidation: a neurobiological perspective.Curr. Opin. Neurobiol. 1995; 5: 169-177Crossref PubMed Scopus (1020) Google Scholar]. Several attempts have been made to bring these seemingly disparate schools of thought together. Namely, some speculate that, rather than assuming the hippocampus supports episodic or spatial memory, its functional contribution may instead be captured by an account that assumes it supports relational encoding in general [73.Schiller D. et al.Memory and space: towards an understanding of the cognitive map.J. Neurosci. 2015; 35: 13904-13911Crossref PubMed Scopus (185) Google Scholar,75.Eichenbaum H. Cohen N.J. Can we reconcile the declarative memory and spatial navigation views on hippocampal function?.Neuron. 2014; 83: 764-770Abstract Full Text Full Text PDF PubMed Google Scholar,76.Milivojevic B. Doeller C.F. Mnemonic networks in the hippocampal formation: from spatial maps to temporal and conceptual codes.J. Exp. Psychol. Gen. 2013; 142: 1231-1241Crossref PubMed Scopus (37) Google Scholar]. Both episodic and spatial memory are forms of relational memory. Phenomenologically, the only aspect that differentiates the two is that episodic memory also includes a temporal component – that is, encoding the order in which events occur. This relational perspective fits well within our meta-map framework of the hippocampus. Namely, modules that make up the meta-map – reflecting, for example, similar places explored in the past or templates for similar but unexplored places – are formed through encoding of statistical regularities in the world. For example, what kind of topology many European cities have, or, if you are rodent, the layout of typical burrow systems. This framework can then easily be extended to accommodate temporal coding that underpins episodic memory formation. For example, if you go to the park in the summer, this event is likely preceded by a trip to the grocery store where you buy drinks and snacks to enjoy in the park. These regularities will be represented in the meta-learning network which then facilitate encoding of such an episodic memory. Our superposition hypothesis can also be directly translated to episodic memory encoding and recall. During encoding, novel sensory stimulations activates a superposition of hippocampal templates, corresponding to potential memories that gradually refine to a single representation that are with to the new event. Similarly, during recall a superposition of encoded memories will be elicited, with the spread of activated memories reflecting the imperfect, and therefore probabilistic, nature of recall. As such, although we present our meta-learning framework in the context of spatial navigation, we do not intend for it to be specific or limited to the spatial domain. Indeed, it has the capacity to incorporate any relational structure given it occurs with enough statistical regularity in the world. Meta-learning – or learning to learn – is a concept that originates in the psychological sciences and in education research [14.Gopnik A. et al.The Scientist in the Crib: Minds, Brains, and How Children Learn. William Morrow & Co, 1999Google Scholar,15.Bransford J.D. et al.How People Learn: Brain, Mind, Experience and School. National Academy Press, 2000Google Scholar]. In recent years meta-learning ideas have found fertile ground in AI [16.Zintgraf L. et al.VariBAD: Variational Bayes-Adaptive Deep RL via Meta-Learning.J. Mach. Learn. Res. 2021; 22: 1-39Google Scholar, 17.Wang J.X. et al.Learning to reinforcement learn.arXiv. 2016; (Published online November 17, 2016)https://doi.org/10.48550/arXiv.1611.05763Google Scholar, 18.Gupta A. et al.Meta-reinforcement learning of structured exploration strategies.arXiv. 2018; (Published online February 20, 2018)https://doi.org/10.48550/arXiv.1802.07245Google Scholar, 19.Beck J. et al.A survey of meta-reinforcement learning.arXiv. 2023; (Published online January 19, 2023)https://doi.org/10.48550/arXiv.2301.08028PubMed Google Scholar]. Particularly, a group of AI methods that are collectively known as meta-reinforcement learning (meta-RL) have been likened to the implementation of learning in the brain [20.Wang J.X. et al.Prefrontal cortex as a meta-reinforcement learning system.Nat. Neurosci. 2018; 21: 860-868Crossref PubMed Scopus (256) Google Scholar]. In the context of spatial learning, meta-RL involves training a recurrent neural network on diverse but related tasks – such as learning to locate food rewards in mazes with different wall configurations [21.Kristopher T.J. et al.A recurrent network model of planning explains hippocampal replay and human behavior.bioRxiv. 2023; (Published online January 19, 2023)https://doi.org/10.1101/2023.01.16.523429Google Scholar]. Under these conditions, the rate of long-term synaptic learning is slowed down since spatial meta-learning can only be achieved over multiple trials, during which a number of different environments are experienced. This results in a learned over-arching (meta-)representation that can encode regularities as well as specificities of distinct environments [17.Wang J.X. et al.Learning to reinforcement learn.arXiv. 2016; (Published online November 17, 2016)https://doi.org/10.48550/arXiv.1611.05763Google Scholar]. This meta-representation can then be used to drive fast environment-specific adaptation without the need of substantial new updates in the weight matrices (i.e., the synaptic structure of the network). From a theoretical point of view, optimal meta-RL can be formalized in terms of information states, which summarise all information collected during an episode [19.Beck J. et al.A survey of meta-reinforcement learning.arXiv. 2023; (Published online January 19, 2023)https://doi.org/10.48550/arXiv.2301.08028PubMed Google Scholar]. A meta-RL agent can then be described as navigating this space of information to plan how to collect new sensory data and use it to acquire reward. This involves the identification of information sources, which acquire value since the resulting information updates can guide the agent toward the rewarding locations. Importantly, the value of information sources can only be evaluated if the agent has an internal representation of its current information state, since the value of acquiring new information is near zero if the animal is certain of its environment but is higher under uncertainty. In recent years, multiple theorists have begun speculating that the computations of the hippocampal–entorhinal system may be captured by a meta-RL network [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar, 23.Whittington J.C.R. et al.The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation.Cell. 2020; 183: 1249-1263 e23Abstract Full Text Full Text PDF PubMed Scopus (116) Google Scholar, 24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar] (see Box 2 for a more extensive discussion of these theories). Specifically, influential theories have postulated that the periodic activity patterns characteristic of cells in the medial entorhinal cortex (mEC, such as grid cells [7.Hafting T. et al.Microstructure of a spatial map in the entorhinal cortex.Nature. 2005; 436: 801-806Crossref PubMed Scopus (2495) Google Scholar]) may support the formation of generalised (meta-) representations [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar], such as representations of geometric relations in 2D environments [23.Whittington J.C.R. et al.The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation.Cell. 2020; 183: 1249-1263 e23Abstract Full Text Full Text PDF PubMed Scopus (116) Google Scholar], but also in nonspatial domains [24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar]. In these frameworks, the hippocampal representations encode unique spaces and experiences by combining relational inputs from mEC with unique sensory information from the lateral EC. However, these models do not elaborate on the nature of these concrete hippocampal representations. In particular, it is not clear how the hippocampus represents situations when no unique learned cognitive map can be recalled. Note that, as explained in the preceding text, the encoding of these ambiguous states is central to the solution of meta-RL tasks since it is required in order to plan exploration and guide information-seeking behaviour. Our framework aims to address these caveats in our understanding.Box 2Links with cognitive spaces and TEM modelsRecent years have seen exciting theoretical developments – some of which incorporate meta-reinforcement learning [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar,23.Whittington J.C.R. et al.The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation.Cell. 2020; 183: 1249-1263 e23Abstract Full Text Full Text PDF PubMed Scopus (116) Google Scholar] – which aim to capture how humans and other animals use their past to guide new learning [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar,24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar,73.Schiller D. et al.Memory and space: towards an understanding of the cognitive map.J. Neurosci. 2015; 35: 13904-13911Crossref PubMed Scopus (185) Google Scholar]. These models by and large suggest that the brain – and particularly the hippocampal–entorhinal system – learns relational structures that are observed frequently in the environment – for example, the 2D structure that underlies space. By representing the underlying relations between elements – be it places, or any other feature – this knowledge can be extrapolated to novel settings and expedite learning. Such a domain-general relational system – sometimes referred to as cognitive spaces [24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar] – is thought to underlie knowledge generalisation and inference. Importantly, the relational representation is thought to be encoded by cells in the mEC; for example, by the periodic firing characteristics of mEC grid cells [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar,24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar], that seem ideally poised to represent structural regularities. In these frameworks, the hippocampus represents unique environments by combining the relational signal derived from the mEC with sensory input arriving from the lateral EC.Our meta-learning framework of the hippocampus builds on these cognitive space models, insofar as it also aims to capture how an animal's learning history may underpin fast and flexible learning. However, our model aims to explain how meta-learning may be operationalised in the hippocampus specifically, and particularly how information states and sources are encoded by hippocampal place cells and replay. Consequently, our framework offers insight into the biological computations that underlie the selection of optimal search policies that reduce uncertainty; which could be formalised as the process by which the correct cognitive space is selected in the aforementioned meta-learning frameworks and what neuronal representations guide this process. These representations can then expedite learning and support accurate behaviour. Being able to capture behaviours that aim to reduce uncertainty and enable optimal policy selection, we believe is an essential stepping-stone in the development of a neurophysiologically plausible model of biological learning. Recent years have seen exciting theoretical developments – some of which incorporate meta-reinforcement learning [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar,23.Whittington J.C.R. et al.The Tolman-Eichenbaum machine: unifying space and relational memory through generalization in the hippocampal formation.Cell. 2020; 183: 1249-1263 e23Abstract Full Text Full Text PDF PubMed Scopus (116) Google Scholar] – which aim to capture how humans and other animals use their past to guide new learning [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar,24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar,73.Schiller D. et al.Memory and space: towards an understanding of the cognitive map.J. Neurosci. 2015; 35: 13904-13911Crossref PubMed Scopus (185) Google Scholar]. These models by and large suggest that the brain – and particularly the hippocampal–entorhinal system – learns relational structures that are observed frequently in the environment – for example, the 2D structure that underlies space. By representing the underlying relations between elements – be it places, or any other feature – this knowledge can be extrapolated to novel settings and expedite learning. Such a domain-general relational system – sometimes referred to as cognitive spaces [24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar] – is thought to underlie knowledge generalisation and inference. Importantly, the relational representation is thought to be encoded by cells in the mEC; for example, by the periodic firing characteristics of mEC grid cells [22.Behrens T.E.J. et al.What is a cognitive map? Organizing knowledge for flexible behavior.Neuron. 2018; 100: 490-509Abstract Full Text Full Text PDF PubMed Scopus (341) Google Scholar,24.Bellmund J.L.S. et al.Navigating cognition: spatial codes for human thinking.Science. 2018; 362eaat6766Crossref PubMed Scopus (233) Google Scholar], that seem ideally poised to represent structural regularities. In these frameworks, the hippocampus represents unique environments by combining the relational signal derived from the mEC with sensory input arriving from the lateral EC. Our meta-learning framework of the hippocampus builds on these cognitive space models, insofar as it also aims to capture how an animal's learning history may underpin fast and flexible learning. However, our model aims to explain how meta-learning may be operationalised in the hippocampus specifically, and particularly how information states and sources are encoded by hippocampal place cells and replay. Consequently, our framework offers insight into the biological computations that underlie the selection of optimal search policies that reduce uncertainty; which could be formalised as the process by which the correct cognitive space is selected in the aforementioned meta-learning frameworks and what neuronal representations guide this process. These representations can then expedite learning and support accurate behaviour. Being able to capture behaviours that aim to reduce uncertainty and enable optimal policy selection, we believe is an essential stepping-stone in the development of a neurophysiologically plausible model of biological learning. The hippocampal contribution to spatial learning is thought to be mediated by the location-specific activity of its principal cells (place cells [3.O'Keefe J. Dostrovsky J. The hippocampus as a spatial map. Preliminary evidence from unit activity in the freely-moving rat.Brain Res. 1971; 34: 171-175Crossref PubMed Scopus (3906) Google Scholar]), which are thought to encode cognitive maps of explored environments [2.O'Keefe J. Nadel L.