Efficient Inference in Structured Spaces

Whittington et al. demonstrate how network architectures defined in a spatial context may be useful for inference on different types of relational knowledge. These architectures allow for learning the structure of the environment and then transferring that knowledge to allow prediction of novel transitions. Whittington et al. demonstrate how network architectures defined in a spatial context may be useful for inference on different types of relational knowledge. These architectures allow for learning the structure of the environment and then transferring that knowledge to allow prediction of novel transitions. Can the circuits responsible for spatial learning during navigation also be used to represent and reason about non-spatial relational structures, and if so, how do they do it? Place cells of the hippocampus and grid cells of the medial entorhinal cortex are key loci for spatial representation in mammals (O’Keefe and Nadel, 1978O’Keefe J. Nadel L. The hippocampus as a cognitive map. Clarendon Press, Oxford1978Google Scholar; Moser et al., 2017Moser E.I. Moser M.B. McNaughton B.L. Spatial representation in the hippocampal formation: a history.Nat. Neurosci. 2017; 20: 1448-1464Crossref PubMed Scopus (142) Google Scholar). In this issue of Cell, Whittington et al., 2020Whittington J.C. Muller T.H. Mark S. Chen G. Barry C. Burgess N. Behrens T.E. The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation.Cell. 2020; 183 (this issue): 1249-1263Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar demonstrate how network architectures defined in a spatial context may be useful for inference on different types of relational knowledge. Their work adds an important brick on the path toward a realization of Tolman’s ideas (Tolman, 1948Tolman E.C. Cognitive maps in rats and men.Psychol. Rev. 1948; 55: 189-208Crossref PubMed Scopus (3308) Google Scholar) about cognitive maps for general-purpose reasoning and a unification of the spatial and non-spatial roles of hippocampus as laid out by Eichenbaum (Eichenbaum and Cohen, 2014Eichenbaum H. Cohen N.J. Can we reconcile the declarative memory and spatial navigation views on hippocampal function?.Neuron. 2014; 83: 764-770Abstract Full Text Full Text PDF PubMed Scopus (291) Google Scholar). Empirically, place and grid cells have been shown to represent locations in physical and abstract spaces. In physical space, grid cells sum instantaneous motion vectors to generate an updated estimate of location. Thus, the grid cell network would track a right-up-left motion from the circle start point in Figure 1A to generate the same ending state as a single up motion from the circle start point. In this way, grid cells can make accurate inferences about location across new trajectories. Place cells build associations between various inputs, including grid states and inputs from external landmarks (Figure 1C). Thus, given the circle landmark, the relevant place cell can activate the grid state for the lower-left corner of the pink loop through a previously formed association, even if the current grid cell state is corrupted. Alternatively, given the grid state for the upper-left corner of the pink loop arrived at along a new path, the associated place cells will activate the representation for the cross landmark (“mental time-travel” or “mind-travel”; Sanders et al., 2015Sanders H. Rennó-Costa C. Idiart M. Lisman J. Grid cells and place cells: an integrated view of their navigational and memory function.Trends Neurosci. 2015; 38: 763-775Abstract Full Text Full Text PDF PubMed Scopus (48) Google Scholar). This ability to predict the landmark that would occur along a never-experienced trajectory is known as “zero-shot” inference. This ability is not merely restricted to the spatial domain and even applies in higher dimensional spaces (Klukas et al., 2020Klukas M. Lewis M. Fiete I. Efficient and flexible representation of higher-dimensional cognitive variables with grid cells.PLoS Comput. Biol. 2020; 16: e1007796Crossref PubMed Scopus (10) Google Scholar). Whittington et al., 2020Whittington J.C. Muller T.H. Mark S. Chen G. Barry C. Burgess N. Behrens T.E. The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation.Cell. 2020; 183 (this issue): 1249-1263Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar combine and operationalize these hypotheses into a working model that performs highly efficient inference across instances of similarly structured spaces. Whittington et al., 2020Whittington J.C. Muller T.H. Mark S. Chen G. Barry C. Burgess N. Behrens T.E. The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation.Cell. 2020; 183 (this issue): 1249-1263Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar build their model—which they refer to as the Tolman-Eichenbaum Machine (TEM)—by designing a modular network of subnetworks with correspondingly segregated inputs and then training it to solve the spatial zero-shot landmark prediction task. The inputs are a series of actions and the landmarks encountered as a result of those actions (Figure 1B). The first subnetwork receives the action inputs and learns how actions cause transitions between locations. This subnetwork learns the structure of the environment (Figure 1C, bottom). The second subnetwork receives landmark inputs and learns which landmarks occur at which locations (Figure 1C, top right). Indeed, after training with landmarks and actions that correspond to movements in an underlying two-dimensional structure, the emergent activity of units in the two subnetworks resembles grid and place cell-like tuning, respectively. Following training in several two-dimensional environments, TEM has learned to represent that structure. At that point, when presented with a new environment of the same underlying structure, TEM can reuse the already learned spatial structure (action-to-transition mapping) and only has to relearn which landmarks occur at which locations, information that is encoded in a new place cell map (Figure 1D). This allows TEM to perform zero-shot learning: to predict the landmarks that would occur on a novel trajectory through the new environment after seeing those landmarks only once on any other trajectory. Previous work had shown that grid cell representations are shared across similar environments (Fyhn et al., 2007Fyhn M. Hafting T. Treves A. Moser M.B. Moser E.I. Hippocampal remapping and grid realignment in entorhinal cortex.Nature. 2007; 446: 190-194Crossref PubMed Scopus (435) Google Scholar; Yoon et al., 2013Yoon K.J. Buice M.A. Barry C. Hyman R. Burgess N. Fiete I.R. Specific evidence of low-dimensional continuous attractor dynamics in grid cells.Nat. Neurosci. 2013; 16: 1077-1084Crossref PubMed Scopus (134) Google Scholar), leading to suggestions that grid cells could enable zero-shot prediction (e.g., Sanders et al., 2015Sanders H. Rennó-Costa C. Idiart M. Lisman J. Grid cells and place cells: an integrated view of their navigational and memory function.Trends Neurosci. 2015; 38: 763-775Abstract Full Text Full Text PDF PubMed Scopus (48) Google Scholar). Whittington et al., 2020Whittington J.C. Muller T.H. Mark S. Chen G. Barry C. Burgess N. Behrens T.E. The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation.Cell. 2020; 183 (this issue): 1249-1263Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar demonstrate the successful use of this architecture to enable solving the zero-shot prediction problem. Whittington et al., 2020Whittington J.C. Muller T.H. Mark S. Chen G. Barry C. Burgess N. Behrens T.E. The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation.Cell. 2020; 183 (this issue): 1249-1263Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar next generalize the zero-shot prediction problem to domains that are structured differently than two-dimensional space by treating relationships in these domains as equivalent to taking actions in the space. They show that the same gross modular networks, when trained on prediction of, for instance, ordinality of items across multiple “ordinality” spaces with a common transition structure but different landmarks (items) can perform zero-shot inference about ordinality in a new instance of such a space. Most interestingly, they show that the same is possible for inference of familial relationships by training similar networks on such tasks (Figures 1A and 1B). This work raises several interesting questions:•Do the structure network units look like grid cells when the network is trained on the non-spatial domains, and do grid cells in the brain show similar response properties on analogous non-spatial tasks?•The same brain makes predictions in many differently structured spaces: TEM is told which structure to use for a given task. How can it detect which spaces have a shared structure when structure type labels are not explicitly given? Sanders et al., 2020Sanders H. Wilson M.A. Gershman S.J. Hippocampal remapping as hidden state inference.eLife. 2020; 9: e51140Crossref PubMed Scopus (20) Google Scholar suggest ways to infer shared hidden states, but do not take into account relational structure. Might there be a useful way to combine the two approaches?•When entering a new environment, Whittington et al., 2020Whittington J.C. Muller T.H. Mark S. Chen G. Barry C. Burgess N. Behrens T.E. The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation.Cell. 2020; 183 (this issue): 1249-1263Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar wipe clean and relearn all cue-location associations, even if a subset of cue-location associations are the same as seen before. How might cue-location association knowledge also be integrated across environments? Though the initial formulation is probabilistic, the development of the model and the tasks are not: it will be interesting to extend them to settings with uncertainty. Overall, Whittington et al., 2020Whittington J.C. Muller T.H. Mark S. Chen G. Barry C. Burgess N. Behrens T.E. The Tolman-Eichenbaum Machine: Unifying space and relational memory through generalisation in the hippocampal formation.Cell. 2020; 183 (this issue): 1249-1263Abstract Full Text Full Text PDF PubMed Scopus (33) Google Scholar demonstrate that the same gross network architectures, when trained on structured spaces, can enable rapid inference and prediction in new exemplars of those spaces. This is achieved by setting up the network architecture to factorize the problem as suggested by grid and place cells into a shared action-based transition component and an association component. The work shows concretely how the brain might learn and understand abstract relationships. The Tolman-Eichenbaum Machine: Unifying Space and Relational Memory through Generalization in the Hippocampal FormationWhittington et al.CellNovember 11, 2020In BriefThe Tolman-Eichenbaum Machine, named in honor of Edward Chace Tolman and Howard Eichenbaum for their contributions to cognitive theory, provides a unifying framework for the hippocampal role in spatial and nonspatial generalization and unifying principles underlying many entorhinal and hippocampal cell types. Full-Text PDF Open Access