Title: Binding in hippocampal-entorhinal circuits enables compositionality in cognitive maps
Venue: NeurIPS 2024
Accepted: yes
Keywords: cognitive maps

compositionality

hippocampus

entorhinal cortex
Abstract: We propose a normative model for spatial representation in the hippocampal formation that combines optimality principles, such as maximizing coding range and spatial information per neuron, with an algebraic framework for computing in distributed representation. Spatial position is encoded in a residue number system, with individual residues represented by high-dimensional, complex-valued vectors. These are composed into a single vector representing position by a similarity-preserving, conjunctive vector-binding operation. Self-consistency between the vectors representing position and the individual residues is enforced by a modular attractor network whose modules correspond to the grid cell modules in entorhinal cortex. The vector binding operation can also be used to bind different contexts to spatial representations, yielding a model for entorhinal cortex and hippocampus. We provide model analysis of scaling, similarity preservation and convergence behavior as well as experiments demonstrating noise robustness, sub-integer resolution in representing position, and path integration. The model formalizes the computations in the cognitive map and makes testable experimental predictions.

Binding in hippocampal-entorhinal circuits enables compositionality in cognitive maps

Abstract

We propose a normative model for spatial representation in the hippocampal formation that combines optimality principles, such as maximizing coding range and spatial information per neuron, with an algebraic framework for computing in distributed representation. Spatial position is encoded in a residue number system, with individual residues represented by high-dimensional, complex-valued vectors. These are composed into a single vector representing position by a similaritypreserving, conjunctive vector-binding operation. Self-consistency between the representations of the overall position and of the individual residues is enforced by a modular attractor network whose modules correspond to the grid cell modules in entorhinal cortex. The vector binding operation can also associate different contexts to spatial representations, yielding a model for entorhinal cortex and hippocampus. Weshow that the model achieves normative desiderata including superlinear scaling of patterns with dimension, robust error correction, and hexagonal, carry-free encoding of spatial position. These properties in turn enable robust path integration and association with sensory inputs. More generally, the model formalizes how compositional computations could occur in the hippocampal formation and leads to testable experimental predictions. 1

1 Introduction

The hippocampal formation (HF), consisting of hippocampus (HC) and the medial and lateral part of the neighboring entorhinal cortex, (MEC) and (LEC), is critical for forming memories and representing variables such as spatial position [1, 2]. Recent work has provided evidence of compositional structure in HF representations, enabling complex representations to be composed by simpler building blocks and their formation rules. Examples include novel recombinations of past experience occurring in replay [3], or the exponential expressivity of the grid cell code [4, 5]. In particular, compositional representations afford high expressivity with lower dimensional storage requirements [6], less complexity in latent state inference, and generalization to novel scenes with familiar parts.

To gain insight into the possible computational principles and neural mechanisms at play in the HF, we take a normative modeling approach. That is, we seek to construct a model built from a set of

neural coding principles that effectively achieves the postulated function of the system. With this approach, we can then explain details about the neuroanatomical and neurophysiological structures in light of their particular contributions to an information processing objective. The resulting model can also lead to new predictions about the neural mechanisms that enable this function.

The postulated function of the HF -as a cognitive map and episodic memory- has a core computational requirement, to represent and navigate space. Here, space is either the actual physical environment or a more abstract conceptual space. We formulate multiple desiderata for an effective representation of space. We then show that a residue number system, incorporated into a compositional encoding scheme, fulfills these desiderata. It is achieved by a modular attractor network that factorizes encoded locations into components of a residue number system. This provides an algorithmic-level hypothesis of hippocampal-entorhinal interactions. A core mechanism of this algorithm is binding , which draws inspiration from work in neuroscience, cognitive science, and artificial intelligence.

2 A normative model for the hippocampal formation

2.1 Principles for representing space

Our first principle is that space is represented by a compositional code that has high spatial-resolution, is noise-robust, and in which algebraic operations on the components can be updated in parallel. Prior work [4, 5] has proposed the residue number system (RNS) [7] as a candidate for fulfilling these requirements. An RNS expresses an integer x in terms of its remainder relative to a set of co-prime moduli { m i } . For example, relative to moduli { 3 , 5 , 7 } , x = 40 is encoded as { 1 , 0 , 5 } . The Chinese Remainder Theorem guarantees that all integers in the range [0 , M -1] , where M = ∏ i m i , are assigned a unique representation. An RNS provides high spatial resolution, carry-free arithmetic operations, and robust error correction [8]. Experimental observations in entorhinal cortex show a discrete multi-scale organization of spatial grid cells [9] that is compatible with the assumption of discrete RNS modules.

The second principle we adopt is that an individual residue value should be encoded by a neural population in a similarity-preserving fashion. In particular, we require that distinct integer values are represented with nearly orthogonal vectors. To achieve this principle, we use a method similar to random Fourier features [10]. Each modulus, with value m i , is assigned a seed phasor vector, g i ∈ C D , whose elements ( g i ) j are drawn uniformly from the m i -th roots of unity (i.e., ( g i ) j = e √ -1 ω ij , with ω ij = 2 π m i k j , and k j chosen randomly from { 0 , ..., m i -1 } ). The representation of a particular residue value a i ∈ { 0 , . . . , m i -1 } is then given by rotating the phases of the seed vector according to [11]:

where we abuse notation slightly to also think of g i as a function that takes a i as input and produces an embedding as described above. The complex-valued vectors can be mapped to interpretable population vectors via a randomized Fourier transform (Figures 6D and S2).

Our third principle concerns the manner in which a unique representation of a particular point in space is formed from the individual residue representations. This requires that we somehow combine the residue vectors for each modulus. Combining via concatenation, though straightforward, is not effective because codes that coincide in subsets of their residue representation would be similar, even when the encoded values are very different. Thus, the method of combining residue codes must be conjunctive . Conjunctive composition is often called binding and is of fundamental importance in neuroscience [12], cognitive science [13], and machine learning [14]. An early proposal for binding is the tensor product of vector representations [15], with the tensor order equal to the number of bound objects.

Here, we implement binding with component-wise vector multiplication, a dimensionality preserving operation that represents a lossy compression of the full tensor product [16, 17]. The resulting compositional vector representation of an integer x ∈ Z using an RNS representation with K moduli, { a 1 , a 2 , .., a K } , is:

We prove in Appendix A.1 that this coding scheme represents distinct integer states using nearly orthogonal vectors, and we show that it generalizes in a natural way to support representation of arbitrary real numbers in a similarity preserving fashion.

Eq. 2 represents individual points along a line. In general, however, a spatial representation involves points in 2D or 3D spaces. Conveniently, vector binding can be also used to compose representations of multidimensional lattices from vectors representing individual dimensions. As we will explain, there is still a choice in this composition that determines the resulting lattice structure. Following earlier proposals [18-20], our fourth normative principle is to choose the lattice structure so that spatial information is maximized, as described in Section 3.5.

The final principle we require is that for computations such as path integration, there should be a simple vector manipulation that results in addition of the encoded variables. Again, vector binding provides this functionality with our coding strategy, because of the following property:

2.2 Modular attractor network for spatial representation

A standard model of grid cell circuits is the line attractor, in which states that represent a consistent location lie on a low-energy manifold [4]. When initialized from a noisy location pattern, the circuit dynamics will generate a denoised location representation. Rather than forming a line attractor model for the entire representational space (Eq. 2), we propose a modular network architecture, so that the compositional structure of a residue number representation can scale towards a large range with fewer memory resources (Section 3.2), in a manner robust to noise (Section 3.3).

A starting point for our attractor network model is the Hopfield network, which acts as an associative memory by storing memory patterns as fixed-point attractors. The Rademacher-Hopfield network [21] is a dynamical system whose state is a vector x ∈ {-1 , +1 } D that obeys the following dynamics:

with X as the matrix of memorized patterns (column vectors of X ). The fixed-point attractor dynamics can be generalized to complex memory patterns z ∈ C D :

where σ is a non-linearity normalizing the amplitude of each complex-valued component to one [22], and Z the corresponding matrix of memorized patterns. The model can also be discretized, such that each component is often quantized to a r -state phasor [23]. The Rademacher-Hopfield model is the special case where r = 2 and the phasors happen to be real-valued.

̸

An r -state phasor network of the form of Eq. 5 is well-suited to serve as an attractor network for each of the residue vectors in an RNS representation of position, with r = m i for modulus i , and the matrix Z (which we shall denote G i ) storing the g i ( a i ) for a i ∈ { 0 , .., m i -1 } . However, we desire a method for representing the whole coding range M := ∏ K i m i without storing all M patterns in one large associative memory. For this purpose we show that a resonator network , a recently proposed recurrent network for unbinding conjunctive codes [24-26], lets us represent this range by storing only n := ∑ K i m i ≪ M patterns. Given a vector encoding of position, p ( x ) , as formulated in Eq. (2), a resonator network will factorize it into its constituent RNS components by iteratively updating each residue vector estimate, ˆ g i . This update is similar to the attractor dynamics of Eq. (5) but made to be consistent with p ( x ) given all other residue estimates ˆ g j = i :

̸

Let us now assume that the input p ( x t ) encodes a spatial position x t using Eq. (2). Given a velocity input q i ( v t ) , estimated from self-motion input, path integration is performed by first running attractor dynamics, then updating attractor states by velocity.

̸

(MEC)

C

C

g

G

(HC)

p

G

q(v)

g

G

(LEC)

4C

G

After velocity updates, one can update the input state p ( x t ) with the conjunctive representation of the current factor estimates:

Further explanation and detail is provided in Appendix B.3.

2.3 Mapping the model to the HF

Although it is not obvious how the components of our normative model should map to the anatomical architecture of HF, we make one proposal as shown in Figure 1. The memory networks for residue representations g ˆ i correspond to grid modules in MEC. Similar to the grid modules, a module for context can be added to the architecture, such as a tag for the identity of a specific environment, with the recurrent synapses C storing tags of different environments.

The context neurons could correspond to the non-grid entorhinal cells, which can contain local, non-spatial information about the environment [27]. The vector p ( x t ) can be linked to place cells in hippocampus. Internal HC circuitry can either buffer the input as in Eq. (6) or allow it to be updated dynamically according to the MEC input (Section 4.1). The mutual interactions between HC and MEC grid modules require projections between these structures. The binding operations that these interactions involve according to Eq. (6) are hypothesized to be implemented by nonlinear interactions between dendritic inputs in HC and MEC neurons.

The model also assumes the ability for sensory cues to provide the initialization signal of the cognitive map, represented by s in Figure 1. For completeness, we adopt the assumption of previous models (e.g., [28]) that heteroassociative memories are formed by the brain that link sensory cues to the hippocampal representations p (Section 4.2). This process would require the system to generate a new context vector c and initialize the cognitive map to a default location in order to learn about new environments. We show that through even a simple heteroassociative mechanism, our modular attractor network can robustly retrieve sensory memories and even protect its compositional structure.

3 Coding properties of the model

3.1 RNS representations have exponential coding range

The compositional RNS vector representation Eq. (2) can encode a coding range of M values using a total of n component patterns for representing the residue of individual modules. The scaling of the coding range is exponential in the number of moduli, K , since if each module has O ( m ) patterns, and the co-prime condition is satisfied, the scaling of the coding range is O ( m K ) . This recovers the expressivity argued by [4, 29].

More generally, it is also exponential in the number of component patterns, n . The optimal coding range is given by the best partition of n into a set of positive { m i } . This optimization is identical to that of finding the maximum order of an element in the group of permutations S n , because the maximum order can be found by finding the longest cycle. The scaling of this value in n is characterized by Landau's function f ( n ) , which is known to converge to exp ( √ n ln n ) as n →∞ [30]. Figure 2A

A

B

C

10

15

12

K=2

K=2

coefficient(αk

2.00

E

range

10

12

二

K=4

range

e

10

K=4

1.75

10

9

K=5

log(Coding I

8

1.50

Coding！

10

6

6

Scaling

1.25

10

3

4

1.00

10

0

0

100

200

2

2.5

5.0

7.5

10.0

2

3

4

Patterns stored (n)

log(Vector dimension)

Number of modules (K)

illustrates how Landau's function is the upper bound to what is achievable for any fixed number of moduli ( K ).

Though other kinds of representations can achieve an exponential coding range, the advantage of the compositional encoding of Eq. (2) comes from the fact that the binding operation implements carry-free vector addition (our fourth principle). This enables updates of the encoded value without requiring further transformations such as decoding, facilitating tasks such as path integration (Section 4.1, Appendix C.3). Binary representations, by contrast, have exponential coding range but require carry-over operations to implement.

3.2 The modular attractor network has superlinear coding range

The exponential scaling of the coding range of the RNS representation is a prerequisite to obtain a large coding range with the attractor network that has to perform computations on this representation, such as input denoising, working memory, and path integration. To estimate the scaling of the coding range in the proposed attractor network (Eq. 6), we study the critical dimension for which the grid modules converge with high probability. Specifically, we empirically estimate the minimum dimension required to retrieve an arbitrary RNS representation with high probability, given a maximum number of iterations (Figure 2B). Remarkably, we find that the number of component patterns n that can be stored is superlinear in the pattern dimension D ; empirically O ( D α ) for some α ≥ 1 . For 2, 3, and 4 moduli, α ≈ 2 . 05 , 1 . 45 and 1 . 23 , respectively (Figure 2C).

These empirical scaling laws are consistent with a simple information-theoretic calculation (Appendix A.2). The minimal amount of bits to be stored for the entire RNS vector encoding scheme is of order O ( M log M ) , and the number of synapses in the attractor network is O ( D K √ M ) . If one makes the cautious assumption of a capacity per synapse of O (1) , the leading order for the coding range M is O ( D α ) , with α = K K -1 .

While the coding range increases with the number of moduli ( K ) for the RNS representation, the superlinear scaling coefficient α K decreases with K for the modular attractor network, reaching maximum superlinearity at the smallest value K = 2 . This reversal is caused by the fact that increasing K decreases the number of synapses, i.e., the memory resource in the attractor network.

3.3 Robust error correction

In addition, we evaluate the robustness of our attractor model to noise. Because the RNS representations are composed of phasors, which are circular variables, we sample noise from a von Mises distribution with two parameters: mean ( µ = 0 ) and concentration pattern κ (Figure 3A). Higher κ values imply less noise; the distribution approximates a Gaussian with variance 1 /κ for large κ . Further tests of model robustness to dropout, limited precision, and ablation are provided in Figure S6.

A

B

Inputnoise

D

Codebooknoise

1.0

1.0

1.0

1.0

K=4

/density

0.8

K=2

0.8

0.8

0.8

Probability

0.6

0.6

0.6

Accurac

0.6

0.4

0.4

Noiseless

0.4

0.4

0.2

0.2

0.2

0.2

K = 4

0.0

0.0

K =2

0.0

0.0

LI-

-T/2

Phase shift

0

TT/2

TT

Codingrange

103

10°

5

Coding range

103

Codingrange

105

We consider three cases: noisy input patterns, noise added to each time step, and noisy weights corruptions of patterns in G i (Appendix B.2). The empirical accuracy of recall varies depending on the type of corruption applied (Figure 3A). We find that for a given dimension D (in this case, 1024 ), increasing noise decreases the maximum coding range that can be decoded with high accuracy (Figure 3B-D). For a fixed noise level, the high-accuracy coding range is largest for input noise, followed by update noise and codebook noise. It is perhaps not surprising that codebook noise has the worst coding range, given that noise added to every stored pattern compounds across the dynamics. Fortunately, the demonstrated robustness to input noise enables sensory patterns to be denoised via heteroassociation (Section 4.2).

3.4 Interpolation between patterns enables continuous path integration

In general, there is a sharp difference between point and line attractors. In our attractor model, the RNS representations of integer values are stored as discrete fixed points. Nevertheless, the attractor network also converges to states that represent non-integer values that are not explicitly stored. In other words, the network smoothly interpolates to points on a manifold of states that represent integer and non-integer values encoded by (2). Figure 4A provides a visualization, showing that the kernel induced by inner product operations retains graded similarity for sub-integer shifts. This kernel enables the modular attractor network to settle to fixed points that correspond to interpolations between integers, and for sub-integer positions to be decoded.

The resolution of decoding is fundamentally limited by the signal to noise ratio. Even so, we find that, up to a fixed noise level, the accuracy regimes of integer decoding and sub-integer decoding coincide. This property enables sub-integer position shifts to be encoded within the states of the network, which, as we will show, results in stable, error-correcting path integration (Section 4.1). We quantify the gain in precision in terms of the bits of information that can on average be reconstructed from a vector (Figure 4D, Appendix B.2). Notably, even a moderate noise level of κ = 8 results in nearly the same information content as in the noiseless case.

3.5 Triangular frames in 2D maximize spatial information

In two-dimensional open field environments, grid cells have firing fields arranged in a hexagonal lattice [31]. Work in theoretical neuroscience shows the optimality of this lattice for 2D environments in terms of spatial information [18-20]. However, the presence of hexagonal firing fields raises a puzzle for residue number systems. Although a crucial property of a RNS is the carry-free property, most implementations of RNS will not perform carry-free updates within a module in non-Cartesian coordinate systems. This generally occurs because the updates of different coordinates must interact due to non-orthogonality.

A

B

K = 4.0

C

K = 2.0

D

1.0

1.0

1.0

20

productl

0.8

0.8

0.8

per vector

15

0.6

0.6

0.6

△x=0.03

10

|Inner

0.4

0.4

△x = 0.07

0.4

二

Noiseless

△x = 0.14

Bits

0.2

0.2

△x=0.33

0.2

5

K=4

△x = 1.0

K=2

0.0

0.0

0.0

0

0

2

4

10°

3

10°

3

10

10

3

10°

5

Modulovalue

Codingrange

Codingrange

Codingrange

Weresolve this issue by showing how to implement a version of vector binding of multiple coordinates in a triangular 'Mercedes-Benz' frame that enables carry-free hexagonal coding. Furthermore, we provide a combinatoric argument for the optimality of triangular frames for R 2 . (A frame is a spanning set for a vector space in which the basis vectors need not be linearly independent.) Our argument relies on the combinatorics of residue numbers, and so for the first time gives an explanation of why the coexistence of RNS and hexagonal codes is optimal.

To form a hexagonal tiling of 2D position requires two steps: first, projection into a 3 -coordinate frame, and second, choosing phases such that simultaneous, equal movements along all three frames cancel out (Appendix A.3). The resulting Voronoi tessellation for different states is pictured in Figure 5A. This encoding enables higher spatial resolution in terms of the number of discrete states: 3 m 2 -3 m +1 for triangular frames, versus m 2 for Cartesian frames. This increased expressivity results in a higher entropy) code for space (Figure 5B). It also results in both a periodic hexagonal kernel and the individual grid response fields being arranged in a hexagonal lattice (Figure 6C).

A

B

11

10

Entropy (bits)

6

8

hexagonal

5

square

oblique

3

10

15

20

25

Modulus

Prior models achieved hexagonal lattices either by pattern formation from circularly symmetric receptive fields (e.g., [32, 33]) arranged on a periodic rectangular sheet or by distorting a square lattice into an oblique one (e.g., [28, 34]). Importantly, oblique lattices have the combinatorial complexity as the square grid and, unlike the construction described above, they do not achieve the same level of spatial resolution (Figure 5B).

4 Testing functionalities of the model

4.1 Robust path integration

A

Horizontal position (cm)

B

35

Truetrajectory

Total distance

20

Decoded position with attractor dynamics

30

Error - no error correction

Decodedposition without attractordynamics

Distance(cm)

Error-grid cell error correction

16

120

Verticalposition

D

2

Time (s)

35

1/

D

Horizontal position(cm)

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Time (s)

Verticalposition

True trajectory

Decoded position (noisy input)

Sensory input

0

0°0

1.0

1.5

2.0

2.5

3.0

()

Given the ability of the attractor model to update its representation of position from velocity inputs, along with its ability to represent continuous space, we evaluate its ability to perform path integration in the presence of noise. We simulate trajectories based on a statistical model for generating plausible rodent movements in an arena [35, 36], and we update grid cell and place cell state vectors according to Equations 7 and 8, respectively.

To evaluate the robustness of the model to error (Appendix B.3), we consider both sources of extrinsic noise (e.g., mis-representations of velocity information), and intrinsic noise (e.g., due to noise in weight updates). The robustness of our model to intrinsic noise is tested by comparing our results to the estimated trajectories obtained without the correction by the MEC modules (Figure 6A and B). We find that our model strongly limits noise accumulation along the trajectory and allows highly accurate integration for a longer period of time (Figure 6A). Consistent with our previous experiments on noise robustness (Figure 3), we find that the model has strong robustness to intrinsic noise, with extrinsic noise resulting in a drift of the estimated position.

We visualize the response fields in different modules and find hexagonal lattices with a module dependent scaling (Figure 6C, Appendix 4.1). In addition, we show that tethering to external cues (e.g., visual inputs), can significantly increase the accuracy of the attractor network. To study this, we associate visual cues to corresponding patches see Section 4.2) and observe that integration of information from sensory visual inputs succeeds in correcting drift due to extrinsic noise (Figure 6D).

4.2 Denoising sensory states via a heteroassociative memory

Finally, we describe a simple extension to our model, in which sensory patterns are fed from the lateral entorhinal cortex (LEC) to update the hippocampal state. This is consistent with theories of

A

60 patterns

B

210 patterns

C

1.0

1.0

1.0

(solid)

0.8

(solid)

0.8

0.8

0.6

0.6

0.6

racy

Accur

0.4

D=256

Accurac

0.4

0.4

二

1pattern

2patterns

D=512

3patterns

0.2

二

D=1024

0.2

0.2

4patterns

5patterns

D=2048

0.0

0.0

0.0

10 patterns

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

0.0

0.1

0.2

0.3

0.4

0.5

Fraction of bits flipped

Fraction of bits flipped

Fraction of bits flipped

memory suggesting that LEC provides the content of experiences to hippocampus [37], as well as with neuroanatomical evidence [38]. Although the structure of the representations of those sensory patterns is unknown, it is theorized that HF is critical to sensory pattern completion [39].

Consistent with this function, recent work [28, 40] has proposed that a heteroassociative scaffold connects sensory patterns to hippocampal activity, allowing robust denoising of sensory states. Though the main focus of our normative model is not sensory denoising, we show that a simple extension to our model (Appendix B.4) robustly retrieves noisy pattern even under high levels of corruption (Figures 7A and B). In Appendix C.3, we also discuss how this capacity for generalization can serve as a model for sequence retrieval and show some preliminary experiments.

In addition to robust denoising of single patterns, our model is also well-equipped to deal with compositions of sensory patterns. Two situations are worth emphasizing: first, we can often unmix multiple sensory states corresponding to a sum of patterns, because the compositional structure of binding between grid modules protects the items in summation (Figure 7C). This differentiates our model from other heteroassociative memories, in which sums of patterns would have multiple equally valid yet incompatible decodings. Second, the context vector module can separate the sensory information corresponding to different environments (Figure S3).

5 Discussion

There are by now numerous theories of the entorhinal cortex and hippocampus, including those that draw upon attractor dynamics and residue number systems. What this paper contributes to the existing body of work is a concrete set of design principles that can be brought together to build a functioning neural system capable of representing space and performing path integration, making the most use of limited neural resources and precision. In particular, a core design principle of this model is a compositional representation of space that achieves a superlinear coding range, which is achieved by a compact, multi-module attractor network. The compositional representation, in turn, is achieved via a vector binding operation, which enables binding multiple scales (moduli) and spatial dimensions, context, and spatial shifts for path integration. This binding mechanism builds on prior work in the field of hyperdimensional computing and vector symbolic architectures [11, 17, 26, 41-43] - and goes beyond it to develop a specific algorithmic hypothesis about structured operations in HF. Our analyses and experiments confirm that the model can achieve important functions of the hippocampal formation and they explain experimental observations, such as hexagonal grid cells, place cells, and remapping phenomena. The model thus contributes to, and greatly benefits from, existing work in theoretical neuroscience on residue number systems [4, 5], continuous attractor network models

of grid cells [4, 32, 44], models of compositionality in the hippocampal formation [45], and the optimality of hexagonal representations in 2D [18, 29].

That biology organized grid cells into multiple discrete modules, rather than pooling all resources into a single module attractor network, has posed a long-standing puzzle to theoreticians: What advantages are conferred by this organization? Our answer is that it provides exponential scaling in dynamic range by combining modules with limited dynamic range multiplicatively . Other recent work has focused on the problem of coordinating representations across multiple modules [28, 34, 46-48], and large scale recordings of the hippocampal formation [49] may provide new opportunities to evaluate their resulting predictions.

Our approach starts from principles of space encoding, in particular, the requirement of compositionality. This strategy is complementary to investigations of the emergence of place and grid cells in artificial neural networks (e.g., [36, 50-57]). These approaches can show the optimality of biological response features under the model assumptions, such as ANN properties, network architecture, training objective and protocol. Here, we emphasize the role of multiplicative binding, a computational primitive that is typically difficult to have emerge in an ANN setting. Early suggestions for realizing conjunctive binding already ventured outside the framework of ANNs [11, 15]. A simple extension of ANNs are sigma-pi neurons [58, 59] that can implement vector binding [60]. Recent work amplifies the view that full conjunctive binding would be a useful inductive bias to augment deep learning architectures [61], and various augmentations of ANNs with dedicated binding mechanisms have been proposed [14, 62, 63].

Our model has clear limitations. The attractor neural network for the cognitive map is still a high-level abstraction of spiking neural circuits in the hippocampal formation. In particular, the phasor states in the model are one linear transform removed from vectors that describe neural population activity. Thus, the mapping between model and neurobiological mechanisms requires an additional step. This disadvantage can be directly addressed by switching to other encoding schemes, such as sparse real- or complex-valued vectors, e.g., [64], for which conjunctive binding operations have been proposed [65, 66]. Although the model is more comprehensive than typical normative models, which usually focus on a single computation, it is far from covering the many other functional cell types observed in the hippocampal formation or contextual modulations observed during remapping. In addition, the current model includes learning only in the heteroassociative projection to LEC. Most observations regarding plasticity in HF are not captured, i.e., signals from reward, or eligibility traces. Finally, our assumptions about inputs to HF from the sensory pathway are simplifying and primarily intended as a proof of concept.

The purpose of the model, to express the fundamental principles of a compositional cognitive map, also leads to testable predictions: First, at the biophysical level, the model predicts multiplicative interactions between dendritic inputs providing the conjunctive binding operation. There are several biophysically realistic ways in which neurons can multiply their inputs [67, 68]. Contextual gating in dendritic branches of hippocampal neurons is consistent with our theory, hippocampal remapping, and neurophysiology of hippocampal dendrites [27]. Our attractor model predicts direct multiplicative interactions between MEC modules, which remains to be tested. Second, the model predicts relatively fixed attractor weights between place and grid cells, with a higher degree of plasticity for the weights between sensory encodings and hippocampal states. Third, we predict that causal perturbations of one grid module can affect the states of other grid modules without involvement of the hippocampus, in a direction that is self-consistent with the update of the attractor state.

Finally, we believe that the proposed modeling approach and the specific attractor model presented have broader applications in neuroscience. For example, the problem of factorization is critical to forming compositional representations of visual scenes, and a closely related attractor neural network can find efficient solutions to such problems [69]. In addition, there are promising ways to map complex-valued attractor neural networks to spiking neural networks [70, 71], which could connect the principles derived here to a concrete implementation on neuromorphic hardware. Such a neuromorphic implementation could yield further quantitative predictions for neuroscience and is an exciting direction for future work.

Acknowledgments

The work of CJK was supported by the Department of Defense (DoD) through the National Defense Science & Engineering Graduate (NDSEG) Fellowship Program. The work of SM was carried out as part of the ARPE program of ENS Paris-Saclay and supported by the European Union's Horizon 2020 Framework Programme for Research and Innovation under the Specific Grant Agreement HORIZONINFRA-2022-SERV-B-01. The work of DK and BAO was supported in part by Intel's THWAI program. The work of CJK and BAO was supported by the Center for the Co-Design of Cognitive Systems (CoCoSys), one of seven centers in JUMP 2.0, a Semiconductor Research Corporation (SRC) program sponsored by DARPA, as well as NSF awards 2147640 and 2313149. DK has received funding from the European Union's Horizon 2020 research and innovation programme under the Marie Sklodowska-Curie grant agreement No 839179. FTS discloses support for the research of this work from NIH grant 1R01EB026955-0.

References

H. Eichenbaum, 'On the integration of space, time, and memory,' Neuron , vol. 95, no. 5, pp. 1007-1018, 2017.

E. I. Moser, M.-B. Moser, and B. L. McNaughton, 'Spatial representation in the hippocampal formation: A history,' Nature Neuroscience , vol. 20, no. 11, pp. 1448-1464, 2017.

Z. Kurth-Nelson et al. , 'Replay and compositional computation,' Neuron , vol. 111, no. 4, pp. 454-469, 2023.

I. R. Fiete, Y . Burak, and T. Brookings, 'What grid cells convey about rat location,' Journal of Neuroscience , vol. 28, no. 27, pp. 6858-6871, 2008.

S. Sreenivasan and I. Fiete, 'Grid cells generate an analog error-correcting code for singularly precise neural computation,' Nature Neuroscience , vol. 14, no. 10, pp. 1330-1337, 2011.

T. E. Behrens et al. , 'What is a cognitive map? Organizing knowledge for flexible behavior,' Neuron , vol. 100, no. 2, pp. 490-509, 2018.

H. L. Garner, 'The residue number system,' in Western Joint Computer Conference (WJCC) , 1959, pp. 146-153.

O. Goldreich, D. Ron, and M. Sudan, 'Chinese remaindering with errors,' in Annual ACM symposium on Theory of Computing (STOC) , 1999, pp. 225-234.

H. Stensola, T. Stensola, T. Solstad, K. Frøland, M.-B. Moser, and E. I. Moser, 'The entorhinal grid map is discretized,' Nature , vol. 492, no. 7427, pp. 72-78, 2012.

A. Rahimi and B. Recht, 'Random features for large-scale kernel machines,' in Advances in Neural Information Processing Systems (NeurIPS) , vol. 20, 2007.

T. A. Plate, 'Holographic recurrent networks,' in Advances in Neural Information Processing Systems (NeurIPS) , vol. 5, 1992.

C. von der Malsburg, 'Am I thinking assemblies?' In Brain Theory , 1986, pp. 161-176.

J. A. Fodor and Z. W. Pylyshyn, 'Connectionism and cognitive architecture: A critical analysis,' Cognition , vol. 28, no. 1-2, pp. 3-71, 1988.

K. Greff, S. Van Steenkiste, and J. Schmidhuber, 'On the binding problem in artificial neural networks,' arXiv:2012.05208 , 2020.

P. Smolensky, 'Tensor product variable binding and the representation of symbolic structures in connectionist systems,' Artificial Intelligence , vol. 46, pp. 159-216, 1990.

T. Plate et al. , 'Holographic reduced representations: Convolution algebra for compositional distributed representations.,' in International Joint Conference on Artificial Intelligence (IJCAI) , 1991, pp. 30-35.

P. Kanerva, 'Hyperdimensional computing: An introduction to computing in distributed representation with high-dimensional random vectors,' Cognitive Computation , vol. 1, pp. 139159, 2009.

A. Mathis, M. B. Stemmler, and A. V. Herz, 'Probable nature of higher-dimensional symmetries underlying mammalian grid-cell activity patterns,' Elife , vol. 4, e05979, 2015.

X.-X. Wei, J. Prentice, and V. Balasubramanian, 'A principle of economy predicts the functional architecture of grid cells,' Elife , vol. 4, e08362, 2015.

F. Anselmi, M. M. Murray, and B. Franceschiello, 'A computational model for grid maps in neural populations,' Journal of Computational Neuroscience , vol. 48, pp. 149-159, 2020.

J. J. Hopfield, 'Neural networks and physical systems with emergent collective computational abilities.,' Proceedings of the National Academy of Sciences , vol. 79, no. 8, pp. 2554-2558, 1982.

A. Noest, 'Phasor neural networks,' in Advances in Neural Information Processing Systems (NeurIPS) , 1987, pp. 584-591.

A. J. Noest, 'Discrete-state phasor neural networks,' Physical Review A , vol. 38, no. 4, p. 2196, 1988.

E. P. Frady, S. J. Kent, B. A. Olshausen, and F. T. Sommer, 'Resonator networks, 1: An efficient solution for factoring high-dimensional, distributed representations of data structures,' Neural Computation , vol. 32, no. 12, pp. 2311-2331, 2020.

S. J. Kent, E. P. Frady, F. T. Sommer, and B. A. Olshausen, 'Resonator networks, 2: Factorization performance and capacity compared to optimization-based methods,' Neural Computation , vol. 32, no. 12, pp. 2332-2388, 2020.

C. J. Kymn et al. , 'Computing with residue numbers in high-dimensional representation,' arXiv 2311.04872 , 2023.

P. Latuske, O. Kornienko, L. Kohler, and K. Allen, 'Hippocampal remapping and its entorhinal origin,' Frontiers in Behavioral Neuroscience , vol. 11, p. 253, 2018.

S. Chandra, S. Sharma, R. Chaudhuri, and I. Fiete, 'High-capacity flexible hippocampal associative and episodic memory enabled by prestructured 'spatial' representations,' bioRxiv , 2023.

A. Mathis, A. V. Herz, and M. B. Stemmler, 'Resolution of nested neuronal representations can be exponential in the number of neurons,' Physical Review Letters , vol. 109, no. 1, p. 018 103, 2012.

E. Landau, 'Über die maximalordnung der permutationen gegebenen grades,' Archiv der Mathematik und Physik , vol. 3, pp. 92-103, 1903.

T. Hafting, M. Fyhn, S. Molden, M.-B. Moser, and E. I. Moser, 'Microstructure of a spatial map in the entorhinal cortex,' Nature , vol. 436, no. 7052, pp. 801-806, 2005.

M. C. Fuhs and D. S. Touretzky, 'A spin glass model of path integration in rat medial entorhinal cortex,' Journal of Neuroscience , vol. 26, no. 16, pp. 4266-4276, 2006.

Y. Burak and I. R. Fiete, 'Accurate path integration in continuous attractor network models of grid cells,' PLoS Computational Biology , vol. 5, no. 2, e1000291, 2009.

N. Mosheiff and Y. Burak, 'Velocity coupling of grid cell modules enables stable embedding of a low dimensional variable in a high dimensional neural attractor,' Elife , vol. 8, e48494, 2019.

F. Raudies and M. E. Hasselmo, 'Modeling boundary vector cell firing given optic flow as a cue,' PLoS Computational Biology , vol. 8, no. 6, e1002553, 2012.

A. Banino et al. , 'Vector-based navigation using grid-like representations in artificial agents,' Nature , vol. 557, no. 7705, pp. 429-433, 2018.

J. R. Manns and H. Eichenbaum, 'Evolution of declarative memory,' Hippocampus , vol. 16, no. 9, pp. 795-808, 2006.

J. J. Knierim, J. P. Neunuebel, and S. S. Deshmukh, 'Functional correlates of the lateral and medial entorhinal cortex: Objects, path integration and local-global reference frames,' Philosophical Transactions of the Royal Society B: Biological Sciences , vol. 369, no. 1635, p. 20 130 369, 2014.

T. J. Teyler and P. DiScenna, 'The hippocampal memory indexing theory,' Behavioral Neuroscience , vol. 100, no. 2, pp. 147-154, 1986.

S. Sharma, S. Chandra, and I. Fiete, 'Content addressable memory without catastrophic forgetting by heteroassociation with a fixed scaffold,' in International Conference on Machine Learning (ICML) , 2022, pp. 19 658-19 682.

R. W. Gayler, 'Vector symbolic architectures answer Jackendoff's challenges for cognitive neuroscience,' in Joint International Conference on Cognitive Science (ICCS/ASCS) , 2003, pp. 133-138.

D. Kleyko, D. Rachkovskij, E. Osipov, and A. Rahimi, 'A survey on hyperdimensional computing aka vector symbolic architectures, part ii: Applications, cognitive models, and challenges,' ACM Computing Surveys , vol. 55, no. 9, pp. 1-52, 2023.

N. S.-Y. Dumont, J. Orchard, and C. Eliasmith, 'A model of path integration that connects neural and symbolic representation,' in Proceedings of the Annual Meeting of the Cognitive Science Society , vol. 44, 2022.

K. Zhang, 'Representation of spatial orientation by the intrinsic dynamics of the head-direction cell ensemble: A theory,' Journal of Neuroscience , vol. 16, no. 6, pp. 2112-2126, 1996.

D. C. McNamee, K. L. Stachenfeld, M. M. Botvinick, and S. J. Gershman, 'Flexible modulation of sequence generation in the entorhinal-hippocampal system,' Nature Neuroscience , vol. 24, no. 6, pp. 851-862, 2021.

N. Mosheiff, H. Agmon, A. Moriel, and Y. Burak, 'An efficient coding theory for a dynamic trajectory predicts non-uniform allocation of entorhinal grid cells to modules,' PLoS Computational Biology , vol. 13, no. 6, e1005597, 2017.

L. Kang and V. Balasubramanian, 'A geometric attractor mechanism for self-organization of entorhinal grid modules,' Elife , vol. 8, e46687, 2019.

H. Agmon and Y. Burak, 'A theory of joint attractor dynamics in the hippocampus and the entorhinal cortex accounts for artificial remapping and grid cell field-to-field variability,' Elife , vol. 9, e56894, 2020.

T. Waaga et al. , 'Grid-cell modules remain coordinated when neural activity is dissociated from external sensory cues,' Neuron , vol. 110, no. 11, pp. 1843-1856, 2022.

C. J. Cueva and X.-X. Wei, 'Emergence of grid-like representations by training recurrent neural networks to perform spatial localization,' in International Conference on Learning Representations (ICLR) , 2018, pp. 1-19.

J. C. Whittington, W. Dorrell, S. Ganguli, and T. Behrens, 'Disentanglement with biological constraints: A theory of functional cell types,' in International Conference on Learning Representations (ICLR) , 2022.

W. Dorrell, P. E. Latham, T. E. Behrens, and J. C. Whittington, 'Actionable neural representations: Grid cells from minimal constraints,' in International Conference on Learning Representations (ICLR) , 2023.

B. Sorscher, G. C. Mel, S. A. Ocko, L. M. Giocomo, and S. Ganguli, 'A unified theory for the computational and mechanistic origins of grid cells,' Neuron , vol. 111, no. 1, pp. 121-137, 2023.

R. Schaeffer, M. Khona, T. Ma, C. Eyzaguirre, S. Koyejo, and I. Fiete, 'Self-supervised learning of representations for space generates multi-modular grid cells,' in Advances in Neural Information Processing Systems (NeurIPS) , vol. 36, 2023.

K. L. Stachenfeld, M. M. Botvinick, and S. J. Gershman, 'The hippocampus as a predictive map,' Nature Neuroscience , vol. 20, no. 11, pp. 1643-1653, 2017.

J. C. Whittington et al. , 'The Tolman-Eichenbaum machine: Unifying space and relational memory through generalization in the hippocampal formation,' Cell , vol. 183, no. 5, pp. 12491263, 2020.

Y. Chen, H. Zhang, M. Cameron, and T. Sejnowski, 'Predictive sequence learning in the hippocampal formation,' bioRxiv , 2022.

J. A. Feldman and D. H. Ballard, 'Connectionist models and their properties,' Cognitive Science , vol. 6, no. 3, pp. 205-254, 1982.

B. W. Mel and C. Koch, 'Sigma-Pi learning: On radial basis functions and cortical associative learning,' in Advances in Neural Information Processing Systems (NeurIPS) , 1989, pp. 474481.

T. A. Plate, 'Randomly connected Sigma-Pi neurons can form associator networks,' Network: Computation in Neural Systems , vol. 11, no. 4, p. 321, 2000.

A. Goyal and Y. Bengio, 'Inductive biases for deep learning of higher-level cognition,' Proceedings of the Royal Society A , vol. 478, no. 2266, 2022.

I. Danihelka, G. Wayne, B. Uria, N. Kalchbrenner, and A. Graves, 'Associative long short-term memory,' in International Conference on Machine Learning (ICML) , 2016, pp. 1986-1994.

A. Ganesan et al. , 'Learning with holographic reduced representations,' in Advances in Neural Information Processing Systems (NeurIPS) , vol. 34, 2021, pp. 25 606-25 620.

M. Laiho, J. H. Poikonen, P. Kanerva, and E. Lehtonen, 'High-dimensional computing with sparse vectors,' in 2015 IEEE Biomedical Circuits and Systems Conference (BioCAS) , IEEE, 2015, pp. 1-4.

D. A. Rachkovskij and E. M. Kussul, 'Binding and normalization of binary sparse distributed representations by context-dependent thinning,' Neural Computation , vol. 13, no. 2, pp. 411452, 2001.

E. P. Frady, D. Kleyko, and F. T. Sommer, 'Variable binding for sparse distributed representations: Theory and applications,' IEEE Transactions on Neural Networks and Learning Systems , vol. 34, no. 5, pp. 2191-2204, 2023.

C. Koch, Biophysics of computation: information processing in single neurons . Oxford university press, 2004.

L. N. Groschner, J. G. Malis, B. Zuidinga, and A. Borst, 'A biophysical account of multiplication by a single neuron,' Nature , vol. 603, no. 7899, pp. 119-123, 2022.

C. J. Kymn, S. Mazelet, A. Ng, D. Kleyko, and B. A. Olshausen, 'Compositional factorization of visual scenes with convolutional sparse coding and resonator networks,' in 2024 Neuro Inspired Computational Elements Conference (NICE) , IEEE, 2024, pp. 1-9.

E. P. Frady and F. T. Sommer, 'Robust computation with rhythmic spike patterns,' Proceedings of the National Academy of Sciences , vol. 116, no. 36, pp. 18 050-18 059, 2019.

J. Orchard, P. M. Furlong, and K. Simone, 'Efficient hyperdimensional computing with spiking phasors,' Neural Computation , vol. 36, no. 9, pp. 1886-1911, 2024.

T. A. Plate, Holographic Reduced Representation: Distributed representation for cognitive structures . CSLI Publications Stanford, 2003, vol. 150.

A. Thomas, S. Dasgupta, and T. Rosing, 'A theoretical perspective on hyperdimensional computing,' Journal of Artificial Intelligence Research , vol. 72, pp. 215-249, 2021.

E. P. Frady, D. Kleyko, C. J. Kymn, B. A. Olshausen, and F. T. Sommer, 'Computing on functions using randomized vector representations (in brief),' in Annual Neuro-Inspired Computational Elements Conference (NICE) , 2022, pp. 115-122.

K. L. Clarkson, S. Ubaru, and E. Yang, 'Capacity analysis of vector symbolic architectures,' arXiv preprint arXiv:2301.10352 , 2023.

J. O. Smith, Spectral Audio Signal Processing . 2011.

B. Komer and C. Eliasmith, 'Efficient navigation using a scalable, biologically inspired spatial representation,' in Annual Meeting of the Cognitive Science Society (CogSci) , 2020, pp. 15321538.

B. Komer, 'Biologically inspired spatial representation,' Ph.D. dissertation, University of Waterloo, 2020.

E. P. Frady, D. Kleyko, and F. T. Sommer, 'A theory of sequence indexing and working memory in recurrent neural networks,' Neural Computation , vol. 30, no. 6, pp. 1449-1513, 2018.

D. Kleyko et al. , 'Efficient decoding of compositional structure in holistic representations,' Neural Computation , vol. 35, no. 7, pp. 1159-1186, 2023.

T. J. Wills, C. Lever, F. Cacucci, N. Burgess, and J. O'Keefe, 'Attractor dynamics in the hippocampal representation of the local environment,' Science , vol. 308, no. 5723, pp. 873876, 2005.

L. Thompson and P. Best, 'Place cells and silent cells in the hippocampus of freely-behaving rats,' Journal of Neuroscience , vol. 9, no. 7, pp. 2382-2390, 1989.

A. O. Constantinescu, J. X. O'Reilly, and T. E. Behrens, 'Organizing conceptual knowledge in humans with a gridlike code,' Science , vol. 352, no. 6292, pp. 1464-1468, 2016.

J. L. Bellmund, P. Gärdenfors, E. I. Moser, and C. F. Doeller, 'Navigating cognition: Spatial codes for human thinking,' Science , vol. 362, no. 6415, 2018.

M. I. Schlesiger et al. , 'The medial entorhinal cortex is necessary for temporal organization of hippocampal neuronal activity,' Nature Neuroscience , vol. 18, no. 8, pp. 1123-1132, 2015.

J. Yamamoto and S. Tonegawa, 'Direct medial entorhinal cortex input to hippocampal CA1 is crucial for extended quiet awake replay,' Neuron , vol. 96, no. 1, pp. 217-227, 2017.

X. Li and P. Li, 'Quantization algorithms for random fourier features,' in International Conference on Machine Learning , PMLR, 2021, pp. 6369-6380.

Supplemental material

A Mathematical derivations

A.1 Similarity-preserving properties of embeddings

In the following section, we examine the similarity-preserving properties of our coding scheme. Recall from Section 2.1 that our crucial desiderata are that: (1) distinct residue values are represented using vectors which are nearly orthogonal, and that (2) the inner product between representations of sub-integer values are reflective of a reasonable notion of similarity between the encoded values. There is a robust literature on this topic both within the Vector Symbolic Architectures community [72-75] and the broader ML community [10], who often study these techniques under the name 'random features.' The methods pursued here fall under these traditions.

To briefly recapitulate the construction of Equation 1: fix some positive integer m , and let P ( k ) denote the uniform distribution over { 0 , ..., m -1 } . Define an embedding g : R → C D using the following procedure: draw k 1 , ..., k D independently from P ( k ) , and set:

where ω = 2 π/m , and i = √ -1 . To simplify analysis, we here assume that m is odd, in which case the above is equivalent to shifting the support of P ( k ) to {-( m -1) / 2 , ..., ( m -1) / 2 } , and defining the embedding g : R → C D component-wise via:

The case that m is even is slightly different, but can be handled using similar techniques and the discrepancy does not affect any of our modeling goals.

Our basic claim is that in expectation with respect to randomness in the draw of k 1 , ..., k D , innerproducts between the embeddings of two numbers a, a ′ recover the periodic sinc-function [76] of their difference. That is:

This accomplishes goal (1) because, for t an integer which is not an integer multiple of m , psinc(t) = 0 . Therefore, distinct integers are represented using vectors which are, in expectation, orthogonal. It also accomplishes goal (2), because psinc(t) ≈ 1 for 0 < | t |≪ 1 . The following theorem demonstrates this property more formally, and provides an approximation guarantee for a specific instantiation of k 1 , ..., k D .

Theorem 1. Fix any D > 0 and δ ∈ (0 , 1) . For any pair a, a ′ ∈ R such that a -a ′ is not an integer multiple of m , with probability at least 1 -δ over randomness in the draw of k 1 , ..., k D :

Proof. Fix any pair a, a ′ ∈ R , and denote for concision t = a -a ′ . Taking an expectation with respect to randomness in k 1 , ..., k D and using a well-known calculation from the signal processing

literature [76]:

The third equality follows from the second by noting that the latter is a sum of a geometric series with common ratio r = exp( ωt ) . The fifth line follows from the fourth by recalling the identity sin( x ) = ( e ix -e -ix ) / 2 i . In the limit of t → 0 , the expression evaluates to 1 , consistent with the normalized inner product of a vector with itself.

To show concentration around this value, consider:

and note that since the complex part of the sum vanishes in expectation, we may consider, without loss of generality, the average of the real-valued quantities: (cos( ωk j ( a -a ′ ))) D j =1 , which are bounded in the range ± 1 . Therefore, by Hoeffding's inequality:

whereupon we conclude that, with probability at least 1 -δ over randomness in the draw of k 1 , ..., k D :

as claimed.

This result can be readily extended to the binding of multiple residue number values. Let g ( a ) = ⊙ K i =1 g i ( a ) , where each g i ( a ) is instantiated independently. Then, by independence, we observe that:

The implication is that E [ g ( a ) ⊤ g ( a ′ ) ∗ ] = 1 if and only if all residue values agree, and zero otherwise. To show concentration around this value, we can again use Hoeffding's inequality, which recovers the same bound on the sufficient dimension.

A.2 Information-theoretic estimate of required pattern dimension

In this section, we describe an information-theoretic estimate on the dimension D necessary to retrieve n patterns within K modules. The main result we aim to show is that D = O ( n ( K -1) /K ) ;

equivalently, the scaling of n for a given D is O ( D K/ ( K -1) ) . This scaling roughly predicts our empirical results of finding the dimension required to achieve high accuracy, suggesting that the attractor network described here performs close to the theoretical bound.

The minimal total amount of information a network needs to store for denoising an RNS representation with coding range M is O ( M log ( M )) . This results from the requirement of content addressability, i.e., for serving as a unique pointer to one of n patterns, each pattern must at least carry information of the order of O ( log ( M )) . For simplicity, we now assume that each module is of size O ( M 1 /K ) . The total capacity of the network is bounded by the number of synapses, which is O ( D ∗ K ∗ M 1 /K ) = O ( D ∗ M 1 /K ) (assuming K is constant), times the capacity per synapse. Under the conservative assumption that the capacity per synapse is O (1) , the dimension is of order O ( e K -1 K log ( M )+log (log ( M )) ) . Thus, the leading order of how D depends on n is O ( M ( K -1) /K ) . If the capacity per synapse is assumed to be larger, O (log ( M )) bits, only the non-leading term cancels and the resulting order of D is still the same.

A.3 Construction of triangular frames

In order to convert a 2 D coordinate x into a 3 D frame y , we first multiply it by a matrix, Ψ whose rows are the elements of a 3 D equiangular frame:

(This particular frame is commonly referred to as a 'Mercedes Benz' frame due to its resemblance to the iconic symbol.) A consequence of working with an overcomplete frame is that there may exist multiple values of y that correspond to the same x . For this frame, the null space of Ψ + is the subspace spanned by [1 , 1 , 1] ⊺ - grounding the intuition that equal movement in all equiangular directions 'cancels out.' It therefore might seem that triangular frames require extra operations to determine if two coordinates are equal, but here we show how to avoid this consequence.

The core strategy is to choose seed vectors g i, 1 , g i, 2 , g i, 3 for each modulus m i that implement this self-cancellation. For a modulus m i , we draw the phasors of seed vectors from the m -th roots of unity. However, we further require that, for each vector component, the three selected phases sum to 0 ( mod 2 π ) . We then form a hexagonal coordinate vector by binding the three seed vectors:

By enforcing that the phases sum to 0 ( mod 2 π ) , we ensure that positions that have an equivalent x coordinate are mapped to the same g i . Observe that Hadamard product binding of phasors is equivalent to summing their phases, and that binding e 0 i corresponds to adding nothing. Hence, a pair of three-dimensional coordinates whose differences are a multiple of [1 , 1 , 1] will be mapped to equivalent vector representations. Finally, we then form the residue number representation for different moduli by binding, as in Eq. 2. The presence of multiple modules and self-cancellation properties complement prior work on the efficiency of hexagonal kernels for spatial navigation tasks [77, 78].

The equivalence of certain 3D coordinates also helps us count the number of states. Clearly, the redundancy means that we have less than m 3 states, but it also shows us that every position in the hexagonal grid can be represented by a 3D coordinate which contains at least one coordinate equivalent to 0 . There is one state where all coordinates are 0 , 3( m -1) states where exactly two coordinates are 0, and 3( m -1) 2 states where exactly one coordinate is zero. Thus, there are 3 m 2 -3 m +1 states for the hexagonal lattice, compared to the m 2 states for the square lattice.

In the case of square lattices in 2 D , all states occupy an equal proportion of space; however, this is not the case for the hexagonal lattice (see Figure 5A). This is because states with more zero-valued coordinates occur slightly more frequently. To estimate the effect of unequal proportions on the entropy, we directly calculate the Shannon entropy of hexagonal lattices for finite size spatial grids of increasing radius l , as an approximation to the infinite lattice. We find that even for l = 1000 , m > 7 the hexagonal code has 99 percent of the entropy of a system that divided all possibilities equally, and that this gap decreases as m grows larger. Thus asymptotically, as m →∞ , the ratio of entropy for hexagonal vs. square grids tends towards log 2 (3) .

B Experimental details

All experiments were implemented in Python involving standard packages for scientific computing (including NumPy, SciPy, Matplotlib). We describe here the parameters and training setup of our experiments in further detail.

B.1 Scaling in dimension

For each number of moduli, K , we seek to find the smallest dimension D for which our attractor model factorizes its input, p , into the correct grid states in a fixed time ( 50 iterations) with high probability (at least 99 percent empirically). In instances where the network states remain similar over time (at least 0 . 95 cosine similarity), we consider that it converged to a fixed point. If such convergence did not occur, we evaluate the accuracy at the last time step.

To evaluate scaling, we first choose our base moduli to be a set of K consecutive primes. We randomly select one of M random numbers to serve as the input and set the grid states to be random. We then evaluate a candidate dimension on the factorization task for a set number of trials ( 200 ) and check accuracy. We compare accuracy by considering whether the amplitude of the complex-valued inner products are highest for the true factor. If the accuracy is above our threshold, we then evaluate performance of a slightly higher dimension (dimensions evaluated are spaced apart on a logarithmic scale). Once a sufficiently high dimension achieves the accuracy threshold, we assume that the scaling is non-decreasing and use the last successful dimension as the first try.

Finally, we fit linear regression to all data points on a log-log scale to estimate the scaling between dimension and problem size. We report the slopes to estimate the scaling coefficients.

B.2 Error correction

General experimental setup. We fix in advance the vector dimension, noise level (determined by 1 /κ ), and number of moduli. Given these parameters, we estimate the empirical accuracy of factorization on an arbitrary input known to correspond to one of the patterns. We use the same method for checking convergence as above, though we increase the maximum number of iterations to 100 . For all experiments in this section, we average over 1 , 000 trials.

In the case of input noise, the vector p is multiplied by a noise vector. In the case of update noise, after every time step, each module of the attractor network is corrupted by a von Mises noise update. In the case of codebook noise, all codebooks are corrupted before the start of any iterations.

Decoding values between integers. In order to test the ability of the modular attractor network to decode at sub-integer resolution, we fix a spatial resolution ∆ x to decode from. In our experiments, we test ∆ x = { 1 / 3 , 1 / 7 , 1 / 15 , 1 / 31 } , and we also report ∆ x = 1 (integer decoding) as a control. Then, using as input a random integer and random multiple of ∆ x , we let the modules of the attractor network settle until convergence (as in other experiments). To evaluate accuracy, we test if the resulting output of the attractor network, ⊙ i g ˆ K i =1 ( t ) , is closer to the ground truth RNS representation than to any other value. We test this with a 'coarse-to-fine' approach: first checking if it is within an integer, and then checking all fractional values within one of that integer. We regard the output as correct if both the integer and fraction match, and incorrect otherwise.

Estimation of information content from a vector. To measure the total resolution of our coding scheme in bits, we factor in both the number of states distinguished ( τ = M ∆ x and the empirical accuracy ( ρ ). To quantitatively estimate this, we report the information decoded in bits according to the following equation [79, 80]:

A consequence of this equation is that the information decoded is 0 when the empirical accuracy is at chance ( 1 /τ ).

B.3 Path integration

General experimental setup. We generate paths using a statistical model simulating rodent twodimensional trajectories in a 50 cm 2 closed square environment [35, 36], with ∆ t = 100 ms. The

path integration method starts from the ground truth first position ( x 0 , y 0 ) which is converted to hexagonal coordinates ( a 0 , b 0 , c 0 ) (see Section A.3) and encoded as an RNS representation p (0) of dimension D = 3 , 000 following the method in Section 2.1, for moduli { 3 , 5 , 7 } . We then factorize p (0) into { g ˆ i (0) } K i =1 to produce the estimated representation ˆ p (0) = ⊙ K i =1 g ˆ i (0) .

At each time step t ≥ 0 , we estimate the position ( x t +1 , y t +1 ) . We give the modular attractor network as input the previous position vector estimate ˆ p ( t ) . It is factorized into the residue components { g ˆ j ( t ) } K j =1 that are then shifted according to the velocity ( da t , db t , dc t ) between ( a t , b t , c t ) and ( a t +1 , b t +1 , c t +1 ) . Namely, for each residue module, we build a velocity vector q j ( t ) = g j, 1 ( da ( t )) ⊙ g j, 2 ( db ( t )) ⊙ g j, 3 ( dc ( t )) that is binded to each residue component g ˆ j ( t ) . The estimated position vector is then the binding of the shifted estimated residue components: ˆ p ( t +1) = ⊙ K j =1 g ˆ j ( t ) ⊙ q j ( t ) . The estimated position (ˆ x t +1 , ˆ y t +1 ) is chosen to be the position ( x, y ) in a grid of 30 × 30 positions mapping the entire environment, corresponding to the highest similarity between p ( x, y ) and ˆ p ( t +1) .

We show the robustness of the path integration dynamics to two different sources of noise. In the case of extrinsic noise (Figure 6D), the hexagonal velocity is corrupted by additive Gaussian noise of variance 0 . 1 . In the case of intrinsic noise (Figures 6A and B), the position vector ˜ p t is corrupted by binding with a vector sampled from a von Mises distribution with concentration parameter κ = 2 .

Response field visualization. Given a moduli m i and a vector g i , we visualize its response field by computing the similarity of the modular attractor output g ˆ i ( t ) and g i along a trajectory. The periodicity in the distribution of random weights and the hexagonal coordinates produce periodic hexagonal receptive fields whose scale depends on m i . Since the inner product between vector states induces a translation-invariant kernel, the response fields for a given moduli are translations of each other.

Connection to sensory cues. Sensory cues are random binary vectors of size N s = D that are associated with positions along the trajectory. When the true trajectory reaches a sensory cue, the hippocampal state ˆ p t is updated using the heteroassociation method described in Appendix B.4

B.4 Heteroassociation

General experimental setup. We evaluate our model's performance for pattern denoising using a heteroassociative learning rule [28, 40]. We consider random binary patterns of size N s = D . We corrupt the patterns by randomly flipping bits with probability p flip ∈ [0 , 0 . 5] and associate them to place cell representations using heteroassociation with a pseudo-inverse learning rule. Let S ∈ R N s × M be the matrix of M patterns to hook to the scaffold and H ∈ C M × D the matrix of M position vectors on which to hook the patterns. We associate a pattern s to a place cell representation p = HS + s , where S + is the pseudo-inverse of S . The model returns a denoised place cell representation ˆ p from which we can estimate a denoised pattern by inverting the heteroassociation projection ˆ s = sgn ( SH + ˆ p ) . Examples of corrupted inputs and reconstructed patterns are shown in Figure S3.

Scaling with dimension. We evaluate the impact that the dimension D has on the denoising performance in Figure 7, for a number of stored patterns M = 60 ( in this case, 3 × 4 × 5 ) and 210 ( in this case, 5 × 6 × 7) . For each dimension D ∈ { 256 , 512 , 1024 , 2048 } , we show the evolution of accuracy for different levels of corruption. For a given dimension D and noise level p flip , we denoise a pattern and consider that the denoising is correct if the denoised pattern is closest to the ground truth pattern (in terms of cosine similarity). We repeat over 500 trials and report the accuracy as well as the average similarity (normalized inner product) between the denoised pattern and its noiseless version.

Superposition of patterns. We show that our model can denoise a superposition of n p patterns one at a time, for n p ∈ { 1 , 2 , 3 , 4 , 5 , 10 } . We fix the dimension D to 2 , 000 and for different values of bit flip probability p flip ∈ [0 , ..., 0 . 5] , we run the model on a superposition s of random binary patterns { s 1 , ..., s n p } of size N s = 2 , 000 : s = s 1 + .... + s n p . We run the model n p times and between each run the denoised pattern is explained away from the superposition [69]. Namely, for run r ∈ { 1 , ..., n p -1 } we denote ˆ s ( r ) the denoised pattern. The input to run r + 1 is then s ( r +1) = s ( r ) -ˆ s ( r ) . We find that the more patterns are superposed, the lower the overall denoising accuracy is. This is due to the fact that when a pattern is incorrectly denoised, explaining away

adds noise or spurious patterns to the representation of the superposition which makes the following denoising steps more difficult.

Comparison to structured patterns. We evaluate our model's ability to denoise structured patterns. We consider the FashionMNIST dataset, from which we select 105 images of size 28 × 28 that we binarize by setting pixel values to be -1 if below 127 , and 1 elsewhere. We compare the denoising performance to the performance on random binary patterns of size 28 × 28 = 784 for fair comparison (Figure S4).

C Additional results

C.1 Further visualizations of grid cell modules

We further visualize the response fields for path integration by showing response fields from different units taken from the same grid module. We simulate a trajectory that traverses the entire environment and represent the activation of different position vectors along the trajectory. For each modulus m i ∈ { 3 , 5 , 7 } , we show the similarity between 4 different vectors g i from module m i and the position vectors along the trajectory. We show in Figure S1 that the different receptive fields of a given module are translations of one another.

module 1

module 2

module3

C.2 Remapping via modulation of context

We demonstrate that the context vector can serve as a model of global remapping in hippocampal place fields, which occurs when different environments are encoded with different populations of cells [27]. The simplest instance of this is when a place field occurs in context A but not context B, consistent with the observed sparsity of hippocampal activity [82]. To model this kind of remapping phenomenon, we consider an instance where there is a gradation of contexts with some phase transition between them; such an instance was studied experimentally [81]. Towards this end, we model linear combinations of these contexts, where the weights each context is given are sigmoid ( x ) , 1 -sigmoid ( x ) , with x varying from -5 to 5 in 8 equally spaced increments, and with sigmoid ( x ) = 1 / (1 + exp ( -x )) . To model hippocampal units, we generate units that prefer one of the two contexts and have a random place field location, using its weight vector, or address, as c ⊙ K i =1 g i , and compare its output to that of the context/grid system at each location and context. It is worth noting that the original experiment of [81] also exhibited instances of rate remapping for

un

Context

some units, and so there is certainly additional complexity underlying remapping that is not captured by our simple model.

C.3 Storing and retracing sequences

We demonstrate that our model can recover sequences by heteroassociation of patterns to positions and path integration in a conceptual space (Figure S5A). This is consistent with the postulated role of the hippocampal formation in performing navigation in conceptual spaces [83, 84], and the role of entorhinal cortex in generating sequences of neural firing in hippocampus [85, 86]. To evaluate our attractor model's fidelity at sequence memorization and retrieval, we simulate trajectories to form sequences of random binary patterns and recall the sequence using the path integration mechanism following the method in Section 4.1, for D = 10 , 000 and moduli { 3 , 5 } . We add extrinsic noise to the velocity input, which accumulates along the trajectory and induces a drift. This implies that patterns at the end of sequences are less well recovered than ones at the beginning (Figures S5B and C).

C.4 Further tests of model robustness

We find that the proposed modular attractor network is also robust to other sources of noise. In particular, we evaluate robustness to synaptic noise, or dropout, decaying synaptic precision, and weight lesions (Figures S6A-C, respectively).

D Broader impacts

The results presented here are primarily addressing fundamental research questions, suggesting computational mechanisms in the brain. These results could lead to experimental design that improves our understanding of circuits in the hippocampal formation, or to artificial intelligence models capable of incorporating compositional structure in navigation tasks. Such impacts are typical of computational neuroscience research. On the other hand, we point out that the explicit compositionality of our modeling approach provides transparency into its operations, which would reduce the risk of unforeseen consequences.

Input

?

%

0

2

2

：

2

2

2

2

Recovered

3

3

4

Input

%

Recovered

3

3

子

3

4

Input

%°

3

2

2

Recovered

3

3

3

3

2

1.0

0.8

0.6

0.4

A

0.2

P(correct basin)-Random patterns

P(correctbasin)-FashionMNiST

0.0

0.0

0.1

0.2

0.3

0.4

0.5

Fraction of bits flipped

A

B

C

1.00

1.00

0.95

0.95

0.90

iccura

Accurac

0.85

0.85

A

0.80

noiseless

0.80

noiseless

noisy velocity (var=0.05)

noisy velocity (var = 0.05)

0.75

noisy velocity (var=0.06)

0.75

noisy velocity (var=0.06)

0

5

10

15

20

25

30

0

5

10

15

20

25

30

Sequenceposition

Sequenceposition

A

Robustnesstodropout

B

Effectsoflimitedprecision

C

Effectsofablation

1.0-

1.0

1.0

1 codeword lesioned

3 codewordslesioned

0.8

0.8

3 bits(D=512)

0.8

★

1modulelesioned

4 bits (D=512)

0.6

0.6

5 bits (D=512)

0.6

Accura

Accurac

Full precision (D=512)

Accurac

p=0.00

3 bits (D=1024)

A

0.4

p=0.05

0.4

4 bits (D=1024)

5 bits (D=1024)

0.4

0.2

p=0.15

p=0.10

0.2

3 bits (D=2048)

4 bits (D=2048)

Full precision (D=1024)

0.2

p =0.20

5 bits (D=2048)

0.0

0.0

Full precision (D=2048)

0.0

102

10

3

10

4

10°

5

10°

3

10

5

0.0

0.2

0.4

Effectiverange

Effectiverange

%of weightslesioned

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Yes, the abstract and introduction stay within the bounds of what we introduce in the paper.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: Limitations are discussed explicitly in the Discussion (Section 5), as well as implicitly throughout the rest of the paper.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The three technical sections, provided in Appendix A.1, Appendix A.2 and Appendix A.3, respectively, provide full proofs or calculations and cite any required background lemmas.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We explicitly wrote Appendix B to disclose any extra pieces of information required to reproduce experimental results. We have also provided implementations in code.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: Open access to the data and code is available at https://github.com/ SoniaMaz8/Hippocampal_enthorinal_circuit .

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: Yes, these details are presented in Appendix B.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: The results are accompanied by error bars and confidence intervals when appropriate for our figures.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: All experiments were performed on CPU with local resources. We do not have precise estimates of the amount of compute operations required, but each individual simulation took less than 3 days of total compute time.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: The authors have reviewed the NeurIPS Code of Ethics and confirm that it conforms to all standards outlined.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Please refer to Appendix D.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The model does not involve any of the examples listed, and does not require additional safeguards.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: All assets are owned and created by the authors unless explicitly stated otherwise.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper involves neither crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper involves neither crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
comments: SUMMARY:
The paper proposes a model for spatial representation in the hippocampal formation using a residue number system (RNS) to encode positions as high-dimensional vectors. These vectors are combined into a single representation through vector binding and maintained by a modular attractor network. The model demonstrates robustness to noise, high-resolution encoding, and effective path integration.

STRENGTHS:
- Solid theory behind every addition to the RNS model to capture HF functionality
- Testable hypotheses for experiments
- Limitation about the bio-plausibility of the proposed model was mentioned in the discussion

WEAKNESSES:
- There were several mention of compositionally as the motivation, but there is no direct analyses to show the effectiveness of model in compositionally
- There is virtually no comparison to other models in terms of coding range, robustness to noise and compositionality
- Code is not provided

QUESTIONS:
- How does the time-scale of unit responses compared to actual neurons? Is the attractor model fast enough to track the changes in the environment?
- The potential prediction for encoding of episodic memory in this framework is not clear to me.

LIMITATIONS:
- Comparison to existing methods, or an ablation study on the proposed method to verify the intended computational role for each component
- Test of the cognitive map aspect of HF
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-09

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a model for spatial representations in the hippocampal formation. The model relies on a residue number system for encoding spatial positions and uses complex-valued vectors to represent individual residues. These vectors are then combined into a unified vector representing spatial position through a conjunctive vector-binding operation that preserves similarities. The model ensures consistency between individual residues and overall position representation through a modular attractor network, which corresponds to the grid cell modules observed in the entorhinal cortex.\n\nSTRENGTHS:\nWhile there has been an ample amount of work addressing the computations in hippocampal formation, this paper introduces several interesting ideas and combines them into a comprehensive framework. This model integrates principles of optimal coding, such as maximizing coding range and spatial information per neuron, with an algebraic framework for computation in distributed representation.\n\nWEAKNESSES:\nWhile theoretically valuable, the approach remains relatively high-level without a realistic evaluation and comparison with behavioral or neural data.\n\nQUESTIONS:\nCan you clarify what you mean by \"carry-free\" hexagonal coding?\n\nCan you reflect on scalability in terms of the number of neurons. For example, regarding the statement, \"In particular, we require that distinct integer values are represented with nearly orthogonal vectors.\" how does this requirement affect the scalability of the approach? \n\nWhat is the numerical precision required for numerical stability in terms of the neural activity and the synaptic weights? Can you discuss how realistic this approach is in the context of real neurons with firing raters under 100Hz?\n\nLIMITATIONS:\nThe authors have adequately addressed the limitations.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a computational model that incorporates a number of properties about encoding of space representation in the system. The mathematical framework appears to be well-justified and carry the desired properties. These properties are related to some of the observations made about the properties of space encoding in hippocampus, however I've found that the paper conflates conceptual similarity (a certain mechanisms seems to have certain properties) and computational/mechanistic similarity (a certain brain mechanisms is _actually_ computing like the model suggest), but more about this later.\n\nThe way I see this work is that is presents an elegant case for encoding of information in mathematical / algebraic sense, but I struggle to connect these properties and the way the system is evaluated to biology. Put another way - I am not sure *why* this particular computational model is a good model for HC? How do we even evaluate if it's close to what the brain is doing? Or maybe this aspect is actually not important to the authors, and the main contribution on this work lies elsewhere? I must admit I might have misunderstood the motivation and the goal of this work, and I will reflect this in my confidence level. To make a better job in the subsequent round of evaluation I would like the authors to explain how do they understand the importance of this work? What is it that main thing that it bring to the table? Apart from mathematical elegance.\n\nThe evaluation of the model is based on simulated trajectories, but the comparison is done \"within\" the proposed model, and does not provide external points of reference to allow the reader to understand if the model is better that some other ones? If what regard? Is it empirically better or worse at explaining known biological quirks of the HF, or the goal was only to model similarity conceptual on an abstract level?\n\nSTRENGTHS:\n* The math is rigorous and there is a clear sense that the constructed mathematical framework is a good match to desired properties.\n\nWEAKNESSES:\n* The task on which the model is tested (inputs, output, goals) was not clearly defined in the paper, from Section 4 we know that it is about path integrations and there are simulated trajectories (generated according to behavioural rules of animals) that the model is compared against, but \n\n* I think there is a dissonance between the claims of the paper regarding neuroscientific impact of this model and the actual comparisons between the model and biology that are brought forward in the paper. If I am correct that these are actually pretty loose, then from here we logically move onto the next question - if the importance is not in that, then what is it in?\n\n* The evaluation of simulated trajectories is not too informative, because it is unclear how trivial or non-trivial it is to show the match between simulated trajectories and the model following them.\n\nQUESTIONS:\n(1) The proposed model is based on arithmetic, element-wise operations over vectors, and modulo operations... Is the claim here that computations similar to these ones are happening in HC, or those are just some operations, that satisfy a number of properties? Basically do you want to say that the mechanism of the model is close to HC, or that just some of the observed characteristics of the model are close to HC?\n\n(2) Following up on (1) - the closeness between the model and HC is, as far as I can tell, only conceptual, right? There were no comparison made against actual empirical measurements of a biological HC during some task?\n\n122: Could you please elaborate what you mean by \"grid modules\", I understand that this is different from grid-like spatial activation patterns? Are you referring to grid cells representing different scales, each scale being a \"module\"?\n\n(3) Why the comparison between the trajectories is done using simulated animal trajectories and not actual ones? I understand that it's impossible to model such a chaotic system as a real mouse running in a grid... but if that is the case, what is the benefit of trying to predict trajectories at all, and using simulated trajectories based on some rules of animal behaviour? I guess I am confused about the chosen way to compare the model with biology.\n\nFigure 6A (1): What exactly is the baseline model that is marked as \"without attractor dynamics\"?\n\nFigure 6A (2): How come the decoded trajectory matched the true one so closely? Is this result impressive or is it trivial in the context of how the model works and how true trajectories were generated?\n\n(4) The predictions listed on lines 316-323 - are they true in biological system? (A) multiplicative interactions between dendritic inputs providing conjunctive binding operation - this one seems to be at the core of potential achievements of the model, but it is very superficially explained, I think it would be great to have a more extensive explanation of what \"multiplicative interactions between dendritic inputs providing conjunctive binding operation\" actually is, how can we see it manifest in biology, and, after that, how your model achieves it. (B) \"Binding between MEC modules\" - what is specifically meant by conjunctive composition and binding in neuroscience context as it pertains to your work? Because if we only mean to say that the brain does combines inputs of the modules, then sure, that's trivial and the fact that a model also does that is kind of expected. If you mean some specific mechanism or form of conjunctive composition - then what is it? How does it manifest in biological HF? Does your model do it in the same way? How can we assess that? (C) \"relatively fixed attractor weights, plastic HC->sensory weights\" - while these are properties of the model, are they properties of the brain?\n\nLIMITATIONS:\nThe authors extensively address the limitation of this work and this provides valuable context to understanding the significance.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a normative model for spatial representation within the hippocampal formation, integrating optimality principles with an algebraic framework. Spatial positions are encoded using a residue number system (RNS) and represented by high-dimensional, complex-valued vectors. These vectors are combined into a single vector representing position through a similarity-preserving, conjunctive vector-binding operation. The model incorporates a modular attractor network, mirroring the grid cell modules in the entorhinal cortex, to ensure self-consistency among these vectors. The paper showcases the model’s robustness, sub-integer resolution, and path integration capabilities through both theoretical analysis and experimental validation.\n\nSTRENGTHS:\nThe use of RNS for spatial representation is a novel approach that maximizes coding range and spatial information per neuron. In addition, the model integrates principles from neuroscience, cognitive science, and artificial intelligence, providing a holistic view of spatial representation in the hippocampal-entorhinal circuits. The authors also provide rigorous theoretical analysis and empirical experiments to support the model’s claims, demonstrating noise robustness and precise spatial representation. The model makes several testable predictions about neural mechanisms, which can guide future experimental research.\n\nWEAKNESSES:\nThe model’s complexity might pose challenges for practical implementation and experimental validation in biological systems. While the model is comprehensive, it remains a high-level abstraction of spiking neural circuits, potentially overlooking finer neurobiological details.\n\nQUESTIONS:\n1. How biologically plausible is the RNS as a coding mechanism in the hippocampal-entorhinal circuits? Are there any existing biological structures that directly support this model?\n2. What specific experiments could be designed to empirically test the predictions made by the model? How feasible are these experiments with current technology?\n3. The model suggests encoding contexts as vectors in the entorhinal cortex. How does it manage the vast diversity and complexity of possible contexts in real-world environments?\n4. While the model shows robustness to noise in simulations, how would it perform under the more complex and varied types of noise encountered in biological systems?\n\nLIMITATIONS:\nThe model abstracts away many neurobiological details, focusing on high-level representations and processes. This could overlook important aspects of the hippocampal-entorhinal circuitry, such as specific neuronal firing patterns and synaptic plasticity mechanisms.  While the modular attractor network is theoretically scalable, it is unclear how well this scalability translates to biological systems. The actual implementation of such a network in the brain might face limitations due to resource constraints and other biological factors.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-22"}