Discretization of continuous input spaces in the hippocampal autoencoder

## Abstract

Abstract The hippocampus has been associated with both spatial cognition and episodic memory formation, but integrating these functions into a unified framework remains challenging. Here, we demonstrate that forming discrete memories of visual events in sparse autoencoder neurons can produce spatial tuning similar to hippocampal place cells. We then show that the resulting very high-dimensional code enables neurons to discretize and tile the underlying image space with minimal overlap. Additionally, we extend our results to the auditory domain, showing that neurons similarly tile the frequency space in an experience-dependent manner. Lastly, we show that reinforcement learning agents can effectively perform various visuo-spatial cognitive tasks using these sparse, very high-dimensional representations.

## 1 Introduction

Ever since the seminal work of [o1971hippocampus] on place cells in rats, decades of animal research have established the hippocampus as a neural system that learns a cognitive map of the environment and uses it for spatial navigation \citepmoser2008place. Further experimental studies have also identified the hippocampus as a key structure in episodic memory formation \citepmoser2015place. Although several attempts have been made to unify these perspectives \citepredish1999beyond, eichenbaum2017integration, a coherent framework that mechanistically relates them remains elusive. Moreover, numerous experiments reporting the instability of place cell activity and their modulation by non-spatial variables \citepfenton1998place, jercog2019heading raise an open question: what are these cells truly encoding?

Efforts to answer this question have shown that place cell-like activity can emerge under various conditions: when agents optimize a predictive coding objective \citeprecanatesi2021predictive, uria2020spatial, ratzon2023representational, gornet2023automated, levenstein2024sequential, chen2022predictive, when networks optimize temporal stability and pairwise decorrelation in processing visual inputs \citepwyss2006model, or when building sparse, compressed representations of environmental states \citepsantos2021entorhinal, santos2021epistemic, benna2021place, ketz2013theta. Notably, the approach where information compression leads to spatial tuning aligns with the earlier hippocampal autoencoder model \citepgluck1993hippocampal, and has been shown to replicate several distinct place cell phenomena following environmental manipulations \citepsantos2021entorhinal, santos2021epistemic.

In this work, we further investigate the mechanisms behind episodic memory formation and the emergence of place cells in the hippocampal autoencoder model. We demonstrate that sparse autoencoders can create discontinuities in the neural population manifold, discretizing arbitrary input spaces into non-overlapping receptive fields, whereby subsets of similar inputs collapse onto single neurons. When applied to visual images, this process generates place cells operating on a very high-dimensional population code. Furthermore, we show that reinforcement learning agents can effectively use such sparse and high-dimensional hippocampal-like representations to solve spatial cognitive tasks.

## 2 Model and Results

### 2.1 Hippocampal-like place cells emerge in sparse autoencoders

We studied the learning of spatial representations by training autoencoders (Figure 1a) with randomly sampled images from four different tasks in the Animal-AI environment \citepbeyret2019animal: Double T-maze, Cylinder, Object Permanence, and Thorndike. We trained two types of autoencoders. "Dense" autoencoders aimed solely to reconstruct the input images, thus preserving input information in their latent space $Z$. In contrast, "sparse" autoencoders had an additional objective beyond input reconstruction: to develop sparse activity patterns in the latent space $Z$. This was achieved using the following loss function:

$$ $\mathcal{L}=\frac{1}{m}\|\textbf{X}-\hat{\textbf{X}}\|_{2}^{2}+\frac{\lambda}{ mn}\|\textbf{I}_{n}-\textbf{Z}^{\text{T}}\textbf{Z}\|_{\text{F}},$ (1) $$

where $m$ denotes the batch size, $n$ the number of neurons in $Z$, and the first term is the mean squared error (MSE) between inputs X and their reconstructions $\hat{\textbf{X}}$, encouraging $Z$ to preserve input information. The second term is an orthonormal activity regularization term, whose strength is controlled by $\lambda$, pushing the Gramian $\textbf{Z}^{\text{T}}\textbf{Z}$ towards the identity matrix $\textbf{I}_{n}$. Since $\textbf{Z}^{\text{T}}\textbf{Z}$ captures the co-activation strengths between neurons in a training sample batch, the orthonormal activity regularization promotes pairwise decorrelation while ensuring equal contribution across neurons. We found this approach yields improved and more reliable results compared to the L1 activity regularization term typically used in sparse autoencoders, particularly in alleviating the dead ReLU problem \citeplu2019dying. For dense autoencoders, $\lambda$ was set to zero, leaving only the reconstruction error. We refer the reader to the Detailed methods section in the Appendix for a complete description of the environments, dataset generation, and parameters used in this study.

Training both types of autoencoders yielded significantly different internal representations of space in their latent space. Dense autoencoders developed many neurons that fired almost everywhere in space, with no defined place fields. In contrast, sparse autoencoders developed a majority of neurons with one or two localized place fields, similar to place cells in the hippocampus (Figure 1b, c). The spatial specificity of sparse autoencoder neurons was also reflected in significantly higher spatial information scores compared to dense autoencoder neurons (Figure 1d). These results demonstrate that single-unit spatial tuning emerges in sparse autoencoders but not in dense autoencoders, despite both types of networks containing the same amount of positional information at the population level, as shown by linear decoding analyses (Figure 1e).

Figure: Figure 1: Hippocampal-like place cells emerge in sparse autoencoders. (a) Autoencoder architecture, featuring the hidden layer or latent space $Z$ (denoted as Fc1) with 1000 neurons. (b) Representative examples of the neurons’ spatial ratemaps for sparse and dense autoencoders. (c) Probability distribution of place field number across environments. (d) Average spatial information per neuron across environments. (e) Normalized average distance error of linear decoding of position with the ratemaps’ population vectors, across environments. The grey dots represent the expected linear decoding errors after performing 1000 random permutations of the ratemaps’ values.
Refer to caption: extracted/2405.14600v1/Figures/figure_1.jpg

### 2.2 Sparse autoencoders discretize and tile the image space with interpretable neurons

Identifying neurons with spatial selectivity similar to hippocampal place cells allowed us to investigate what these neurons encode. Given that their spatial selectivity must arise from some form of visual selectivity, we explored whether they also exhibit localized receptive fields in image space.

We created an image space by extracting semantically-relevant image embeddings of all samples using CLIP and further reducing the dimensionality to a 2D space with UMAP. We then searched for clusters in this 2D image space by running the DBSCAN algorithm on the points corresponding to images that maximally activated a particular neuron (see Figure 2a and Detailed methods in the Appendix). These clusters formed convex hulls (i.e., patches) that corresponded to the neuron’s receptive fields in the image space. When pooled together, receptive fields across neurons partitioned and covered the entire image space (see example in Figure 2b). Furthermore, we computed an overlap metric to estimate the redundancy across the neurons’ receptive fields. We observed that sparse autoencoder neurons tiled the image space in a minimally-overlapping manner, in contrast to dense autoencoder neurons, whose overlap tended to be significantly higher (Figure 2c).

Additionally, we performed unit clamping experiments, setting neurons to their maximal recorded value while others were set to zero, and then decoded their activity back to images. The generated images showed a striking resemblance to the training images, making these neurons highly interpretable (Figure 2d). These results establish a solid relationship between episodic memory formation and spatial coding.

Figure: Figure 2: Sparse autoencoders discretize and tile the image space with interpretable neurons. (a) Images taken in one of the environments (’Cylinder’), encoded with CLIP and further reduced to two dimensions with UMAP. Points of different colors correspond to the images that maximally activate each example neuron (above the 50% threshold of the maximum neuron’s recorded activity). Clusters of maximally activated images are extracted with DBSCAN, making up the convex hulls. (b) Convex hulls for all neurons in a sparse autoencoder trained with images from the ’Cylinder’ environment. The overlap metric corresponds to the expected overlapping area (in %) of two randomly chosen hulls (see Detailed methods in the Appendix for further details). (c) Average overlap in 2D image space of sparse and dense autoencoders, across tasks and for a range of threshold values of maximal activation. (d) Example interpretable neurons in the sparse autoencoder. The corresponding neuron in latent space $Z$ is set to its maximum recorded value across the dataset, while all other neurons are set to zero. Then, the enforced activity vector $Z$ is deconvolved into an image by passing it through the decoder.
Refer to caption: extracted/2405.14600v1/Figures/figure_2.jpg

### 2.3 Population structure in sparse autoencoders is grounded on mixed selectivity

Having linked the formation of episodic memories with the discretization of the image space in sparse autoencoders, we explored the population structure of the latent space representations. Inspired by [stringer2019high] on the dimensionality of the population code in the mouse visual cortex, we examined the dimensionality in our autoencoders. Dimensionality was estimated by performing PCA on $Z$ and computing the linear fit of the resulting eigenspectrum in log-log space, yielding a power-law exponent, $\alpha$. High $\alpha$ values indicate low-dimensional codes, while low $\alpha$ values suggest high-dimensional codes. An $\alpha\approx 1$ indicates a high-dimensional but smooth (i.e., no discontinuities) underlying manifold, as seen in neural responses in the visual cortex \citepstringer2019high.

We found that dense autoencoders had dimensionality scores close to 1, similar to [stringer2019high], whereas sparse autoencoders exhibited much higher dimensional representations (Figure 3a), aligning more with the efficient coding hypothesis \citepbarlow1961possible. The almost-flat eigenspectrum suggests that sparse autoencoders’ population activity indeed encodes fine stimulus features. Moreover, the orthonormal activity regularization also disrupted the input-output similarity preservation typically seen in dense autoencoders (Figure 3b), further supporting the idea of a sharp discretization of the image space by sparse autoencoders.

Borrowing concepts from sparse dictionary learning \citeplewicki2000learning, we considered the decoder weights to be the dictionary of kernels, and the sparse neuron activities to be the coefficients that use the dictionary to reconstruct the inputs. Dense autoencoders exhibited orthogonal kernels, while sparse autoencoders showed non-orthogonal kernels despite highly uncorrelated activity patterns (Figure 3c). This indicates that orthogonal activity does not imply orthogonal kernels, and that sparse autoencoder neurons learned similar feature combinations, indicative of mixed selectivity in neurons \citepfusi2016neurons. These findings suggest that mixed selectivity and high dimensionality are closely linked to the formation of detailed episodic memories.

Figure: Figure 3: Population structure in sparse autoencoders is grounded on mixed selectivity. (a) Eigenspectrum decay in latent space representations (first two rows) and images from the environments (third row). Parameter $\alpha$ corresponds to the power law exponent from linear fitting in log-log space. (b) Input-output similarity for sparse and dense autoencoders, with data pooled across environments. Correlation scores correspond to Spearman’s rank coefficients, and fitting curves have been generated with a locally weighted scatter-plot smoother (LOWESS) for improved visualization. (c) Pairwise Pearson correlation scores between all neurons’ activity in latent space, pooled across environments (left) and pairwise kernel similarity in the decoder weights (layer Fc2), representing the similarity density across "words" in the learned "dictionary" (right).
Refer to caption: extracted/2405.14600v1/Figures/figure_3.jpg

### 2.4 Zero-shot learning of place cells in sparse autoencoders

A typical observation in the hippocampus is that place cells can be identified within the first minutes of an animal being exposed to a new environment \citepfrank2004hippocampal. Given that we have shown that sparse autoencoders exhibit very high dimensionality (Figure 3a) and single neurons tend to encode small clusters of samples (Figure 2), we investigated the extent to which neurons that learned to encode samples in one environment could generalize to encoding unseen environments and exhibit zero-shot place cells. Therefore, we trained sparse autoencoders in one environment and tested them across all others. Strikingly, neurons developed place fields with distributions very similar to those in their training environment (Figure 4a). Furthermore, the average spatial information across neurons and the mean decoding error of the rate maps were very similar, with no significant degradation compared to the training environment (Figure 4b). These results suggest that the network’s circuitry learned to cluster and collapse similar samples onto single neurons in a more generic manner, beyond the specific details of the dataset.

Figure: Figure 4: Zero-shot place cells in sparse autoencoders. (a) Probability distributions of place field number when testing a model within its training environment (light blue) or across unseen environments (dark blue). (b) Average spatial information per neuron, pooled across models and testing environments. (c) Normalized average distance error of linear decoding of position with the ratemaps’ population vectors, across models and testing environments. The grey dots represent the expected linear decoding errors after performing 1000 random permutations of the ratemaps’ values.
Refer to caption: extracted/2405.14600v1/Figures/figure_6.jpg

### 2.5 Sparse autoencoders discretize and tile the input frequency space in an experience-dependent manner

If the hippocampus functions as a generic, modality-independent episodic memory system, our findings with the sparse autoencoder should generalize to other input modalities, such as sound. Indeed, "place cell"-like activity in the hippocampus has been reported for tasks involving "navigating" the sound frequency space, with neurons developing localized receptive fields around particular sound frequencies \citeparonov2017mapping. To investigate whether a similar effect could be observed within our framework, we trained autoencoders to compress and encode sound waves uniformly sampled from a linearly-varying frequency signal (Figure 5a).

We observed the emergence of frequency-specific receptive fields in sparse autoencoders, but not in dense autoencoders (Figure 5b). These receptive fields tiled the entire frequency space in a linear manner. However, when sampling was biased towards certain frequencies, the neurons’ receptive fields became denser and clustered around those frequencies (Figure 5d). Additionally, similar to our previous findings with visually interpretable neurons, we found that individual neurons encoded particular frequencies so that these representations were readily decodable via clamping (Figure 5c). These results demonstrate that sparse autoencoders can discretize arbitrary input spaces to support episodic memory formation.

Figure: Figure 5: Sparse autoencoders discretize and tile the input frequency space in an experience-dependent manner. (a) Data samples are generated by applying a uniformly distributed sliding window to a linearly-varying frequency signal. The samples are fed into a convolutional autoencoder, analogous to the one used for vision (more details can be found in the Detailed Methods section of the Appendix). (b) Unsorted and sorted receptive fields by peak activity location for both sparse and dense autoencoders. Latent space activity $Z$ responding to pure tone test inputs was convolved with a Gaussian kernel (sigma of 0.5 Hz), and then normalized by the maximum per neuron in the sorted plots. Lanczos interpolation was applied to all plots for improved visualization. (c) Decoded output signals after setting the corresponding neuron in latent space to its maximum recorded value across the dataset, while all other neurons were set to zero. (d) Sorted receptive fields in a sparse autoencoder trained with an unbalanced dataset. The data samples were generated with a sliding window that was not uniformly distributed in the frequency space, but whose density followed a Gaussian distribution centered at 45 Hz. Dashed vertical lines denote one standard deviation $\sigma$ from the mean.
Refer to caption: extracted/2405.14600v1/Figures/figure_4.jpg

### 2.6 DQN agents learn effectively with sparse, very high-dimensional representations

Representations used to build episodic memories are likely also employed for behavioral learning in the brain. Therefore, we investigated whether hippocampal-like representations emerging in sparse autoencoders would be suitable for reinforcement learning. To test this, we employed DQNs incorporating either sparse or dense autoencoders to solve a range of tasks in the Animal-AI environment, which inherently require spatial navigation skills (see Figure 6a, and Detailed methods in the Appendix for further details on the tasks, model, and training parameters).

Very high-dimensional representations (such as those based on efficient coding) are known to be highly sensitive to slight input perturbations and are thought to generalize poorly to new, unseen samples \citepnassar20201. Thus, one might expect that DQNs would struggle with tasks requiring generalization to unseen samples during training in non-stationary environments. Contrary to this expectation, we found that DQNs using sparse autoencoders were not systematically worse than those using dense autoencoders (Figure 6b). Although sparse autoencoders seemed to perform worse than dense autoencoders in two of the four tasks tested here (Object Permanence and Thorndike), they were superior in one of them (Cylinder) and had matched performance in the remaining one (Double T-maze). Therefore, while further testing is definitely needed to obtain a more reliable picture of their relative performance, these results suggest that, in practice, hippocampal-like representations can be suitable for reinforcement learning, despite their very high dimensionality.

Figure: Figure 6: DQN agents learn effectively with sparse, very high-dimensional representations. (a) Overhead images captured from above the virtual arena of the four tasks (from left to right): Double T-maze, Cylinder, Object Permanence, and Thorndike. (b) Performance (average reward across episodes) of DQN agents using sparse and dense autoencoders across tasks.
Refer to caption: extracted/2405.14600v1/Figures/figure_5.jpg

## 3 Discussion

##### Optimization objectives underlying place cell emergence

We have demonstrated the distinct emergence of place cells in autoencoders with orthonormal activity regularization (Figure 1). Notably, sparse compression alone was sufficient to develop spatial tuning, without the need for a predictive objective \citeprecanatesi2021predictive, ratzon2023representational, uria2020spatial, gornet2023automated, levenstein2024sequential, chen2022predictive. While predictive coding may explain other features of the hippocampus, such as place-cell theta sequences \citepdragoi2006temporal, prediction does not appear to be necessary for the emergence of realistic place cells. We speculate that models optimized for next-input predictions likely learn compressed representations of the environment implicitly as part of the predictive objective. Furthermore, by training the autoencoders with randomly sampled and shuffled images, we have shown that neither temporal contiguity of samples nor any temporal correlations are required to develop place cells. This finding suggests that while predictive learning capturing temporal input correlations might correspond to experience-dependent theta sequences in the hippocampus, the formation of compressed state representations might correspond to time-independent learning processes at the gamma frequency scale \citeplisman2005theta. Additionally, we have demonstrated that sparse autoencoders can learn localized receptive fields of the input space while breaking the relationship between input similarity and latent space similarity. This finding contrasts with previous research that emphasized the preservation of input-output similarity matching for learning localized receptive fields \citepsengupta2018manifold, qin2023coordinated. Overall, it appears that sparse compression alone is sufficient to learn localized receptive fields, that in turn manifest as place cells when applied to the visual domain.

##### Sparse and very high-dimensional codes

A slowly-decaying eigenspectrum, where fine details are over-weighted, represents codes that create discontinuities by disrupting the locality of the manifold structure supporting the input space distribution \citepnassar20201. We demonstrated that this discontinuity can be induced by an orthonormal activity regularization objective, facilitating the creation of event memories by discretizing the image input space (Figure 2). Our results suggest that very high-dimensional codes underlie the formation of place cells (Figure 3), aligning with the efficient coding hypothesis \citepbarlow1961possible, which posits that the brain maximizes information by eliminating correlations in sensory inputs, leading to sparse coding \citepolshausen1996emergence. Indeed, it has been shown that hippocampal neurons in rodents become sparser with prolonged exposure to the environment \citepratzon2023representational. Moreover, the storage of social memories in mice has been linked to high-dimensional representations in the hippocampus \citepboyle2024tuned. Importantly, sparsity has been shown to control a generalization-discrimination trade-off \citepbarak2013sparseness, which could explain why the hippocampus relies on sparse representations, well-suited for progressive discrimination of similar environments and events. Our study shows that smooth place cell maps can coexist with and emerge from extremely high-dimensional sparse codes \citepchettih2023barcoding. We therefore predict that the dimensionality of the population code along the sensory hierarchy should decrease to support the learning of invariant sensory representations \citepfroudarakis2020object, and then increase sharply at the apex, in the hippocampus, to enable the formation of detailed memories based on the specific combination of such invariant representations. This role of the hippocampus aligns with our observation of mixed selectivity, i.e., neurons learning similar feature combinations (Figure 5c), which in turn has been proposed to enable high-dimensional representations important for higher cognition areas \citepfusi2016neurons, bernardi2020geometry.

##### Circuit mechanisms underlying memory formation

The surprising observation of zero-shot learning of place cells (Figure 4) suggests that the sparse autoencoders learned to cluster similar samples onto single neurons in a generic manner. We hypothesize that the responsible circuits might correspond to known hippocampal processes, mainly pattern completion and pattern separation \citeprolls2013mechanisms. On the one hand, neurons are pushed to collapse across-sample variability by clustering samples based on similarity, an effect akin to pattern completion. On the other hand, sparsity also imposes sharp discontinuities between clusters in neuronal space, even when they might be close in input space, a process akin to pattern separation. The combination of both processes is reminiscent of the "locality-sensitive hashing" (LSH) algorithms used in the computer science field for fast image search \citepkulis2009kernelized. Future work will shed light on the learned circuit mechanisms behind such a LSH in sparse autoencoders, and their potential mapping to pattern separation and completion.

##### Place cell over-representation near reward areas

We have shown that the development of localized receptive fields can be generalized to other input modalities, such as sound. Crucially, we used this simplified framework to demonstrate that receptive field distribution tends to be modulated by the input sampling distribution (Figure 5d). Importantly, it has been observed that the density of place fields increases near reward areas \citepmamad2017place. This has led researchers to seek external reward signals in the hippocampus that would modulate the place cell map \citepkaufman2020role. However, based on our results, we propose an alternative explanation based on oversampling: animals tend to spend more time within rewarded areas due to consummatory behaviors, hence biasing sensory sampling and learning. Additionally, in line with this hypothesis, place cell trajectories leading to rewards or goals tend to be replayed more often than unrewarded past trajectories \citepambrose2016reverse, which would further reinforce the sampling bias.

##### Biological plausibility

Although it has been claimed that error backpropagation and gradient descent are mechanisms that could be implemented in the brain \citeplillicrap2020backpropagation, particularly in the hippocampus \citepsantos2021epistemic, we believe that such strong assumptions are unnecessary to map our model and observations to the real hippocampus. The orthonormal activity regularization term used in our sparse autoencoders could be realized by combining strong lateral inhibition (promoting pairwise decorrelation) and homeostatic plasticity (ensuring that neurons maintain equalized firing rates over time). Additionally, the orthonormal term can be thought of as sparse whitening in ReLU-like neurons (i.e., pairwise decorrelation and variance normalization in the low-firing rate regime), a mechanism proposed to be realized in the brain by an overcomplete basis of inhibitory interneuron projections \citepduong2023adaptive, duong2023statistical, lipshutz2022interneurons. Therefore, we contemplate several alternative mechanisms whereby the main objective driving our sparse autoencoders could be realized in brain circuits.

##### Limitations

While our reinforcement learning experiments suggest that DQNs can make use of very high-dimensional representations to solve complex tasks, the present study is limited in scope (Figure 6). We tested only a few tasks (four tasks within the Animal-AI testbed) and a single model (DQN). To gain a more comprehensive understanding of the suitability of hippocampal-like representations for behavioral and policy learning, further testing is required with a broader range of tasks and models, especially in non-stationary environments where unseen samples are the norm. Additionally, future research should explore how these sparse autoencoders could enhance reinforcement learning algorithms that rely on discrete representations, potentially enabling algorithms based on, e.g., the successor representations \citepdayan1993improving, to extend beyond simplified grid worlds.

## Code Availability

All the code necessary to reproduce the figures and results presented in this paper has been made publicly available at: [https://github.com/adriamilcar/HashingBrain](https://github.com/adriamilcar/HashingBrain).

## Acknowledgments and Disclosure of Funding

We want to thank Raimon Bullich Vilarrubias for suggesting to apply the model to the frequency domain, and Oscar Guerrero Rosado for aiding in the design of the models’ diagrams.

## NeurIPS Paper Checklist

- 1.
Claims
- Question: Do the main claims made in the abstract and introduction accurately reflect the paper’s contributions and scope?
- Answer: [Yes]
- Justification: There is no claim in the abstract that is not strongly supported by the results in the Model and results section.
- Guidelines:
•
The answer NA means that the abstract and introduction do not include the claims made in the paper.
•
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
•
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
•
It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
- 2.
Limitations
- Question: Does the paper discuss the limitations of the work performed by the authors?
- Answer: [Yes]
- Justification: Limitations are noted in the Discussion section, especially regarding the limited number of experiments in the reinforcement learning tasks.
- Guidelines:
•
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
•
The authors are encouraged to create a separate "Limitations" section in their paper.
•
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
•
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
•
The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
•
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
•
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
•
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren’t acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
- 3.
Theory Assumptions and Proofs
- Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
- Answer: [Yes]
- Justification: All mathematical assumptions of the models are clearly stated in the Detailed methods section in the Appendix.
- Guidelines:
•
The answer NA means that the paper does not include theoretical results.
•
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
•
All assumptions should be clearly stated or referenced in the statement of any theorems.
•
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
•
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
•
Theorems and Lemmas that the proof relies upon should be properly referenced.
- 4.
Experimental Result Reproducibility
- Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
- Answer: [Yes]
- Justification: All the algorithms, metrics, and parameters used for all models an analyses are clearly stated in the Details methods section in the Appendix.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
•
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
•
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
•
While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a)
If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b)
If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
- 5.
Open access to data and code
- Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?
- Answer: [Yes]
- Justification: A section of Code availability was created to share our open-source code that contains all the necessary code, data, and instructions to reproduce the results and figures.
- Guidelines:
•
The answer NA means that paper does not include experiments requiring code.
•
Please see the NeurIPS code and data submission guidelines ([https://nips.cc/public/guides/CodeSubmissionPolicy](https://nips.cc/public/guides/CodeSubmissionPolicy)) for more details.
•
While we encourage the release of code and data, we understand that this might not be possible, so “No” is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
•
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ([https://nips.cc/public/guides/CodeSubmissionPolicy](https://nips.cc/public/guides/CodeSubmissionPolicy)) for more details.
•
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
•
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.
•
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
•
Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
- 6.
Experimental Setting/Details
- Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
- Answer: [Yes]
- Justification: All the procedures, hyperparameters, optimizers etc. are clearly stated in the Detailed methods section in the Appendix.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
•
The full details can be provided either with the code, in appendix, or as supplemental material.
- 7.
Experiment Statistical Significance
- Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
- Answer: [Yes]
- Justification: All the plots where data is averaged or pooled across conditions include error bars or shaded areas around the means for clear visualization of significant differences.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
•
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
•
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
•
The assumptions made should be given (e.g., Normally distributed errors).
•
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
•
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.
•
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
•
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
- 8.
Experiments Compute Resources
- Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
- Answer: [No]
- Justification: The experiments performed in this paper only require standard computers with standard GPUs. Therefore, we considered it irrelevant to report such information given the small scale nature of our study.
- Guidelines:
•
The answer NA means that the paper does not include experiments.
•
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
•
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
•
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn’t make it into the paper).
- 9.
Code Of Ethics
- Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics [https://neurips.cc/public/EthicsGuidelines](https://neurips.cc/public/EthicsGuidelines)?
- Answer: [Yes]
- Justification: Our research has been done without violating any aspect in the NeurIPS code of ethics.
- Guidelines:
•
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
•
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
•
The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).
- 10.
Broader Impacts
- Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
- Answer: [N/A]
- Justification: The nature of our experiments and results are quite theoretical and more related to the understanding of the computations in the brain than to any practical, deployable application. Therefore, we considered it irrelevant to add a discussion about positive or negative societal impacts.
- Guidelines:
•
The answer NA means that there is no societal impact of the work performed.
•
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
•
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
•
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
•
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
•
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
- 11.
Safeguards
- Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
- Answer: [N/A]
- Justification: The type of data (images from an open, virtual environment) and models (simple autoencoders) used in this paper do not posses any risk we are aware of.
- Guidelines:
•
The answer NA means that the paper poses no such risks.
•
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
•
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
•
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.
- 12.
Licenses for existing assets
- Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
- Answer: [N/A]
- Justification: This paper does not use any particular existing asset that requires special consideration besides citation of openly available, standard python libraries. The code and data was generated by us.
- Guidelines:
•
The answer NA means that the paper does not use existing assets.
•
The authors should cite the original paper that produced the code package or dataset.
•
The authors should state which version of the asset is used and, if possible, include a URL.
•
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
•
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
•
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, [paperswithcode.com/datasets](paperswithcode.com/datasets) has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
•
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
•
If this information is not available online, the authors are encouraged to reach out to the asset’s creators.
- 13.
New Assets
- Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
- Answer: [N/A]
- Justification: No particular assets are being released with this paper.
- Guidelines:
•
The answer NA means that the paper does not release new assets.
•
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
•
The paper should discuss whether and how consent was obtained from people whose asset is used.
•
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
- 14.
Crowdsourcing and Research with Human Subjects
- Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
- Answer: [N/A]
- Justification: There are no experiments with human subjects included in this paper.
- Guidelines:
•
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
•
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
•
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
- 15.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
- Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
- Answer: [N/A]
- Justification: There are no experiments with human subjects included in this paper.
- Guidelines:
•
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
•
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
•
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
•
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

## Appendix A Appendix

### A.1 Detailed methods

#### A.1.1 Model’s architecture and training

##### Visual autoencoder (Vis-AE)

To compress the images from the Animal-AI enviroment, we employed a convolutional autoencoder (ConvAE) that maps the $3\times 84\times 84$ to a latent space with 1000 neurons, effectively compressing the images by a factor of $\sim$21. The details on the architecture can be found in Table 1.

**Table 1: Architecture of the Visual Autoencoder (Vis-AE)**
| Layer | Type | Act. Func. | Filters/Units | Kernel Size | Stride/Padding |
| --- | --- | --- | --- | --- | --- |
| Input | Input | - | $3\times 84\times 84$ | - | - |
| Conv1 | 2D Conv. | ReLU | 16 | $4\times 4$ | 2/1 |
| Conv2 | 2D Conv. | ReLU | 32 | $4\times 4$ | 2/1 |
| Conv3 | 2D Conv. | ReLU | 64 | $4\times 4$ | 2/1 |
| Reshape | Reshape | - | 6400 | - | - |
| Fc1 ($Z$) | Linear | ReLU | 1000 | - | - |
| Fc2 | Linear | ReLU | 6400 | - | - |
| Conv4 | Trans. 2D Conv. | ReLU | 32 | $4\times 4$ | 2/1 (out. pad. 1) |
| Conv5 | Trans. 2D Conv. | ReLU | 16 | $4\times 4$ | 2/1 |
| Conv6 | Trans. 2D Conv. | Sigmoid | 3 | $4\times 4$ | 2/1 |
| Output | Output | - | $3\times 84\times 84$ | - | - |

##### Audio autoencoder (Aud-AE)

To compress the frequency signals from the synthetic sound dataset, we employed another ConvAE that maps one-second time series data (with a resolution of $10^{-3}$ s) to a latent space with 100 neurons, effectively compressing the signals by a factor of 10. The details on the architecture can be found in Table 2.

**Table 2: Architecture of the Audio Autoencoder (Aud-AE)**
| Layer | Type | Act. Func. | Filters/Units | Kernel Size | Stride/Padding |
| --- | --- | --- | --- | --- | --- |
| Input | Input | - | $1\times 1000$ | - | - |
| Conv1 | 1D Conv. | ReLU | 16 | $100$ | 2/1 |
| Conv2 | 1D Conv. | ReLU | 32 | $100$ | 2/1 |
| Conv3 | 1D Conv. | ReLU | 64 | $100$ | 2/1 |
| Reshape | Reshape | - | $2624$ | - | - |
| Fc1 ($Z$) | Linear | ReLU | 100 | - | - |
| Fc2 | Linear | ReLU | $2624$ | - | - |
| Conv4 | Trans. 1D Conv. | ReLU | 32 | $100$ | 2/1 |
| Conv5 | Trans. 1D Conv. | ReLU | 16 | $100$ | 2/1 |
| Conv6 | Trans. 1D Conv. | Tanh | 1 | $100$ | 2/1 |
| Output | Output | - | $1\times 1000$ | - | - |

##### Loss function

The loss function to be minimized in both autoencoders (Vis-AE and Aud-AE) includes a mean squared error (MSE) term as a reconstruction error to force the latent space to preserve the input information, and a orthonormal activity regularization term that promotes sparse representations in the latent space $Z$:

$$ $\mathcal{L}=\frac{1}{m}\|\textbf{X}-\hat{\textbf{X}}\|_{2}^{2}+\frac{\lambda}{ mn}\|\textbf{I}_{n}-\textbf{Z}^{\text{T}}\textbf{Z}\|_{\text{F}},$ (2) $$

where X is the input, $\hat{\textbf{X}}$ is the output (i.e., the reconstructed input), $\lambda$ is the regularization coefficient ($10^{3}$ by default), $\textbf{I}_{n}$ is the identity matrix with shape $n\times n$, Z is the middle layer’s activity matrix of shape $m\times n$, $m$ being the batch size, and $n$ the number of hidden units. Therefore, the Gramian $\textbf{Z}^{\text{T}}\textbf{Z}$ (with shape $n\times n$) captures the pairwise co-activation strengths between neurons in latent space. The symbols $\|\cdot\|_{2}$ and $\|\cdot\|_{\text{F}}$ denote the squared L2-norm and the Frobenius norm, respectively. The orthonormal activity regularization term promotes pairwise neuron decorrelation while achieving an equalized contribution across neurons, alleviating the dying ReLU problem. For comparisons with the dense AE, $\lambda$ was simply set to 0. We have found this orthonormal activity regularization to give improved and more reliable results than the L1 activity regulation term used in standard sparse autoencoders, especially in preventing the dead ReLU problem.

##### Data generation and training of Vis-AE

For the visual experiments, datasets were generated by sampling a total of 10000 images in each of four Animal-AI environments: "doubleTmaze", "permanence", "cylinder", and "thorndike", at random locations within the arena (excluding the 10% of the space closest to each wall) and with random angles, following uniform distributions. Training was conducted using batches of 256 images for 10000 epochs, with independent training runs per environment. The Adam optimizer with no weight decay was used to train the network and the learning rate was set at $10^{-4}$. In addition, the regularization strength $\lambda$ was set to $10^{3}$ for the sparse autoencoder, and 0 for the dense autoencoder. All weights were initialized using Xavier initialization except for $Z$ (i.e., Fc1), whose weights were initialized following a random asymmetric initialization to minimize the dying ReLU problem \citeplu2019dying. An early stop of 0.0005 in reconstruction loss was used to compare sparse and dense autoencoders with similar reconstruction capabilities.

##### Data generation and training of Aud-AE

For the synthetic audio experiments, training consisted of batches of 256 one-second audio slices of varying frequencies. A sliding window of 1 second (with 1 ms shift) was applied to a linearly-varying frequency signal of total time 100 seconds, moving from 10 to 80 Hz, hence resulting in a total of 99001 samples. The sampling frequency was set at ($10^{4}$ Hz, so that the kernel size (1000) matched to one full cycle at the lowest input frequency (10 Hz). Training was conducted for 1000 epochs using the Adam optimizer with a learning rate of $10^{-4}$, with no weight decay. Here, the regularization strength $\lambda$ was set to $10^{4}$ for the sparse autoencoder and 0 for the dense autoencoder. The weights were initialized as with the visual-AE, with Xavier initialization in all layers except for the the latent space $Z$ that followed a random asymmetric initialization. An early stop of 0.002 in reconstruction loss was used to have a fair comparison between sparse and dense autoencoders.

#### A.1.2 Spatial tuning

##### Firing ratemaps

To generate ratemaps from latent space activity, we first created a grid of $60\times 60$ bins (or $30\times 30$ for computing spatial information scores) for each neuron. For each bin in the grid, we summed the neuron’s activity values for images sampled within that bin, generating an activity map in space. Then, an occupancy map was generated to account for the variability in the number of images sampled at each spatial bin (sampling density), which was used to normalize the values in the activity map. Finally, Gaussian smoothing was applied to each neuron’s normalized activity map, using a standard deviation of 3 bins. The resulting maps were normalized to their corresponding maximum values, yielding smooth ratemaps representing spatially-distributed neural activity.

##### Place field identification

To identify and quantify place fields in each neuron’s ratemap, we first binarized the ratemap by setting pixels with activity below 20% of the maximum activity to zero (inactive bins) and those above to one (active bins). Clusters were identified by grouping adjacent active bins, forming a cluster if a group of active bins was completely surrounded by inactive bins. Clusters not meeting the size criteria for place cells (between 3% and 50% of the total number of bins, 3600) were discarded. The remaining clusters were considered place fields.

##### Spatial information

Spatial information (SI) scores measure the amount of information a neuron’s firing rate ($\nu$) conveys about the agent’s position (r). For each neuron, we first normalized its ratemap (using a $30\times 30$ bin grid) by the overall mean activity $\bar{\nu}$. Then, we computed an occupancy map that was normalized by the total number of samples to reflect the proportion of "time" spent in each bin of the ratemap, denoted as $p(\textbf{r})$. Finally, we applied the formula introduced in [skaggs1992information] to compute the SI scores:

$$ $\text{SI}=\sum_{\textbf{r}\in\textbf{R}}\frac{\nu(\textbf{r})}{\bar{\nu}}\log_ {2}\left(\frac{\nu(\textbf{r})}{\bar{\nu}}\right)p(\textbf{r}).$ (3) $$

The average SI across all neurons in the latent space $Z$ provides an estimate of the degree of spatial tuning that the model has developed.

##### Spatial position decoding

The spatial decoding error measures the expected error of a linear decoder using latent space activations $Z$ to predict the spatial position r. We fit a linear regression model with $Z$ as the independent variables and r as the dependent variables, predicting positions as $\hat{\textbf{R}}=Z\mathbf{W}$. Then, we compute the mean squared error (MSE) between the predicted positions $\hat{\textbf{R}}$ and the actual ones R:

$$ $\text{MSE}=\frac{1}{n_{\text{samples}}}\|\textbf{R}-\hat{\textbf{R}}\|_{2}^{2}.$ (4) $$

Finally, the average spatial decoding error (MSE) is re-scaled by dividing it by the maximum distance in the environment, that is, the diagonal of the arena, computed as $d=s\sqrt{2}$, with $s$ being the side length.

#### A.1.3 Interpretability

##### Visualizing and quantifying the network’s tiling of the image space

We employed the CLIP neural network \citepradford2021learning to encode images (resized from $3\times 84\times 84$ to $3\times 224\times 224$) into 512-dimensional vectors. These vectors were subsequently reduced to a two-dimensional representation using UMAP \citepmcinnes2018umap, with 10 neighbors and a minimum distance of 0.1, enabling the visualization of the high-dimensional image space.

Neurons in the hidden layer of the autoencoder that exhibited strong activation in response to specific images—those triggering activations exceeding a certain % of their maximum activation across the dataset—were mapped to points in the 2D image space. We then identify clusters of points using the DBSCAN algorithm \citepester1996density, with radius $\epsilon$ of 1 and minimum samples of 4. These parameters are very dataset-dependent and were thus selected and validated via extensive visual inspection to ensure reliable cluster identification. Convex hulls were constructed around these clusters using the Quickhull algorithm \citepbarber1996quickhull to delineate their spatial boundaries. This allowed us to identify the regions of the input space that each neuron encodes in their activations, i.e., their receptive fields.

Let $\{H_{i}\}$ denote the set of convex hulls corresponding to each neuron’s activated image space. The average overlap metric, $\overline{O}$, was calculated as follows:

$$ $\overline{O}=\frac{1}{\binom{k}{2}}\sum_{i<j}\frac{\text{Area}(H_{i}\cap H_{j} )}{\text{Area}(H_{i}\cup H_{j})},$ (5) $$

where $\text{Area}(H_{i}\cap H_{j})$ represents the area of intersection between hulls $H_{i}$ and $H_{j}$, $\text{Area}(H_{i}\cup H_{j})$ is the area of the union of hulls $H_{i}$ and $H_{j}$, and $k$ is the total number of hulls. The hull calculations were performed using the Shapely Python library \citepgillies2013shapely. The metric $\overline{O}$ thus represents the average proportion of overlap relative to the union for each pair of hulls and ranges from 0 (no overlap) to 1 (complete overlap), thereby providing a quantitative measure of the redundancy in the neurons’ receptive fields across the image space.

##### Neuron clamping and decoding

To test whether neurons in $Z$ were directly interpretable based on their single-neuron activity (therefore obviating population codes), we conducted clamping experiments. This involved setting the activation of a specific neuron $i$ in $Z$ to its maximum activation value observed across the dataset $\mathcal{X}$, while setting the activations of all other neurons to zero. This is represented as $z_{i}^{\prime}=(0,\dots,0,x_{\text{max}},0,\dots,0)$ where $x_{\text{max}}=\max(\{z_{i}|z=f(x),x\in\mathcal{X}\})$ and $f(x)$ represents the encoding function mapping $\mathcal{X}$ to $z$. Then, $z_{i}^{\prime}$ is processed by the decoder $g(z_{i}^{\prime})$ (with $g(x)$ representing the decoding function mapping $z$ to $\mathcal{\hat{X}}$) to yield an output signal (image of audio wave, depending on the AE).

##### Population code dimensionality

The dimensionality of the population code was estimated by computing the power-law exponent $\alpha$ of the latent space activity $Z$ \citepstringer2019high. We performed PCA on $Z$ and computed the linear fit of the resulting eigenspectrum in log-log space over the range of the first 10 to 100 principal components. Since the exponent $\alpha$ provides an estimate of how fast the population activity eigenspectrum decays as new dimensions are added, high $\alpha$ values are indicative of low-dimensional codes, whereas low $\alpha$ values indicate high-dimensional codes.

#### A.1.4 Reinforcement learning experiments

##### Animal-AI Testbed

The Animal-AI testbed is a comprehensive platform designed for evaluating the cognitive and learning capabilities of AI agents in a variety of tasks that simulate real-world challenges \citepbeyret2019animal. This testbed provides diverse environments where agents must use visual cues and navigate complex structures to achieve specific goals. The visual inputs from these environments are standardized to a resolution of 84 by 84 pixels, and agents can perform actions defined by a 2-dimensional vector of integers: the first component goes from 0 to 2 and corresponds to not moving, moving forward, or moving backwards, respectively; and the second component also goes from 0 to 2 and corresponds to not rotate, rotate left, or rotate right, respectively. To encourage efficient behavior, a standard frameskip of 4 is applied, and the reward value decreases by 0.001 at each step. Episodes terminate either when the agent obtains the reward or after 1000 frames.

We evaluate our reinforcement learning agents using four distinct benchmarks within the Animal-AI testbed: the Double T-maze, Object Permanence, Cylinder, and Thorndike tasks. Each of these tasks presents unique challenges that require the agent to apply different strategies and cognitive abilities.

- •
Double T-maze. Each episode starts with the agent positioned randomly at one of the corners of the maze, and the objective is to navigate to the center to obtain the reward. The center contains the only positive reward (+3) available in the environment. Due to the high and opaque maze walls, the agent cannot directly see the reward and must explore the maze to find it.
- •
Object Permanence. At the beginning of each episode, the agent observes a large reward (+3) falling behind a wall until it is completely occluded. The agent must then navigate to the hidden reward, avoiding a small and visible reward (+1) along the way.
- •
Cylinder. This task involves an opaque cylinder with a medium-sized reward hidden inside. The agent begins outside the cylinder and must navigate into the cylinder to obtain the reward (+2).
- •
Thorndike. The task tests the agent’s ability to escape from a closed box to reach a reward located outside the box. The box is semi-transparent, allowing the agent to see the reward from inside. The only exit is blocked by a movable obstacle that the agent must push to escape. A medium reward (+2) outside the box is the sole positive reward available.

##### Model

To evaluate the performance of our sparse autoencoders in reinforcement learning scenarios, we used a standard Deep Q-Network (DQN) architecture \citepmnih2015human with modifications to the input layer. Instead of feeding raw pixel data from the Animal-AI environments, we used the compressed representations of 1000 units generated by the Visual Autoencoder (Vis-AE).

The loss function optimized by the Deep Q-Network (DQN) is the Mean Squared Error (MSE) between the predicted Q-values and the target Q-values, calculated using the Bellman equation:

$$ $\mathcal{L}(\theta)=\mathbb{E}_{(s,a,r,s^{\prime},d)\sim\text{ReplayBuffer}} \left[\left(r+\gamma\cdot(1-d)\cdot\max_{a^{\prime}}Q_{\text{target}}(s^{ \prime},a^{\prime};\theta^{-})-Q_{\text{main}}(s,a;\theta)\right)^{2}\right]$ (6) $$

where $Q_{\text{main}}$ is the main Q-network with parameters $\theta$, $Q_{\text{target}}$ is the target Q-network with parameters $\theta^{-}$, $s$ is the current state, $a$ is the action taken, $r$ is the reward received, $s^{\prime}$ is the next state, $d$ is a boolean indicating whether $s^{\prime}$ is a terminal state, and $\gamma$ is the discount factor. This loss function aims to minimize the difference between the Q-value predicted by the main network and the target Q-value, which is computed based on the reward and the maximum Q-value of the next state predicted by the target network. The training of the DQN was performed by using the RMSprop optimizer. The target network was periodically updated with the weights of the main DQN to stabilize training. The DQN was trained with the following hyperparameters: a learning rate of 0.00025, a discount factor ($\gamma$) of 0.99, an update frequency of 4 steps, and a target network update frequency of 2500 steps. The $\epsilon$ for the epsilon-greedy policy started at 1 and decayed linearly to 0.1 over 25000 steps. The replay buffer size was set to 25000, with a batch size of 32 for experience replay. The details on the architecture can be found in Table 3.

**Table 3: Architecture of the Deep Q-Network (DQN)**
| Layer | Type | Act. Func. | Units |
| --- | --- | --- | --- |
| Input | Input | - | 1000 |
| Fc1 | Linear | ReLU | 100 |
| Fc2 | Linear | ReLU | 50 |
| Fc3 | Linear | ReLU | 25 |
| Fc4 | Linear | ReLU | 9 |
| Output | Output | - | 9 |

##### Training and performance metrics

Each reported experiment tested two DQN agents, Sparse and Dense, which differ only in their use of different Vis-AE models (sparse and dense autoencoders, respectively) to obtain compressed representations from the environment observations as input. The two agents were evaluated across the four Animal-AI tasks described earlier. Each model run lasted 5000 episodes, and to ensure statistical reliability, each model played each task between 20 and 27 times. The reported average performance metric was calculated using a sliding window of 20 episodes.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: * How were the images sampled when training the autoencoder?
* Could you be more specific about what exactly didn't work with using an explicit sparsity penalty (l1 norm)?
* How should I think about multi-modal signals related to navigation (e.g. vestibular, proprioceptive, and motor information)? Is the paper suggesting they are irrelevant for generating place cells?
soundness: 1
strengths: 1. Interesting use of the Animal-AI environment to study how phenomena commonly found in biological brains (place cells) might emerge.
2. Paper is clearly written (minus a few missing experimental details)
summary: This paper models the formation of hippocampal-like spatial representations by training sparse autoencoders on visual images from 3D environments. By applying an orthonormal activity regularization to encourage sparsity, the authors observe place cell-like behavior, where certain neurons respond selectively to specific regions in the input space. They argue that this sparsity enables the model to discretize continuous spaces into distinct memory-like representations. The study also extends to audio data, showing similar localized tuning, and evaluates how these representations impact reinforcement learning agents. The paper argues that such sparse coding mechanisms could illuminate how episodic memory and spatial representations emerge in the hippocampus.
weaknesses: 1. Confounding Visual and Spatial Representations: The study compares sparse autoencoder representations derived from static visual images to hippocampal place cells, effectively confounding visual encoding with spatial encoding. The hippocampus integrates both sensory cues (such as vision) along with self-motion cues (vestibular, proprioceptive, and motor information). The analogy to hippocampal place cells is therefore weak, as the model solely captures visual features, which may not directly translate to spatial encoding in biological brains.
2. "Sparsity" Enforced by Design, Not Necessarily Task-Relevant: The network's sparse representations are enforced through an orthonormal regularization term, which directly promotes decorrelation (not sparsity!) in the latent space. This enforced structure is interesting but does not convincingly link the sparsity to improved performance on the core task of reconstructing visual inputs. The paper lacks an analysis on whether the sparse representations improve or impair reconstruction accuracy, raising questions about whether these representations are useful for encoding spatial features or are merely artifacts of the regularization. I think the paper should compare both dense (no penalty), decorrelated (the orthonormal penalty used in the paper), as well as sparse (l1 penalty) networks; and show the performance on the task (reconstruction error) for each, ideally for a range of regularization strengths (values for ). How do the results in the paper depend on either (a) the choice of penalty used or (b) the regularization penalty strength ()?
3. Unclear Image Sampling and Environmental Modeling: The training dataset appears to consist of randomly sampled images from a virtual environment, but the paper provides little detail on the spatial structure of these samples. If the images are not derived from a structured, agent-driven navigation through space, it is unclear how the autoencoder captures spatial rather than purely visual information. This choice weakens claims that the model learns spatial representations akin to the hippocampal encoding of navigated environments. More rigorous sampling or agent-driven exploration would be necessary to substantiate claims of spatial learning. This would also ideally include some understanding of how the results depended on the choice of agent-driven exploration used.
4. Inadequate Evaluation of Representation Utility: The paper’s core claims focus on the emergence of hippocampal-like representations, but the practical utility of these representations remains untested in terms of spatial or episodic memory tasks. While place cell-like activity is demonstrated in the autoencoder’s neurons, it is unclear if these representations are suitable for real-world tasks that would benefit from spatial memory, such as pathfinding or goal-directed behavior. The RL experiments suggest the opposite (that the penalty is harmful), if I understand them correctly.

---

confidence: 5
contribution: 3
initial_rating: 3
presentation: 4
questions: In general, I liked the paper and applaud the innovative use of autoencoders and the clarity of the authors’ representations (pun intended). However, the biological relevance and the application of terms like ‘memory’ and ‘LSH’ feel overstated, making the results overly speculative. If the claims were substantially backed up with well-stated limitations or if terminology were revised to better match the model's demonstrated capabilities (e.g., ‘representation’ rather than ‘memory’) and the other stated weaknesses were addressed, I would be inclined to increase my score.
soundness: 1
strengths: #### **Originality**
This paper introduces an innovative approach by using sparse autoencoders with orthonormal regularization to simulate place cell-like behavior, bridging neuroscience and machine learning. This combination of techniques is a creative application in hippocampal modeling, especially as it applies across sensory modalities (visual and auditory) and introduces a new metric for discretization of input spaces.

#### **Quality**
The experimental design is thorough and well-documented, including comparisons across dense and sparse models, detailed model architecture, and training procedures. The experiments are contextualized within neuroscience theories like pattern separation/completion, lending scientific depth and grounding to the study. However, while rigorous, the experiments are limited to specific, controlled environments, which may impact generalizability.

#### **Clarity**
The paper is generally clear in explaining complex ideas, with visuals that effectively illustrate neuron activity maps and spatial encoding properties. The appendix offers additional transparency regarding methods, adding to reproducibility. Nonetheless, some technical details—such as orthonormal regularization’s biological plausibility—could be expanded to improve accessibility.

#### **Significance**
This work has notable implications for both neuroscience and AI, providing a potential model for hippocampal-like representation and spatial coding that might inform further studies in spatial cognition. Its cross-modal approach (visual and auditory) hints at a generalizable model for sensory representation, though further validation across a broader range of tasks and sensory modalities would strengthen its practical impact.
summary: This paper investigates how sparse autoencoders, equipped with orthonormal activity regularization, can simulate place cell-like activity in artificial neural networks, potentially providing insights into hippocampal encoding mechanisms. Through comparisons between dense and sparse autoencoders trained on images of 3D environments, the authors demonstrate that sparse autoencoders generate discrete, interpretable representations by mapping similar inputs to localized receptive fields with minimal overlap. This mechanism is similar to locality-sensitive hash, creating high-dimensional representations that resemble spatial maps, similar to place cells in the hippocampus. Additionally, the study generalizes these findings to the auditory domain, showing that sparse autoencoders can produce frequency-specific receptive fields, suggesting modality-independent encoding.

The authors introduce a novel metric to quantify the discretization of image space and present results that reinforcement learning agents benefit from these high-dimensional sparse representations in some environments. The paper concludes with a discussion on the biological plausibility of orthonormal regularization, suggesting it could be implemented through neural processes like lateral inhibition and homeostatic plasticity. Overall, the work bridges computational neuroscience and artificial intelligence by proposing a model where sparse input compression supports episodic memory formation and spatial representation in a hippocampal-inspired framework.
weaknesses: ### **Inaccurate use of terms**
The paper’s use of terms like "memory" and "locality-sensitive hashing (LSH)" misrepresent its functionality, suggesting capabilities that the model doesn’t fully achieve.

The term "memory" implies persistence, flexible retrieval, and time-based storage. Episodic memories, in particular, include context, time, and location (what, when, where). The model, however, lacks mechanisms for retaining and recalling information across time, relying instead on transient representations accessed by direct activation (neuron clamping). Referring to these as "representations" or "spatial encodings" would more accurately reflect their function.

LSH typically involves hashing for efficient, similarity-based retrieval, using probabilistic mappings to preserve approximate similarity and enable efficient search. The model’s rigid clustering lacks LSH’s probabilistic flexibility and retrieval efficiency, as it does not include true hashing or indexing. Describing the model as “localized spatial encoding” or “input-space tiling” would avoid implying LSH-like efficiency or robustness.

__Suggestion__: Replace "memory" with "representation" and "LSH" with “localized encoding” to better match the model’s capabilities. If these terms are retained, clearly specify that the model only partially mimics these concepts. Compare to methods like receptive field mapping or population coding for a more accurate portrayal.

### **zero-shot learning**
The authors claim that sparse autoencoders exhibit zero-shot learning of place cells across novel environments, but there is a limited exploration of why or how this occurs. The model’s capacity for generalization appears promising but has not been properly examined. Analyzing the network's internal structure or activity patterns when exposed to novel environments could reveal mechanisms that enable this generalization. Running ablation studies on the architecture (e.g., removing the orthonormal regularization) or visualizing the learned feature space across environments would provide insights into which aspects drive the process. Also, how does the decoding look like during zero-shot learning?

Moreover, when the model encounters unseen data, interpreting its representations as "memory" becomes challenging, as true memory implies both stored information and the ability to generalize or recall similar contexts flexibly. When the model encounters unseen data, it generates new, sparse representations in the latent space without direct prior storage or memory of these specific inputs. This process resembles on-the-fly encoding rather than recall from stored experiences, suggesting that the model is generating new representations rather than accessing a pre-existing memory.

In zero-shot learning, the model's response to new data could reflect a form of pattern generalization based on similarities to previously encoded features. While this may resemble aspects of memory, it lacks associative recall or the ability to adapt previously stored information to novel contexts, a hallmark of true memory.

The representations formed for new inputs suggest flexible encoding rather than dynamic recall, as they are generated without modification of or access to previously stored states. This is akin to a flexible encoding mechanism rather than a memory system, as it adapts to new data but does not actively retrieve, compare, or contextualize it based on stored knowledge.

### **bio plausability and regularization**
The paper claims that sparse autoencoders with orthonormal regularization simulate place cell-like properties in the hippocampus, but the biological plausibility of this regularization remains speculative. Orthonormal regularization is uncommon in neural modeling and lacks a clear parallel with hippocampal processes.

Moreover, it is not yet clear if the place cell-like properties are specific to this regularization or could be achieved with alternative, potentially more biologically plausible methods.
Implementing and testing additional regularization strategies, such as L1 or L2 regularization combined with decorrelation constraints, could reveal whether place cell emergence depends specifically on orthonormal regularization. This would not only validate the robustness of the current approach but also explore alternative mechanisms that might be more feasible in biological systems. There are several propositions in the literature see e.g. [Schaeffer et al.](https://arxiv.org/abs/2311.02316)

### **Interpretabilty**
The interpretability claims based on clamping neurons to their maximum observed activity are primarily qualitative, which may limit reproducibility and objectivity.
Including quantitative metrics—such as cosine similarity or structural similarity index (SSIM) scores—between the decoded outputs and training images would provide a more rigorous assessment of interpretability. If visualizations remain subjective, a standardized metric would enhance reproducibility and add credibility to the interpretability claims.

### **Sensory modalities**
While the authors extend the model to the auditory domain, the tested signals are relatively simple (frequency alone), which limits understanding of the model’s adaptability to more complex auditory stimuli.
Testing with more complex auditory signals, such as temporal sequences or multi-dimensional auditory stimuli (e.g., varying intensity, background noise), would demonstrate if the model’s generalization holds for richer sensory environments.  Moreover, the study from Aronov focused on a goal-directed task.

### **RL**
The paper’s reinforcement learning (RL) results show that sparse autoencoders perform inconsistently, which the authors attribute to the brittleness of high-dimensional representations. However, the exact limitations and causes of brittleness are not thoroughly analyzed.

A more detailed analysis of the conditions under which sparse representations lead to brittle behavior could provide actionable insights for improving stability. Testing across a wider range of RL architectures (beyond DQN) or incorporating additional stability measures, such as other regularization techniques, dropout, or noise injection, could yield strategies to mitigate brittleness and enhance adaptability in RL tasks.

---

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: * Abstract: I hope the authors could shorten the abstract and extract important message they want to deliver in this paper. Currently, the abstract is a pretty detailed summary of all the experiments. I don’t feel this level of details is necessary for the abstract.
* I wonder if the results (e.g., place fields) in the experiments are a function of \lambda? For example with a smaller \lambda i.e., weaker sparsity constraint, will the rate maps become more band-like? You may want to include these as appendix, just to show the sensitivity of the results to a quite important hyperparameter. 
* I found the results in Fig 2d on memory quite interesting. I wonder if the authors have come across Radhakrishnan et al. 2020 that shows that autoencoders are associative memory models (like Hopfield Nets). This may offer you an interpretation of your results that in sparse autoencoders, individual neurons are basically “attractor neurons”.
* Is it possible to map the sparse autoencoders to the hippocampal circuit? For example, dentate gyrus is known to perform some kind of orthogonalization. How can it be related to the circuit mechanism implied by the sparse autoencoder?

Overall, I will suggest a score 5 to this paper. However, if they authos can address the concerns raised above, particularly on highlighting the contribution better, I'm happy to raise it to 6.

References: 
Radhakrishnan et al. 2020: Overparameterized neural networks implement associative memory
soundness: 3
strengths: * The authors have conducted a comprehensive set of experiments, including:
   * spatial navigation with visual inputs: I really appreciate the usage of Animal-AI environments for the experimentation, which distinguishes this paper with earlier ones like Benna and Fusi (2021) that only performed the experimentation on simulated tasks with low-dimensional inputs.
   * In-depth analysis of the latent structure of the autoencoder.
   * Investigation of frequency-sensitive “place cells”, which shows the generalizability of the model.
   * Reinforcement learning agents.
* The writing is clear and easy to follow.
* There is no obvious error in the paper.
summary: This paper demonstrates a sparse autoencoder-based modelling approach to understand the encoding mechanism of place cells. Through a variety of experiments on spatial navigation tasks with visual stimulus, sound frequency and reinforcement learning agents, the authors have demonstrated that sparse autoencoder can be used to explain the computational mechanism behind the discretization of continuous, high-dimensional inputs i.e., place cells in the hippocampus, and that memory encoding and place cells are closely linked.
weaknesses: * Main contribution is unclear: 
  * Despite the comprehensive experiments presented in this work, I found the main contribution poorly defined, especially given that earlier works (e.g., Benna and Fusi 2021, Santos-Pata et al. 2021) have followed similar autoencoder-based approaches. 
  * For example, the authors stated in the final sentence of abstract that the findings demonstrate how sparsity gives rise to interpretable memories and thus establishes a link between memory and place cells. However, it has been shown in multiple models that sparsity is a necessary condition for obtaining place cells, regardless of the model (e.g. Chen et al., 2024, Benna and Fusi, 2021), and the second part of the claim is exactly what has been studied in Benna and Fusi 2021. Given these, what is the novel contribution of this paper?
  * One possible contribution that the authors can highlight is that the paper provides a full-scale investigation of 1) the necessity of sparsity and 2) the relationship between memory and space representation with experiments on higher-dimensional inputs of different modalities. However, this is only my opinion, and the authors may have other contributions they wish to highlight. In that case, please make sure they are sufficiently highlighted in the paper.
  * Related to this, I feel the paper could be improved by having a ‘Related Work’ section, which highlights what has been done and what hasn’t been done in earlier literature, and how this paper addressed these gaps.
* Lack of key experimental details in the main text:
  * I agree many experimental details (e.g., training information like learning rates) don’t have to be included in the main text. However, this paper has missing experimental details that affects understanding. Here’s an incomplete list of them:
    * Section 2.1: I hope the authors can use a few sentences to describe what exactly does the agent do in these 4 animal-ai environments, or what exactly the task is, as some readers might be unfamiliar with the dataset.
    * Section 2.2: In the first sentence of the second paragraph the authors mentioned “embedding of all samples” – as a reader I’m not sure what all samples refers to. I assume they are the images sampled from the tasks? However, which task did you use to get the samples? One of them or all? Also, although CLIP, UMAP, DBSCAN are well-known embedding/dim-reduction methods, you should cite relevant literature.
    * Section 2.4: The authors mentioned that the autoencoder is trained on one environment and tested on all others. Which environment is the training one and which are the testing ones? Did you do a cross-validation like testing? If not, I think it is preferable to do so as it will demonstrate the robustness of the results better.
    * Section 2.5: I struggle to understand the purpose of Fig 5d and the sentence associated with it: why do we need to bias the sampling towards certain frequencies?
    * Section 2.6: The authors moved the whole task description of these RL experiments to the Appendix, so it is quite unclear to me what exactly is the performance in Fig 6b measured on. It would be much nicer to include in the main text what the tasks and metrics are.

* Redundant experiments: I suggested in the previous point that the paper could benefit from adding more experimental details. This might be challenging given the limited number of pages. However, I believe there are some redundant experiments in this paper that can be removed to give space to more details for other, more informative experiments. For example, I didn't find the results in Fig 3 very unexpected/intriguing. Since orthogonality constraint is imposed on the autoencoder, it is unsurprising that the eigenspectrum of the sparse autoencoder is a flat line, because it is exactly what orthogonality means i.e., neurons will be more independent and explain equal amounts of variance in the data. I understand that the authors wanted to reproduce the results with mice in Stringer et al. 2019 but keep in mind the causality of the results: in mice, the eigenspectra reveal a coding mechanism, whereas in autoencoders, the coding mechanism (orthogonality/sparsity) is first imposed so the observed eigenspectra are expected. I’m wondering if the authors are willing to remove this experiment, or simply move it to the Appendix.

Overall, given the final two points above, I suggest the authors to think about the allocation of paper content on different experiments more carefully. Specifically, they may want to consider removing contents related to Fig3 and add more details to other experiments.

Meta-review: This paper models the formation of hippocampal-like spatial representations by training sparse autoencoders on visual images from 3D environments. By applying an orthonormal activity regularization to encourage sparsity, the authors observe place cell-like behavior, where certain neurons respond selectively to specific regions in the input space. They argue that this sparsity enables the model to discretize continuous spaces into distinct memory-like representations. The study also extends to audio data, showing similar localized tuning, and evaluates how these representations impact reinforcement learning agents. The paper argues that such sparse coding mechanisms could illuminate how episodic memory and spatial representations emerge in the hippocampus.

The paper presents an interesting application of sparse autoencoders to simulate place cell-like behavior, providing valuable insights into how neural representations could emerge in the hippocampus. The approach is clearly written and utilizes a relevant, well-defined experimental setup that demonstrates promising results in both visual and auditory modalities.

The primary weakness lies in the confounding of visual and spatial representations. By using static visual images instead of a more dynamic, agent-driven approach to spatial navigation, the authors fail to accurately capture the integration of sensory and self-motion cues, weakening the connection to hippocampal place cell behavior. Additionally, the enforced sparsity via orthonormal regularization is not shown to improve reconstruction accuracy or task performance, raising concerns about the utility of these representations for spatial encoding. Furthermore, the paper lacks key experimental details, such as how the images were sampled, which hampers the understanding of how the model captures spatial information. Finally, the biological plausibility of the orthonormal regularization method and its comparison with other regularization techniques are not fully explored, limiting the paper’s impact. 

Given these issues, the paper does not convincingly demonstrate how the proposed model advances our understanding of hippocampal encoding and memory formation, leading to the conclusion that it should be rejected.

Decision: Reject