Geometry of naturalistic object representations in recurrent neural network models of working memory

Authors: xiaoxuan  lei
takuya  ito
pouya  bashivan
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 20:45:56

Abstract: Working memory is a central cognitive ability crucial for intelligent decision-making. Recent experimental and computational work studying working memory has primarily used categorical (i.e., one-hot) inputs, rather than ecologically-relevant, multidimensional naturalistic ones. Moreover, studies have primarily investigated working memory during single or few number of cognitive tasks. As a result, an understanding of how naturalistic object information is maintained in working memory in neural networks is still lacking. To bridge this gap, we developed sensory-cognitive models, comprising of a convolutional neural network (CNN) coupled with a recurrent neural network (RNN), and trained them on nine distinct N-back tasks using naturalistic stimuli. By examining the RNN’s latent space, we found that: 1) Multi-task RNNs represent both task-relevant and irrelevant information simultaneously while performing tasks; 2) While the latent subspaces used to maintain specific object properties in vanilla RNNs are largely shared across tasks, they are highly task-specific in gated RNNs such as GRU and LSTM; 3) Surprisingly, RNNs embed objects in new representational spaces in which individual object features are less orthogonalized relative to the perceptual space; 4) Interestingly, the transformation of WM encodings (i.e., embedding of visual inputs in the RNN latent space) into memory was shared across stimuli, yet the transformations governing the retention of a memory in the face of incoming distractor stimuli were distinct across time. Our findings indicate that goal-driven RNNs employ chronological memory subspaces to track information over short time spans, enabling testable predictions with neural data.

Main paper: 
 introduction: 
 Working memory (WM) -the ability to store and manipulate information over short periods -is a central cognitive capability that enables a wide spectrum of behaviors [Baddeley, 1992] . Over the past few decades, various experimental, computational, and theoretical techniques have been adopted to study WM from both cognitive and neural perspectives. However, several fundamental issues remain unresolved, including the key question of how high-dimensional sensory information is encoded, maintained, and modulated according to specific task demands.A significant body of work has been dedicated to modelling the computations underlying WM. These include many classic models from cognitive science [Meyer and Kieras, 1997a , Ritter et al., 2019 , Baddeley et al., 2021] , neuroscience [Miller et al., 1996 , Wang, 1999 , Emrich et al., 2013] , and more recently deep learning [Yang et al., 2019, Ehrlich and Murray, 2022] . Recent approaches have focused on using artificial neural networks, such as recurrent neural networks (RNNs), to understand and model WM due to their ability to learn the complex cognitive tasks that are commonly used to study WM in humans. However, prior studies were limited in three aspects. First, most works used abstract categorical inputs, such as color and location of moving dots that are represented as binary (or one-hot vector) inputs [Panichello and Buschman, 2021 , Yang et al., 2019 , Piwek et al., 2023] . While training models with such categorical inputs is easier in practice, the resulting models offer limited insights into how real world, naturalistic stimuli (which are embedded in high-dimensional spaces) are processed. Second, most prior work has considered single or a few cognitive tasks [Xie et al., 2022 , Mante et al., 2013 , Piwek et al., 2023] , limiting the generality of these models in explaining the neural computations underlying working memory. Lastly, while some prior work has explored how object features are encoded into population activity [Xie et al., 2022] , it remains unclear how information in working memory is sustained across time to support concurrent encoding, retention, and retrieval during a dynamic task-and how these processes might align with classic cognitive theories [Luck and Vogel, 1997 , Alvarez and Cavanagh, 2004 , Franconeri et al., 2013] .To address these limitations, we investigated how task-optimized RNNs manipulate the multidimensional properties of naturalistic visual inputs during different stages of WM (Figure 1d ). Specifically, we examined: 1) How do task-optimized RNNs select task-relevant properties of naturalistic objects during WM? 2) What computational strategies do RNNs employ to dynamically maintain object properties in the face of incoming (distractor) information? To address these questions, we trained multi-task models on a collection of N-back tasks using naturalistic stimuli, and analyzed the RNNs' latent space during concurrent encoding, retention, and retrieval of information. We specifically chose the N-back task, given that it requires the dynamic encoding, retention, and retrieval of information in the face of incoming distractor information. This is in contrast to prior studies, which primarily studied WM tasks such as delayed-match-to-sample tasks, which focus on WM maintenance during stable fixation periods. The nature of the N-back task makes it an excellent testbed to evaluate how naturalistic object features are maintained in a dynamic environment that requires on-the-fly WM updates and decision-making.Our main contributions are as follows:• We trained different classes of gated and gateless RNN models on a suite of WM tasks and developed decoder-based analyses to study the geometry of naturalistic object representations during different stages of WM.• We found that task-relevant and -irrelevant object properties are simultaneously encoded in multi-task RNN models, while only gateless RNNs produced shared and reusable representations across tasks.• We found that object features are less orthogonalized in the RNNs' hidden dynamics compared to perceptual representations.• We found that RNNs solve the N-back task by using chronological memory subspaces to separate object representations presented across time. This finding supports resource-based models of working memory [Alvarez and Cavanagh, 2004, Franconeri et al., 2013] and challenges the classic slot-based model [Luck and Vogel, 1997] . 
related works: 
 Models of working memory. Originally rooted in cognitive science, the notion of WM was first formally defined and popularized by Baddeley [1992] who proposed a cognitive system consisting of modality-specific buffers and a shared executive module to control the information flow in and out of the memory buffers. This initial work, along with most early models, portrayed WM as a memory system with three key features: flexibility of information representation, limited capacity, and limited temporal span.Subsequent models based on this perspective were largely akin to the architecture of the Von-Neumann computers, comprising of input and output channels, volatile memory components, and a central processing system that continuously executed pre-specified computer code according to task goals [Cowan, 1988 , Meyer and Kieras, 1997b , Anderson, 2013] . However, experimental work in neuroscience has showed that the underlying biological circuitry of WM consists of a wide network of brain areas with diverse roles that do not adhere to the clean-cut modules specified in those previous models [Sreenivasan and D'Esposito, 2019] . Compte et al. [2000] was one of the first studies that showed RNNs with closely matched connection parameters to those measured from the brain can reliably store information in the presence of distractors. Later work showed that these networks are not only capable of performing many classic WM tasks but also replicate neural signatures that were previously observed in animals' brains during these tasks, indicating that similar neural computations may be used by both systems [O' Reilly and Frank, 2006 , Mante et al., 2013 , Xie et al., 2023 , Finkelstein et al., 2021 , Masse et al., 2019] . More recent work proposed various neural network architectures that combine linear attention mechanisms with slot-based memory modules and feedback mechanisms as models of working memory [Hwang et al., 2024 , Loynd et al., 2020] .Neural network models in neuroscience. Neural network models have been increasingly used in computational neuroscience to model neural computations during different behaviors. These models are valuable tools that incorporate ideas from a diverse range of cognitive architectures, while also making testable behavioral or neural predictions. Various classes of neural network models have now been used to simulate neural activity in sensory brain regions [Schrimpf et al., 2018 , Kell et al., 2018 , Khaligh-Razavi and Kriegeskorte, 2014 , Yamins et al., 2014 , Bashivan et al., 2019] , language comprehension [Hasson et al., 2020 , Schrimpf et al., 2021] , and decision making [Mante et al., 2013 , Yang et al., 2019] . Furthermore, others have used RNNs to study the dynamical motifs that underlie various behavioral signatures such as memorization, integration and selection of information, and attention. For example, Mante et al. [2013] used goal-directed RNNs to simulate the dynamics of neural populations in the macaque prefrontal cortex during a context-dependent decision making task and found specific dynamical mechanisms for the selection and integration of task-relevant inputs. Yang et al. [2019] identified specialized functional clusters, mixed selectivity, and compositional representation in multi-task RNNs. The shared dynamic motifs across tasks were further extended and described mathematically in Driscoll et al. [2022] . 
methods: 
 Tasks. We considered N-back tasks (N ∈ {1, 2, 3}) based on one of three distinct object properties (i.e. feature; F ∈ {Location, Identity, Category} (denoted as L, I, C), resulting in a total of 9 N-back task variants (Figure 1b ). Naturalistic stimuli were generated using 3D object models from the ShapeNet dataset (rendered examples in Figure A1a ) [Chang et al., 2015] , comprising 4 object categories, each with 2 unique identities rendered from various view angles, and presented at 1 of 4 possible locations. We consider two validation approaches: validating on novel view angles, and validating on novel identities. The training and validation novel angle datasets differed in their viewing angles, necessitating view-invariant processing by the model. In contrast, the validation novel identity includes unseen identity sampled from categories same as the training dataset.Model Architecture. We considered a two-stage model that delineates perceptual and cognitive processes (Figure 1c ). At the first stage, the model processes sequences of images, utilizing an Ima-geNet [Deng et al., 2009] pre-trained ResNet50 [He et al., 2016] model to derive visual embeddings from each image input. All object features including category, identity, and location were highly decodable from these activations (category: 100.00%, identity: 99.57%, location: 100.00%; 2-fold cross-validation). A point-wise convolutional layer reduces the dimensionality of the 2048-channel feature map from ResNet's penultimate layer (layer 4.2 ReLU) to match the RNN's hidden size.Next, the vectorized embeddings are concatenated with a task index vector and processed by a fully connected layer (matching the RNN's latent size) with layer normalization [Ba, 2016] , serving as input to the RNN. The RNN's output is then fed through another fully connected layer and projected to one of three possible responses: match, non-match, or no action.Each network is trained to perform one (single-task-single-feature) or multiple tasks (multi-feature or multi-task or both). After training, we analyzed activations from the penultimate layer of ResNet50 (i.e. the perceptual space), as well as the RNN activations during the stimulus presentation and subsequent timesteps (i.e. encoding and memory space respectively, denoted as E and M as shown Each object property is encoded in a high dimensional latent subspace of this model; Right: RNN model represents each object property in its encoding latent subspace and retains some or all of the properties within its memory subspaces at later time points.in Figure 4a ). We considered three recurrent architectures for the second stage of the model including the vanilla RNN, GRU [Chung et al., 2014] , and LSTM [Hochreiter and Schmidhuber, 1997] .Model Training Each model architecture was trained within three scenarios that differed in their training diet:• Single-Task-Single-Feature (STSF): trained on a single n-back task based on a single object feature (e.g.1-back location).• Single-Task-Multi-Feature (STMF): trained on a single choice of N for all three feature variations (e.g. 1-back L, or C or I).• Multi-Task-Multi-Feature (MTMF): encompassing all choices of N (1,2,3) and features (L, I, C).Model parameters were trained using AdamW [Loshchilov and Hutter, 2017] optimizer with an initial learning rate of 3e -5 and a Multi-step learning rate decay with γ = 0.1 for every 100 iterations. Batch size was 256 and all trials were generated on-the-fly using the iWISDM package [Lei et al., 2024] . Hidden layers of RNN modules are initialized with Kaiming Initialization [He et al., 2015] .For details of decoding analysis and Procrustes analysis, please refer to Appendix .1 
experiments: 
 All models reached > 95% accuracy on train and > 90% on validation set with novel object angles. Generalization to novel object instances was substantially weaker (Figure A1b, c ). Model performance increased with the number of model parameters, and with identical parameter count, vanilla RNNs accuracy was lower compared to their gated counterparts (Figure A1c ). The ensuing analyses utilized data collected from models with 512 units for vanilla RNNs, and 256 units for GRUs and LSTMs, to ensure comparable model performance as well as comparable model parameters.We first examined how RNN modules represent various object properties such as location, identity and category in their latent space. In particular, we investigated the following two questions:1) Do recurrent networks preserve object properties that are not necessary for the task? In order to perform a task, recurrent networks must maintain information about the task-relevant object properties. However, maintaining information about the task-irrelevant factors is not necessary from the perspective of the task objective. Therefore, RNNs may either selectively maintain taskrelevant information or maintain full object representations, recalling task-relevant information when prompted.We trained decoders (i.e. classifiers) to predict each object property from the RNN hidden state activity from the first timestep of each trial (e.g. F = L i vs. F = L j̸ =i decoders, total 4 location decoders). Cross-validated decoding accuracies are shown in Figure 2b for STSF, STMF, and MTMF GRU models. Unsurprisingly, the task-relevant object properties are fully retrievable in all models.Further tests revealed a causal relationship between the subspaces encoding task-relevant information and the network's generated response (Fig. A7 ; see section .1 for details.) However, while taskirrelevant object features are not generally well-preserved in STSF models (Figure 2b , left), they are much better preserved in STMF and MTMF models (i.e. decoding accuracy > 85%; Figure 2b middle and right). This finding was consistent across all three RNN architectures (Figure A2a ) and across time points (Fig. A5 ). These results suggested that all RNNs maintained a full representation of objects in their latent spaces regardless of which object properties were required for performing the task.2) Are object properties encoded within a subspace that is shared across different tasks or distinct ones within each task? Having observed that both task-relevant and -irrelevant information are retained by multi-feature RNNs, we next asked whether RNN encoding of object properties is task-dependent or -independent. To probe this, we trained decoders to predict object properties from the RNNs' activations during one task, and tested the decoder on RNN activations when performing another task (i.e., cross-task decoding). We quantified the generalization performance of MTMF models across all three architectures (Figure 2a ). We found that gated RNNs (GRU and LSTM, Figure A2b ) utilized highly task-specific subspaces for encoding object properties, while vanilla RNN encoded object properties within a subspace that was shared across all task-variations (Figure 2c ). This suggests that gated RNNs tend to learn task-specific representations that do not generalize across tasks, potentially impacting their ability to generalize to new tasks.To improve their performance, RNN weights might form structured and separable representations for each task-relevant feature. For this to be true, the RNN latent space may orthogonalize feature representations beyond their perceptual representation (see schematics in Figure 3a ). To quantify orthogonalization, we calculated the angles between all pairs of decision hyperplanes using cosine similarity (bootstrapped 10 times). We then summarized these angles into a single orthogonality measure by computing the Frobenius norm of the difference between this matrix and the identity matrix (which represents complete orthogonalization). We defined this measure as the orthogonalization Wij = 1 -abs cos(W i , W j ) O = E triu( W ) (1)where W i is the normal vector of the decision hyperplane that separates points assigned with feature value i from the rest, cos(W i , W j ) is the cosine similarity between the two normal vectors. We take the absolute value since the relative direction does not matter. triu(.) is the upper triangle operator.We evaluated the degree of representation orthogonalization within the perceptual and encoding spaces. Contrary to our hypothesis, we observed that relative to the perceptual space, the RNN latent space slightly de-orthogonalize the axes along which distinct object features are represented (Figure 3b ). Similar results were found when PCA was used to equalize dimensionality between the two spaces (Figure A3 ). Although more orthogonalized representations generally facilitate structured and enhanced separation of task-relevant features, our behavioral results still indicate optimal performance. One possible interpretation of these results is that the reduced orthogonalization in the RNN latent space produces a more efficient (lower dimensional) representation. (In contrast, increased orthgonalization in the representational space would increase the overall dimensionality of the object representations.) In practice, only a subset of dimensions need to contain orthogonalized representations for successful task performance. In turn, this would make it easier to train the read out weights of the RNN to produce the correct task responses. 4.3 Neural mechanisms of concurrent encoding, maintenance, and retrieval in RNN models of WMHaving examined the encoding of objects in the RNN's latent space, we next investigated how RNN dynamics enable simultaneous encoding, maintenance, and retrieval of information. Performing our N-back task suite required the RNN to keep track of prior objects' properties while simultaneously encoding incoming stimuli with minimal interference.We reasoned that the RNN may implement one of three possible mechanisms to perform the N-back working memory task suite (Figure 4e ):• H1: Slot-based memory subspaces [Luck and Vogel, 1997] . Where the RNN latent space is divided into separate subspaces that are indexed in time within the sequence. Each object is encoded into its corresponding subspace (i.e. slot) and is maintained there until retrieved. By definition, the subspace assigned to each memory slot is distinct and "sustained" in time.• H2: Relative chronological memory subspaces. Where the RNN latent space is divided into separate subspaces that each maintains object information according to their age (i.e. how long ago they were encoded). Such a mechanism will require a dynamic process for updating the content of each memory space at each time step during the task.• H3: Stimulus-specific relative chronological memory subspaces. This is similar to the relative chronological memory hypothesis but with independent subspaces assigned to each object. Each observation in the sequence is thus encoded into a distinct subspace and encoding of each stimulus is in turn distinctly transformed into associated memory representations.To identify the hypothesis that best matches the computations performed by the RNN, we analyzed how the RNN latent subspace encodes and transforms each object property (E (S,T ) ) across time into memory (M (S,T ) ) (Fig. 1d ).We first tested whether object information is maintained in a temporally stable subspace (i.e. sustained working memory representation) which aligns with H1 prediction (i.e. E (S=i,T =1) ?= M (S=i,T =k) , k ∈ {2, 3, 4...}; Fig. 4a ). For this, we trained decoders to predict the value of each object property using the RNN unit activity during the encoding phase (i.e. Encoding Space) and evaluated its generalization performance in consecutive steps (i.e. Memory Spaces). We reasoned that if the object information is encoded in a subspace that is stable across time (as in a memory slot), the decoders' generalization performance should be high and comparable to its performance during the encoding phase. Contrary to H1's prediction, we found that the decoders do not generalize well (Figure 4b ), suggesting that the object information is not stably encoded in a temporally-fixed RNN latent space. However, we observed that in STMF and MTMF models, the cross-time decoding accuracy is consistently higher during recall (Figure 4b and c ). Interestingly, this suggests that the object representation is partially realigned with its original encoding representation when it is retrieved (Fig. 4b ).Next, we examined whether the object encoding space is shared between incoming stimuli, irrespective of the specific object or time (H2 vs. H3; i.e. E (S=i,T =i) ? = E (S=j,T =j) , for i ̸ = j). We thus fitted classifiers to decode each object property using the hidden activity from the encoding space of each stimulus within the sequence (i.e. decoding S = i from E (S=i,T =i) ), testing it on the stimuli appearing at other time steps (i.e. decoding S = j from E (S=j,T =j) ). We performed this analysis for all object properties and for all models and all tasks. The validation and generalization accuracies were almost identical (Figure 4d ), suggesting a stable encoding representation (E (S=i,T =i) = E (S=j,T =j) ) consistent with H2. In other words, each object in the sequence was encoded according to its chronological age (i.e., when it was placed into memory), regardless of the object property.Having examined how the RNN latent space allows concurrent encoding, retention, and retrieval of information, we next investigated what transformations underlie the conversion of information from one subspace to another. Specifically, we inquired whether the transformation of feature subspaces across timesteps is stable with respect to the same encoded stimulus (i.e. T i = T i+1 ; see Fig. 4e ). As detailed in Appendix .1, we adopted the orthogonal Procrustes analysis to obtain rotation matrix R S,T to characterize the transformation. The orthogonal Procrustes analysis is a statistical shape analysis which discovers simple rigid transformations that superimpose a set of vectors/points onto another. We used this analysis to inquire whether each set of object feature decoders can be rotated to align with the set of decoders for the same object properties, but across time. Thus, the above test can be reformulated asEQUATIONAdditionally, we also tested if the transformation is consistent across stimuli within the sequence:EQUATIONWe first checked whether the feature representation subspace transformation across timesteps are structured (Figure 4f -left). In other words, we evaluated how well the Procrustes analysis can capture the rotation transformation of feature representation subspaces. The reconstructed decision hyperplanes resulting from rotating the original decoders by the Procrustes transformation matrix R were highly accurate (Fig. 4f -right), indicating that a rotation was able to capture the transformation performed by the RNN across time (also see Figure A4 ).Next, we tested Eqs. 2 and 3 by swapping the rotation matrix at R (S=i,T =j) with R (S=i,T =j+1) or R (S=i+k,T =j+k) respectively, and plotted the accuracy of reconstructed decision hyperplane for MTMF models in Figure 4g . We reasoned that if these rotation operations were shared across time steps and stimuli, swapping them would not significantly affect the decoding accuracy. Across all model architectures and tasks, we found that replacing R (S=i+k,T =j+k) consistently yields good accuracy, whereas replacing R (S=i,T =j+1) does not. Similar results were obtained in models trained on larger N (N=1-4), stimuli with naturalistic texture backgrounds, and more number of categories and identities (Fig. A6 ). These results suggest that while the transformation remains consistent across different stimuli, it is not stable over time. Structured transformation preserves the topology (i.e. the transformation can be captured solely by a common scaling factor and a rotation matrix). Unstructured transformation: does not preserve the topology. Right: Decoding accuracy for fitted decoders (solid line) and reconstructed decoders (dotted line) using the rotation matrix R (i,i) from the Procrustes analysis. The small accuracy gap between fitted and reconstructed decoders suggests a structured transformation. g) Decoding accuracy of the reconstructed decoder when the original rotation matrix is substituted with another (indicated by the x-axis labels). Rows and columns corresponds to object properties and MTMF network architectures respectively. 
discussion and conclusions: 
 We investigated how naturalistic objects are represented in recurrent models during a dynamic and difficult WM task. In contrast to most prior work that build computational models of WM using abstract categorical stimuli, which typically study WM during a stable fixation/delay period, we trained a range of sensory-cognitive models to perform N-back tasks using naturalistic stimuli. We found that models trained on multiple features and multiple tasks retained object information regardless of their task-relevance. While prior studies investigating WM tasks using RNNs have identified shared representations and dynamical motifs across related tasks [Yang et al., 2019 , Driscoll et al., 2022] , we found that representations were largely task-specific in gated RNNs such as GRUs and LSTMs. The representational differences between gated and gateless RNNs provides an opportunity for future work to adjudicate between these recurrent architectures and their cognitive significance (i.e., gated vs. gateless) with empirical neural data.While increased orthogonalization could in theory enable better task performance through the formation of increased separation of object representations in the recurrent module, we found the opposite. Despite reduced orthogonalization of task-relevant features in the RNN, this did not appear to influence task generalization behavior. One possible interpretation of this is that the reduced orthogonalization in the RNN latent space produces a more efficient (lower dimensional) representation. In practice, only a subset of latent need to contain orthogonalized task-relevant representations representations for successful task performance. Having fewer latent dimensions that orthogonalize task-relevant representations could make it easier to decode/read out the correct response information from this latent subspace. Clarifying this distinction further will be important for future work.Lastly, we found that RNNs solve the N-back task by leveraging chronological memory subspaces to maintain information about different objects distinct. This finding is consistent with the "resource" model of working memory [Alvarez and Cavanagh, 2004, Franconeri et al., 2013] , which proposes that memory resources are flexibly distributed across all items. This is contrast to the "slot-based" model [Luck and Vogel, 1997] , which suggests that memory is composed of discrete, independent slots for each item. Furthermore, the observed dynamics align with the conceptual framework in Whittington et al. [2023] , where different memory slots can transfer information between each other during working memory. Our experiments reveal that similar slot-like subspaces naturally emerge in task-optimized RNNs, with information transfer between them guided by RNN transformations based on task demands. However, as these subspaces are carved out of a shared neural space defined by the RNN units and dependent on chronology, they are not necessarily non-overlapping (unlike that assumed in Whittington et al. [2023] ). Such possible overlaps may account for previous findings on the influence of memory load on working memory performance [Ma et al., 2014] . Altogether, these results provide testable predictions to evaluate the neural basis of WM in humans in future work. 
limitations: 
 Our study has several limitations:1. Task-Specific Findings: Our results are specific to the N-back task structure, and it remains uncertain whether similar computational strategies would emerge in other working memory tasks. 2. Analysis for Novel Objects: While we observed reduced performance with novel objects, we did not analyze the representational geometry associated with these stimuli or investigate the reasons for diminished performance. Our use of naturalistic stimuli was intended to avoid imposing the representational geometry typical of abstract inputs, as seen in previous studies (e.g., Piwek et al. [2023] , Yang et al. [2019] ). 3. Architectural Constraints: Our findings are restricted to commonly used RNN architectures, such as vanilla RNNs and LSTMs. Therefore, we cannot make definitive claims about how different neural network architectures might affect representational geometry. Further research could explore whether other architectures yield similar or distinct patterns. 4. Impact of Network Scaling: We did not explore how scaling the network size might influence the results. It is possible that increasing the network size could alter the strategies employed by models to perform dynamic WM tasks. In the Procrustes analyses (see the Procrustes analysis section below), the vector w is derived by rotating the original decoder using the Procrustes transformation and the bias term is obtained from the original decoder. When d ≥ 0, the corresponding data point is assigned to one class, and to the alternative class when d ≤ 0.The set of N vectors {w 1 , w 2 , . . . , w N } in D-dimensional space R D , represent N decoders in that space. These vectors span a subspace of R D that is the set of all possible linear combinations of these vectors. Mathematically, the subspace S spanned by these vectors is defined as: S = span{w 1 , w 2 , . . . , wN} = i = 1 N α i w i | α i ∈ R . .Procrustes analysis Procrustes analysis [Ross, 2004] is a powerful method for identifying shape correspondence that relies on the orthogonality of the rotation matrix. Here, we consider a shape spanned by vertices specified by the normal vectors of decision hyperplanes obtained by classifying We then perturbed the hidden states, obtained during the presentation of the first stimulus, along the direction of the corresponding decision hyperplane. The distance moved is proportional to the normalized norm vector (x-axis, distance moved in units). The perturbed hidden states were passed through the recurrent module along with the paired stimulus to compute the probabilities of the three possible actions.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [No] Justification: Although we did not provide explicitly information about required computational resources, this is not a computational heavy project.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines?Answer: [Yes] Justification: We comply with NeurIPS Code of Ethics.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [NA] Justification: This question is not applicable to our work.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA] Justification: This questions is not applicable to our work.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.12. Licenses for existing assets Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [NA] Justification: This question is not applicable to our work.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided. • If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. 
 
 Appendix: 38th Conference on Neural Information Processing Systems (NeurIPS 2024).We used Scikit-Learn's implementation of SVC for fitting the classifier parameters. 


Decision: In this study, the authors developed sensory-cognitive models, comprising of a convolutional neural network (CNN) coupled with a recurrent neural network (RNN), and trained them on nine distinct N-back tasks using naturalistic stimuli. The representational geometry and dynamics were investigated using latent subspace analysis. The reviewers had several concerns about the motivation/task design/writing. In particular, the reviewers asked for additional control experiments to strengthen the claims here. The rebuttal went very well and all reviewers gave positive comments. I am inclined to accept the paper.

Reviews: review: paper_summary: paper_summary: This work involves training RNNs on the n-back task with naturalistic images fed into the RNN through a CNN front-end. Through a large number of permutations comprising different task requirements, different combinations of tasks and different architectures, the authors study how task relevant and task irrelevant information is processed in the RNN back-end.
strength_weakness: strength_weakness: This work is critically lacking the control experiment of simply training an RNN without natural images or CNNs. The aim of this work is to understand how RNNs process natural images in working memory, but I feel that these results (such as representational orthogonalization) can be reproduced if you simply use randomly generated vectors as inputs. I foresee that this would be a logical doubt from many readers, including myself.

One reason to think this way is that the output of the final layer of a pretrained CNN is essentially uninterpretable (and hence no different from being a random vector of numbers) beyond how it can be clustered according to its softmax boundaries. This is especially so when CNNs have location invariance due to their shared convolution across space, and produce very similar outputs at the final layer (from which the RNN is connected to) for images from the same category.
questions: questions: There seems to be a lack of details in the methods. For example, are the models only trained/tested on 6 time steps?
limitations: limitations: In the checklist, the authors claim that "We point out the limitations of our model and analysis methods in the discussion." But I really cannot find any sentence in Section 5 that describes any limitation of this work at all.
paper_summary: paper_summary: This paper examines the mechanisms of working memory in recurrent neural networks (RNNs) trained with naturalistic objects and N-back tasks, a classic task in cognitive and neuroscience. The aim is to study more complex, ecologically valid stimuli than the abstract categorical input that previous studies of working memory in RNNs have used. Additionally, the choice of N-back tasks provides a setting in which encoding of new memories, maintenance of prior ones in the face of distracting stimuli, and retrieval must all be balanced. For example, in a 2-back task, I must maintain a memory of the stimulus I saw 1 trial ago, and protect it from interference, as I encode the current stimulus and retrieve the stimulus from 2 trials ago to compare them. Here, the authors investigate what strategies RNNs use to dynamically maintain this object information. Moreover, they investigate RNNs' ability to flexibly switch between multiple tasks (1-, 2-, or 3-back tasks) as well as multiple object properties (identity, category and location). 
The findings can be summarized as follows:
1. Information about both task-relevant and task-irrelevant object properties (as measured through a linear classifier's decoding accuracy from the latent vector) is present in multi-task RNNs, but not in single-task RNNs, where only the single property that is relevant for the task is decodable.
2. Information about object properties is orthogonalized in the RNNs' latent space, meaning that the normal vectors of the hyperplanes separating different values of the features used in the task (identity, category and location) are close to orthogonal.
3. In multi-task settings, a vanilla RNN encodes object properties in a subspace that is shared across tasks, while gated RNNs (LSTM and GRU) keep separate subspaces for different tasks.
4. In N-back tasks that require to simultaneously encode, maintain, and retrieve information in working memory, RNNs implement a shared encoding space for all stimuli that appear for the first time, and they do not maintain consistent codes for different objects across time.
5. The transformations of feature subspaces across time can be well-approximated by a rotation matrix, and they are consistent across stimuli, but not stable over time. In other words, the rotation matrix that predicts the transformation of one stimulus from one timepoint to the next can predict the transformation of another stimulus at equivalent processing stages (e.g. between the initial encoding of the stimulus and the next timestep) but it cannot generalize to other processing stages.

Overall, this paper is a thorough investigation of the mechanisms underlying working memory in RNNs in a complex, ecologically valid task setting.
strength_weakness: strength_weakness: - The paper contains several typos: for example, at line 106 "dynamical dynamics", lines 238-240 "We thus fit classifiers to decode each object property using the hidden activity from Encoding Space of the stimuli first appears at one timestep testing it on stimuli first appears other time steps.", line 244 "we next investigated how what transformations underlied the conversion of information", line 258: "architecutre". I would recommend the authors to go through the manuscript and correct these errors, as they make reading the paper unnecessarily harder.
- Several details of the model and training are left out: for example, figure 1c shows a hidden layer between the RNN's latent space and the output. What is the size of this layer? What is the training loss? I imagine it is cross-entropy, but this information should be made explicit. Also, how was the identity of the task encoded? Was a dummy (constant, or random) task identity vector also provided to the single-task single-feature model? How many iterations was the model trained for? How long were the sequences in the training and validation sets?
- The generalization to novel object views and novel object instances is briefly mentioned, but not developed any further. The authors should add an explanation of what can be inferred from these results, and how they related to other findings in the paper. Otherwise, I see little reason for including them.
- I am not sure what conclusions can be drawn from the orthogonalization found in the RNNs' latent space. The authors seem to attribute it to the dynamical encoding of stimulus information implemented by the RNNs, but the comparison between the perceptual space (penultimate layer of ResNet) and the latent space of the RNN does not seem appropriate, especially since the RNN is trained, while the ResNet (if I understand correctly) is frozen. It seems natural then that any network (even an additional feedforward layer) explicitly trained for a task would represent stimuli along orthogonal dimensions that corresponds to the task-relevant features. It becomes hard then to interpret the finding of orthogonalized representations.
- While the paper convincingly characterize the subspaces used by the RNNs to perform the N-back tasks, it does not directly test their causal relevance. This seems like it would be relatively straightforward: by shifting the network's representation along the direction of the normal vector to a given hyperplane, it should be possible to "push it" towards giving a specific answer. For example, once the hyperplane separating "car" from other categories has been computed, the RNN's hidden state can be shifted along the normal vector's direction to (falsely) make it output "match" if the object seen N timesteps ago was a car (in a category N-back task). A similar analysis would validate the notion that the subspaces found by the decoding analysis are actually used by the networks in solving the task. 
- While the paper contains several analyses looking at _cross_-decoding, in which a classifier trained on a timestep in the sequence is evaluated on another timestep, there is no plot showing the _within-timestep_ decoding accuracy. Such an analysis would be helpful for characterizing the dynamics of the information contained in the representation: for example, is the representation of an object simply maintained in memory, iteratively enhanced, or does it degrade in time? Looking at decoding accuracy over time, or also at the orthogonalization index over time, would help to adjudicate between these possibilities. The fact that representation transformations across timesteps are well approximated by rotation matrices suggests that stimulus discriminability is roughly constant (at least until recall, as shown in fig. 4f) but measuring this discriminability directly as decoding accuracy would help to interpret the finding of rotation-like dynamics in functional terms. As decoding accuracy seems to be in general close to 100%, perhaps a continuous measure such as distance from the classification hyperplane would be more sensitive in revealing these dynamics.
- In general, I feel that the paper does not provide a cohesive story to bind the different results together and interpret them. I think the findings are really interesting, but it is not clear what the take-home message is in terms of what they teach us about working memory in humans, animals or artificial systems. For example, what does the relative chronological memory subspace proposed in H2 (and confirmed by the results) mean in functional terms? What is the functional usefulness of having an encoding-specific subspace shared across stimuli, for example? Or what does the finding of rotation-like dynamics tell us in functional terms? One possible interpretation that I take away from this is that _protection against interference_ is a major principle determining RNNs' dynamical codes. For example, having an encoding-specific subspace can help to maintain the current stimulus separate from the ones being held in memory. Is this also your interpretation of the results? The results, as I said, are very interesting by themselves, but providing your own (even if speculative, that is completely fine) interpretation of what principles underlie these results would substantially strengthen the paper. Relatedly, what is the link between the present results and the broader literature on working memory in neuro- and cognitive science? Do these results allow us to distinguish between theories that have been previously proposed, or do they require a new explanatory framework? I appreciated the "Related works" section, which gave a nice overview of research in this field, but I felt the Discussion was not as thorough in placing the current findings in context.
questions: questions: No questions, aside from those mentioned in the "weaknesses" section.
limitations: limitations: The paper does not explicitly address the limitations of the current approach. In particular, to what extent the current findings generalize to other working memory tasks, and to what extent they reflect working memory operations in biological systems, is not clarified.
paper_summary: paper_summary: The manuscript uses various RNN architectures as "model animals" to study the representation and processing of naturalistic stimulus during several working memory (K-back) tasks. Unlike prior work, the current study considers various contexts for the cues, making the representation by the RNNs inherently higher dimensional. They find that when RNNs are required to perform a task with multiple contexts, all RNNs kept track of irrelevant stimuli; yet only vanilla (not gated) RNNs used shared representations across tasks. Moreover, the authors find distinct features are stored in orthogonal subspaces, a finding consistent with the prior experimental literature. They conclude with a study of temporal stability in the face of incoming distractors.
strength_weakness: strength_weakness: - Some relevant citations are missing, please see below.
- The presentation, for me, was a bit dense. I was not fully able to follow Section 4.3, though I believe I understood the general takeaways. The authors should work significantly on the presentation.
questions: questions: I believe the manuscript is novel, convincing, and impactful. Some methodological details are missing, but I am sure the authors can add them during the revisions. Please find my comments/questions below:

- Could you please define what is meant by a latent space, preferably in mathematical terms? I think the authors refer to subspaces defined by the decoder weights, but this should be stated more clearly since latent subspace has different meanings in neuroscience.

- I believe there are at least two seminal works that need to be cited. Please see [1] and [2]; and several citations therein. Specifically, the claim "This is in contrast to prior studies, which primarily studied WM tasks such as delayed-match-to-sample tasks, which don’t evaluate WM maintenance in the face of incoming (and distracting) information." is not correct as both of these works considered attractors. There are more works that should be cited. I believe the authors can find them through a short search, anchoring on these two papers.

- "For this, we trained decoders to predict the value of each object property using the RNN unit activity during the Encoding Space and evaluated its generalization performance in consecutive Memory Spaces." I am wondering if this is a fair test, since the representation may settle into a steady-state after a few time steps. I am not exactly sure how to test this in a way that can account for transient response to die out, so I will leave it to the authors if you wish to address this or not.

Overall, I ended with more exciting questions after reading the manuscript than I started with, which is a mark of a paper that I believe deserves a publication in NeurIPS. I would kindly ask the authors to perform *substantial* edits to improve the presentation so that it is easier to follow the details of Section 4.3.

Citations:

[1] Masse, N. Y., Yang, G. R., Song, H. F., Wang, X. J., & Freedman, D. J. (2019). Circuit mechanisms for the maintenance and manipulation of information in working memory. Nature neuroscience, 22(7), 1159-1167.

[2] Finkelstein, A., Fontolan, L., Economo, M. N., Li, N., Romani, S., & Svoboda, K. (2021). Attractor dynamics gate cortical information flow during decision-making. Nature neuroscience, 24(6), 843-850.
limitations: limitations: NA
paper_summary: paper_summary: The paper presents a study of Working Memory (WM) in Recurrent Neural Network (RNN) models.
The main contribution is the study of the latent space dynamics of RNNs during WM-related tasks with respect to naturalistic stimuli, instead of abstract categorical stimuli that are commonly used. The paper analyses gated and non-gated RNNs, and characterises how the geometric properties of their latent spaces change during the trials, as new stimuli are integrated in the WM.
strength_weakness: strength_weakness: * Very small dataset (4 categories, 2 identities per category, 4 possible locations). The N back window also seems limited (how does WM scale for N>3?)
* Stimuli are more realistic, but there is space for improvement (i.e., realistic backgrounds instead of pitch black?)
* The discussion of the three hypotheses in subsection 4.3 is unclear to this reviewer.

If these weaknesses are addressed, the rating may be increased.
questions: questions: * Does the training from ImageNet transfer to this dataset?
* The normal vectors to the decision boundaries can be taken to be unit norm. In a high dimensional space, however, unit-normed vectors are, with high probability, orthogonal to each other. How does this impact the validity of the orthogonality index?
* It is unclear to this reviewer the reasoning behind the usage of Procrustes analysis for studying the evolution of the decision boundaries.
limitations: limitations: The dataset and the task extent seem to be undersized with respect to what seems reasonably possible. Including larger N-back windows and more objects could be beneficial.

Comments: title: Thank you
Thank you!
Response to authors
Last comment
Controls
Early response
updated section 4.3 cont'd
updated sec 4.3
comment: We thank the reviewer for the detailed feedback. Below are our responses to some of the questions:
1. **Causal relevance test**. Thank you for pointing out possible roots of the observed high variability in the results of this additional experiment. We will perform the updated version of the experiment using 1 vs. 1 decoders and pending results will include them into the revised manuscript. 
2. > does Fig. 1b in the attachment indicate that the accuracy was still close to 100% also several timesteps after the time of the response? 

We would like to clarify that the analysis presented in Fig. 1b of the attached PDF pertains to decoding features of the present stimuli, rather than features of stimuli from memory. As a result, it is expected that the information is decodable irrespective of the timestep. 
We realized that the reviewer may have asked about memory decoding where the object properties of past stimuli are decoded from each time step. For results specifically related to memory decoding, please refer to Fig. 4f in the manuscript, where the solid line represents the decoding of task-relevant features from memory. The dotted lines that represent the decoding accuracy after the rotation transformation almost exactly overlaps with the solid lines. 

3. **Relation to Whittington et al. 2023.** We thank the reviewer for further inquiring about the differences between our findings and those of the model in Whittington et al.. The model proposed in Whittington et al. 2023 assumes that the RNN hidden space consists of a fixed number of “activity slots” with fixed, slot dimensionality and read out weight (reading out of a single slot). Furthermore, the dynamics of the hidden state are limited to identity or zero matrices that can be used to create an exact copy of the contents from one slot to another. While there are conceptual similarities between this theory and our findings from task-optimized RNNs, our findings highlight that 1) strict assumptions on the dimensionality of the subspaces, readout weights, and the dynamics need not be fixed and suitable values could emerge through learning on tasks. 2) Moreover, the proposed mechanism in Whittington et al. is insufficient for performing tasks that require comparison of multiple stimuli (e.g. n-back task) as in such situations the network needs to produce an output by comparing its memory content with perceptual inputs but the proposed model consistently reads out of a single memory slot. We will extend the discussion about the differences and similarities between that work and ours in the revised version of the manuscript.
Thank you for your candid feedback and thoughtful comments throughout the review process. In response to the reviewer's suggestion, we will revise the phrasing from "penultimate layer" of the CNN to "convolutional layer" to avoid any potential confusion in the final revisions. We appreciate your time and effort in helping us improve our work!
I thank the authors for the clarification. I was working with the "default" CNN-RNN architecture which involves connecting the flattened layer of the CNN to an RNN. (Such models are commonly known as convolutional RNNs, and there are easily hundreds of papers on this model if you search "convolutional recurrent neural network" on google scholar). Even in neuroscience this architecture has been proposed, such as in [Xie et al. 2023](https://www.biorxiv.org/content/10.1101/2023.03.30.534982v1.abstract). 

Now that the authors have clarified that they are, in fact, connecting to a convolutional layer, I can accept that there is some biological plausibility in this model. Convolutions were originally inspired by complex cells in the first place, and the layered structure is analogous to the sequential processing along the visual pathway. In fact, I respect the authors for not following the norm in the name of biological plausibility.

I went back to the main text again, and I see that the authors described this as the "penultimate layer" of the CNN, which is more generally referring to either the pre-softmax layer or the pooling layer -- this can be made clearer and emphasized, because this practice (rightfully) detracts from the norm as mentioned above.

Overall, I am glad that this has been cleared up. With this clarification, the control I requested is still important but not absolutely critical like I previously insisted. I will increase my score by 1 and I wish the authors the best of luck.
Thank you for taking the time to review our responses again and for your thoughtful feedback. We're glad our message was communicated clearly and that you found our work valuable. We also appreciate the score adjustment! Thanks once more for your time and effort in reviewing our work.
We sincerely appreciate the reviewer’s continued engagement in discussing our work. We would like to clarify some final points that had been previously raised.
1.  We would like to clarify an important aspect of our experiment that may have caused some confusion. Specifically, **we use the output from the layer 4.2 ReLU, which is distinct from the logit/softmax layers.** (The dimensions of the ReLU layer are 512 * H/32 * W/32, where H and W represent the height and width of the image, respectively, with H = W = 224. As in contrast, the dimensionality of the logit layer is 2014*C, where C is the number of categories). We mention this because the examples provided by the reviewer in their recent comments seem to assume a vector representation from the logit/softmax layers.

2. > If this is the stance that this work takes, and even puts "naturalistic" in the title, then this work must prove itself to give different results using abstract encoding.  

We would like to clarify that in the present work, we are not claiming the representational geometry is different from those in RNNs trained with abstract inputs as we are not explicitly comparing the two cases. Our title contains the term “naturalistic” because we simply used naturalistic inputs to investigate how those multidimensional representations are encoded and memorized by RNNs. Regarding the reviewer’s comment about our statement on lines 41-45: *“While training models with such categorical stimuli is easier in practice, the resulting models offer limited insights into how real world, naturalistic stimuli (which are encoded in high-dimensional spaces) are processed by these models.”* The motivation behind lines 41-45 was to simply point out that the understanding of how networks function with naturalistic stimuli in a generalizable manner are incomplete, and we investigate this matter throughout the manuscript. However, in the light of the discussion, we will revise these sentences in accordance with the reviewer’s suggestions.

3. > The authors wrongly assume the results in 2e cannot be replicated with abstract inputs

We apologize that we initially misunderstood the reviewer’s request for the additional analysis. While we agree that the analysis in Fig. 2e of the rebuttal can be theoretically replicated for RNNs trained on abstract inputs, the measure of orthogonality will be 1 in the input space (to the RNN), which is already at ceiling. The reason is because abstract (i.e., one-hot) vectors are orthogonal to each other, which would make it trivial to compare orthogonality between the abstract input space (with orthogonality at 1) with the RNN’s latent space. This triviality is not the case for naturalistic images, particularly since our task requires generalization to view angles and stimulus identities. 

We once again thank the reviewer for their time.
I thank the authors for their rebuttal, and apologize for the delay in responding. Generally, their responses to my points were helpful in clarifying my doubts. I only have further comments on the following points:

- **Orthogonalization results.** As I had written in my initial review, I wasn't sure how to interpret the orthogonalization finding in the first place, so I also don't know how to interpret the updated finding of *lower* orthogonalization in the RNN. It is indeed counterintuitive, as I would think that training with a task would lead to more orthogonalization along task-relevant dimensions. Either way, I don't think this finding is central to the paper's message, so I believe you could simply remove it.

- **Causal relevance test.** I do appreciate that the authors have run this analysis. However, given the high level of variability in the network's response across trials, I do not think any conclusion can be made from this analysis about the causal relevance of the network's subspaces. The significance of the trends shown in Fig. 1c of the attachment could be simply checked using a measure of correlation between percentages of shift and probability of each given response. I doubt that any of the correlations shown in Fig. 1c will be significant. One possible reason for the high variability might be that the authors subsampled "match" trials and tried to push them towards giving a "mismatch" response. This means that the network's state was pushed towards the "all" direction of the "one vs. all" classifier, which probably does not correspond to any particular category, hence the high variability. It is possible that trying to push a "mismatch" response towards a "match" response (i.e. pushing the network towards a specific category) would lead to more reliable results. However, my intuition might be wrong and I understand that the authors don't have the time to run this analysis. If the authors wish to include this additional analysis as it is, then, they should describe it as inconclusive rather than try to infer any causal effect from it.

- **Within-timestep decoding.** The finding of constant decoding accuracy for task-relevant features (and slightly degrading accuracy for task-irrelevant ones) is quite straightforward, and I think it would make a nice addition to the paper. In particular, it provides a nice contrast to findings in RNNs that had to categorize objects without any memory requirements, such as [1, 2], in which decoding accuracy was found to be increasing across time. This suggests a potential distinction between dynamic encoding strategies in perception and working memory. One thing that is not fully clear to me: does Fig. 1b in the attachment indicate that the accuracy was still close to 100% also several timesteps after the time of the response? If so, this would be interesting to highlight. To make this figure clearer, I would suggest adding vertical lines corresponding to the response times in the different conditions, so that it is clear when the accuracy is being measured before or after the response.

- **Relation to other models.** I thank the authors for their clarification. While I'm familiar with the general idea of slot-based models, I haven't had the time to go through the more explicit model proposed by Whittington et al. (2023). However, from a quick skim of that paper it seems like they do address temporally-structured tasks as well (beyond just delayed response)? Particularly, in their Figure 5, they describe a model that shows some similarity to what the authors propose here, whereby stimuli are encoded in a given slot and then rotated, such that the initial slot can be occupied by the new stimulus to be encoded. I would appreciate it if the authors could clarify the distinction between their model and the one in Whittington et al. (2023). In general, I agree about the importance of across-tasks comparisons, and I am glad that the authors plan to add a comparison with a delayed match-to-sample task. I think that will be a nice addition to the paper.

To conclude, as the authors have clarified several points but not made major changes to the paper, I plan to keep my score. Again, apologies for not leaving enough time for more discussion with the authors, but I believe any further revisions would have been minor anyway.

[1] Spoerer, C. J., Kietzmann, T. C., Mehrer, J., Charest, I., & Kriegeskorte, N. (2020). Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision. PLoS computational biology, 16(10), e1008215.

[2] Thorat, S., Aldegheri, G., & Kietzmann, T. C. (2021). Category-orthogonal object features guide information processing in recurrent neural networks trained for object categorization. arXiv preprint arXiv:2111.07898.
I thank the authors for the response. Also thanks for the clarification on the number of inputs, although it really should not be this tough to figure this out. As I carefully said in the response, the *maximum* dimensionality is 192, nothing about the actual dimensionality (in the extreme case you can put everything in one line and it will be 2-D and there will still be a decoder that separates them). I summarize the author's points below, for additional discussion with reviewers later.

1. The authors cited the Brainscore series of works to justify that the output of CNNs are biologically plausible, hence the input to the RNN is biologically plausible.

2. The model can generalize to novel view angles which abstract inputs cannot.

3. Experiments such as Figure 2e (rebuttal PDF) cannot be done with abstract inputs.

4. There are some mechanistic differences in the methods of this work and Piwek et al. 2023

5. The authors concede that some findings can be replicated with abstract inputs but disagree that their input is not naturalistic.

(Points 1 and 2) I will give a very simple example. Suppose the task has 2 categories, cats and dogs. Here is how the CNN works:

Case 1
- Natural image of cat is the input
- The final layer of the CNN is [12.534, -1.245]
- After softmax, the output is [1,0] implying the classification is a cat

Case 2
- Natural image of dog is the input
- The final layer of the CNN is [-3.142, 4.123]
- After softmax, the output is [0,1] implying the classification is a dog

The authors are arguing that the vectors [12.534, -1.245] and [-3.142, 4.123] represent naturalistic inputs of the dog and cat into the RNN. The model can "generalize" to new angles because the CNN will give the correct classification regardless of view angles. In addition, BrainScore looks at the *convolutional* layers, not the post-flattened layer being biologically realistic.

(Point 3) The authors wrongly assume the results in 2e cannot be replicated with abstract inputs, because the trained weights $W_{xh}$ can manifest the same results (and its even emergent!). Once again this could have been avoided if the control had simply been done (I understand it's too late now).

(Point 4) I acknowledge the differences (clearly the simulations cannot be exactly the same when the tasks are different), but the point is that RNNs in literature have been trained on abstract inputs, and this paper specifically highlights this as a problem in lines 41-45. If this is the stance that this work takes, and even puts "naturalistic" in the title, then this work must prove itself to give different results using abstract encoding.

(Point 5) I can't argue with the author's opinions. I have made my case and the other reviewers should as well after reading the series of exchanges here.

For reasons above, I will keep my score.
**Regarding the total number of stimuli**: We rendered 192 stimuli from the 3D dataset ShapeNet. These stimuli were then positioned in one of the four quadrants on the frame to generate N-back task frames, as location is a key feature of interest.

**192 stimuli identities and dimensionality**: We believe the reviewer may be referring to categorical encoding (binary encoding/one-hot encoding etc) when suggesting that 192 stimuli identities would have 192 dimensions. However, our definition of dimensionality pertains to the neural representation space. As illustrated in Figure 1a of the attached PDF, the dimensionality of the CNN embeddings is 92, not 192, as the total number of stimuli identities might suggest. This reduction in dimensionality reflects the CNN's invariance to viewing angles, mixed selectivity, and the hierarchical nature of features like category and identity. Manually constructing an abstract code that encompasses all these aspects is challenging. Additionally, we use CNN embeddings rather than hard-coded stimulus identities as input to the RNN because CNNs pretrained on ImageNet exhibit significant representational similarity to the human visual system (Yamins et al., 2014 etc).

Furthermore, we would like to emphasize that the models we trained can generalize to novel view angles within the same identity and to novel identities within the same categories. This suggests that the model is truly learning the task rather than overfitting the data. We find it unlikely that this level of generalization would emerge from a model trained with an abstract input space.

Moreover, we are interested in the relative orthogonalization level within the perceptual and encoding spaces. As shown in the updated Figure 2e of the attached PDF, RNN representations are orthogonalized, with the orthogonalization index for the encoding space nearing 1. However, a critical point we wish to address is whether RNNs orthogonalize task-relevant feature spaces more due to task requirements compared to CNN space. If we were to use hardcoded stimulus identity encoding as RNN input, such a comparison analysis would not be feasible. Unlike Piwek et al. (2023), who utilized feature-sensitive neurons with circular normal tuning functions to encode categories or identities, our feature spaces are not one-dimensional continuous variables.

Our claim differs from Piwek et al. (2023), which demonstrates an orthogonal-to-parallel transformation of the cued versus un-cued color planes from the pre-cue to retro-cue period. In contrast, our feature spaces remain relatively orthogonal in both the perceptual and encoding spaces. The input construction in Piwek et al. (2023) assumes an orthogonal relationship between the concatenated encodings of two locations and the colors (using 17 feature-sensitive units for each possible location, following a circular normal tuning function). Therefore, the orthogonal representation at the beginning of each trial is expected. In our case, we did not assume that CNN embeddings are orthogonal, and any representational geometry we observe is an emergent property.

Finally, our use of naturalistic encoding is not intended to suggest that results obtained using abstract encoding are incorrect. Rather, it aims to remove input hypotheses that are manually introduced into model design. While Piwek et al. (2023) could potentially achieve similar representational dynamics using a model that takes RGB encoding of color conjugated with location as input, our naturalistic approach allows us to explore questions beyond the constraints of input encoding design. We agree with the reviewer that some of the same findings may be replicated if non-naturalistic inputs (such as random vectors) were to be used as input to the RNN but we respectfully disagree with the reviewer that the input to the RNN in our experiments is not naturalistic. In our experiments, the RNN receives a vector representation of a naturalistic stimulus on every step which is processed by a biologically-plausible vision neural network. We remain open to further comments by the reviewer.

We hope this response addresses the reviewer’s concerns, and we look forward to further feedback! Thanks!

[1] Yamins, Daniel LK, et al. "Performance-optimized hierarchical models predict neural responses in higher visual cortex." Proceedings of the national academy of sciences 111.23 (2014): 8619-8624.
It sounds like there are 192 $\times$ 4 different stimuli instead of 192? I will just arbitrarily assume its 192 in my argument below but replace it with the correct number if I'm wrong.

I would normally consider what the authors did as a strawman argument. Noise aside, 192 stimuli identities would have a maximum of 192 dimensions. Comparing them to 5000 randomly generated vectors makes absolutely no sense at all. But I would like to give the authors the benefit of the doubt and clarify myself:

Let $x$ be the final layer of the CNN. $W_{xh} x$ is the input to the RNN, where $W_{xh}$ is the set of weights that transforms CNN outputs into RNN inputs. I am saying if you replace $x$ with 192 random vectors, each representing one identity of your stimuli, and $W_{xh}$ is trainable, it is possible to get the same results. Nothing before $W_{xh} x$ matters. 

The title of the paper contains the phrase "naturalistic object representations in recurrent neural network models", and I am challenging the very fact that the input to the RNN is not naturalistic. In fact, prior literature has shown that WM orthogonalization can occur with random inputs [1]. It is extremely important that the authors distinguish their model from a model that does not have a CNN, yet capable of producing the same phenomenon.

[1] https://doi.org/10.1371/journal.pcbi.1011555
Dear Authors,

It seems to me that my message went across. As initially noted, I did really like your work. After having gone through other reviews in my batch, I had a little bit more time to look at your responses to me and other reviewers. While I agree with Reviewer VPbf that additional controls could strengthen your study, I don’t believe they are essential for publication. Future work will ultimately determine whether your findings generalize under further controls. I feel more confident that you will be able to address the issues I raised in my original review, so I will gave you the score that you requested.
Thank you for your feedback! We appreciate the time you took to review our manuscript and thank you once again for your support  in our work!
Thank you for your candid feedback and for taking the time to review and discuss our work. We respect your decision and will take your feedback into account for our future work.
We initialize the random vector input to match the dimensions of our naturalistic input, 224×224×3, drawn from a standard Gaussian distribution. For the input frames, we place one of the 192 possible stimuli in one of the four location quadrants, introducing slight variations from frame to frame to add randomness. Specifically, we add a Gaussian random variable to the sampled central location coordinates within the selected quadrant when placing the stimulus on the frame. As a result, each frame with the same stimulus in the same quadrant is not identical in pixel space. To address input dimensionality, we perform PCA on both the random vector inputs and naturalistic inputs, selecting the first k principal components that account for at least 90% of the variance. This analysis is conducted using 5,000 task trials, which explains why the dimensionality of the random vector inputs exceeds 4,000.
I thank the authors for the response, and I will reply again with my final decision after carefully reviewing the results. I just have a simple question for the authors:

The authors stated in another reply to another reviewer that the total stimuli count is 192. Can I ask how the dimensionality of random vectors, in Figure 1A of the rebuttal PDF, is more than 4000?
Dear Authors,

I understand your desire and ambition to continue the discussion to increase my current score. However, likely unknowingly, you are utilizing a rebuttal strategy known as "wearing down the reviewers." Such a strategy might work for less experienced reviewers, but I simply used it to update my priors with more evidence for a bad behavior I suspected (more on this later). In my latest response, I made it crystal clear that 6 was the highest score I am comfortable giving, the best response to this would be to thank me for my time and wish me well as well.

Since you want to continue, let me explain why I cannot go beyond 6. The initial submission feels incomplete, as if it is the version that you could get done until the deadline (also the assigned number of 20855 suggests it was one of the last submitted papers). 

To some extent, I assigned a solid chance for it being a placeholder for the extensive revisions you planned to perform after the deadline has passed. As several reviewers noted, methodology is incomplete, results are rushed, and several definitions missing. It is *inappropriate* to submit a half-finished work to a conference, in which reviewers are already overburdened. It is even more inappropriate to wear those reviewers down with extensive revisions. Thus, though this work was initially appropriate for a poster thanks to the novel science, the possibility of it being a placeholder prevented me from giving a score higher than 6 for a spotlight recommendation.

Now, with your latest response that seems to add major revisions to your work, my priors are updated and I now more strongly suspect that this is indeed what happened. I do not believe this type of behavior should be rewarded, but **I will leave it to the AC to decide if such major revisions would be appropriate for the NeurIPS conference.**

Due to the reasons described above and my increased suspicion, I will **decrease** my score down to 5. To be perfectly clear and candid, my score can only go down from here, not up. I hope you will take this as a learning experience for future and once again wish you all the best.
*Continued from previous comment* 

Having examined how the RNN latent space allows concurrent encoding, retention, and retrieval of information, we next investigated what transformations underlied the conversion of information from one subspace to another. Specifically, we inquired whether the transformation of feature subspaces across timesteps is stable w.r.t the same encoded stimulus (i.e. Ti = Ti+1; **see Fig. 4e)**. As detailed in Appendix B A.2, we adopted the **orthogonal** Procrustes analysis to obtain rotation matrix RS,T to characterize the transformation. **The orthogonal Procrustes analysis is a statistical shape analysis which enables discovering simple rotation transformations that superposition a set of vectors/points onto another. We used this analysis to inquire whether each set of object feature decoders can be rotated to align with the set of decoders for the same object properties but at a different time step.** Thus, the above test can be reformulated as

*Equation (2)*

Additionally, we also tested if the transformation is consistent across stimuli **within the sequence,** that is to say:

*Equation (3)*

Before delving into testing 2 and 3, we first checked whether the feature representation subspace transformation across timesteps are structured or not (Figure 4f, left). In other words, we evaluated whether Procrustes analysis can capture the rotation transformation of feature representation sub- spaces **in the first place. The reconstructed** decision hyperplanes **resulting from rotating the original decoders by the Procrustes transformation matrix R were highly accurate (Figure 4f-right) which indicated that a rotation operation was able to well capture the transformation performed by the RNN across time steps** (also see Appendix A.1. Figure A4). Further, we tested 2 and 3 by swapping the rotation matrix at RS=i,T=j by RS=i,T=j+1 or RS=i+k,T =j+k respectively and plotted the accuracy of reconstructed decision hyperplane for MTMF models in Figure 4g. **We reasoned that if these rotation operations were shared across time steps and stimuli, swapping them should not significantly affect the decoding accuracy.** Across all model architecture and model operating modes, we found that replacing RS=i+k,T =j+k **consistently yields** good accuracy, **whereas replacing** RS=i,T =j+1 **does not. These results** suggest that while the transformation remains consistent across **different** stimuli, it is not stable over time.
We understand the reviewer's continuing concern in terms of the clarity of the revised text given that updates to the manuscript are not allowed at this stage. To help the reviewer with their judgement, below we include all of the text from the revised section 4.3, hoping that the reviewer can appreciate the improvements made to the clarity of the new revised section. We highlighted the parts of the text with substantial changes. 

# Section 4.3 

Having examined the encoding of objects in RNN latent space, we next investigated how RNN dynamics enable simultaneous encoding, maintenance, and retrieval of information. **Performing our N-back task suite required the RNN to keep track of prior objects’ properties as well as the incoming stimuli with minimal interference.** We reasoned that the RNN may implement one of three possible mechanisms to perform the n-back working memory task suite (Figure 4e).

- H1: Slot-based memory subspaces **(Luck and Vogel 1997, Whittington et al. 2023)**. Where the RNN latent space is divided into separate subspaces that are indexed by time within the sequence. Each object is encoded into its corresponding subspace (i.e. slot) and is maintained there until retrieved. **By definition, the subspace assigned to each memory slot is distinct and “sustained” in time.**

- H2: Relative chronological memory subspaces. Where the RNN latent space is divided into separate subspaces that each maintains object information according to their age (i.e. how long ago they were observed, **for example memory of previous observation or prior to last observation). Such a mechanism will require a dynamic process for updating the content of each memory space at each time step during the task.**

- H3: Stimulus-specific relative chronological memory subspaces. Which is similar to the relative chronological memory hypothesis but with independent subspaces assigned to each object. **Each observation in the sequence is thus encoded into a distinct subspace and encoding of each stimulus is in turn distinctly transformed into associated memory representations.** 

To identify the hypothesis that best matches the computations performed by the RNN, we analyzed how the RNN **latent subspaces that encodes each observed object property (E(S,T ) ) is transformed across time into memory representations (M(S,T ) )(Fig. 1d).**

We first tested whether object information is maintained in a temporally stable subspace **(i.e. sustained working memory representation)** which aligns with H1 prediction (i.e. E(S=i,T =1) =? M(S=i,T =k), k ∈ {2, 3, 4...}; Fig. 4a). For this, we trained decoders to predict the value of each object property using the RNN unit activity during the encoding **phase (i.e. Encoding Space)** and evaluated its generalization performance in consecutive **steps (i.e. Memory Spaces)**. We reasoned that if the object information is encoded in a subspace that is stable across time **(as in a memory slot),** the decoders’ generalization performance should be high and comparable to its performance during the encoding phase. Contrary to H1’s prediction, we found that the decoders do not generalize well (Figure 4b), suggesting that the object information is not **stably** encoded in **a temporally-fixed** RNN latent space. However, we observed that unlike STSF models, in STMF and MTMF models, the decoding accuracy at the step where the object information needs to be recalled for comparison with the most recent object (i.e. the executive step) is consistently higher than other time steps (Figure 4b and c). This suggests that the object representation is partially **realigned** with its original encoding representation at the step where a comparison is to be made **(Fig. 4b; slightly higher decoding accuracy at the executive step).** 

Next, we examined whether the object Encoding Space is shared between all incoming stimuli **within the sequence** (H2) or not (H3) (i.e. E(S=i,T =i) =? E(S=j,T =j)) **– the primary difference between H2 and H3 hypotheses.** We thus fitted classifiers to decode each object property using the hidden activity from **encoding phase (i.e. Encoding Space) of each stimulus within the sequence (i.e. decoding S=i from E(S=i,T =i)), testing it on the stimuli appearing at other time steps (i.e. decoding S=j from E(S=j,T =j)).** We performed this analysis for all object properties and for all model operating modes (i.e. each individual task). The validation and generalization accuracies were almost identical (Figure 4d), suggesting a stable encoding representation **(E(S=i,T =i)=E(S=j,T =j))** consistent with H2. **In other words, each object in the sequence was encoded within the same RNN latent subspace regardless of its order in the sequence.**

*Continued in the next comment*
I thank the authors for thoroughly answering to my questions.
In particular, the discussion of the datasets (their size of the stimuli, the N-back extent and the independence from the background) addresses my doubts entirely, and is convincing.
I also appreciate the revised explanations for section 4.3, which were a major point of confusion.
Finally, the discussion of the orthogonalisation index was very useful to help my understanding. As for the apparent contradiction in the results obtained when considering the revised definition, the authors’ hypothesis about PCA being unable to capture the high dimensional nature of the perceptual space seems reasonable.
As a result of the thorough answers that were provided, I will increase my rating.
I thank the authors for the rebuttal. As noted in my response, I already believe this work to be a suitable contribution to NeurIPS. Both of my concerns were related to the writing, which the authors have committed to addressing. Yet, I have no way of checking the final version, so my current score is the highest I am willing to give due to the poor presentation of the initial submission, which is what I am asked to judge by the guidelines. I wish the authors all the best!