Neuron Platonic Intrinsic Representation From Dynamics Using Contrastive Learning
Authors: Wei Wu
Can Liao
zizhen Deng
Zhengrui Guo
Jinzhuo Wang
Venue: ICLR 2025 Poster (2025)
Keywords: representation learning
biology
neuroscience
contrastive learning
Primary area: applications to neuroscience & cognitive science
TL;DR: Obtaining Intrinsic Representations of Single Neurons from Dynamics via Contrastive Learning

Abstract: The Platonic Representation Hypothesis posits that behind different modalities of data (what we sense or detect), there exists a universal, modality-independent representation of reality. Inspired by this, we treat each neuron as a system, where we can detect the neuron’s multi-segment activity data under different peripheral conditions. We believe that, similar to the Platonic idea, there exists a time-invariant representation behind the different segments of the same neuron, which reflects the intrinsic properties of the neuron’s system. Intrinsic properties include the molecular profiles, brain regions and morphological structure, etc. The optimization objective for obtaining the intrinsic representation of neurons should satisfy two criteria: (I) segments from the same neuron should have a higher similarity than segments from different neurons; (II) the representations should generalize well to out-of-domain data. To achieve this, we employ contrastive learning, treating different segments from the same neuron as positive pairs and segments from different neurons as negative pairs. During the implementation, we chose the VICReg, which uses only positive pairs for optimization but indirectly separates dissimilar samples via regularization terms. To validate the efficacy of our method, we first applied it to simulated neuron population dynamics data generated using the Izhikevich model. We successfully confirmed that our approach captures the type of each neuron as defined by preset hyperparameters. We then applied our method to two real-world neuron dynamics datasets, including spatial transcriptomics-derived neuron type annotations and the brain regions where each neuron is located. The learned representations from our model not only predict neuron type and location but also show robustness when tested on out-of-domain data (unseen animals). This demonstrates the potential of our approach in advancing the understanding of neuronal systems and offers valuable insights for future neuroscience research.

Published as a conference paper at ICLR 2025
NEURON PLATONIC INTRINSIC REPRESENTATION FROM DYNAMICS USING CONTRASTIVE LEARNING
Wei Wu
Can Liao
Zizhen Deng
Peking University
University of Georgia can. www.luugaaa............ edu
Chinese Academy of Sciences Institute of Automation 188038356122 68 63 com
weiwu@stu. pku edu.
Zhengrui Guo
Jinzhuo Wang = Peking University
The Hong Kong University of Science and Technology Beijing Institute of Collaborative Innovation zguobol connect. ust hk
wangjinzhuo@pku.co edu. en
ABSTRACT
The Platonic Representation Hypothesis posits that behind different modalities of data (what we sense detect), there exists universal, modality- representation of reality. Inspired by this hypothesis, we treat each neuron as a detect the neuron's multi-segmen activity data under dif-
system, where ferent peripheral conditions. We believe that, similar to the Platonic idea, there ex- ists time-invariant representation behind different segments of the same neuron, which reflects the intrinsic properties of the neuron's 's system. Intrinsic properties include the molecular profiles, location within brain regions and morphological structure, etc. The optimization objective for obtaining intrinsic neuronal repre- sentations should meet two criteria: the representations of recording segments from the same neuron must exhibit higher similarity compared to those from dif- ferent neurons; (II) the representations should generalize effectively to out-of- domain data. To this end, we propose the NeurPIR (Neuron Platonic Intrinsic Representation) framework, which leverages contrastive learning by treating seg- ments from the same neuron as positive pairs and those from different neurons negative pairs. In the implementation, we adopt VICReg, which only uses pos- itive pairs while indirectly separating dissimilar samples through regularization terms. To validate the efficacy of our method, we first conducted tests on simu- lated neuronal population dynamics data generated by the Izhikevich model. The results confirmed that our approach accurately captured the neuron types as de- fined by the preset hyperparamettrs.. Subsequently, we applied our method to two real world neuron dynamics datasets, which included neuron type annotations derived from spatial transcriptomics and the location of each neuron within brain regions. The representations learned from our model not only accurately pre- dicted neuron types and locations but also demonstrated robustness when tested out-of-domain data (data from unseen animals). This finding underscores the potential our approach in furthering the understanding of neuronal systems and offers valuable insights for future neuroscience research. Code is available at https///gttubbcoo///////sss///ne
INTRODUCTION
Unraveling the intricacies of neuronal activity and the information encoded within neural dynam- ies stands as monumental challenge in the field of neuroscience. Plato's cave allegory and Pla- tonic Representation Hypothesis Huh et al. (2024) suggests the existence of universal, modality- independent representation of world that transcends the modalities through which we perceive it. Drawing inspiration from this philosophical concept, we propose novel perspective on neuronal
Corresponding Author

---

Published as a conference paper at ICLR 2025
activity, treating each neuron as a distinct system capable of generating multi-segment activity data under various peripheral conditions, the multi-segment activity data of individual neuron as what we perceive.
Our approach, NeurPIR, is based on the premise that, similar to the Platonic idea, a time-invarian representation exists that underlies the diverse activity segments of the same neuron. This represen- tation is hypothesized to encapsulate the intrinsic properties of the neuronal system, offering a stable and consistent framework for understanding neuronal function.
To extract intrinsic representations, we formulated an optimization objective based on two key crite- ria. First, activity segments from the same neuron should exhibit greater similarity than those from different neurons, ensuring that our method effectively captures the unique signatures of individ- ual neurons. Second, the derived representations should demonstrate high generalizability, enabling their application to out-of -ut-of-domain data, such as neuronal activity from different species or experi- mental conditions not included in the training set.
To achieve these objectives, we employed contrastive learning, powerful machine learning tech- nique that treats segments from the same neuron as positive pairs and those from different neurons as negative pairs. This approach leverages the contrast between similar and dissimilar samples to learn effective representations. For our implementation, we utilized VICReg, a variant of contrastive learning that focuses exclusively on positive pairs while incorporating regularization terms to indi- rectly separate dissimilar samples. This method ensures that the learned representations are both discriminative and robust variations in the data. However, achieving the above requires the fol- lowing two innovative designs for neuron data: Firstly, we use CEBRA [Schneider et al.] (2023b) to integrate the single neuronal peripheral information (such as activity of neighboring neuronal populations and behavioral data for each segment of a single neuron). This process encodes the peripheral information associated with each segment of an individual neuron. While CEBRA has been widely recognized for its ability to produce high-performance learnable latent embeddings for jointly representing surrounding neuronal data and external information, it was previously applied to groups of neurons rather than focusing on individual neurons. Secondly, the approach aims to produce consistent representation of a neuron's s identity across varying experimental conditions, using these conditions as positive instances in a contrastive learning paradigm. While multi-session data naturally provides such instances, the reality is that many neurons are recorded in only a single session. To address this limitation, we designed data augmentation method tailored to neuronal data. This method involves extracting segments of varying lengths from the same session to serve as positive sample pairs. In this way; the learned representation can capture intrinsic properties across different time scales.
To rigorously assess the effectiveness of the representations learned by NeurPIR, we model neuron population dynamic data using the Izhikevich model [zhikevich| (2003), where different neurons are assigned distinct hyperparameters representing different firing modes as intrinsic properties. These neurons are randomly connected to form a network, and after stimulating the network, we obtain neuron population data. The representations learned by through supervieedd learning on this neuron population data have been confirmed to contain hyperparameter information. In recognition of the noise and complexity in real-world neuronal datasets, we turn to a publicly available dataset of mouse brain neuron populations. These neurons, after recording dynamic activities, are separated and labeled with cell type annotations using transcriptomics. The representations learned by on this dataset can be efficiently utilized for downstream cell type prediction tasks. In addition, using another real public dataset containing ten mice and 39 datasets, we demonstrate that the intrinsic information contained in the representation obtained by is consistent across animals, and that this representation has strong cross-domain capability when performing downstream tasks.
RELATED WORK
Single Neuron Models: Single neuron models are essential in understanding the fundamental prop-
erties of neuronal dynamics and behavior. The Leaky Integrate-and-Fire model Liu & Wang (2001) developed in the 1950s, is a minimalistic model that simulates the integration of synaptic inputs and the leakage of membrane potential over time. The Hodgkin Modgkin-Huxley model Nelson & Rinzel (1955), introduced in 1952, provides a detailed description of action potential generation using complex equations to represent ionic currents across the neuronal membrane, offering deep insights into neu-
2

---

Published as a conference paper at ICLR 2025
ronal excitability. The -aaguuo model Izhikevich & FitzHugh (2006), proposed in 1961, simplifies the Hodgkin-Huxles model to focus on excitability and action potential dynamics while reducing computational complexity. Finally, the Izhikevich model, developed in 2003, combines simplicity with versatility, effectively capturing a wide range of neuronal firing patterns and bal- ancing computational efficiency with biological realism. The hyperparameters set for these models can be viewed as prior assumptions about the inherent properties of the neurons, In the subsequent sections, we select the Izhikevich model to generate a synthetic dataset for testing, using the set hyperparameters as the true labels.
Neural Latent Representation Learning: Neural Latent Representation Learning has been pivotal in transforming high high-dimensional neuronal data into lower- swwer-dimensional embeddings that encapsu- late instantaneous information [Bengio et al. (2013). remarkable impact has been made in neuro- science from the linear dimensionality reduction techniques such as Principal Component Anal- ysis (PCA) | Maćkiewiez & Ratajezak (1993) to the non-linear visualization methods like Uniform Manifold Approximation and Projection (UMAP) Melnnes et al. 2018) and t-Distributed Stochastic Neighbor Embedding (I-SNE) Kobak & Berens 2019) and most recently, to the advanced, data- driven deep learning strategies. The focus was first reducing the dimensionality of neuronal data alone. It later expanded to include joint dimensionality reduction with behavioral information and external stimuli (e.g.. pi- VAE Prakash (2024) and BLEND Guo al. (2025)). CEBRA represents a culmination of these advancements, integrating various techniques into a unified framework. CE- BRA is used in this study to integrate the stimuli information experienced by each neuron as input for NeuPRINT is method used to extract invariant information from neurons Mi et al. (2023); how- ever, still adheres traditional neuron modeling approaches, which can only implicitly represent neurons. This results in challenging training processes and suboptimal representation performance.
Contrastive Learning for Voice Representations: The basic approach of the solution presented in this paper draws inspiration from similar tasks, such as extracting inherent representations of speakers from voice data Torres al. (2024). This type of work has been implemented on song- singer datasets, where contrastive learning is used to bring together the voices of the same singer while pushing apart the voices of different singers. However, while we borrowed the underlying idea, the specific methods had to be uniquely designed to accommodate the characteristics of neural data.
3 METHOD
3.1 GOAL
Our goal is to develop method for learning intrinsic neuron representations from neuron population data. These representations should meet three key criteria: (i) neurons with similar functional roles should exhibit greater similarity in their representations compared to those with different roles; (ii) the learned representations should be robust to variations neuronal activity patterns caused by different stimuli environmental conditions; and (iii) the representations should be adaptable and generalizable to new and unseen neuronal activity patterns.
3.2 ARCHITECHTURE
The ideal embedding space for neuron representations should cluster recording segments of the same neuron while also ensuring semantic consistency by placing similar neurons close each other within the space. In line with the criteria outlined in Section 3.1 We conducted advanced contrastive learning loss functions, VICReg Bardes et al.) (2021). We also carefully designed the data sampling methods to generate multi-segments activity data for each neuron for training purposes.
Data sampling: Without loss of generality, we take bi-segment data as example. Consider two scenarios: 1) If neuron has recordings from multiple sessions, we randomly extract two segments (Xeeg> xeeg)) from different sessions. If neuron has recordings from only single session, we randomly extract two non-overlapping segments (Xeegr Xbeg) from that session. We repeat this process B times for batch size of B, obtaining positive pair batch (X!1) x
3

---

Published as a conference paper at ICLR 2025
Running Speed, etc.
Xst
Xst
Xbe
Neuron Index
Stimulus(Xst)
CEBRA
Bchavior(Xbe)
Xse:
Xse: 2
Session
Session(Xsc)
(VICReg)
Each Segment SI Embedding
Contrastive Learning For Segment Embedding
Neuron Recording Segment
Encoder
Positive pair
Negative pair Neuron Intrinsic Representation
Figure Optimization objective for obtaining intrinsic representations of neurons as follows: clips(segments) from the same neuron should have a higher average similarity than clips from dif- ferent neurons. Figure illustrates how this objective can be achieved using a contrastive learning approach, where different recording segments from the same neuron are treated positive pairs and segments from different neurons are treated as negative pairs. Each neuron is considered sep- arately, and the Stimulus, Behavior and Session treated as surrounding information(SI), which acts as auxiliary variables. The surrounding information for each neuron is processed and encoded using CEBRA.
Model: We should integrate surrounding information into each segment. In neurobiology, the stim- ulation received by the neuron corresponding to a segment (Xst), the animal's behavior at the corre- sponding time (Xbe), and the session information (Xae) are all the surrounding information (SI) of this segment. CEBRA is self-supervised learning algorithm designed to obtain interpretable and consistent embeddings of high-dimensional recordings using auxiliary variables. In simple terms, it serves as an encoder for the dynamic activity information of neuronal populations. Its advantage lies in its ability to encode both neuronal activity data (N*T) and corresponding auxiliary variables, such behavior and external stimuli (M+T), into lower-dimensional representation (D*T), where represents the latent space dimensions. In CEBRA, does not correspond individual neurons, but rather provides dimensionality reduction method that combines nigh-dimensional temporal activity data of neuronal populations with corresponding auxiliary variable information, aiming to uncover the relationships between neuronal activity and these variables.
Our work focuses on obtaining time-invant inv-iiatriant representations for individual neurons. We innova- tively CEBRA preprocessing step for input data from from the viewpoint individual neurons, we leverage CEBRA to integrate peripheral informa- tion-suce the activity of neighboring neuronal populations and behavioral data associated with
perspective. Specifically,

---

Published as a conference paper at ICLR 2025
each segment of a single neuron. Xst consists of two components: peripheral neuronal activity and visual stimulation. The dimensions of xt are n + 1: The n dimensions are the activity of the surrounding neurons relative to the neuron you re considering, because there will be coordinates for each neuron in the dataset, so for each neuron, we'll take the 47 nearest neurons in the experiment. The last dimension visual stimuli, and we have relatively simple way of dealing with visual stim- uli: Blank visual stimuli are represented by O, Drifting Grating visual stimuli by 1, Natural Scenes visual stimuli by and each time point represented by fixed number. In the future, could try to encode the video of visual stimuli so that it changes over time, but we currently only encode it according to the type of visual stimulus. The dimension of Xbe 1: running speed each time point. The dimension of X is I: session number. The dimensions of X.i are 10: The total dimension of Xee X sn and Xst 50, we use CEBBRA project them to the low dimension
Xai CEBRA(X Xbe, Xse)
(I)
We then denote the pair (Xai, Xeeg )oo the same time as X in the following sections. The encoder G()) maps the extracted input X into latent representation H'' which is then aggregated into time- invariant feature embeddings H using adaptive average pooling. These embeddings H are further mapped into lower-dimensional space Z by projection layer P(-). The full model F(-) combines feature extraction, encoding, and projection. During training. F(X) produces the projections Z. After training. the projection layer is removed, retaining only the embeddings H. The similarity between embeddings is measured using cosine similarity.
Contrastive Learning- vICReg: VICReg aims enhance the quality of learned embeddings by incorporating three types of losses: variance loss, invariance loss, and covariance loss.
The invariance loss, ensures that embeddings of segments from the same neuron close:
Cunaar (1()
1 Σ -( N
(2)
The variance loss regularizes the standard deviation of the embeddings be near a target value f4, which helps prevent embedding dimensions from becoming non-informative. Given dj(z) E BR vector of batch values dimension j. the variance loss is defined as:
1 D L (2) = D Σ max (0, p ((,()S(d(()..)))
(3)
3=1
where D represents the number of dimensions in z, and S denotes the regularized standard deviation, S(x,e) e) - / Var(<)= +
The covariance loss promotes orthogonality among embedding dimensions by decorrelating them:
1 [(C()))
Loov (z) D2
(4)
where C(z) =
1 N-I Em (z, 2) (z, =) is the covariance matrix of z, and ž = N En
4 EXPERIMENTS
4.1 DATA
Since the neuron intrinsic property is hardly available in vivo neuronal recordings, we applied to synthetic data where we can access the ground-truth intrinsic property. To make the synthetic data exhibit dynamics similar to that of real neurons, we simulated the data following the Izhikevich model. The Izhikevich model is a spiking neuron model that combines biological realism with computational efficiency. It is designed to capture the rich dynamics of real neurons while remaining computationally simple. The model is defined by the following differential equations:
Simulated Data:
dV
0.04V +5V + 140 u +I,
(5)
dt
5

---

Published as a conference paper at ICLR 2025
du a(bV u),
di
where V is the membrane potential of the neuron, u is a recovery variable, 7 is the input current, and a, b, and are hyperparameters. The model also includes spike-reset mechanism:
V c u t u u+b
if V 30 mV, then
a. b, and d can be regarded as intrinsic properties of each neuron.
We utilized rare, real-world multimodal dataset Bugeon al. (022 to test which comprises two main components: 1) in vivo long-term recordings of multiple neurons in the mouse primary visual cortex using two-photon imaging technology. and 2) spatial transcriptomics of ex vivo slices from the recorded brain tissue to measure the expression levels of 72 selected genes in these neurons. Based on gene expression levels, neurons were assigned two main types of labels: (i) excitatory and inhibitory classification, and (ii) subclass labels (Lamp5, Pvalb, Vip. Sneg, and Sst) for some inhibitory neurons. This dataset contains recordings from four mice (Mouse A, B, C, and D). Mouse A has data from 6 sessions, Mouse B from 3 sessions, Mouse C from 5 sessions, and Mouse D from 3 sessions. During each session, the mice were exposed to three types of visual stimuli: Blank, Drifting Gratings, and Natural Scenes. A total of 9,278 unique neurons were recorded in the dataset.
Real Data Bugeon:
Real Data Steinmetz: The Steinmetz dataset comprises 39 Neuropixels recordings, capturing data from 400 to 700 neurons across various regions of the mouse brain of 10 mice during a vi- sual behavior task Steinmetz al. (2019). This dataset is excellent for exploratory analyses and is well-supported by extensive tutorial resources [] alongside a wealth of experimental and behavioral variables included within.
4.2 EVALUATION
The process is divided into steps: (i) perform self-supervised contrastive learning on the dynamic data of all neurons to obtain a representation for each neuron; (ii) Employ 5-fold cross-validation approach use these neuron representations input classifier for predicting the pre-defined neuron type labels from the simulation.
Evaluation on Simulated Data:
The process is divided into three steps: (i) perform self- supervised contrastive learning on the dynamic data of all neurons from four mouse in the real dataset obtain a representation for each neuron; (ii) Based on neurobiological prior knowledge and spatial transcriptomic gene expression information, assign cell type labels to each neuron. The labels fall into two categories: (a) excitatory and inhibitory, and (b) Lamp5, Pvalb, Vip. Sneg, and Sst; (iii) Implement a 4-fold (with the folds based on the identity of the mice) cross-validation approach where the neuron representations are used as input to a classifier to predict the neuron's type labels for both categories (a) and (b).
Evaluation on Real Data Bugeon:
Out-of-Domain Evaluation Steinmetz dataset: The process is divided into three steps: (i) Perform self-supervised contrastive learning the dynamic data of all neurons from all mice in the real dataset to obtain a representation for each neuron; (ii) As before, assign location within brain regions labels the neurons; (iii) Implement 10-fold (with the folds based the identity of the mice) vross-validation approach where the neuron representations are used as input to a classifier to predict the neuron's location. During neurodevelopment, where the position of a neuron crucial for its differentiation, maturation, and connectivity. The location can influence the neuron's gene expression, synaptic connections, and ultimately its function Patel & Poo (1982). In this sense, the location is an intrinsic property because it defines role within the nervous system.
4.3 COMPARISON OF METHODS
LOLCAT: This method et al. (0023)) follows a supervised learning paradigm. It directly uses activity data from subset of neurons train classifier to predict neuron labels, and then validates on the remaining neurons. Consequently, the representations learned in the intermediate

---

Published as a conference paper at ICLR 2025
layers of the model contain only label information and do not fully capture the intrinsic properties of the neurons.
PCA: This method employs self-supervised approach. Principal Component Analysis (PCA) duces the dimensionality of each neuron's ' activity data by projecting it onto a lower-dimensional space, thereby providing representation based on the most significant components of the activity data.
UMAP: Similar to PCA, Uniform Manifold Approximation and Projection (UMAP) is self- supervised method that reduces the dimensionality of each neuron's 's activity data. UMAP preserves local and global structures in the data create a meaningful lower-dimensional representation.
This self-supervised method involves implicit representation learning through back- propagation. Due its implicit nature, the model can be difficult converge and may require substantial training data and time to achieve effective results, and the representations may not align well with the Platonic Representation Hypothesis.
NeurPrint:
To demonstrate that the representations learned only from neuronal activity capture invariant prop- erties, we designed verification process using simulation data. Based on the Izhikevich model, we defined the neurons' time-invariant hyperparameters as their intrinsic properties and applied stimulation to generate activity data (Figure 2). Our model relies solely on activity data to derive time-invariant representations of neurons and validates these representations by showing their abil- ity to distinguish predefined intrinsic hyperparameters. To evaluate the effectiveness of our method, we classify intrinsic hyperparameters (five categories representing five neuronal firing modes) us- ing the obtained intrinsic representations and compare its performance with four other algorithms: PCA. UMAP, NeuPRINT, and LOLCAT. The classification performance is assessed across multiple neuron types using three common metrics: Precision, Recall, and F1-score. Higher values of these metrics indicate better classification performance.
From the results presented in Table 4.3 we observe that: (i) our method consistently achieves the highest performance across most neuron firing modes, particularly in the Regular Spiking (RS) and Fast Spiking (FS) categories, where it significantly outperforms all baselines with Fl-scores of 0.884 and 0,881, respectively; (ii) although some baselines, such as LOLCAT and NeuPRINT, demonstrate competitive results in certain neuron firing modes like Low-Threshold Spiking (LTS), their performance falls short compared to NeurPIR. For instance, in the case of Chattering (CH) neurons, NeurPIR achieves an F1-score of 0.671, surpassing LOLCAT's 0.652 and NeuPRINT''s 0.611. These results underscore the robustness and accuracy of NeurPIR across varying neuron firing modes, establishing it the most effective method in this comparison. Furthermore, these neuron firing modes are determined by preset hyperparameters, further validating that the intrinsic representations learned by NeurPIR effectively capture hyperparameter information.
4.4 REAL DATA NEURON PLATONIC INTRINSIC REPRESENTATION CONTAINS MOLECULAR INFORMATION
In this section, we utilized a public multimodal dataset, referred to Bugeon, train and evaluate our model. We compared the performance metrics of neuron type classification for three method- ologies: NeuPRINT, LOLCAT, and the proposed NeurPIR (PCA and UMAP were excluded as they cannot process behavioral information). The evaluation metrics included precision (Prec.), recall (Rec.), and F1 score, which are essential for assessing the classification effectiveness across various neuron types, categorized into subclasses and classes.
The results for the subclasses Lamp5, Vip, Pvalb, and Sst are summarized in Table A.I For the Lamp5 subclass, NeurPIR achieved the highest F1 score (0.569 + 0.014), demonstrating balanced classification performance. Similarly, in the Vip subclass, NeurPIR outperformed others with an Fl score of 0.662 + 0.035, highlighting its superior accuracy in classifying Vip neurons. For Pvalb neurons, NeurPIR again led with an Fl score of 0.604 + 0.043, reflecting its effectiveness in this subclass. In the Sst subclass, although NeuPRINT achieved the highest precision (0.704 + 0.168), its large standard deviation indicates variability. NeurPIR showed the highest F1 score (0.492 + 0.080), demonstrating robustness despite the challenges in classifying Sst neurons.
Overall, the results demonstrate the effectiveness of NeurPIR in accurately classifying distinct neu- ronal subtypes, achieving improved precision and Fl scores compared to existing methodologies.
7

---

Published as a conference paper at ICLR 2025
Firing Modes
Metric
PCA
UMAP
NeuPRINT
LOLCAT
NeurPIR
Prec.
0.689
0.473
0.836
0.783
0.872
regular spiking (RS)
Rec.
0.918
0.590
0.908
0.910
0.898
FI.
0.786
0.525
0.870
0.841
0.884
Prec.
0.534
0.332
0.646
0.681
0,678
intrinsically bursting (IB)
Rec.
0.375
0.310
0.645
0.573
0.693
FI.
0.440
0.320
0.644
0.620
0.684
Prec.
0,506
0.335
0.648
0.616
0.722
chattering (CH)
Rec.
0.603
0.248
0.580
0.698
0.630
F1.
0.548
0.285
0.611
0.652
0.671
Prec.
0.778
0.602
0.826
0.853
0.853
fast spiking (FS)
Rec.
0.620
0.638
0.820
0.733
0.913
F1.
0.689
0.618
0.823
0.787
0.881
Prec. 0.970
0.944
0.968
0.993
0.990
low-threshold spiking (LTS)
Rec.
0.935
0.965
0.993
0.983
0,990
FI.
0.952
0.954
0.980
0.988
0.990
Table 1: This table presents the performance metries precision (Prec.), recall (Rec.), and Fl five methods (PCA, UMAP, NeurPrint, LOLCAT, and NeurPIR) for different neu- types: regular spiking (RS), intrinsically bursting (IB), chattering (CH), fast spiking (FS), and low-threshold spiking (LTSS). The results indicate that and NeurPrint consistently achieve higher precision and Fl scores for most neuron types, while UMAP shows relatively lower performance, particularly for IB and CH neurons.
These findings highlight that the neuronal representations generated by our method effectively en- code neuron type information. Since neuron types are determined by molecular characteristics, this suggests that the intrinsic representations also capture molecular information.
Neuron Type
Metric
LOLCAT
NeuPRINT
NeurPIR
Subclass
Prec.
0.460 + 0.064
0.590 + 0.073
0.607 + 0.049
Rec.
0.418 + 0.084
0.562 + 0.093
0.587 = 0.067
Fl.
0.401 + 0.056
0.553
0.080
0.582 + 0.059
Class
Prec.
0.619 0.015
0.711 = 0.044
0.755 + 0.023
Rec.
0.584
0.707 + 0.047
0.749 + 0.040
F1.
0.552 0.018
0.706 0.047
0.747 = 0.016
Table 2: Average performance metrics for neuron type classification. Subclass averages are cal- culated across Lamp5, Vip, Pvalb, and Sst, while Class averages are calculated across Ex and In. Metrics include precision (Prec.), recall (Rec.), and Fl score, with errors represented as standard deviations.
Method
Domain
Precision
Recall
In-Domain
0.701 + 0.093
0.674 + 0.084
0.694 + 0.072
LOLCAT
Out-of-Domain
0.659 0.075
0.644 + 0.092
0.652 + 0.073
In-Domain
0.764
0.091
0.744 0.081
0.743 0.071 0.662 + 0.072
NeuPRINT
Off-of-Domain
0.665 = 0.079
0.678 + 0.092
In-Domain
0.764 0.093
0.749 + 0.072
0. 0.758 + 0.066 0.708 + 0.054
NeurPIR
Out-of-Dommal
0.701 = 0.081
0.672 + 0.074
Table 3: Average performance metrics for In-Domain and Out-of-Domall data across all regions (Vis, Thal, Hipp, Mid) for each method. Metrics include precision, recall, and Fl score, with errors represented standard deviations.
8

---

Published as a conference paper at ICLR 2025
4.5 REAL DATA SHOWS ROBUSTNESS ON OUT-OF- -F-MAAIN (UNSEEN ANIMAL) DATA
In this section, we focus on validating the consistency of the intrinsic representations generated by the model across different animals. The Steinmetz dataset, which includes neural activity data from ten rats, provides neuron labels based on their respective brain regions. It can be intuitively seen from Figure 3 that the response patterns of neurons the same brain area are significantly different in different mice, but we hope that the representation obtained from the model still contains consistent information, like consistent brain area information. This corresponds to the evaluation of the generalizability of the representations obtained by the model cross cross-modal (here, cross- animal) data in deep learning. We use a task of classifying location within brain regions to validate the intrinsic representations. We use NeurPIR perform self-supervised training on all the neurons from the all mice to obtain the intrinsic representations. For the downstream task of brain region classification, we used 10-fold valssavaii (with folds based on the identity of the mice).
As shown in Figure 4.4 the model's generalizability is evaluated through brain region classification across different neural representations. In-domain validation results demonstrate that consistently outperforms other methods like LOLCAT and NeuPRINT across all location within brain regions, including visual cortex (ViS), thalamus (Thal), hippocampus (Hipp), and midbrain (Mid). Specifi- cally, achieves validation Precision close to 0.80 in most regions, with NeuPRINT slightly trailing behind.
When examining out-of-domain performance (cross-animal), we notice a general drop in accuracy for all methods. However, NeurPIR still retains a higher degree of accuracy across all location within brain regions compared to NeuPRINT and LOLCAT, showing the model's ability to capture more robust intrinsic representations across different animals. This consistent outperformance across both in-domain and out-of-domain tests suggests that the representations obtained by better generalize across animals while still preserving critical brain region information.
CONCLUSION AND DISCUSSION
In this paper, we present a novel and scalable approach for extracting and leveraging the intrinsic properties of neurons. This approach holds significant potential for re-evaluating existing neuro- science data and enhancing our understanding of neural computation. Future research may focus on further enhancing its ability handle an even more diverse range of datasets and applying it to other domains where extracting intrinsic properties from complex systems is of great importance. Limitations: (i) The representation learned by our method can only distinguish neurons with sub- stantial differences in essential attributes. For instance, when neurons are associated with more refined brain-area labels, it becomes challenging to differentiate them, necessitating more data for training support. (ii) The learned neuron representations only support data collected from the same technology. The generalization cross platform data, such as two-photon data and neurpixel data, remains to be explored. (iii) Over very long timescales, some of the short-term invariant properties of neurons may change. This can be exploited to study the changes in neuronal properties during the progression of diseases like Alzheimer's ' disease.
REPRODUCIBILITY STATEMENT
To enhance the reproducibility of this study, we provide Appendix section comprising 4 sub- sections that offer detailed supplementary information. Appendix A.3 presents the pseudo-code of Synthetic Data. Appendix A.4 presents python code for downloading and organizing the steinmetz dataset. Appendix A.5 presents the pseudo-code of sownstream task. Appendix A.6 presents De- scription and Function across firing types / neuron types / brain regions in this paper.
ACKNOWLEDGEMENT
This research supported by National Natural Science Foundation of China (6220071694), Na- tional Key Research and Development Program of China (2024YFF050740X0) and Young Elite Sci- entist Sponsorship Program by Beijing Association for Science and Technology (BYESS2023026).

---

Published as a conference paper at ICLR 2025
REFERENCES
Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Varitace-ivarrancc-ovvaaiine regularization for self-super learning- arXiv preprint arwiv:220S.c...mm 04906, 2021.
Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence, 35(8):1798-188) 1828, 2013.
Stephane Bugeon, Joshua Duffield, Mario Dipoppa, Anne Ritoux, Isabelle Prankerd, Dimitris Ni- coloutsopoulos, David Orme, Maxwell Shinn, Han Peng, Hamish Forrest, al. A transcriptomic axis predicts state modulation of cortical interneurons. Nature, 607(7918),330-333) 2022.
Zhengrui Guo, Fangxu Zhou, Wei Wu, Qichen Sun, Lishuang Feng, Jinzhuo Wang, and Hao Chen. Blend: Behavior- guided neural population dynamics modeling via privileged knowledge distilla- tion. 2025. URL wtt.t:tps///// arxiv. http://www...../b////00...00
Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis. 2024. Utth:tss::///// //axivv..iivv.v/aa org/ abs/ 20os/240 2200/ 2405.07987
Eugene Izhikevich. Simple model spiking neurons. IEEE Transactions on neural networks, 14(6):169-1572, 2003. 2003.
Eugene M Izhikevich and Richard FitzHugh. Fitzhugh-nagumo model. 2006.
Scholarpedia, 1(9):1349,
Dmitry Kobak and Philipp Berens. The art of using t-sne for single-cell transcriptomics. Nature communications, 10(1):5416, 2019,
Ying-Hui Liu and Xiao-Jing Wang. Spikeeryy adaptation of generalized leaky integrate- and-fire model neuron. Journal of computational neuroscience, 10:25 2001.
Andrzej Mačkiewicz and Waldemar Ratajczak. Principal components analysis (pea). Computers & Geosciences, 19(3):303-32, 1993.
Leland Mclnnes, John Healy, and James Melville. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv preprint arXiv:1802.03420. 2018.
Lu Mi, Trung Le, Tianxing He, Eli Shlizerman, and Uygar Sümbül. Learning time-invariant rep- resentations for individual neurons from population dynamics. Advances in Neural Information Processing Systems, 36:46007-46000 2023.
Mark Nelson and John Rinzel. The hodgkin huxlley model. The book of genesis, 2, 1995.
N. Patel and M. M. Poo. Orientation of neurite growth by extracellular electric fields. Journal of Neuroscience, (4)):483449) 1982.
Rohan Prakash. Interpreting latent manifolds of high-dimensional neural activity using pi-vae. Mas- ter's thesis, New York University Tandon School of Engineering, 2024.
Aidan Schneider, Mehdi Azabou, Louis McDougall-
David Parks, Sahara Ensley, Kiran Bhaskaran-Nair, Tomasz Nowakowski, Eva L Dyer, and Keith B Hengen. Transcriptonic cell type structures vivo neuronal activity across multiple timescales. Cell reports, 42(4), 2023a.
Steffen Schneider, Jin Hwa Lee, and Mackenzie Weygandt Mathis. Learnable latent embeddings for joint behavioural and neural analysis. Nature, 617(7960):30--36, 2023b.
Nicholas A Steinmetz, Peter Zatka-Haas, Matteo Carandini, and Kenneth Harris. Distributed coding of choice, action and engagement across the mouse brain. Nature, 576(7786):266-73, 2019.
Bernardo Torres, Stefan Lattner, and Gael Richard. Singer identity representation learning using self-supervised techniques. arXiv preprint arXiv:2401.05064, 2024.
A APPENDIX
10

---

Published as a conference paper at ICLR 2025
A.1
DETAIL PERFORMANCE METRICS FOR NEURON TYPE/LOCATION CLASSIFICATION.
Neuron Type
Metric
NcuPRINT
LOLCAT
NeurPIR
Subclass
Prec.
0.481 + 0.035
0.344
0,487 = 01,007
Lamp5
Rec.
0.667 = 0.037
= 0.684 0.094
0.028
Fl.
0,559 = 0.036
0.400
0.569 # 0.014
Prec.
0.614 0.044
0.008
0.657 1657
Vip
Rec.
0.652 = 0.034
0.406
0.668 = 0.027
F1.
0.632 = 0.034
0.000
= 1
Prec.
0.559 0.046
0.028
0.602 0.042
Pvalb
Rec.
0.604 + 0.017
0.403 # 0.017
0.607 = 0.055
F1.
0.580 + 0.032
14415
0.604 = 0.043
Prec.
0.704 # 0.168
0.477 = 0.130
0.681 = 0.072
Sst
Rec.
0.323 = 0.084
0.170 = 0.077
0.388 = 0.080
F1.
0.441 # 0.105
0.248 = 0.100
0.492 = 0.080
Class
Prec.
0,685 = 0.050
0.555 0.010
0.720 = 0.025
Ex
Rec.
0.774 = 0.020
0.854 = 0.009
0.817 = 0.031
F1.
0.726 + 0.036
0.673 = 0.008
0.765 = 0.009
Prec.
0.737 # 0.037
0.682 0.020
0.790 = 0.020
In
Rec. 0.640 + 0.073
0.314 = 0.027
0.680 0.048
F1. 0.685 = 0.058
0.430 = 0.028
0.729 = 0.023
Table 4: Performance metrics for neuron type classification, 4-fold cross-validation was used, with the folds based on the identity of the mice This table delineates the precision (Prec.), recall (Rec.), and Fl score for various neuron types categorized into subclasses (Lamp5, Vip, Pvalb, Sst) and classes (Ex, In). The metries demonstrate the comparative performance of these methods in identi- fying and classifying distinct neuronal subtypes.
11

---

Published as a conference paper at ICLR 2025
Region
Method
Precision
Recall
F1
neu
0.923 # 0.029
0,614 = 0.052
0.736 + 0.042
Vis
0.751 0.047
0.467 0.105
0.572 0.086
LOLCAT
0.836 # 0.049
0.584 0.450 # 0.059
0.775 0.014
0,425 = 0.053
0.547
NeurPIR
0,900 = 0.016
0,652 = 0.024
0.756 0.013
0.825 0.017
0.514 0.012
0.633 0.013
NeuPRINT
0.758 0.025
0.762 # 0.005
0.760 = 0.013
Thal
0.669 = 0.025
0.724 # 0.019
#
LOLCAT
0.717 0.047
0.738 0.013
0.025
0.698 0.073
0.681
0.688
NeurPIR
0.765 # 0.025
0.772 0
4.768
0.725 0.049
0.761 # 0.027
NeuPRINT
0.708 0.033
0.785 # 0.013
0.744 0.023
Hipp
0.626 = 0.026
0.647 # 0.016
LOLCAT
0.668 0.034
0.742 0.703 0.018
0.595 # 0.028
0
0.044
NeurPIR
0.723 0.018
0.019
0.751
0.644
0.743
0.689 = 0.017
NeuPRINT
0.669 # 0.026
0.814 # 0.005
0.734 0.017
Mid
0.611 0.012
0.748 = 0.031
0,672 # 0,013
LOLCAT
0.584 0.018
0.768 0.010
# 1
0.569 0.024
0.726
NeurPIR
0.669 0.028
0.789 # 0.018
172400.00
0.634 = 0.027
0.740 = 0.013
0.683 # 0.021
Table Performance Comparison of Methods Across location within brain regions (In- Di-Dittttibuton The upper part of each indicator represents In Domain and the lower part represents Out of Domain.
vs
A.2 IYYPERPARAMETER OF FIVE FIRING MODES OF NEURONS WHEN SIMULATING DATA
12

---

Published as a conference paper at ICLR 2025
0.26
0.2
0.28
0.38
0.08
-Man Meare Edlely
174 22.0
35.
Figure 2: the hyperparameters of five firing modes of neurons when simulating data; b: the neuron
firing: the index of regular spiking (RS) is 0-400, the index of intrinsically bursting (IB) is 400-800, the index of chattering (CH) is 800- 1200, the index of fast spiking (FS) is 1200-1600. the index of low-threshold spiking (LTS) is 1600-2000; c: the results of the visualization of neuronal activity data using UMAP.
A.3 SIMULATION OF NEURONAL POPULATION USING THE IZHIKEVICH MODEL
1. Define the Izhikevich model parameters for each neuron type:
Regular Spiking (RS)
Intrinsically Bursting (IB)
Chattering (CH)
Fast Spiking (FS)
Low-Threshold Spiking (LTS)
2. Initialize the total number of excitatory (N.) and inhibitory (N,) neurons.
3. Assign neuron types to indices:
25% Regular Spiking (RS)
25% Intrinsically Bursting (IB)
25% Chattering (CH)
25% Fast Spiking (FS)
4. Initialize synaptic connection matrix
5. Set initial values for membrane potential v and recovery variable
6. For each time step t from 0 to T:
Calculate input current T.
If t > 0, add synaptic contributions from previously fired neurons.
Update membrane potential and recovery variable u using Fuler's method.
Check for spikes and record firing events.
Reset membrane potential and increment recovery variable for fired neurons.
13

---

Published as a conference paper at ICLR 2025
Store current and potentials for analysis.
7. Update neuron activity data.
8. Plot firing events.
9. Print shapes of activity arrays for each neuron type.
A.4 PYTHON CODE FOR DOWNLOADING AND ORGANIZING THE STEINMETZ DATASET
@title Data retrieval
import os, requests
fname []
for j in range (3)
fname append stwiimeta_parttddd..te.. i
url = -
url.append httppp//hhhhpp/s///o../////////a url. (r1.append""httpp hhttp//////off////////////oooaaa
for j in range (len(u))))
if not os.path.isfile (fname [j]): try:
r requests. (url(i])
except I
onError: i
print
else:
if r.status_code != requests.codes
else:
with open (fname [i], "wb") as fid:
write (r. content)
@title Data loading alldat np ap.array ([)) for j in range (len (fname)):
alldat np. hstack (aaldatt,
np. 1pal (' isteiimetz_partdddnnk $ j,
allow_pickle-True) [' ('dat']))
14

---

Published as a conference paper at ICLR 2025
A.5 PSEUDO-CODE OF DOWNSTREAM TASK
Algorithm Neural Network Classification with K-Fold Pross-Vallidation
1: Import necessary libraries: NumPy, scikit-lcam, Seaborn, Matplotlib
2:
Ignore warnings
3:
4: procedure MAIN
5: Initialize labels as NumPy array
6:
Encode labels using LabelEncoder Standardize neuron features using StandardScaler
7:
8:
9: Initialize StratifiedKFold with splits
Initialize statistics dictionaries for precision, recall, F1-score Initialize empty list for confusion matrices
11:
12:
13:
for each fold in K-Fold do
14:
Split data into training and test sets Create MLPClassifier model
15:
16:
Fit model to training data Predict labels for test data Generate classification report and confusion matrix Append confusion matrix to list
17:
18:
20:
for each cell type in classes do
22:
Record precision, recall, and F1-score end for
24:
end for
26:
for each cell type in classes do
27:
Calculate and print average metries
28:
end for
29:
30:
Compute cumulative confusion matrix
31:
Print cumulative confusion matrix
32:
33:
Optional: Plot cumulative confusion matrix using Seaborn
end procedure
15

---

Published as a conference paper at ICLR 2025
A.6
STEINMETZ DATASET
a
Brain areas present in each dataset
area
thl
ND9
midorain
Con
Cori
Torssmann
Porssmom
cossmann
canch
sere canaaea 12 sereen
gE uiree
1. unte
B6 Lale
to 11
8
22 2 24 tr.
2
25
12 26 aiae
51
01
dataset
b
3.0
T1eae (nivg tate (r-
4.5
2.5
4.0
2.25
2.0
3.5
2,00
1.5
3,0
1.50
100
130
200
50
100
150
200
250
50
1.00
150
200
250
right
vis
thal
hipp
mutther
10p4 le 1igtte Nggeet (M4
Frn0 1991
mean ng rate 111t
bocs
meen
Figure 3: Steinmetz dataset contains 39 subdataset from 10 mice. Each subdataset records 400- 700 neurons each from across the mouse brain during a visual behavior task. Each neuron with its brain area label. b: show of part of subdataset3. show of part of subdataset26.
A.7 DESCRIPTION AND FUNCTION ACROSS FIRING TYPES NEURON TYPES BRAIN REGIONS IN THIS PAPER
Firing Types:
1. Regular Spiking (RS) Neurons:
Description: Regular Spiking neurons are characterized by their ability to fire action potentials at regular, predictable rate in response to sustained depolarizing stimulus. These neurons typically exhibit a linear relationship between input and output, meaning they can respond to small inputs with consistent firing pattern.
Intrinsically Bursting (IB) Neurons:
Intrinsically Bursting neurons produce bursts of action potentials in response to a depolarizing stimulus, addition to regular single spikes. This type of neuron displays a unique pattern of activity where series of action potentials is generated in quick succession followed by period of quiescence.
Description:
3. Chattering (CH) Neurons:
Description: Chattering neurons exhibit a high- fighffreuuenc, sustained firing pattern. These neu- are characterized by their ability to fire high rate in bursts, often exhibiting very rapid oscillatory behavior with minimal latency between spikes.
4. Fast Spiking (FS) Neurons:
16

---

Published as a conference paper at ICLR 2025
Description: Fast Spiking neurons type of inhibitory interneuron known for their ability to fire action potentials at very high frequencies (often greater than 100 Hz). They exhibit rapid and precise spiking in response to stimuli and critical for the regulation of network activity.
Twwrssheholl Spiking (LTS) Neurons:
Description: Low-Threshold Spiking neurons are characterized by their ability fire action po- tentials at relatively low levels depolarization. They are often described as having a "sensitive" or "easy-to- ressy-to-trigger" firing threshold, which enables them respond to subtle changes in membrane potential.
Neuron Types:
1. Lamp5 Neurons (Lysosomal-AAsooiittedd Membrane Protein 5):
Description: Lamp5 neurons type of GABAergic interneuron that expresses the Lamp5 pro- tein, which involved cellular trafficking and autophagy. These neurons are often found in regions of the brain involved in cortical circuits, particularly in the cortex.
2. Vip Neurons (Vasoactive Intestinal Peptide-expressing neurons):
Description: Vip neurons type of inhibitory interneuron that express vasoactive intestinal peptide (VIP), a neuropeptide involved in the modulation of neural circuits. These neurons typically have broad inhibitory effects in cortical regions and influence the activity of other interneuron types.
3. Pvalb Neurons (Parvalbumin-eepressing neurons):
Description: Pvalb neurons subtype of fast-spiking inhibitory interneurons that express the calcium-binding protein parvalbumin. These neurons are well known for their ability to fire action potentials at extremely high frequencies with minimal delay.
4. Sst Neurons (Somatostaaiin-xxpessing neurons):
Description: Sst neurons another type of inhibitory interneuron that express somatostatin, neuropeptide that inhibits neurotransmitter release. These neurons are typically involved in modu- lating cortical circuits and are especially important for regulating synaptic plasticity.
Brain Regions:
1. Vis (Visual Cortex): Occipital lobe, primarily in the calcarine sulcus and surrounding regions.
Hipp (Hippocampus): Medial temporal lobe, beneath the cerebral cortex.
3. Thal (Thalamus): Deep within the brain, just above the brainstem, the center.
Mid (Midbrain): Between the forebrain and hindbrain, just above the pons and below the thala-
mus.
17


Reviews:
[{"review_id": "w6ScNlZCyt", "reviewer": "Reviewer_eGKZ", "summary": "The paper introduced NeurPIR, a self-supervised contrastive learning approach to learn intrinsic representation for each neuron from population dynamics. The method leveraged CEBRA and VICReg for representation learning, and was evaluated using synthetic and two mouse datasets, showing ability to learn representations indicative of neuronal intrinsic properties that are decodable by downstream classifiers.", "questions": "* Line 54 and 55: the paper is motivated to make segments of activity from the same neuron or similar neurons to converge, while dissimilar neurons diverge. However, VICReg only uses positive pairs, and adds regularization to prevent representation collapse. How does using VICReg help push dissimilar neurons apart according to the motivation? \n* Figure 1: description in the texts and accompanying caption to understand this figure are missing. The figure does illustrate negative pairs, however, VICReg does not use negative pairs as mentioned above. How are negative pairs processed by the NeurPIR model?\n* Line 134: the goal was to learn intrinsic neuronal representations on neuron population data, but it does not seem that activity of other neurons are used in the model (equations 1 to 4). How does the model use population dynamics to learn neuronal representations?\n* Line 149: what is the length of one segment? Is there a chance that two randomly selected segments overlap with each other?\n* Line 155: what is session information $X_{se}$? What are dimensions of $X_{st}$, $X_{be}$, $X_{se}$, $X_{si}$?\n* Line 161: can the author provide more details on what is adaptive average pooling?\n* Line 202: how the target value $\\mu$ was set?\n* Figure 4: how about precision, recall, and F1 scores (to be consistent with Tables 1 and 2)?\n* It would also be helpful to provide additional details on the architecture design and training process, e.g. hyper-parameters, training time, etc.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "* The method combines CEBRA and VICReg to incorporate surrounding information and learn enhanced representation with contrastive learning, which is a novel approach.\n* The paper is well motivated and tackles an important problem in neuroscience.\n* The writing is clear and logical.", "weaknesses": "* The usefulness of the out-of-domain evaluation on Steinmetz dataset is questionable. It seems as described on line 266, the self-supervised contrastive learning is performed on all neurons of all mice, including the test mice, i.e. the self-supervised model and classifier have to be retrained everytime new mice come in. Can any part of the model at least be reused during test time?\n* Model architecture is not clearly explained by figures or texts. Some details of the methods are missing (elaborated in Questions).\n* Some ablation studies are missing that would otherwise be helpful to understand the method in greater detail. For example, an ablation on which surrounding information included in the CEBRA framework has the most impact, or an ablation on different choices of contrastive learning methods besides VICReg might be helpful. \n* Results in tables and figures do not have errorbars. Adding sensitivity analyses would be helpful to quantify how significant the improvements of NeurPIR over the baselines are.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "uMvyEIL3Bd", "reviewer": "Reviewer_mMop", "summary": "This work proposed NeurPIR, which focus on learning a platonic representation from neural activities data to reflect the inherent properties and neuronal identity, relating to molecular information. The goal of this work is to learned representations robust to variations due to external stimuli and experiment conditions. It utilized the self-supervised multi-segment contrastive learning strategy from CEBRA, and learning representations for neurons with compare data from different segments, different behavior information, session information, and for neurons share similar functional roles to align closely in their representations. And they further aggregate the representation with adaptive average pooling to extract time-invariant representations. They further incorporate VICReg loss to enhance the prediction. The work is evaluated on three benchmarks: Izhikevich simulation model, spatial transcriptomics data with neural activities, neuron location with out-of-domain data, and compared with two existing baselines NeuPRINT and LOLCAT.", "questions": "1. Time complexity of NeuPIR compared to other baselines?\n2. How many data samples are required to learn effective embedding?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The work is evaluated on three representative benchmarks, including one synthetic dataset, and two neural datasets. And it is compared against with two baselines and demonstrating SOTA performance in most of the tasks.\n2. The work utilized a novel, distinct and effective method to utilize contrastive learning strategy to learn the platonic representation, compared to NeuPRINT with learning time-invariant representations with a neuron-wise look-up table for dynamics forecasting during self-supervised learning, or LOLCAT with label-guided representation with end-to-end supervised learning. \n3. The out-of-domain evaluation on unseen mice is an important question, which increases the soundness of the evaluations.", "weaknesses": "1. The core of the proposed approaches similar strategy as CEBRA to utilize the contrastive self-supervised approach for representation learning. The major difference from CEBRA is that it utilizes the adaptive average pooling to aggregated into time-invariant embedding, which might not necessarily guarantee the converge of the representation based on the data sampling. Further investigation could be done on how to guarantee converge based on different sampling strategy, or the requirement of amount of data to affect the predictive performance of the downstream tasks.\n2. The evaluation on Steinmetz dataset is evaluated on decoding brain region with the learned intrinsic representations is related to analyze the invariant properties of the neurons, while brain region is only coarsely reflecting information from neuronal level, would it be possible to evaluate on more fine-grained information such as spatial location of individual units?\n3. Ablation studies of VICReg loss should be performed.\n3. Sensitivity analysis (i.e. error bar) are not included in Fig 4, the effect of data shuffling, random initialization, etc. could be reported.\n4. Figure quality (i.e. font size, resolution) in the paper as limited, presentation and writing could be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YdEai6gRpV", "reviewer": "Reviewer_bVqe", "summary": "This paper proposes a contrastive learning method (NeuPIR) for analyzing single-neuron activity data, with the goal of obtaining a representation which preserves property-level similarity of neurons (e.g. cell type). The method combines a variational autoencoder (CEBRA) with a contrastive loss (VICreg). NeuPIR is applied to a synthetic dataset and real neural datasets with cell type and brain area information, and compared against a few other methods of feature extraction (PCA, UMAP, NeuPRINT, & LOLCAT).", "questions": "* The abstract says \"PRH posits that representations of different activity segments of the same neuron converge, while segments from inherently dissimilar neurons diverge\" (lines 19-21). What representations this is referring to is not clear in context.\n* The introduction should be more specific about what the method actually is, what the contrastive objective is, etc.\n* Identifying cell type seems like a paradigmatic task where compressing neural activity into binned firing rate loses important information which may be contained in the spike train.\n* The method section should provide more information about what CEBRA is.\n* The cross-animal generalization experiment (5.3) is interesting but as predicting cell type is the more relevant problem I would be interested to see a similar generalization experiment with a cell type dataset as in 5.2.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* Understanding the role of cell type diversity is a major challenge for neuroscience. A contrastive approach like this one, which can in principle be trained without labels, is likely to be the way forward given the difficulty of obtaining cell type information and activity simultaneously.\n* Furthermore, learning representations which preserve cell property information is a application of obvious interest to the ICLR community.\n* This paper performs the right experimental evaluations for its method, starting with a synthetic dataset, then moving to a real dataset where ground truth cell type information is available, and finally a real dataset with only brain region information, and compares against appropriate competitor methods.", "weaknesses": "* The framing in terms of the \"Platonic Representation Hypothesis\" [1] feels like hype chasing. What is being presented here is just a contrastive learning method which is meant to identify similarity and differences in properties of neurons. This is not actually related to the PRH in a meaningful sense, which is about how representations _of the world_ converge across models in completely different domains. I am willing to raise my score if the framing of the paper (primarily title, abstract, introduction) is substantially revised to take this into account.\n* The details of the method are not clear to me. CEBRA [2], at least as proposed, takes in a window of activity across a population of neurons (among other covariates) and maps it to a single latent point. I believe that here CEBRA is being applied to the activity of single neurons and their covariates but this is an important distinction which is not made explicit in the text.\n* I am not convinced that other methods are being fairly compared to NeuPIR. There is a lack of detail about how hyperparameter selection occurred in the experiments which makes this difficult to evaluate. For instance, LOLCAT fails to label any neurons at all as Sst in Table 2 which suggests it wasn't tuned correctly for the task.\n* The method is not significantly original as it is combining the pre-existing CEBRA architecture [2] with the VIC contrastive loss [3], making this nearly a pure applications paper. This is not necessarily a flaw but does put the burden of innovation on the value of its scientific findings.\n\n[1] Huh, Minyoung, et al. \"The platonic representation hypothesis.\" arXiv preprint arXiv:2405.07987 (2024).\n[2] Schneider, Steffen, Jin Hwa Lee, and Mackenzie Weygandt Mathis. \"Learnable latent embeddings for joint behavioural and neural analysis.\" Nature 617.7960 (2023): 360-368.\n[3] Bardes, Adrien, Jean Ponce, and Yann LeCun. \"Vicreg: Variance-invariance-covariance regularization for self-supervised learning.\" arXiv preprint arXiv:2105.04906 (2021).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vWzpEYH5aL", "reviewer": "Reviewer_Yi7j", "summary": "The paper proses a novel method called NeurPIR for extracting individual neuron representation based of neuron population recordings in a self-supervised fashion. For this purpose NeurPIR essentially combines a neural data specific sampling method, average pooled CEPRA embeddings, and a VICReg contrastive loss in a cohesive manner. The method is evaluated on a suite of synthetic and real neuron population activity recordings, where it superior performance to alternative methods.", "questions": "1) How exactly are the experiments on the Steinmetz dataset supporting the claims of neuron intrinsic representation learning?\n2) For the Bugeon Dataset: why was only data of mice A used?\n3) For the Steinmetz Dataset; why wan’t also e.g. a 10-fold crossvalidation used (folds along the mice identity)?\n4) Would you expect you method to also perform well if the number of Izhikevich neuron “types” was greatly increased (e.g. 10,20,30 categories)?\n5) Will/are the exact labels used for three experiments, and in particular for Bugeon be available for others to be able to reproduce the experiments exactly?\n6) Would you expect different results if max-pooling was used instead of mean-pooling?\n7) Is there any more literature / papers exactly doing single neuron characterization based on neuron population activity (possibly on other datasets)? It seems to be challenging to find more related literature.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper tackles a relevant, yet extremely challanging task: extracting individual neuron characteristics from neuron population data.\n- The paper convincingly demonstrates the proposed methods ability to do so to some extent, in particular compared to other available methods.\n- The related literature, methods and experiments section is very detailed and well written, aiding interpretability of the results and reproducibility.", "weaknesses": "* 1\\) The Steinmetz dataset experiment: likely doeant support the claim of being able to extract neuron internal represenations, as the method was trained to predict the location of the neurons, arguably an external neuron property?\n   - Even if two exactly same neurons were integrated into two different brain regions they are likely identifiable through their activity as the differences in input to the two regions typically significantly deviates.\n   - To claim exactraction of neuron intrinsic properties, the above effect would have to be significantly smaller than neuron intrinsic properly related differences, which is not validated.\n* 2\\) The Conclusion: is crucially missing a paragraph on the limitations of the proposed method and the experimental findings. The paper would significantly benefit from it.\n* 3\\) The Abstract: the first half is very confusing to read and doesn’t make it clear what the paper is actually about (for someone from a general computational neuroscience background). The paper would therefore greatly benefit from simpler and more concrete wording there.\n   - Examples of terms that were not really helpful to me : “decoupling of intrinsic properties, “time-varying dynamics”, “dynamic activities”, “varying signals”, etc. What property, whos dynamics, what activity, which signal?\n   - In particular “intrinsic properties” should be immediately followed by examples (later given), and the first mention of PRH feels out of place and its unclear how it relates to the sentences surrounding it. \n   - Furthermore, in the context of computational neuroscience “what information is conveyed by neural activities” most often implies figuring out what neurons try to communicate to process information, which the paper is not about. \n   - Finally, a statement like “NeurPIR captures the preset hyperparameters of each neuron” implies precise recovery e.g. $b = 0.2$. The proposed method was rather shown to capture rough categorical differences instead. More appropriate would could be e.g. “NeurPIR captures the class/category/type of each neuron”.\n* 4\\) The Figures: have barely readable label sizes and legends, or missing annotation.\n   - Figures 2 and 3 feature barely readable label sizes and legend\n  - Figure 1 could use more labels that help relate the model description to the images in the figure. (e.g. X, H, Z, P, F, CEBRA, VICReg)\n* 5\\) The Tables (or their descriptions): would benefit from some aggregate metrics across all categories.\n* 6\\) For Reproducibility: manual labeling like in the Bugeon is hard to reproduce, and its unclear to me from the text whether these will be / are provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)