Title: Generalized and Invariant Single-Neuron In-Vivo Activity Representation Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: neuroscience

neural activity

single neuron

in vivo

neurobiology

self-supervised learning

domain adaptation
Abstract: In computational neuroscience, models representing single-neuron in-vivo activity have become essential for understanding the functional identities of individual neurons. These models, such as implicit representation methods based on Transformer architectures, contrastive learning frameworks, and variational autoencoders, aim to capture the invariant and intrinsic computational features of single neurons. The learned single-neuron computational role representations should remain invariant across changing environment and are affected by their molecular expression and location. Thus, the representations allow for in vivo prediction of the molecular cell types and anatomical locations of single neurons, facilitating advanced closed-loop experimental designs. However, current models face the problem of limited generalizability. This is due to batch effects caused by differences in experimental design, animal subjects, and recording platforms. These confounding factors often lead to overfitting, reducing the robustness and practical utility of the models across various experimental scenarios. Previous studies have not rigorously evaluated how well the models generalize to new animals or stimulus conditions, creating a significant gap in the field. To solve this issue, we present a comprehensive experimental protocol that explicitly evaluates model performance on unseen animals and stimulus types. Additionally, we propose a model-agnostic adversarial training strategy. In this strategy, a discriminator network is used to eliminate batch-related information from the learned representations. The adversarial framework forces the representation model to focus on the intrinsic properties of neurons, thereby enhancing generalizability. Our approach is compatible with all major single-neuron representation models and significantly improves model robustness. This work emphasizes the importance of generalization in single-neuron representation models and offers an effective solution, paving the way for the practical application of computational models in vivo. It also shows potential for building unified atlases based on single-neuron in vivo activity.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors point out that many computational neuroscience models have the problem of poor generalizability across batches and propose a model-agnostic method to solve this problem. The method involves adversarial training, where a representation encoder and a batch discriminator are trained jointly so that batch information is removed from the representation encoder. Then, the authors proceed to evaluate their method with 2 different datasets and 4 different types of representation encoders (including transformer-based models, contrastive learning models, VAE-based models, and supervised models). The proposed adversarial training method can improve cross-stimulus and cross-animal generalizability.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n* The paper points out an important problem in computational neuroscience and proposes a simple yet effective method to solve the problem\n* The paper is well-motivated and easy to follow\n* The proposed approach is model-agnostic and seems easy to incorporate into other model training as well\n\nWeaknesses:\n* The paper has limited information on the details of adversarial training\n\nQUESTIONS:\n1. I am wondering how the batch labels are encoded during training. The paper says the batch label encodes \"experimental metadata (e.g., animal ID, stimulus condition, recording session)\". Is it done via a cross-entropy loss on a classification objective?\n2. Could you provide more details on how you chose the hyperparameters via cross-validation?\n3. Adversarial training is difficult to converge. Do you observe that some representation encoders are easier to work with than others in adversarial training?\n4. Regarding the discriminator, have you tried different sizes/depth/architecture, and do those choices lead to better/worse/similar performance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper is interested in the training of an embedding model of single-neuron categorization from functional recording. The authors provide a simple adversarial method to make the model generalize better to new stimulus conditions (or new animals). The method could be applied to enhance generalization across stimulus conditions. animal identity or others.\n\nThe key idea is to train the embedding model against a classifier in an adversarial fashion. While the classifier is optimized to classify stimulus conditions (or animals) from the embedding, the model is constantly optimized to make it harder for the classifier. In this way, the trained embedding holds no \"information\" about the stimulus condition (or animal identity). In practice, this appears to enhance generalization across animals. This technique is related to Domain adversarial neural networks outside of neuroscience, or gradient reversal training (cited by the authors).\n\nThe method is tested on three different embedding models of single neuron data (LOLCAT, NeuPRINT, and VAE). All three models use a different loss to construct an embedding of the neural activity. From the embedding, the performance is reported as the accuracy of a downstream cell type classifier from the embedding representation. It first confirmed that when one stimulus condition (or animal) is left out of training, the decoded performance of a trained cell-type classifier is lower when tested on the left-out stimulus (or animal). This defect is indeed reduced with the proposed method: during the training of the embedding, the drop in accuracy when tested on a new stimulus condition or animal (out of the training set) is reduced with the described method.\n\nSTRENGTHS AND WEAKNESSES:\nThe method is remarkably simple and could turn out to be very useful to reinforce the generalization capability of a model.\n\nStrengths:\n- The method is simple and seems easy to try in practice. It is appropriate to use that for neuroscience in this context.\n- The method can be applied broadly to pretty much any deep learning model, so the scope of applicability is large\n- The method appears to generalize to new animals or conditions, even with a fairly low number of animals in the training set\n\nWeaknesses:\n- The model is demonstrated on cell type decoding, generalization across animals, and brain area decoding. Yet the method is very general, so the benefits could have been demonstrated on more quantitative metrics. For instance, using the trained model at hand (VAE, etc), one could test if the validation metrics of these models generalize across datasets. \n- Similarly, I see no reason why focusing on single-cell functional activity. The method is likely to be usable as well with many cells or other contexts. It is not clear to me why single-cell data is specifically interesting in this context.\n- Unfortunately, the paper is somewhat hard to understand because of the writing. The English is grammatically correct, but the paragraph structure and choice of words are sometimes hard to follow when reading from top to bottom. In terms of structure, the different datasets are presented one after the other. Then, the experimental settings for all experiments are described. Then the results for all three experiments are presented. It means one needs to constantly switch between the experimental context, which is hard to follow. Regarding the wording, the concepts that are described as \"batch\", \"invariant\", and \"out-of-domain\" are often assumed to be understood by the reader without prior definition or motivation. Even if these terms are common, one needs more explanation to understand what is meant in the context of the paper.\n\nQUESTIONS:\n- Is the enhanced generalization also measurable for the VAE validation metrics ?\n\nLIMITATIONS:\nThe impact of the paper is limited by the writing style, which is not of the best quality.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 2, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nCurrent computational models of single-neuron activity commonly face generalization issues, often failing to accurately generalize to new stimulus types and unseen subjects.  The authors of this paper seek to address this limitation faced by current models by proposing an adversarial training strategy that encourages a model to predict single-neuron representations (usable for some downstream task such as predicting cell type) meanwhile fooling a discriminator that explicitly tries to predict batch-specific attributes that the representation model is susceptible to overfitting.  The authors demonstrate the efficacy of this proposed approach on two datasets: predicting cell type from using the V1-CellType dataset and predicting anatomical brain region using the IBL Brain-Wide Map dataset.  When training single-neuron representation models (LOLCAT, NEMO, and VAE models) with the proposed adversarial training strategy, the model generalizes much better (as compared to the same models trained without the adversarial learning strategy) to data from stimuli or subjects (animals) excluded from the training data.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Out-of-distribution robustness has historically been challenging in single-neuron representation models.  The results provided in this paper demonstrate the proposed approach as a relatively simple and flexible way to achieve greater out-of-distribution generalization, across subjects and stimulus types.\n- The proposed framework is generalizable and would be applicable to a wide variety of neuroscience modeling tasks.  It is not specific to any architecture, it can be applied to different batch-level attributes, etc.\n- Intuitive motivation and logical framework to solve this problem\n- Text and figures are clear and well-presented\n\nWeaknesses:\n- Technical novelty is limited: novel application but adversarial training with a discriminator to mitigate model biases (e.g., protected attributes) has been proposed in a variety of prior works\n- In depth evaluations beyond aggregate classification accuracies would are lacking.  For instance, the discussion section indicates that certain neuronal subtypes are poorly predicted, and it would be helpful if the paper showed this and any potential tradeoffs that resulted from using the adversarial training strategy.  Additionally, for transparency, I would like to see cross-animal evaluation results when different animals are used in the test set.\n- Related to the above, further analysis and probing into the efficacy of this method under constraints like dataset size, number of batch-level attributes, etc. would be appreciated.  For instance, would we still see similar improvements in generalization when only 2 stimulus types are used in training?  Does the method remain effective beyond classification tasks (e.g., predicting spiking activity from a subset of neurons to a given stimulus)?\n\nQUESTIONS:\n- In regard to weakness bullet point 3: How does expect this technique would scale with the number of batch-level attributes?  Neuroscience datasets are typically very limited in this regard.  A concrete experiment, for instance, may vary the number of stimulus types or animals available in training and evaluate as was done in the main text (evaluating performance across the remaining held-out stimulus types or animals).\n- Does this approach also boost in-distribution performance?  That is, if all stimulus types or animals are available in training (but we still maintain a held-out test split, as in the \"random split\" baselines of the paper), does the adversarial training still improve generalization?\n- Minor suggestion about data presentation in tables 1-3: In each table, the way the relative performance change is presented is unintuitive to me.  It seems that percent change in standard cross-attribute model results are presented relative to the random-split model, whereas cross-attribute + adversarial training results are presented relative to the standard cross-attribute model.  It would be more clear for both of these performance results to be presented relative to the random-split baseline.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-26"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper \"Adversarial Training for Generalized and Invariant Single-Neuron In-Vivo Activity Representation\" addresses out of domain generalization problems in neuron models that present single neurons for clustering and cell type identification. The authors establish an out of domain test protocol and show that combining existing methods with domain adversarial training improves the out of domain robustness.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- Biological clustering and cell type identification are important problems for neuroscience. Showing that existing methods suffer from out of domain generalization problems addresses an important problem. \n- I like that the paper is straightforward and doesn't try to be fancy when it's not necessary: Takes existing methods, shows that there is a problem, combines it with existing domain adversarial methods which largely fixes the problem. \n\nWeaknesses:\n- There is no novel algorithm here, but I don't think this is a major issue as the contribution is rather the finding and the fix with straightforward algorithms. \n- The paper could be improved in clarity of writing. Unless I completely misunderstood the paper, the main purpose is not to predict neuronal activity but rather to train an embedding to identify cell types. This is not very clear from the introduction and should be clarified. If the paper is about encoding model that predict neuronal activity, then a lot of the relevant literature is missing (a few examples are listed below).\n- The paper could also profit from another iteration of proofreading and structure refinement. For instance: On page 5 there is the paragraph on Theoretical Motivation. This paragraph then states that it does not \"want to introduce it too theoretically\" and refers to appendix A. Appendix A5 with the title \"theoretical motivation\" is 4 lines long, does not include any theoretical motivation and is not really informative by itself. I would ask the authors to make the presentation of the material more coherent. \n\nI. Ustyuzhaninov, S. A. Cadena, E. Froudarakis, P. G. Fahey, E. Y. Walker, E. Cobos, J. Reimer, F. H. Sinz, A. S. Tolias, M. Bethge, A. S. Ecker Rotation-invariant clustering of functional cell types in primary visual cortex International Conference on Learning Representations (ICLR), 2020\n\nEric Y. Wang, Paul G. Fahey, Zhuokun Ding, Stelios Papadopoulos, Kayla Ponder, Marissa A. Weis, Andersen Chang, Taliah Muhammad, Saumil Patel, Zhiwei Ding, Dat Tran, Jiakun Fu, Casey M. Schneider-Mizell, R. Clay Reid, Forrest Collman, Nuno Maçarico da Costa, Katrin Franke, Alexander S. Ecker, Jacob Reimer, Xaq Pitkow, Fabian H. Sinz, Andreas S. Tolias Foundation model of neural activity predicts response to new stimulus types Nature\n\nPolina Turishcheva, Max Burg, Fabian H. Sinz, Alexander Ecker Reproducibility of predictive networks for mouse visual cortex NeurIPS\n\nKonstantin-Klemens Lurz, Mohammad Bashiri, Konstantin Friedrich Willeke, Akshay Kumar Jagadish, Eric Wang, Edgar Y Walker, Santiago Cadena, Taliah Muhammad, Eric Cobos, Andreas Tolias, Alexander Ecker, Fabian Sinz Generalization in data-driven models of primary visual cortex ICLR \n\nA Spectral Theory of Neural Prediction and Alignment Abdulkadir Canatar, Jenelle Feather, Albert Wakhloo, SueYeon Chung\n\nQUESTIONS:\nPlease see weaknesses. In short:\n\n- Clarify better what your paper addresses. Predicting neuronal activity or learning neuronal fingerprints. \n- If its predicting neuronal activity: Then you need to account for related work (I listed examples above). \n- Improve clarity of the presentation of the material.\n\nLIMITATIONS:\nLimitations are addressed in the final paragraphs.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe manuscript addresses a practical weakness of recent single-neuron in-vivo representation models: their tendency to over-fit \"batch\" factors such as stimulus set, recording session, animal identity or hardware platform. The authors (i) define a rigorous OOD protocol, (ii) propose a model-agnostic adversarial remedy, (iii) demonstrate compatibility with LOLCAT, NeuPRINT, VAE and NEMO, and (iv) show robust improvements across mouse datasets.\n\nSTRENGTHS AND WEAKNESSES:\nThis is a technically solid, clearly written contribution that offers an actionable remedy and a much-needed evaluation protocol for OOD robustness in neural representation learning. Methodological novelty is modest, but the work's practical value and clean empirical validation make it a worthwhile addition to the literature.\n\nQUALITY\n\nStrengths:\n* Model-agnostic intervention (the gradient-reversal discriminator plugs into 4 different baselines)\n* Both accuracy/ROC and UMAP are reported, and improvements are consistent\n\nWeaknesses:\n* Only two mouse datasets are used; no check on primate or synthetic benchmarks, so the claim of \"cross-lab variation\" is still speculative\n* No results on alternative batch-invariance methods (e.g., mix-style, batchnorm-free) or sensitivity to $\\lambda$ (adversarial weight)\n* While embeddings preserve coarse cell-type groups, the paper does not test whether trial-level neural tuning curves remain interpretable after adversarial training\n\nCLARITY\n\nStrengths:\n* Writing is concise; the motivation for OOD splits is immediately clear\n* Architecture diagrams and tables are clear\n\nWeaknesses:\n* Key implementation details (optimizer, $\\lambda$ schedule, discriminator capacity) sit in Appendix D; surfacing them earlier would aid replication\n* The paper cites “single A100 @ 10 h” but omits batch size and sequence length, which is important for those without A100s\n\nSIGNIFICANCE\n\nStrengths:\n* addresses a practical painpoint for neuroscientists\n* provides a model-agnostic evaluation protocol for single-neuron representation work\n\nWeaknesses:\n* Impact is confined to neural-representation learning; broader ML domains have long used domain-adversarial training\n\nORIGINALITY\n\nStrengths:\n* First paper (to my knowledge) to marry domain-adversarial training with single-neuron time-series models *and* to quantify OOD gaps systematically across stimulus and animal factors\n* The dual OOD axes (stimulus-held-out vs animal-held-out) create a novel benchmark\n\nWeaknesses:\n* Gradient-reversal layers are standard in domain adaptation; the technical novelty lies more in application than in method\n* No new theoretical insights into why adversarial erasure of batch cues preserves task-relevant neuron structure\n\nQUESTIONS:\n* What is the motivation for picking a test set that is visually and statistically so unlike the training stimuli? I couldn't find a reference to Figure 1 in the main text. Is the goal to probe the most extreme distribution shift, or does this particular stimulus have biological relevance?\n* Primate cortex is notably different from mouse cortex, in terms of hierarchy, computation, and level of success that the deep learning community has had at modeling it. Will this translate to primate cortex and beyond? Which aspects of the adversarial objective do you expect to transfer or fail?\n* Currently the contribution is framed for single-neuron representation learning. Please explain explicitly how the findings might feed back into mainstream AI.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}