Title: Task-Optimized Convolutional Recurrent Networks Align with Tactile Processing in the Rodent Brain
Venue: NeurIPS 2025
Accepted: yes
Keywords: NeuroAI

Temporal Neural Networks

Tactile Perception

Somatosensory Cortex
Abstract: Tactile sensing remains far less understood in neuroscience and less effective in artificial systems compared to more mature modalities such as vision and language.
We bridge these gaps by introducing a novel Encoder-Attender-Decoder (EAD) framework to systematically explore the space of task-optimized temporal neural networks trained on realistic tactile input sequences from a customized rodent whisker-array simulator. 
We identify convolutional recurrent neural networks (ConvRNNs) as superior encoders to purely feedforward and state-space architectures for tactile categorization. 
Crucially, these ConvRNN-encoder-based EAD models achieve neural representations closely matching rodent somatosensory cortex, saturating the explainable neural variability and revealing a clear linear relationship between supervised categorization performance and neural alignment.
Furthermore, contrastive self-supervised ConvRNN-encoder-based EADs, trained with tactile-specific augmentations, match supervised neural fits, serving as an ethologically-relevant, label-free proxy.

For neuroscience, our findings highlight nonlinear recurrent processing as important for general-purpose tactile representations in somatosensory cortex, providing the first quantitative characterization of the underlying inductive biases in this system. 
For embodied AI, our results emphasize the importance of recurrent EAD architectures to handle realistic tactile inputs, along with tailored self-supervised learning methods for achieving robust tactile perception with the same type of sensors animals use to sense in unstructured environments.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper tackles rodent whisker-based touch from both an ML and a neuroscience angle.\nThe authors (i) port the WHISKiT Physics simulator to a 30-whisker mouse array and generate two large ShapeNet-derived datasets of realistic force-and-torque sequences; (ii) introduce a PyTorch \"Encoder-Attender-Decoder\" search space that spans feed-forward, state-space, and convolutional recurrent (ConvRNN) encoders plus optional attention modules; (iii) train 62 model variants under supervised categorization and contrastive self-supervised objectives that use force/torque-specific augmentations; and (iv) compare each layer's representations to barrel-cortex population recordings via noise-corrected RSA.\n\nThey find that (i) ConvRNN encoders (i.e. IntersectionRNN) outperform ResNet and SSM baselines on tactile categorization, with (ii) the best ConvRNN-based models “saturating” explainable neural variance, matching or exceeding inter-animal consistency without any fitted read-outs. Task accuracy and neural alignment are linearly related, and contrastive SimCLR models (with tactile augmentations) equal supervised models in neural predictivity, offering a label-free proxy. GPT-style attenders give modest but consistent gains over no-attention or Mamba variants.\n\nSTRENGTHS AND WEAKNESSES:\nThis paper delivers an end-to-end demonstration from realistic whisker physics through large-scale training to barrel-cortex validation. It is allegedly the first work to reach the noise ceiling in tactile cortex and thereby may illuminate key somatosensory inductive biases. The open release of code, simulator, and data further boosts value for both neuroscience and robotics. However, the current draft may over-claim: training on passive ShapeNet sweeps but testing on a six stimulus active-whisking set risks spurious ceiling-level fits, and architecture rankings lack formal statistics.\n\nQUALITY\n\nStrengths:\n* From a biomechanically faithful simulator to large-scale model training and direct neural comparison, the experimental flow is coherent and correctly executed.\n* 60+ Encoder-Attender-Decoder variants cover feed-forward, state-space, recurrent and attention architectures under both supervised and self-supervised regimes.\n* Split-half reliability filtering, noise-ceiling correction and layer-wise RSA are the accepted best practices for small-stimulus neural datasets.\n\nWeaknesses:\n* Models learn on passive sweeps of ShapeNet objects but are judged on active whisking to six simple shapes. This mismatch may inflate RSA by chance; an ablation including these test shapes in the training mix (held out for validation) is missing.\n* Bar plots lack formal significance tests; we do not know if IntersectionRNN > ResNet is reliable across mice\n* Six objects restrict the dimensionality of representational geometry and make it easier to “hit the ceiling.” Confidence intervals on the noise ceiling itself are not shown.\n* Search budget (GPU-days, hyper-parameter tuning strategy) is only sketched, making it hard to gauge efficiency or carbon cost.\n\nCLARITY\n\nStrengths:\n* Figures are information-dense yet readable; architecture, dataset and result diagrams are clear.\n* The summary of tactile augmentations vs. image-style augmentations is easy to follow and motivates the SSL results\n\nWeaknesses:\n* Introduction and related-work sections are bloated with robotics-hardware history that obscures the main contributions.\n* Key implementation details (augmentation hyper-parameters, exact RSA equation, IntersectionRNN definition) are relegated to the appendix. Some should appear in the main text.\n\nSIGNIFICANCE\n\nStrengths:\n* First convincing “goal-driven” account of somatosensory cortex\n* demonstrates that recurrent encoders plus force/torque-aware contrastive learning outperform common feed-forward baselines\n\nWeaknesses:\n* Impact on mainstream ML may be limited. The architectural lesson (“use ConvRNNs for smooth temporal signals”) is incremental relative to earlier vision work.\n* The neural dataset’s narrow scope means conclusions about “saturating variance” could be overturned once richer recordings emerge.\n\nORIGINALITY\n\nStrengths:\n* Combines three elements that have not previously appeared together: (i) a 3-D physically correct whisker array, (ii) a systematic temporal-network search that spans ConvRNN, SSM and attention, and (iii) noise-corrected RSA on barrel-cortex data.\n* Shows that SimCLR with bespoke tactile augmentations rivals supervised learning. This is a novel result in somatosensation and a direct answer to an open question from Zhuang et al. 2017\n\nWeaknesses:\n* The Encoder-Attender-Decoder framing is more a tidy taxonomy than a fundamentally new modelling concept; each module comes from prior work.\n* Attention module analysis is modest. Gains are small and mechanistic interpretation remains speculative\n\nQUESTIONS:\nFollowing up on my comments above, the paper would benefit from a targeted robustness ablation, stronger significance tests, and a leaner, more transparent methods presentation. Specifically:\n* Models are trained on passive sweeps of 9k ShapeNet meshes but evaluated neurally on active whisking to six cup-like shapes. Ceiling-level RSA might therefore hinge on an accidental projection onto a tiny stimulus sub-space. This needs at least one ablation to show robustness. For instance: retrain (or at least fine-tune) the best ConvRNN-EAD on a dataset that includes the six concave/convex objects (held out only for validation) and report RSA, to verify it remains near ceiling after ablation.\n* The lack of statistical testing undermines confidence in architecture ranking. We cannot tell if IntersectionRNN > ResNet is reliable across mice. To address this, the authors should provide hierarchical statistics (e.g. linear mixed-effects with random intercept per mouse) or paired permutation tests on RSA scores.\n* Six stimuli yield a noisy ceiling estimate. Please report split-half reliability distribution; give bootstrapped 95 % CIs for the inter-animal RSA. Add SEM bars or violin plots for the model-to-mouse RSA distribution (n = 11). Clearer ceilings would raise confidence. Revealing wide uncertainty without acknowledging it would lower significance.\n* Several critical details are buried or missing. Please move to main text or clearly state in appendix: full definition of IntersectionRNN cell and unroll scheme, exact tactile augmentation parameters (rotation range, flip prob., temporal reversal prob.), and optimizer/learning-rate schedule/batch size/etc.\n\nLIMITATIONS:\nMostly yes. Related to my comments above, the training-test domain gap should be better acknowledged/addressed.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work focuses on rodent tactile processing, and evaluates neural networks trained on biomechanically-realistic force / torque tactile sequences. The results show that recurrent ConvRNNs (IntersectionRNNs) are better at tactile categorization and neural alignment than feedforward ResNet and attention-based models. The models show impressive neural alignment, saturating the explainable neural variability and surpassing inter-animal consistency benchmarks with both supervised and contrastive self-supervised objectives.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths  \n- First application of model-brain-alignment methods to tactile processing across a broad set of model architectures and both supervised and self-supervised training objectives\n-\tNovel Encoder-Attender-Decoder parameterization\n-\tDirect neural comparison between a large set of models and rodent somatosensory cortical responses\n-\tTraining models on a synthetic dataset (simulated objects interacting with biomechanically-realistic rodent whisker model) is creative and provides a strong validation of the bio-mechanical model (given the success of the resulting convRNN models in predicting real neural data) \n\nWeaknesses  \n- The alignment with real rodent neural data appears to be based on RDMs computed from just 6 stimuli. This might be the best dataset available, but this dataset would seem to provide a rather coarse measure and should temper claims about the strength of the alignment between models and real neural data. Reading the abstract I was excited by the finding that the neural alignment scores saturated the noise ceiling, but less surprised that this was the case after reading that the analysis focuses on 6 stimuli. The results are still promising, but I would not necessarily expect “saturating the noise ceiling” to hold up under a larger, more varied dataset.\n\nQUESTIONS:\nResults Figure 4. How can the animal-to-animal scores be so low (<.20) and the model-to-animal scores so high (approaching 1.0, noise adjusted)? How could one model RDM be almost perfectly correlated (noise adjusted) with each individual animal RDM (without any tuning to individuals), but those animal rdms be so poorly correlated with each other?\n\nResults Figure 4. Can a model score well just by distinguishing convex from concave? Is there reliable variation between near-medium-far (within convex only; or within concave only), and if so, can the models capture this variation as well? This would at least suggest that the models are capable of capturing “fine-grained differences in real neural data” (within the constraints of this particular real-neural dataset). Relatedly, since the RDMs are so small, it would be useful to depict them in a Figure, to clarify what the “neural target” is for these analyses\n\nFor task-performance, is it fair to use “top-5 classification” as your measure of task-performance for the self-supervised models, since this is not in fact the task that the model was trained on? I suppose the more neutral measure of task-performance would be some version of the task loss (maybe somehow normalized to allow comparison across tasks), or even a complementary “instance-recognition” measure (how well individual samples are discriminated from the rest, which is more in-line with the objective of the self-supervised models.\n\nRegarding the strong neural alignment of the self-supervised models and its parity with the supervised models, I would find these results more compelling if it could be clearly demonstrated that there’s some fine-grained signal here (not just getting the convex vs. concave distinction, assuming that’s PC1 for these neural data, but also getting the near-medium-far variation within convex, or separately within concave).\n\nWere the rodent neural data combined across brain regions? Why not do separate analyses for S1, S2? Or are the RDMS too similar between these regions to bother with the breakdown?\n\nMinor  \n- How was the explainable variance ceiling estimated?\n-\tDid you identify the best-fitting layer in separate split of data than what you used to provide the final score for that layer? If so, this could be clarified in the final paragraph of the methods (pg. 6, just before Results).\n-\tCan the authors include an explanation for what the IntersectionRNN is?\n-\tHow ethologically valid do you think the simclr variant is? Could you have had the simulated rodent actually “sample the object twice” and still get strong learning?\n-\tAre there any other “precision measurements” or “fine-grained” analyses afforded by the rodent neural dataset, e.g., tuning of individual neurons?\n-\tIsn’t the importance of an inductive bias of “non-linear recurrent hierarchical network” expected from prior work on rodent tactile processing?\n-\tI didn’t really understand how this work might impact robotics exactly. How might these insights be integrated into robotic systems? Are tactile robotics not already using recurrent CNNs? Or is the idea that recurrent EAD systems are new and should be used?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an Encoder-Attender-Decoder (EAD) framework for modeling tactile perception in rodents by training temporal neural networks on realistic tactile sequences simulated from a rodent whisker array. The authors build the ShapeNet Whisking Dataset which is a synthetic tactile dataset generated by simulating a rodent’s whisker array interacting with 9,981 3D objects from ShapeNet across 117 categories. Using the WHISKiT Physics simulator, the authors modeled forces and torques at the bases of 30 whiskers during object sweeps with varied speeds, heights, rotations, and distances, creating rich temporal tactile signals reflective of real whisker sensing. The authors systematically compare convolutional recurrent neural networks (ConvRNNs), feedforward ResNets, and attention/state-space models. The results emphasize the importance of recurrent processing and self-supervision for capturing tactile representations, advancing both scientific understanding and robust tactile perception in embodied AI.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. The paper is easy to follow, with major contributions mentioned clearly in the Introduction.\n2. The motivations behind this paper are of significant importance, in that they build a systematic study of encoder, attention, and decoder architectures for inferring sensory semantics from simulated rodent whiskers, which can pave the way for architecture design in robotic tactile sensing.\n3. The authors present the ShapeNet Whisking Dataset which is a synthetic tactile dataset generated by simulating a rodent’s whisker array interacting with 9,981 3D objects from ShapeNet across 117 categories.\n4. The author’s insights around contrasting usefulness of recurrent and attention-based architectures, in that the temporally smooth signals from tactile inputs benefit from convolutional and recurrent mechanisms that integrate information locally over time, hold in the presented results.\n\nWeaknesses:\n\n1. While this study itself is meaningful, the connection to a real-life application, say, in robotics, is far-fetched. Is there any evidence that the takeaways may hold for any available sensing modalities such as force/torque or vision-based touch sensing from GelSight?\n2. Although the simulations are biologically realistic, models are trained on synthetic data rather than real tactile sensor readings, leaving a gap between simulation and potential real-world heterogeneity in tactile data.\n\nQUESTIONS:\n1. Did you explore other recurrent variants (e.g. vanilla RNNs, bi-directional RNNs), and how did they perform?\n2. ShapeNet is a human-centric dataset. How do you think the neural alignment results would change if the dataset contained ecologically relevant rodent objects (e.g. burrow structures, food items)?\n3. How transferable are your ConvRNN-based models to real robotic tactile sensors such as GelSight, given mechanical differences?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper systematically explores neural network architectures for tactile perception inspired by the rodent whisker system. The authors propose an \"Encoder-Attender-Decoder\" (EAD) framework and train various models using realistic simulated tactile data from virtual whisker interactions with 3D objects. They demonstrate that convolutional recurrent neural networks (ConvRNNs), outperform other architectures on tactile object categorization tasks. These ConvRNN models exhibit remarkable alignment with neural recordings from rodent somatosensory cortex, surpassing average inter-animal consistency. This research quantitatively identifies nonlinear recurrence as a key inductive bias in rodent tactile processing, guiding future development in embodied tactile AI systems.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n[S1] The proposed Encoder-Attender-Decoder framework enables systematic exploration of neural architecture space, moving beyond ad-hoc architecture selection. It facilitates comprehensive comparisons of multiple models against biological data, highlighting essential inductive biases relevant to rodent tactile processing.\n\n[S2] The paper provides valuable insights and findings without explicitly fitting models to neural data. The ConvRNN encoders effectively saturate explainable neural variance in rodent somatosensory cortex, and demonstrate a clear correlation between task performance and neural fit.\n\nWeaknesses:\n\n[W1] In Figure 4, nearly all models—including randomly initialized versions—outperform the a2a baseline regarding neural fit. Additionally, certain untrained models outperform their trained counterparts. The method used to evaluate neural fit and the impact of model training need further investigation.\n\n[W2] The neural data task involves only six distinct stimuli (concave/convex at three distances), potentially simplifying variance saturation compared to scenarios with more diverse stimulus sets.\n\nQUESTIONS:\nSee Weaknesses.\n\nAdditional question: Are there any models that perform lower than the a2a baseline in neural fit? For example, untrained different architectures\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 4, "ORIGINALITY": 4, "DATE": "2025-07-03"}