Coupled Transformer Autoencoder for Disentangling Multi-Region Neural Latent Dynamics
Authors: Ram Dyuthi Sristi
Sowmya Manojna Narasimha
Jingya Huang
Alice Despatin
Simon Musall
Vikash Gilja
Gal Mishne
Venue: ICLR 2026 Poster (2026)
Keywords: multi-region neural recordings
shared/private disentanglement
transformer sequence models
coupled autoencoders
latent variable dynamics
Neuropixels
neural dynamics
representation learning
Primary area: applications to neuroscience & cognitive science
TL;DR: We introduce a coupled transformer autoencoder that separates shared from private neural dynamics in simultaneous multi-area neuronal recordings.

Abstract: Simultaneous recordings from thousands of neurons across multiple brain areas reveal rich mixtures of activity that are shared between regions and dynamics that are unique to each region. Existing alignment or multi-view methods neglect temporal structure, whereas dynamical latent-variable models capture temporal dependencies but are usually restricted to a single area, assume linear read-outs, or conflate shared and private signals. We introduce Coupled Transformer Autoencoder (CTAE)—a sequence model that addresses both (i) non-stationary, non-linear dynamics and (ii) separation of shared versus region-specific structure, in a single framework. CTAE employs Transformer encoders and decoders to capture long-range neural dynamics, and explicitly partitions each region’s latent space into orthogonal shared and private subspaces. We demonstrate the effectiveness of CTAE on a controlled synthetic dataset and two high-density electrophysiology datasets of simultaneous recordings from multiple regions, one from motor cortical areas and the other from sensory areas. CTAE extracts meaningful representations that better decode behavior variables compared to existing approaches.

Published as a conference paper at ICLR 2026
COUPLED TRANSFORMER AUTOENCODER FOR DISENTANGLING NAMICS
NEURAL LATENT DY-
Ram Dyuthi Sristi UC San Diego rsristi@ucad. edu
Sowmya Manojna Narasimha
Jingya Huang
UC San Diego
UC San Diego
smnarasiinaa@ceea www../ahhauccde..ooooo
jih201. www.@gmail..ccc.c. com
Alice Despatin
Simon Musall
RWTH Aachen University & Jülich alice.
RWTH Aachen University & FZ Jülich s www.mmsllff-jjuullll de
de
Vikash Gilja
Gal Mishne UC San Diego gmi gmishne@ucad.co
UC San Diego
vgilja@ucsd.edu
ABSTRACT
Simultaneous recordings from thousands of neurons across multiple brain areas re- veal rich mixtures of activity that shared between regions and dynamics that unique to each region. Existing alignment or multi-view methods neglect temporal structure, whereas dynamical latent-variable models capture temporal dependencies but are usually restricted to single area, assume linear read-outs, or conflate shared and private signals. We introduce Coupled Transformer Autoencoder (CTAE) sequence model that addresses both (i) non-stationary, non-linear dynamics and (ii) separation of shared versus region-specific structure, in a single framework. CTAE employs Transformer encoders and decoders capture long-range neural dynamics, and explicitly partitions each region's 's latent space into orthogonal shared and private subspaces. We demonstrate the effectiveness of CTAE controlled synthetic dataset and two high-density electrophysiology datasets of simultaneous recordings from multiple regions, from motor cortical areas and the other from sensory areas. CTAE extracts meaningful representations that better decode behavior variables compared to existing approaches.
INTRODUCTION
The advent of high-density electrophysiology probes, e.g., Neuropixels, and volumetric calcium imaging has enabled recording large-scale, high-resolution, multi-region neuronal datasets. This shift from single-area recordings to distributed circuits reveals both globally coordinated signals and region-specific specialization (Jun et al. [017]] Machado et al. 2022). Individual neurons both receive input from and project multiple distant areas; behaviorally relevant codes are hypothesized to be broadcast across widespread circuits rather than confined to canonical locus (Machado al. 2022). This perspective challenges conclusions drawn from single-area studies and raises the question of which computations are local, which are distributed, and how global brain states are coordinated. Disentangling the shared components that mediate inter-area interactions from private signals unique to each region is therefore essential-both for mechanistic insight and for designing causal experiments such as targeted optogenetic inactivation or electrical stimulation of an upstream circuit.
Thus, recent work has focused on analyzing distributed circuits to recover latent activity patterns both within and across regions. A successful multi-region latent model faces three simultancous challenges: (i) latent trajectories must evolve smoothly in time to respect neural autocorrelation; (ii) they must accommodate the non-stationary, nonlinear dynamics that real circuits display; and (iii)

---

Published as a conference paper at ICLR 2026
they must separate shared from region-specinng structure without a parameter explosion as the number of areas grows.
Even within single brain region, recordings of neural population activity reveal mixture of struc- tured responses and substantial trial-to-trial variability, introducing a challenge when interrogating the behavior of neural circuits (Cunningham & Yu 2014). A widely adopted framework- neural latent
dynamics (Vyas et al. 2020] Churchland & Shenoy (024) proposes that these high-dimensional responses reflect the evolution of a consistent, low-dimensional trajectory, supported by stable patterns of co-variability across neurons. Classical PCA or factor analysis (FA) uncover low-dimensional Yu 2014) but ignore time. Moreover, because observed spikes are subject to high Poisson noise, the resulting latents can exhibit interpret and may not reflect meaningful underlying processes. Consequently, latent variable models that incorporate temporal smoothness dynamical constraints have been developed, based on either linear Gaussian Process (GP) models (Yu et al., 2009) or nonlinear deep learning models (Pandarinath) et al. 2018 Ye & Pandarinath 2021). Yet, when these single-area tools are applied naively to multi-area data, e.g., by concatenating the recordings, they often fail. Inter-area delays warp the latent space, differences in correlation structure cause shared factors absorb private variance, and the larger or more active region can dominate the mixture weights. To address these challenges, recent studies have investigated both predictive models (Zandvakili & Kohn 2015 Semedo al. 2019 Perich et al., 2018) and joint latent variable (Gokcen et al. 2022 ?) approaches for inter-regional activity. These efforts reveal that only a selective subset of latent dimensions actively participates in inter-area communication. Such findings motivate extensions to the latent dynamics framework, proposing that communication is mediated through persistent, low-dimensional subspace- referred to as a communication subspace (Semedo et al. 2019) that is distinct and orthogonal to private, region-specifer dynamics Fig. [
structure Cuntninhmun
dynamics that are difficult to
More recently, joint latent variable models inspired by canonical correlation analysis (CCA) have been developed to capture correlated latent dynamics across regions while simultaneously learning region-specific latent components. One approach focused on neural data has been linear approaches
that generalize GP-based models to the multi-region setting (Gokcen et al., 2022; 2023 2024 Li disentangling methods (Lee & Pavlovic
ct al. 2024). Alternatively, nonlinear autoencoder-base
2021 Koukuntla et al. 2024) have been developed to analyze multiview point cloud data and partition latents into shared and private factors. However, these treat time points as i.i.d. samples and therefore discard dynamics.
This work. We introduce the Coupled Transformer Autoencoder (CTAE), an end-to-end framework for modeling simultaneous recordings from multiple brain areas. Transformer- (Vaswani et al. 2017) encoders and decoders act as flexible priors capable of capturing stsssttttio,,,, long-range neural dynamics; each region's latent space is split into orthogonal shared and private subspaces, with reconstruction losses preserving region-specific information and lightweight alignment loss matching the shared representations. Because the latents are behaviour-agnostic downstream decod- ing wegtheet kinematics, forces, or cognitive variables- can be carried out on the same embedding without retraining. Our contributions are as follows:
Non-stationary multi-region modelling. CTAE extracts shared and private representations that capture long-range, nonlinear dynamics absent.
Scalable architecture. mixing weight allows the shared subspace to extend trivially beyond two regions without an exponential increase in parameters.
Generic downstream utility. The behaviour-agnostic latent space supports diverse decoding tasks (e.g. position, velocity, cognitive state) using simple linear read-outs.
Empirical validation. On neural recordings from MI-PMd (Utah Arrays) and SC-ALM (Neu- ropixel Probes), CTAE achieves higher shared-variance capture than existing works, while revealing interactions consistent with anatomy.
RELATED WORK
Single-region latent-variable models. Linear techniques-include PCA Pearssn 1901) | and FA (Harman 1976) are widely used to analyze large-scale recordings (Cunningham & Yu! 2014). Caussian-Process Factor Analysis (GPFA) adds a smooth GP prior to FA to enforce temporal
2

---

Published as a conference paper at ICLR 2026
continuity (Yu et al. 2009). GPFA's stationarity and linear read-out assumptions relaxed by Latent Factor Analysis via Dynamical Systems (LFADS) (Pandarinath et al. 2018) and the Neural Data Transformer (NDT) (Ye & Pandarinath] 2021). which exploit nonlinear recurrent generators or self-attention to capture non-stationary and nonlinear dynamics. Subsequent efforts pursued identifiability with switching LDS (Linderman et al. 2016)) Glaser et al. 20200. GP-SLDS (Hu et al. 2024), locally linear manifold models in DFINE Abbasspouazad et al. 2024), and variational neural
state-space models such VIND (Hernandez al. 2018) and S4-based sequence layers (Gu et al. 2022).
Multi-region latent models. Multi-region methods relying on GP models include Delayed Latents Across Groups (DLAG) (Gokcen et al. 2022), its nuutiipopulation generalization mDLAG (Gokcen eet al. 2023), multi-view GPFA extensions (Gokeen et al.) 2024). and Multi-Region Markovian Gaussian Process (Li et al. 2024). These inherit smooth- assumptions and linear read-outs that struggle with non-stationary or long-range dependencies. Classical multiset CCA (Hotelling 1936) and lensor-decomposition methods (Cichocki et al., 2015) have also been applied multi-area recordings, however these are linear techniques. Current-Based Decomposition (CURBD) (Perich al. 2020) infers directed currents between many regions from data-constrained RNNs, yet it does not model explicit -aivedt split. Similarly, multi-modal models have been recently developed to jointly analyze neural recordings and behavior Vahidi et al. (2025): Gondur et al. (2024); Yi et al.) (2025). These models could potentially be adapted to analyze multi-region recordings, however we expect Multimodal GP- VAE Gondur et al. (2024) and Shared-AE Yi et al. (2025) to scale poorly to more than 2 regions due to exponential growth in experts pairwise loss terms. Both also operate on i.i.d. samples or short windows rather than full neural time series.
Multiview autoencoders Multi-view representation learning has progressed beyond classical CCA by incorporating additional losses and constraints to more effectively identify shared latent represen- tations. However, these methods often require adaptation to account for the specific statistical and temporal properties of neural data. Correlation-based alignments cannnnnas-ccnonicalll correlation analysis (CCA) and its deep variant (DCCA) (Andrew et al.) 2013) maximise instantaneous correlation but provide guarantee of capturing all shared variance; Deep CCA Autoencoders (DCCAE) add reconstructions at the risk of mixing private with shared information (Wang et al.] 2016).. Repre- sentative examples include Deep Coupled Auto-encoder Networks (Wang al.] 2014), Coupled Autoencoders for domain adaptation (Wang & Breckon, 2023), Correlated Autoencoders for audio- visual retrieval (Feng et al. 2014)) Deep Correlation Autoencoders (DCCAE) (Wang al. (0166). Split-brain Autoencoders (Zhang et al. 2017). shared-private domain adaptation AES (Bousmalis 2016) and cycle-consistent multiview AEs (Wu et al. 2018). SPLICE (Koukuntla et al.
2024) extends this line with measurement network penalties to sharpen disentanglement, however it scales poorly itss auxiliary measurement networks grow exponentially with region count-while
DMVAE assumes one global shared component across all views, precluding subset-specific latents. Multimodal VAE (MVAF) (Wu & Goodman] 2018), Disentangled Multimodal VAE (DMVAE) (Lee) & Pavlovic 2021). Joint Multimodal VAE (Sutter et al., 2021), Mixture-of Mixtert-of-Experts MVAE (Shi et al., 2019) learn shared/private factors across images, text and audio but treat each sample independently, ignoring sequential dependencies.
Across these categories, no existing method jointly satisfies (i) non-stations non-stationary nonlinear dynamics, (ii) temporal continuity and (iii) scalability to more than two regions without incurring a parameter explosion as the number of areas increases.
PROBLEM FORMULATION
Let (1) E MIX, (2) E R2NT denote simultaneous neural population recordings from two brain regions, each acquired over time steps and, N1 and N₂ channels respectively (generalization to more than 2 regions is in Appendix B We assume that the observed neural activity at each time step t is a nonlinear transformation of latent variables (Fig 2). which include: (i) shared latent dynamics within a communication subspace (Semedo et al. 2019) spanned by neural trajectories that are correlated across regions, and (ii) private latent dynamics that capture region-specific processes lying in subspaces orthogonal to the shared component. To effectively capture the temporal structure
3

---

Published as a conference paper at ICLR 2026
Neural Activity
Latent Dynamics
Subspace Disentanglement
Region (lii
Region I (1,
Shared Latent
ext
62*
min L
aligument
Region Private Lateot
est
Region 2 X g(S.
Figure 1: Observed neural activity across time from brain regions, denoted as ( and X modeled as nonlinear function of underlying latent dynamics specific to each region. In the illustration, [er, E2. Cg] span the latent subspace for region and [c1, 62 64] for region 2. Inter- regional communication is mediated by shared latent trajectories S within the common subspace [2], which drive correlated population activity. Following the output-null/potet hypothesis, assume shared and private dimensions are orthogonal, allowing clear recovery of shared dynamics from region-specific processes.
of neural activity, model these latent variables as functions of the entire observed neural activity.
x (S..., P(.)) X S( ( fort E [1....,
where denotes the sequence of indices between and t, S. E Rdd represents the shared dynamics between the two regions at time and p(") Rd, and (( e Rd2 capture dynamics specific to regions and 2, respectively, and d,, d,, and d2 denote the dimensionalities of the shared and private representations.
Our goal is to recover the shared and private latent dynamics S, P(), and ( given the observed neural activity (1) and (2)
Bold upper-case letters denote matrices (e.g., X). bold lower-case denote latent vectors (e.g., w), andFFnnicante reconstructions. Let 1, denote a vector of all-ones of length n and let O, denote vector of all zeros of length n. [I())]] denotes the Frobenius norm. O denotes element-wise multiplication. Expectation is over batches of trials unless stated otherwise.
Notations.
4 COUPLED TRANSFORMER AUTOENCODERS FOR TWo REGIONS
Our model is comprised of coupled autoencoder built on transformer models, that disentangles shared and private representations. We have designed loss functions to recover latents that maximally represent the neural activity, where the shared representations are aligned across all regions, and each disentangled sub-space orthogonal to the others. For clarity, we describe the two-region case the main text; the extension arbitrary number of regions is provided in Appendix
4.1 MODEL ARCHITECTURE
We design separate causal Transformer-based encoder-decoder pairs, denoted by (E. (D and (EE?2) (2))) ), for each of the two brain regions. Each encoder is Transformer stack with self-attention layers that capture long-range, nonlinear temporal dependencies within region. The recorded spike trains are Saussian-smoothhe encoder. Each decoder employs standard Transformer cross-atlention reconstruct the original
to obtain continuous firing-rate estimates which are the inputs to the
4

---

Published as a conference paper at ICLR 2026
Observed Spilee X
Simoothed Spikkz XPn
Inferred Latent
Weight Maak
Pased Latent
inferred Eatnn Rate g (i"("""
Region
Encoder Ag
Decrder D
Private Region
Weighned Lanane Poston
Sharid Across Regions
W
Private Region
Region
Rncader
Decadar
Figure 2: CTAE architecture. CTAE is composed of a coupled autoencoder, where the encoders and decoders are causal transformers designed to reconstruct neural activity for each region r. The inputs the network are estimated spike rates from each region. A weight mask per region w(() is used to disentangle the shared representation (violet) from the region-speciff latents (red and blue) in the encoder outputs Z The latents are recovered via end-to-end training.
firing rates from its region's latents. Processing the full multichannel time series from each session, the encoders produce latent representations partitioned into two distinct components: shared latent sequences capturing dynamics common both regions and private latent sequences specific each region:
(1)
E (x(1))
2 2 (x
Z
(22
E
DxT
Here D is the total number of latent dimensions. To indicate which of the latent dimensions correspond shared and private representations, we introduce weight vectors W, and w2 {0, 1).
Region-specific weight masks. Let the latent dimension be partitioned as D = de + d1 + d2. Define three contiguous index I, {l...., d.). In {d++I,.... "d,+tt...dd..d... and In +1,+ D). We then construct binary weight vectors that indicate which subset of the latent dimensions correspond the shared to region-private Define W1, W2 E {0, 1 as
ld,
O
W2
1d,
O
1
(3)
that W1 activates the shared and region-private of region dimensions, whereas wa activates the shared and region-pil eegion-private dimensions of region Throughout the paper we treat w, as fixed; the latent space dimensions (da, d1, d2) are treated as hyper-parameters and tuned validation set. Importantly, these masks do not hard-code the actual interaction structure: they only specify an upper bound on how many shared private latents the model may allocate. During training, dimensions unsupported by the data naturally collapse to negligible variance. While fixed masks provide clarity and control in the two-region setting, they could also be initialized from anatomical priors or made learnable so that the model itself infers which latents are shared versus private. We leave these scalable extensions to future work.
Weighted latent fusion. Broadcasting the masks across time, we aggregate the two latent trajectories dimension- wise via a masked average for each dimension in the latent:
w [r Z
(1) d + w2(dZ
Z, [d]
w w2(d]
Equation equation leaves the private blocks unchanged (they receive weight from only their own region)
p(1) Zz, 2 2 ZiZ
(222
(5)
5

---

Published as a conference paper at ICLR 2026
while averaging the shared block across regions:
$ = Zz,
2
11 + 2/221
(6)
This design will implicitly force alignment of the shared latents of both regions, via minimizing the reconstruction losses we define the next section.
Region-specifie decoding, Each decoder receives only the latent dimensions relevant to its own region:
x = DD (ww,1)0Z))
where (F, IR DxT is outer product such that each weight duplicated for each dimension along all time points. Thus, the element-wise product zeroes out unrelated latents, forcing decoder r rely exclusively the subset of dynamics meaningful for its region.
This architecture (i) aligns both regions in a common latent space of dimension D, (ii) preserves region-specific structure via fixed relevance masks, and (iii) enforces cross-region consistency through the fusion rule in Eq. equation 4 The loss function balancing reconstruction accuracy and alignment We illustrate our architecture in Fig. 2
objectives is detailed in Section 4.2
4.2 TRAINING ORJECTIVE
We use four loss functions to recover the shared and private latent representations with CTAE:
Reconstruction loss. The first loss is reconstruction loss, so that each transformer autoencoder faithfully reproduces its own region's 's activity:
Lree
Σ
Shared-only reconstruction. To ensure that all structure common to both regions is routed into the shared block, we define a loss such that each decoder reconstructs the neural activity from its region using only the shared representations. Let (() = WI @w = [1d, 0s,1 be the intersection mask that selects only the shared dimensions. The loss is
2
Lshared
Σ DID
where ((*) zeroes out all private coordinates from the inputs to the decoder, thus forcing the model to encode every cross-region regularity inside the shared subspace. Without this constraint, shared information can shift into private subspaces, leading inaccurate representation.
Alignment loss. The shared latents are meant to capture only dynamics common to both regions. To enforce this, we align each encoder's shared output their average, ensuring consistency across regions and preventing region-specific variance from leaking into the shared space:
Calign Σ (w, @ Z (w,(w,,) (w,
Z
(10)
Orthegonality loss. To encourage each latent coordinate to capture distinct, non-redundant structure, we penalize correlations between all rows the fused latent matrix Z E RxT Let G = - 1ZZ" denote the empirical Gram matrix of latent trajectories, whose off-diagonal entries encode row-wise correlations. We drive only the off-diagonal entries of G towards zero,
Lorth = ||G diag( G) 1.
(11)
so that every pair of latent dimensions shared or private becomes approximately orthogonal, promoting global disentanglement.
Finally, the complete objective minimizes the weighted sum of the four losses:
Lror + Aatign Calign + Ashared Lshared + Aorth Larth-
(12)
The weights Agharod Aatign Aurth are selected on a held-out validation set. Methodology for training the architecture is summarized in Algorithm 1

---

Published as a conference paper at ICLR 2026
5 EXPERIMENTS!
We analyze two real-world datasets of simultaneous neural recordings from motor circuit and from multisensory circuit. In addition, we include controlled synthetic dataset in Appendix C We demonstrate the efficacy CTAE extracting shared and private dynamics comparison to
DLAG (Gokcen et al.) 2022). We also provide comparison Deep CCA (Andrew et al. 2013) in Appendix Implementation details for each experiment are in the appendix.
5.1 MOTOR CIRCUIT: M1-PMD
We applied CTAE to simultaneous neural recordings from dorsal premotor cortex (PMd) and primary motor cortex (M1) in macaque monkeys performing a standard delayed center-out reaching task with eight outward targets (Perich al., 2018), additional details in Appendix E. The dataset consisted of 208 total trials across 8 reach conditions, with spike-sorted data from 66 and 52 putative neurons in PMd and MI, respectively, recorded via 64-channel Utah arrays. Each trial spanned 3s, with spikes and behavioral variables (e.g., position) binned in 100 ms bins, resulting in 30 time points per trial.
Historically, PMd and MI have been jointly analyzed due their complementary roles in movement preparation and execution. During the instructed delay period between stimulus onset and the go cue, both regions exhibit preparatory activity without inducing muscle output (Kaufman al., 2014 Churchland & Shenoy 2024), potentially representing task goals i.e., target identity (Byron et al. Lara et al.] 2018). Following the "go' cue, both PMd and MI generate execution-related activity that descends to motor neurons and enables decoding of continuous hand kinematics, including
position and velocity (Churchland & Shenoy 2024 Elsayed et al. 2016 Gilja et al.) 2012 Byron
al. 2010
We applied CTAE to the dataset for unified modeling of joint latent subspaces and identified:
A shared subspace of dimension d,, capturing correlated dynamics across PMd and MI.
Private subspaces of dimensions dymu and dMI, capturing region-specific activity.
Fig. visualizes subset of the CTAE inferred neural latent dynamics (the set of all latents is in Fig. 14p.
Each panel shows a distinct latent dimension evolving over time. The shared latent subspace captures both condition-invarian and condetion-dependent structure, encoding aspects such reach direction and temporal progression. Region-specifie latents display more complex dynamics, likely reflecting local circuit processes.
Shared
0.5
0.5
05
0.0
Private M1
Private PMd
Figure 3: M1-PMd dataset. CTAE shared (top) and region-private (bottom) latents. Dotted vertical line indicates the "go" cue each trial.
To interpret the latent dynamics Symamics-shared 2 and private (PMd), (MI), we evaluated the inferred latents using two decoding tasks with simple linear decoders:
Continuous decoding hand position: Predict hand position at each time point after go-cue (last seconds for each trial) using linear regression: uhunnd W zz We plot in Figure 44 the predictions from neural activity, from CTAE latents and from DLAG latents.
Discrete decoding target condition: Classify the reach condition (one of 8 targets) using multi-
class logistic regression: f c exp(w) | x+by) for é = 1,..., C. We present in Figure 4B the
exp(w, z+b;)
confusion matrices across conditions for neural activity, CTAE and DLAG latents.
Code for CTAF available in httpp:://ittu...ccm com/M shne labb ctae-multinee
7

---

Published as a conference paper at ICLR 2026
Continuous Position Decoding
(b) Target Direction Classification
Ground truth
PMd
Reacch Direction
MI o 54)
Pid 563
P10.7
P12.07
P-0322
DLAG shared
DLAG prisze Mi
DLAG prizate MNd
DLAG Stared (3 34
DLAG PRuste MI (3:47)
DLNG Phote PHG I 25)
150.3
(50.4
Figure 4: M1-PMd dataset. a) Ground truth hand position in top-left corner. Hand position decoding from neural activity in M1 and PMd (top). from CTAE latents (middle) and from DLAG (bottom). b) Confusion matrices for reach direction classification (order of plots same as in A). Classification accuracy in parentheses.
We hypothesized that the shared latent space would encode the majority of behaviorally relevant information-partiicn target ddentity-while PMd might contribute higher-order planning signals, and M1 reflect finer temporal structure related to movement execution.
Our results confirmed that CTAE's shared latent factors accounted for the majority of variance associated with reaching behavior (Fig. a).. In other words, the shared PMd-M1 subspace captured the dominant task-relevant signals. In contrast, DLAG tends distribute behaviorally relevant variance across private PMd and private MI latents in a directionally anisotropic manner. This reflected in both private latents predicting certain movement directions but not others, and also discrete target classification showing high accuracy for only a subset of reach directions. Such dtrection-speciff "fragmentation" of kinematic information across private latents inconsistent with established findings that both PMd and M1 encode reach kinematics through coherent, low- dimensional manifold. This pattern suggests that DL.AG may inadvertently leak shared information into private subspaces, whereas CTAE's mask structure and regularization prevent such leakage. We also compare to DeepCCA in Appendix H. and to analyzing the concatenation of data from both regions with single Transformer AE in Appendix [] and demonstrate that both baselines have significantly lower continuous and discrete prediction accuracy than both CTAE and DLAG shared latents. These findings demonstrate that CTAE simultaneously decode continuous behavioral trajectories Fig. a. and classify discrete behavioral states Fig. b. more effectively than prior approaches.
We present time-wise prediction of the reach in Fig. 19 Importantly, our results support the view that motor planning and execution signals are largely embedded in shared population dynamics across PMd and MI, while private activity may facilitate flexible, context-dependent processing as adaptation to perturbations without interfering with ongoing execution.
To evaluate the contribution of the individual loss functions to the CTAE model we perform an ablation study (full details in Appendix(F] and report the prediction accuracy in Tab. []] We demonstrate that removing any of the three loss functions results in decreased performance.
Ablation study:
5.2 MULTISENSORY CIRCUIT: SC-ALM
We evaluated CTAE on a new dataset of simultancous recordings from the superior colliculus (SC) and anterolateral motor cortex (ALM) in mice trained multisensory discrimination task (see
8

---

Published as a conference paper at ICLR 2026
Model
Shared
Private M1
Private PMd
CTAE
0.69 (0.03)
0.22 (0.02)
0.21 (0.03)
CTAE without alignment loss
0.61 (0.02)
0.16 (0.02)
0.2 (0.02)
CTAE without orthogonality loss
0.31 (0.02)
0.28 (0.02)
0.29 (0.02)
CTAE without shared only reconstruction loss
0.34 (0.01)
0.36 (0.01)
0.37 (0.01)
Table 1: Ablation study of CTAE loss functions. Each entry indicates the model's accuracy in
predicting the reach direction. The values in the brackets indicate standard deviation across 5-folds.
Appendix E.2 for task details). Using high-density Neuropixels probes, neural activity was recorded
from superficial and deep layers of SC together with ALM while animals integrated visual and tactile stimuli to identify the target side (left vs. right). We fit our multi-region CTAE (R > 2: Appendix BB) to this three-region dataset (superficial SC, deep SC, ALM). This multi-region analysis infers a wide range of multi-region interactions: shared-across-all pairwise-shared, and region-private (Fig.
visual
tactile
multisensory
right
left
shared SupSC-CteepCC St 11l sharedd SupSC-DeeptC: #13
thered DeeOSC-AUM #3
shared PAPP #2
shared DeepSC-NL e3
DE NLM
A Monnenly
Wmmning
www
0
Time (s)
stimuli
delay start
response
Figure 5: SC-ALM dataset. Representative latents. Each panel shows the condition-averagee time course of one latent (title = subspace and latent index). Left panels: stimuli-vin visual (blue), tactile (orange), multisensory (green). Right panels: target side-right (red), left (purple).
loop implicated in linking sensory inputs to motor planning. SC integrates visual and tactile information and contributes to orienting behavior (Stein & Meredith 1993: Cang & Feldheim 2013). whereas ALM encodes preparatory and choice- related activity during decision- -making tasks (Guo et al. 2014 Li et al., 2015). We hypothesized that shared subspaces across superficial and deep SC layers and between deep layers of SC and ALM would preferentially capture task-relevant features, such as stimulus type and target side, thereby mediating the transformation from multisensory evidence accumulation to decision making.
SC and ALM are key components
Figure 6 shows decoding performance of stimuli and target side across private and shared subspaces. For the raw neural activity, Deep SC best discriminates both stimulus and target. Superficial SC activity correctly distinguishes target side but poorly distinguishes between stimulus type for visual and multi-sensory trials, and poorly distinguishes between target sides for tactile trials. contrast, ALM activity correctly distinguishes target side but poorly distinguishes between stimulus type for tactile and multi-sensory trials, and poorly distinguishes between target sides for visual trials. However, with CTAE we identify shared dynamics between regions that better identify stimulus and target. The shared subspaces between superficial and deep SC, and between deep SC and ALM, yielded higher decoding accuracy than region--rric privvate latents. The shared subspace between superficial SC and ALM has little shared dynamics meaningful for discrimination. Our results indicate that deep SC plays more central role in this circuit, with shared dynamics reflecting evidence accumulation with superficial SC and choice with ALM (Fig. [8). For comparison, we applied mDLAG to the same dataset (Fig. T]. In contrast to CTAE, mDLAG shared and private latents do not exhibit any improvement over predicting from raw activity, and decoding performance remains diffuse across subspaces. This likely reflects the linear observation assumption mDLAG, which may limit its ability to capture nonlinear multi-region interactions present in this task. Overall, these findings highlight role for inter-regional subspaces in integrating sensory inputs within SC and propagating them to ALM to support decision-related processes.

---

Published as a conference paper at ICLR 2026
BAW)
SupSC (0
RAW
DeaeSC 1O.
RAW
ALM
visual right-
0.1
0.0
0.0
visual left 0.0
0.0
0.1
52
0.9
0.0
0.1
0.3
0.0
06
81
91
RAW
tactile right
0.8
tactile left 0.0
0.0
0.4
0.4
03
50
0.0
64
0.1
0.9
0.0
0.0
0.0
0.1
01
044
01
93
Row normollzed probobiiity
multisensory right
multisensory left
0.0
0.4
0.0
0.0
0.2
6/4
0.0
63
0.0
0.0
0.1
6.7
0.0
0.0
0.1
03
03
0.4
retuate Supder (0 501
Irbuate DeepSC 10
AL
visuall right
0.4
CTAE PRIVATE
visual left
0.
0.0
0/0
64
0.1
0.1
02
044
tactile right- 00
01
611
0.0
0.1
tactile left 0.0
0.0
0.6
0.5
0.1
60
0.1
6.1
0.1
05
0.1
0.1
0.1
02
04
02
911
0.2
multisensary right
multisensory left
0.0
0.4
0.0
0.0
0.2
64
0.0
62
0.0
03
02
0.3
0.1
02
01
02
02
0/
shared SupSt-Deeedt in
shared DewpSr-AA -AM (0.59)
shored Sustr-Al M shared SupSC--eepSS--ll
Visuall right
0.0
0.0
0.
CTAE SHARED
visual 01
0.
tactile right 00
tactile left
00
02
0.
0.1
multisensary right 01
multisensory left
0.0
0
0.0
00
0.
63
01
0.1
0.5
0.1
0.3
0.0
0.
02
84
6.8
02
0.1
02
0.4
visual right
visual left
tactle right
tactilc left
multisensory right
multisensory lent
visual right
visual left
right
tactile left
multisensory right
multisersory left
visual right
vicuaa left
tactle right:
tactile left
multisensory right
multisensory left.
visual right
visual left.
lactle right
tactlie left
muttisensory right
multisansory left
Figure 6: Row-normalizer confusion matrices for decoding stimulus x target side. (top) Raw activity from SupSC, DeepSC, ALM; (middle) CTAE private: region-specifie latents; (bottom) CTAE shared latents (pairwise and 3-way). Panel titles show mean 5-fold accuracy.
6 CONCLUSIONS
In this paper, we presented a new approach for recovering shared and private latents from multi- region neural recordings. Our framework relies on a Transformer-basee autoencoder architecture for nonlinear modeling of the relationship between neural activity and latent dynamics, and loss functions designed to recover orthogonal private and shared latents that maximally capture the information in the shared activity between the brain regions. By explicitly separating shared and private sources of variability, CTAE facilitates for addressing deeper questions about how distributed neural populations collectively drive flexible behavior and adapt through experience (Perich al., (2018). This work demonstrates effective joint latent inference across cortical areas and establishes a foundation for future studies on generalizable, population-Ieved neural computations across behavioral contexts.
robust comparisons across contexts and offers foundation
There are multiple exciting future directions building on our proposed CTAE model. First, while our paper focuses on neural recordings, our coupled-trane transformerr autoencoder design is not specific to neural data and can be applied to general multiview time series data. Particularly, it can be applied to analyze neural and behavioral recordings to identify features shared between neural activity and behavior, and neural features that are not reflected in behavior and vice versa. Second, currently our approach is scalable to more than two regions by generalizing the orthogonality loss to pairs of brain regions. By making the disentanglement weight vector learnable, the dimensions of each of the subspaces can be identified from the training data. Finally, training CTAE directly on spikes with Poisson/dispersion-.e aware observation model, i.e. incorporating likelihood loss matched to spike generation (c.g., Poisson or negative-binomial) will enable directly analyzing unsmoothed spike counts.
10

---

Published as a conference paper at ICLR 2026
7 ACKNOWLEDGMENTS
This research is partially supported by a Simons Foundation Pilot Extension Award 00003245 (G.M. and R.S.), National Science Foundation EFRI 2223822 (G.M., V.G., S.M.N. and J.H.) and Deutsche Torschungseemeinschat (DFG, German Research Foundation) as part of the SPP2205 533396241 (A.D. and S.M.). This work utilized computational resources provided by the National Research Platform (NRP) Nautilus, supported by the National Science Foundation.
8 CONFLICT OF INTEREST
V.G. holds shares in Neuralink Corp. and is Chief Scientific Officer and an options holder at Paradromics, Inc.
REFERENCES
Hamidreza Abbaspourazad, Eray Erturk, Bijan Pesaran, and Maryam M. Shanechi. Dynamical flexible inference of nonlinear latent factors and structures in neural population activity. Nature Biomedical Engineering, 8(1):85-108, 2024. doi: 10.1038/s41555
URL https
7/doi 10. 1038/ 023 01106-1
Epub 2023-12-11.
G. Andrew, R. Arora, Bilmes, and K. Livescu. Deep canonical correlation analysis. In Proc. of the 30th International Conference on Machine Learning, pp. 1247-1255, 2013.
Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan.
Domain separation networks. Advances in neural information processing systems, 29, 2016.
M Yu Byron, Gopal Santhanam, Maneesh Sahani, and Krishna V Shenoy. Neural decoding for motor and communication prostheses. In Statistical Signal Processing for Neuroscience and Neurotechnology. pp. 219-263. Elsevier, 2010.
Jianhua Cang and David A Feldheim. Developmental mechanisms of topographic map formation and alignment. Annual review of neuroscience, 36(1):51-77, 2013.
Mark Churchland and Krishna V Shenoy. Preparatory activity and the expansive null-space. Nature Reviews Neuroscience, 25(4):213-25) 2024.
Andrzej Cichocki, Danilo Mandic, Lieven De Lathauwer, Guoxu Zhou, Qibin Zhao, Cesar Caiafa, and Huy Anh Phan. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. IEEE signal processing magazine, 22(2):145-11) 2015.
John Cunningham and Byron M. Yu. Dimensionality reduction for large-scale neural recordings. Nature Neuroscience, 17011::50001109 2014. doi: 10.1038/mm.3776 URL https: :/ttps:///oil
org/10. 100./10.0.nn3333 3776
Gamaleldin Elsayed, Antonio H Lara, Matthew Kaufman, Mark M Churchland, and John Cunningham. Reorganization between preparatory and movement population responses in motor cortex. Nature communications, 7(1):13239, 2016.
Fangxiang Feng. Xiaojie Wang, and Ruifan Li. Cross-modal retrieval with correspondence au- toencoder. In Proceedings of the 22nd ACM international conference Multimedia, pp. 7-16, 2014.
Vikash Gilja, Paul Nuyujukian, Cindy A Chestek, John P Cunningham, Byron M Yu, Joline M Fan, Mark M Churchland, Matthew T Kaufman, Jonathan C Kao, Stephen Ryu, and Krishna V Shenoy, A perforreffrrancc neural prosthesis enabled by control algorithm design. Nature Neuroscience, 15(12)1175--1757 2012.
Joshua Glaser, Matthew Whiteway, John P Cunningham, Liam Paninski, and Scott Linderman. Recur- rent switching dynamical systems models for multiple interacting neural populations. In Advances in Neural Information Processing Systems, volume 33, pp. 14867-14875 14878. Curran Associates, Inc., 2020.
11

---

Published as a conference paper at ICLR 2026
E. Gokcen, J. H. Macke, and B. M. Yu. Disentangling the flow of signals between populations of neurons with delayed latents across groups. Nature Computational Science, 2:441 452, 2022.
Evren Gokcen, Anna Ivic Jasper, Alison Xu, Adam Kohn, Christian K. Machens, and Byron M. Yu. Uncovering motifs of concurrent signaling across multiple neuronal populations. Advances in Neural Information Processing Systems 36 (NeurIPS 2023), pp. 24283-24295, 2023, URL net forum -irpiidd333300 3to.iiidduuuggg.s3330
httpp:://ppeney///
Evren Gokcen, Anna Jasper, Adam Kohn, Christian K Machens, and Byron M Yu. Fast multi group gaussian process factor models. arXiv preprint arXiv:2412.16773, 2024.
Rabia Gondur, Usama Bin Sikandar, Evan Schaffer, Mikio Christian Aoi, and Stephen L Keeley. Multi-modal gaussian process variational autoencoders for neural and behavioral data. In The Twelfth International Conference on Learning Representations, 2024.
Albert Gu, Karan Goel, and Christopher Ré. Efficiently modelling long sequences with structured state spaces. In International Conference on Learning Representations, 2022.
Zengcai V Guo, Nuo Li, Daniel Huber, Eran Ophir, Diego Gutnisky, Jonathan T Ting, Guoping Feng. and Karel Svoboda. Flow of cortical activity underlying a tactile decision in mice. Neuron, 81(1): 179-194, 2014.
Harry H Harman. Modern factor analysis. University of Chicago press, 1976.
Daniel Hermandez, Antonio Khalil Moretti, Ziqiang Wei, Shreya Saxena, John Cunningham, and Liam Paninski. Nonlinear evolution via ppatially-dependee linear dynamics for electrophysiology and calcium data, arXiv preprint arXiv: I8/iv:88/..00 1.02459, 2018.
H. Hotelling. Relations between two sets of variates. Biometrika, 28(3/4):311-77 1936.
Amber Hu, David M. Zoltowski, Aditya Nair, David Anderson, Lea Duncker, and Scott W. Linderman. Modeling latent neural dynamics with gaussian process switching linear dynamical systems. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL httttttt///aaatt
org/ hbs//bb//tt aa////bb//488 2408 03330
J. J. Jun, N. A. Steinmetz, J. H. Siegle, al. Fully integrated silicon probes for high-density recording of neural activity. Nature, 551:232-2366 2017. doi: 10. 1038/nature24636..
Matthew T Kaufman, Mark M Churchland, Stephen Ryu, and Krishna V Shenoy. Cortical activity in the null space: permitting preparation without movement. Nature Neuroscience, 17(3):400444, 2014.
S. Koukuntla, J. Julian, J. C. Kaminsky, et al. Unsupervised discovery of the shared and private geometry in multi-view data, arXiv preprint. 2024.
Antonio H Lara. Gamaleldin F Elsayed, Andrew J Zimnik, John P Cunningham, and Mark M Churchland, Conservation of preparatory neural events in monkey motor cortex regardless of how movement is initiated. elife, 7:e31 826, 2018.
M. Lee and V. Pavlovic. Private-shared disentangled multimodal variational autoencoder learning hybrid latent representations. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, 2021. 11Xi:22012213024.
Nuo Li, Tsai- Wen Chen, Zengcai V Guo, Charles R Gerfen, and Karel Svoboda. A motor cortex circuit for motor planning and movement. Nature, 519(7541):51-56) 2015.
Weihan Li, Chengrui Li, Yule Wang, and Anqi Wu. Multi-region Markovian Gaussian process: An efficient method discover directional communications across multiple brain regions. In International Conference on Machine Learning, pp. 281 28112-28131 28131. PMLR, 2024.
Scott W. Linderman, Andrew C. Miller, Ryan P. Adams, David M. Blei, Liam Paninski, and Matthew J. Johnson. Recurrent switching linear dynamical systems, 2016. URL https: : htt///axvvv.../// org/ abs/161 08466 arXiv preprint.
12

---

Published as a conference paper at ICLR 2026
T. A. Machado, I. V. Kauvar, and K. Deisseroth. Multiregion neuronal activity: the forest and the trees. Nature Reviews Neuroscience, 23(1 1:683-704, 2022. doi: 121103//44159--022-0663666
C. Pandarinath, D. J. O' Shea, J. Collins, et al. Inferring single-trial neural population dynamics using sequential auto-encoders. Nature Methods, 15:805-80 815,-115,7700 2018.
Karl Pearson. Lii. on lines and planes of closest fit to systems of points in space. The London, Edinburgh, and Dublin philosophical magazine and journal of science, 2(11):559-572, 1901.
Matthew G Perich, Juan A Gallego, and Lee E Miller. A neural population mechanism for rapid learning. Neuron, 100(4):964-97,, 2018.
Matthew G Perich, Charlotte Arlt, Sofia Soares, Megan E Young, Clayton P Mosher, Juri Minxha, Eugene Carter, Ueli Rutishauser, Peter Rudebeck, Christopher D Harvey, and Kanaka Rajan. Inferring brain-wide interactions using data-constrained recurrent neural network models. BioRxiv, pp. 2020-12, 2020.
João D Semedo, Amir Zandvakili, Christian K Machens, Byron M Yu, and Adam Kohn, Cortical areas interact through a communication subspace. Neuron, 102(1))249-2299cc 2019.
Yuge Shi, Brooks Paige, Philip Torr, et al. Variational mixture-of-experts autoencoders for multi- modal deep generative models. Advances in neural information processing systems, 32, 2019.
Barry E. Stein and M. Alex Meredith. The Merging of the Senses. MIT Press, Cambridge, MA, 1993.
Thomas M Sutter, Imant Daunhawer, and Julia E Vogt. Generalized multimodal elbo. International Conference on Learning Representations, 2021.
Parsa Vahidi, Omid G Sani, and Maryam M Shanechi. Braid: Input-driven nonlinear dynamical modeling of neural-behavioral data. arXiv preprint arXiv:2509 2025.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Fukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
Saurabh Vyas, Matthew D Golub, David Sussillo, and Krishna V Shenoy. Computation through neural population dynamics. Annual review of neuroscience, 43(1):249-275, 2020.
Qian Wang and Toby P Breckon. Generalized zero-shot domain adaptation via coupled conditional variational autoencoders. Neural Networks, 163:40-52, 2023.
W. Wang, R. Arora, K. Livescu, and J. Bilmes. On deep multi- representation learning: Objectives and optimization. In Proc. of the 33rd International Conference on Machine Learning, pp. 1083- 1092, 2016.
Wen Wang. Zhen Cui, Hong Chang, Shiguang Shan, and Xilin Chen. Deeply coupled auto-encoder networks for cross-view classification. arXiv preprint arXiv:1402.2033, 2014.
Lin Wu, Yang Wang, and Ling Shao. Cycle-consistent deep generative hashing for cross-modal retrieval. IEEE Transactions on Image Processing, 18(4)::602-1112 2018.
Mike Wu and Noah Goodman. Multimodal generative models for scalable weakly-supervise learning. Advances in neural information processing systems, 31, 2018.
Joel Ye and Chethan Pandarinath. Representation learning for neural population activity with neural 01210 arXiv preprint.
data transformers, 2021.
Daiyao Yi. Hao Dong, Michael James Higley. Anne Churchland, and Shreya Saxena. Shared-ae: Automatic identification of shared subspaces in high-dimensional neural and behavioral activity. In The Thirteenth International Conference on Learning Representations, 2025.
B. M. Yu, J. P. Cunningham, G. Santhanam, et al. Gaussian procces factor analysis for low-dimensional single-trial analysis of neural population activity. In Advances in Neural Informa- fion Processing Systems, pp. 1881-1888, 2009.
13

---

Published as a conference paper at ICLR 2026
Amin Zandvakili and Adam Kohn. Coordinated neuronal activity enhances corticocortical communi- cation. Neuron, 87(4):827-88) 839, 2015.
Richard Zhang, Phillip Isola, and Alexei A Efros, Split-brain autoencoders: Unsupervised learning by cross-channel prediction. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 1058-1067, 2017.
14

---

Published as a conference paper at ICLR 2026
A CTAE ALGORITHM OUTLINE FOR R = 2 BRAIN REGIONS
Algorithm 1: Training Coupled Transformer Autoencoders (CTAE) for two regions
Input: Paired recordings ( x cencoder-dccooo parameters 0, Φ. latent dimensions (d,, d1, d2), loss weights Ashared; Aalign Aorth, optimiser O and learning rate ?)
Output: Trained parameters 0" ,6'
Compute weights: W1 [1de 1dy Owg]
W2 - [1d,, Od, 1.n]]
w(*) W1 W2
while not converged do
Encoder forward pass: " <- El (x()).
Z c- (x(1)
Fuse latents: Z +- (Eq. equation
Extract S, p(U) p(2) (Eq. equation
Data reconstruction: x0) +- D (wilf)
x ((221())))
Compute losses: L. free + Saharedfsbare + Aalign Catign + Aorth Corth
(Eq. equation eequuaionn 11 Parameter update: 0, Φ T o, Vo,oLra
10 return 0" 0",6"
B EXTENDING CTAE TO R > 2 BRAIN REGIONS
In the main text, we presented CTAE with two-region formulation and reported results datasets involving either two or three regions. Here, we provide the generalization of our framework to 2 regions.
B. PROBLEM FORMULATION REGIONS)
For clarity, we present the formulation below for three regions. However, this construction naturally extends more than three regions.
Let (1) RNXT X E R (3) RNxT denote simultaneous neural population recordings from three brain regions, each acquired over 7 time steps with N1, N₂, Ng channels, respectively. We assume that the observed activity in each region arises from latent dynamics that decompose into:
1. Region- dynamics p() that capture computations unique to each region [1,2,3) 3). lying in subspaces orthogonal to shared components.
2. Pairwise shared dynamics {(i) that span communication subspaces shared between each pair of regions (i,j) E {(1, 2), (2, (1,3)), reflecting coordinated activity patterns specific to that pair.
3. Globally shared dynamics (123) that capture population trajectories common to all three regions, representing fully shared circuit-level computations.
Formally, the neural activity at each time step is modeled as nonlinear functions of these latent processes:
x A
((12) (12)
(13)
(123)
P(
1:t
1:t
14
x (2) o
(12)
(23)
(123)
(13)
lat
14
x (3) = h (
1st lit
14
where S E dh denotes the latent state shared between regions and j. ((13)) E Rdis the fully
shared latent state across all three regions, and ((r) dRdr the region-private latent state of region
15

---

Published as a conference paper at ICLR 2026
Our objective is recover these latent representations ( p(r)} from the observed neural activity (x x x such that the model disentangles private, pairwise shared, and fully shared dynamics in a principled way. For more than three regions, this formulation extends by including shared latent variables for all possible combinatorial subsets of regions, with the training algorithm remaining unchanged. The training algorithm introduced below applies to this general multi-region setting.
B.2
GENERAL MULTI- REGION SETTING
To move beyond two regions, we extend the formulation with a general mechanism that assigns latent dimensions all possible combinatorial subsets of regions. This extension builds directly on the same architectural backbone introduced in Section 4 but augments it with an enhanced weighted latent fusion mechanism. This mechanism enables CTAE to disentangle latents shared by arbitrary subsets of regions while preserving strictly region-private structure. All other components pee-region causal Transformer encoders/decoders and the optimization procedure- remain unchanged. The key modifications are described below.
Encoder outputs. For each region r E {1...., R...., R) we keep a dedicated mncodrr-deooded pair ( D) and obtain
((r) E (x E D
Region-specific weight masks. A single binary matrix encodes which latent dimensions are associ- ated with each region uses:
W [ W2 wR] E {0,1)RXx,
Whr, =
region r claims dimension d.
A dimension is private when exactly one row in W is active and shared when two or more rows are active, thereby accommodating every subset pattern (for instance, when R the binary code 11 indicates a dimension used by regions and 2 but not 3; 101 corresponds to regions and 3; and 111
denotes a dimension shared by all three). Relative to the two-region mask 3 W expands the notion of "shared" from "both regions" to "arbitary subset of regions". The membership matrix W can be (i) fixed a priori from anatomical knowledge, (ii) selected by appropriate number of latent dimensions to each subset pattern (analogous to the two-region masks in Eq. equation 3), or (iii) treated as a fully learnable variable and optimised jointly with the rest of the network.
search that allocates an
Weighted latent fusion. We fuse the region-spection specific latents with a masked average
Σ R
Wir.
[d]
4.d
(14)
E W
[[r.d] d]
so that private dimensions stay unchanged, whereas any dimension claimed by multiple regions is forced align across them.
Region-specific decoding. Each decoder receives only the dimensions it owns,
(r) (r)) ((w,17) Z),
exactly as in the two-region case. Hence the extension does not change model capacity but merely broadens how "shared" information is defined.
This design (i) embeds all regions in one latent space of size D, (ii) preserves region-specific structure through fixed relevance masks, and (iii) enforces cross-region consistency via Eq. equation [4
B.3 TRAINING OBJECTIVE
We retain the four losses used for two regions--rconsstuuttionn, shared-only reconstruction, alignment, and orthogonality-whie replacing binary masks with the matrix W.
Reconstruction. The reconstruction loss is unchanged, only we now sum over all regions.
R
Lree
Σ
X (r)
(15)
r=1
16

---

Published as a conference paper at ICLR 2026
Shared-only reconstruction. To guarantee that all cross-region regularities live inside shared dimensions, we mask out every coordinate used by fewer than two regions. Let s = (E, 22) E
{0, 1) and w! = Wr ① s. We then reconstruct each region from the shared subspace alone:
R
Sharred
EI ((w ((w(")
2) - 1
(16)
When this term reduces to Eq. (11) in the main text, but for R > 2 now supervises every shared subset simultaneously.
Alignment. Each region's encoder estimate should coincide with the fused latent over the dimensions it owns:
Latign Σ (w, Z (w, о 1F
(17)
Private coordinates contribute zero because they match by definition; shared coordinates are actively driven together, extending the simple two-region difference penalty to an R-way match.
Orthogonality.
fotth zz diag( , zz )|
(18)
Identical to the two-region version, this term keeps all latent dimensions private or shared-mutually decorrelated.
Total loss.
L L + Eataaee + Aalign + North
(19)
with Ashared, Aalign Aorth selected on validation set. The objective reduces exactly the two- region formulation when R = 2; for R > 2 it differs only in how masks are constructed and applied, maintaining conceptual continuity while capturing richer patterns of shared neural dynamics.
C SYNTHETIC NONLINEAR COUPLED DATASET
We construct synthetic two-region dataset in which the ground-truth shared and private latent trajectories are known. We generate paired sequences ( E RNTxxD and x RNXxxD; from latent trajectories stated, we use = 1000, 32, d, =, d1 d 3, D =2.
R NxTxd, p(1) R NxTxd, and p(2) RRNxxxd Unless otherwise
The latent trajectories S,, P! and p!(2) are generated as sinusoidal processes with randomly sampled amplitudes, phases, and integer frequencies. Concretely, for each trial f1..... N) and latent dimension k; we sample an amplitude di,a NO 1), phase di Uniform(O, 2m), and frequency Si, ~ Uniform (2, 3, 4), and define
24,1, 0i (2T 21RR),
T t t=
T.
(20)
The shared latents and private dimensions p(1) and p(2) are generated independently using the above construction with dimensionality da, di and d2, respectively. Each region linearly mixes shared and private latents:
H = a.AA a.A
where A!r)) D-xd and (r Dexde have iid. Gaussian entries scaled by 1/ l/vd. We set a, = ap = 2.0.
We then introduce nonlinear distortion by applying elementwise mapping:
(r)
f,llan = tanh(H! + +H)
The resulting (r) Gaussian noise:
t,llaan can be interpreted as synthetic log-firing rates. We then add independent
x((r) ((r)
iellean e(r)
e( ~N1,00071
17

---

Published as a conference paper at ICLR 2026
aa)
CTAE Latents
(b)
DLAG Latents
1.0
0.10
Shared
Shared
Learned
|corr
Learned
|corr]
0.5
0.05
Private
Private
Private
Private
Shared
Private 2
Shared
Private 1
Private 2
Private
True
True
Figure 7: Absolute Pearson correlation between aligned learned latents and ground-trute latents on the synthetic dataset. (a) CTAE. (b) DLAG. Red dashed lines indicate shared and region- region-specific partitions.
3
1.5
Shared
WW
0.0
-1.5
true
CTAE letents
1.5
0.0
-1.5
10
20
30
20
30
10
20
30
Time (bin)
Figure 8: Latent trajectories the synthetic dataset. Rows correspond to shared, private region and private region latents. Blue: ground truth. Orange: aligned CTAE latents.
with 0.01.
After training, we extract the learned latent trajectories from each model and align each latent subspace (shared and private) to the corresponding ground-truth latents using linear transformation estimated by least-squares regression on a held-out subset of trials. This alignment accounts for the fact that latent representations are only defined up lincar transformation.
We then compute the absolute Pearson correlation between each aligned learned latent dimension and each true latent dimension. Figure shows the resulting correlation matrices. For CTAE, the correlations exhibit clear block structure: dimensions assigned to the shared subspace align with the true shared latents, and region-specific dimensions align with their corresponding private latents, with minimal or no cross-block correlation. In contrast, DLAG does not produce a structured alignment; correlations are weak and do not follow the shared/private partition. DLAAG assumes linear observation model, whereas in this dataset the observations are nonlinear functions of the latent variables. This model mismatch limits its ability to recover the underlying shared and private structure. Figure 8 further illustrates representative latent trajectories after alignment. CTAE closely matches the ground- truth shared and private dynamics across time, while preserving separation between shared and egioo-specific components. Overall, CTAE recovers latent dimensions that closely match the true shared and private dynamics under nonlinear observations, whereas DLAG fails to recover the underlying structure in this setting.
18

---

Published as a conference paper at ICLR 2026
D TRAINING AND HYPERPARAMETER DETAILS
To ensure consistency in model capacity across regions, we adopt identical causal Transformer-hass encoder-decoder architectures for each region, using an equal number of encoder and decoder layers to maintain architectural symmetry. We then performed grid search over:
Number of layers. We varied the total number of Transformer layers L E (1, in both encoder and decoder to trade off model capacity against overfitting risk and computational cost.
Latent dimensions. Private and shared dimensions, d (5, 10, 15). Note Here, the specified latent dimensionality d should be treated as an upper bound on bottleneck capacity. After training with the designed regularizers, redundant factors collapse to near-zero explained variance, yielding effective dimensionality den S d that approximates the data's intrinsic dimensionality.
Loss weights.
Shared-space: Ashared 1,2
Alignment: Aatign E {0, 0.05, 0.1, 0.5)
Orthogonality: Anoth {0, 0.001, 0.01,
Learning rate. 1) E (10-3, (10-4 10-43
Warm-up schedule for orthogonality loss. To prevent an overly strict orthogonality constraint from hindering early representation learning, we Aorth for the first epochs allowing the model to focus reconstruction and shared/private reparation-then linearly ramp it up its target Aorth value over the next epochs. This gradual increase stabilizes training by delaying the full orthogonality penalty until the autoencoders have already learned meaningful features. We explored e E {100, 500, 1000). Formally,
t t e,
orth = t-e Aorth
e e< t - 2e,
Aorth:
t> 2c.
Positional encoding. We use fixed, deterministic sinusoidal signals into each input embedding to convey temporal order, following (Vaswani al.) (017). Specifically, for time step and embedding dimension index (with model dimension dmoodl):
PE(t, 21) = sin mmodd"
PE(t, 2i + 1) = cos 10000 ffmoddl
These encodings provide both absolute and relative position information without adding any learnable parameters, enabling the self-attention layers to distinguish different time steps.
All models were trained for 10 10000 epochs, and the run minimizing the total loss in equation 12 on held-out validation set was selected. Table 2 summarizes all the hyperparameters and their corresponding search ranges and Table 3abllerreports the optimal settings for the two real and one synthetic datasets studied. For the corresponding DLAG and mDLAG baselines, we matched the total number of latent dimensions those used by CTAE for each dataset (Table 3) ensure fair comparison.
Table 2: Hyperparameter search ranges.
Hyperparameter
Values
# Transformer layers L latent dims. dj = d2 d.
[1,2,3]
{5, 10, 15)
Ashared
5,,1,2,5)
Aalign
{0, 0.05, 0.1, 0.5)
Aorth
(.,0.001, 0.01,
Learning rate 1)
{10-3, 1100-,110-4 (100,500,1000)
Warm-up epochs for orth. loss
Compute Resources. All models were trained using UUUDA-acceleaater implementations in Py Torch.
Training was performed on NVIDIA Quadro RTX 8000 GPUs, supplemented by additional compute resources from National Research Platform (NRP) Nautilus.
19

---

Published as a conference paper at ICLR 2026
Table Selected hyperparameters for MI-PMO SC-ALM and Synthetic datasets.
Parameter
M1-PMd
SC-ALM
Synthetic
latent dims
15
dg
4, d1
d2
# Layers L.
3
1
Ashared
5
Aalign
0.5
0.1
0.1
Aorth
0.01
0.01
0.01
Learning rate ?
10-4
10-4
10-3
Warm-up (orth. loss)
100 epochs
1000 epochs
100 epochs
E DATASET DETAILS
The motor circuit dataset used this study is publicly available and was originally released with (Perich et al. 2018). Both datasets were preprocessed using standard neural signal processing pipelines. Specifically, spike trains were first binned at 100 ms for both MI-PMd dataset and SC- ALM dataset. The binned spike counts then smoothed using Gaussian filter (kernel size of and 2 for M1-PMd and SC-ALM, respectively) to estimate instantaneous firing rates, which were used input the CTAE model.
E. MOTOR CIRCUIT DATASET: CENTER-OUT ARM REACHING TASK
We use the dataset from Perich et al., 2018). which includes neuronal recordings from Utah Array in MI and PMd: see Fig. 9 Left). In this task, monkey performs center-out arm reaches using planar manipulandum. Neural activity was recorded from the primary motor cortex (M1) and dorsal premotor cortex (PMd) via Utah arrays, yielding spike-sorted activity from 52 putative units in M1 and 66 PMd (Fig. 9 Right). Each trial begins with a target onset, followed by a variable delay period 103 s) for movement preparation. After "go" cue, the monkey initiates a reach one of eight evenly spaced targets a 2D workspace, holds briefly, and returns to center. The dataset includes 208 trials across target conditions, with approximately 26 repetitions per condition.
Center-out Arm Reaching Motor Cirouit, M1-PMd
Trail Averaged PMd Neural Activity
Avaraged Neural
20
Neure
Neurmn
6-5 Sul
c)
Averaged Activity
Mid Neuron:
PMd Neu 2
MI Neuron: 3
MI Neuron: 4
Time
Time 1s1
Figure 9: Motor study. a) Schematic of the behavioral tasks performed by non-human primates in the datasets used Scction Center-Out Arm Reach Task: The monkey controls a manipulandum to perform instructed delay reaches toward of cight peripheral targets, with neural activity recorded from M1 and PMd using Utah arrays. b) Population firing rates in M1 and PMd. Heatmaps shows the trial-averaged z-scored firing rates for individual neurons (rows) across time. c) Condition-averaal averaged firing rates of individual neurons in MI and PMd. Each panel shows trial-averaged firing rates for single neurons, grouped and colored reach direction. Colormap follows that in Fig.
20

---

Published as a conference paper at ICLR 2026
E.2 SC-ALM DATASET: MULTISENSORY TASK
The anterolateral motor cortex (ALM) and the superior colliculus (SC) are important parts of a cortico-subeortical loop that transforms multisensory inputs into behavioral decisions. To study the role of these areas in multisensory integration and decision-making we trained mice in a multisensory discrimination task, where animals had integrate visual and tactile information over time identify the target stimulus side. We then performed simultancous neural recordings in ALM and SC, using high-density Neuropixels probes, in task sers-peffrming animals., see Fig. TO]left).
Each trial had a total duration of 7 seconds, comprising I-second baseline, then 3 seconds of probabilistic stimuli, followed by a 0.5-second delay without any stimulus and finally, the spouts moved in front of the mouse, allowing up to 2.5 seconds for a licking response. The mice were rewarded when responding on the side where more sensory stimuli were presented during the stimulus period. The stimulus period stimulus could be presented to the mouse. One to six stimuli were presented on a given side, with the
composed of six periods of 500 milliseconds each, during which
each stimuli likelihood of presentation determining the difficulty of the task. Depending trial type, a stimulus could either be a visual grating moving along a screen (visual trial), an air puff of 100ms the vibrissae one side of the mouse (tactile trial), congruent multisensory manner (multisensory trial). Mice were trained to respond by licking the side on which more stimuli were shown. An example trial is illustrated in Fig. 10 center). An example of a multisensory trial timeline is presented in Fig. right).
both, in simultaneous
We analyzed a single session from an expert animal that achieves 70% correct choice accuracy (564 trials in total), with approximately 80-100 trials per stimulus type and per target direction
Screen (visual)
Stimulus
Delay 0.5s
6 x 0.5s
Left stim
Visual
Right stim
Left stim
Water reward
Tactile
Right stim
Spouts
G0
A Airpuffs
Licks (L)
(tactile)
Water
Response
Figure 10: Multisensory study. Left: Schematic of the 2-alternative forced choice task performed by the mice, trained lick on the side where more stimuli was presented. The mice received a water reward through the target spout. Right: Schematic of the task timeline in multisensory trial with left-side target. The stimulus phase in grey, the delay and response phase are in white, separated by a vertical line and the movement of the spouts begins at the start of the response period.
F ABLATION STUDY: INDIVIDUAL LOss CONTRIBUTIONS
We evaluate the contribution of each loss term by training four CTAE variants:
1. Full CTAE: all losses active (shared-only reconstruction, alignment, orthogonality, standard reconstruction).
2. No Shared-only Reconsruction: drop the loss that reconstructs inputs using only the shared subspace.
3. No Alignment: remove the term that aligns shared latents across regions.
4. No Orthogonality: remove the orthogonality constraint between shared and private latents.
For each variant, we (i) visualize ablated interactions where meaningful, and (ii) quantify reach- direction decoding accuracy from each latent subspace.
21

---

Published as a conference paper at ICLR 2026
mDLAG PRIVATE
visual SupSC (0.18),
private DeepSC (0.25)
private (0.23)
1.0
Row-normalized
visual right
probability
tactile left
0.5
tactile right
multisensory left
multisensory right
0.0
SupSC-DeepSC (0.29)
DeepSC-ALM (0.20)
SUpSC-ALM (0.23)
SupSC-DeepSC-ALM (0.26)
mDLAG SHARED
visual left
visual right
tactile
tactile right
multisensory left
multisensory right
visual right
tactile left
tactile ht
multisensory left multisensory right
visual left
tactile left
tactlle right
multisensory left multisensory right ht
visual left visual right tactile left
multisensory left multisenap ht
visual right tactile left
tactlle right
multisensory left
multisensory right
visual rig
tactile rig
visual
Figure 11: low-normalized confusion matrices for decoding stimulus x target side. (top) mDLAG private: region-specifer latents; (bottom) mDLAG shared latents (pairwise and 3-way). Panel titles show mean 5-fold accuracy.
F.1 SHAREEDONLY RECONSTRUCTION LOss ABLATION
The shared-only reconstruction loss ensures that the shared subspace alone can reconstruct the original neural input; encourages capturing all common dynamics in that subspace. Without it, information shifts into private latents, reducing interpretability of the shared space. As shown in Table [ removing this loss component leads to dramatic drop in shared- latent decoding accuracy, accompanied by corresponding increase in private-latent accuracies.
F.2 ALIGNMENT LOSS ABLATION
The alignment loss penalizes discrepancies between shared latents from each region, enforcing that they capture the same underlying dynamics. To highlight the significance of this term, we performed loss ablation experiments and analyzed the latents when alignment loss was enforced. In Fig. 12a), the alignment loss is enforced, forcing the latents to have high similarity and significant overlap. When the alignment loss relaxed in Fig. 12b), the shared latents diverge and sometimes cancel each other, indicating breakdown in cross-region correspondence. Correspondingly, T shows that removing the alignment loss degrades shared-latent decoding accuracy, while private-latenl accuracies remain largely unchanged.
F.3 ORTHOGONNALITY LOSS ABLATION
The orthogonality loss enforces that shared and private latents carry non-redundant information by minimizing their dot products. evaluate the importance this loss, ablation experiments with relaxed orthogonality constraints were used during model training. The degree of disentanglement is quantified using the dot products between latents. Fig. 13 a) the orthogonality loss is enforced in the model and this constrained the latents to be maximally disentangled from each other. In Fig. [3 b) the orthoginality loss is relaxed in the model and there is a significant overlap between the latents, indicating breakdown in disentanglement. Correspondingly, Table shows that removing the orthogonality loss degrades shared-laten decoding accuracy, while private- latent accuracies remain largely unchanged.
22

---

Published as a conference paper at ICLR 2026
Shared
0.5
0.0
0.3
0.0
-0.5
b)
Zi
Alignment of shared latents in M1-PMd. Each subplot visualizes a single latent. Blue and orange traces indicate shared latents obtained from M1 and PMd respectively. Black represents the mean of blue and orange traces. (a) Alignment loss is enforced in the model. The shared latents have significant overlap. (b) Alignment loss is relaxed in this model. The latents deviate significantly from each other.
Figure 12:
a)
b)
1.0
Shared
0.02
0.01
0.01
0.19
0.17
0.14
Private M1
0.01
0.01
0.27
0.17
0.5
Private PMd
0.02
0.22
0.0
Shared
Private M1
Private PMd
Shared
Private M1
Private PMd
Figure 13: Orthogonality between CTAE latents M1-PMd. The red lines are used to differentiate between the shared and eegion-private MI and region-private PMd latents, in that order. The values in the blocks indicate the mean of dot product value between the latents. (a) Latents from model with orthogonality loss enforced (b) Latents from model with orthogonality loss relaxed
G ANALYSIS OF LATENTS AND ATTENTION MAPS
G.1
OVERVIEW OF LATENT DYNAMICS
The CTAE model extracts both shared and region-private latents from neural activity, and does so at single-trial resolution. Shared and region-private latents for the motor dataset are obtained from the best performing CTAE model and visualized in Fig. [4] The region-private latents capture neural activity patterns that are specific to the region and thereby, reflect local dynamics and computations. The shared latents capture activity patterns that common to both regions. The representation of task variables such as reach direction in motor task was analyzed by averaging latent trajectories across trials grouped by task variables. In both regions in Fig. [4]] distinet latents exhibit systematic variation across the task variables, indicating that the latents are inherently capturing behaviorally relevant information.
23

---

Published as a conference paper at ICLR 2026
Fig. 15 visualizes a subset of the CTAE inferred neural latent dynamics (from the set of all latents in Fig. 14). Each panel shows state-space trajectories for different latent groups across task epochs. Immediately after target onset, trajectories the shared subspace Z(*) diverge from a common baseline toward target-dependent fixed points, yielding explicit goal code code-consistent consistent with prior reports of a PMd-M1 preparatory subspace. During movement, variance in Z increases along axes orthogonal to the preparatory subspace, indicating a shift in PMd-MI coupling as the task transitions to execution (Fig. L5 We also observe that PMd-private latents maintain a stable target representation during the preparatory phase (Fig. (5C). consistent with staging signals that are subsequently reflected in the shared space; after the go cue, this private structure weakens, no longer reflecting clear separation of different targets, consistent with transient reorganization or monitoring
movement unfolds (Fig. 15 15)).
Shared
0.5
Reachh directions
1s
0.5
0.0
-0.5
Private
0.5
0.0
-055
0.5
0.0
-0.5
Private PMd
0.5
0.0
0.5
0.0
-0.5
Figure 14: CTAE latent dynamics MI-PMd. Fach subplot visualizes single neural latent, with colors indicating trial-averaged latents for each reach direction. Red dashed line indicates go-cue. (Top) Shared latents capture neural dynamics present in both MI and PMd. (Middle) MI-private latents capture dynamies specific to MI (Bottom) PMd-private latents capture dynamics specific to PMd.
In Fig. 16 and Fig. we present all the latents for the SC-ALM dataset, where we trial-average the latents within target side and stimulus type, respectively. In general, most latents exhibit oscillatory patterns that, to varying degrees, are tuned to the stimulus presentation times; superficial SC latents tend to exhibit higher-freqqencc oscillations. Shared latents for Deep SC and ALM, and region- specific ALM latents, exhibit separation between the target sides at the delay and response. Superficial SC latents and shared superficial and deep SC latents exhibit separate activity patrern for tactile trials comapred to visual and multisensory trials. In contrast, Deep SC and ALM shared latents exhibit patterns that separate all three stimulus types.
We next evaluate temporal decoding of the target variable (e.g.. reach direction, target side) from the latents, to better determine the dynamics represented by the latents in shared and private subspaces.
24

---

Published as a conference paper at ICLR 2026
Ground Truth Postton
PMd Private Latents
Labert
Loteent
Lotent Dam
Latent Dim
Laters
Preparstory
Shared M1-PMd Latents
0.1
Later Dm
Latent Dim 04
Latent Dim
Latens Din
Latone g
Lotent Dimn
5.5. Lateos Dam
Movement Phase
Figure 15: State-space views of CTAE latents across task epochs. Each subplot shows trajectories in dimensional latent subspace. Colors denote trial-averaged trajectories for each reach direction in (A). (B) Shared PMd-MI subspace: trajectories evolve over the entire trial, circle denotes 'go' cue. (C) PMd-private subspace during the preparatory phase. Left and right plots are two different 3D subspaces within the private latents. (D) PMd-private subspace during the movement phase. Together, these panels highlight distinct PMd-MI interaction modes that differ between preparation and execution. Left and right same in (C).
25

---

Published as a conference paper at ICLR 2026
Target side negh
WA MANNO MA WA
Pharo SupSc.Co DeepSc 180
100 ghared DeegSt ALA
private S OR 2000
private -03
shared SupSC Dww.sc----C-- AUA
Share AUASC-UU
DeepsC
Figure 16: SC-ALM: CTAE latents averaged across target side.
26

---

Published as a conference paper at ICLR 2026
multtseenza
Modality toctile
visual
shared SupSC. Deepsi 100
ghared DeegSC AUM ao0
2.00 private SupEC 100
shared SupSC- Dee.-DC----CC-- -UA
shared SupSC-AUX AUM
private AUM
Figure 17: SC-ALM: CTAE latents averaged across stimulus type.
27

---

Published as a conference paper at ICLR 2026
G.2 TEMPORAL DECODING ANALYSIS FROM CTAE LATENT SUBSPACES
Time-resolved decoding. To localize when task information appears in each CTAE latent subspaces, considered per-trial, per-time bin latents from CTAE's private and shared (pairwise and all-shared) subspaces. At every time bin, the input features are the latent dimensions in that subspace (i.e., private, all shared, pairwise shared) within window of 5 time bins. For each subspace ran 5-fold stratified cross- cross-validation with a single multinomial logistic-regression classifier (with feature standardization) trained once per fold on the full time-flattened feature matrix. Time-resolved accuracy was then obtained without re-training: at test time we fixed the classifier and masked all features outside temporal context centered at each time point (a symmetric sliding window of five bins, i.c., =2 bins). We report mean + s.d. across folds at every time bin and overlay event markers for cue onsets, delay start, and delay end.
Findings: SC-ALM dataset. Across subspaces, the accuracy of the shared latents progressively improves as the response approaches, indicating that cross-area information integrates toward the decision and movement execution. Specifically, the shared subspace across all 3 regions demonstrates a positive trend towards the response period with the highest accuracy at the response period and slight increase in accuracy after each stimulus time. Superficial SC private subspace shows the highest prediction accuracy prior to the delay period, consistent with sustained maintenance of task-relevant signals. In contrast, ALM and the Deep SC-ALM shared subspace peak after the response, suggesting strong encoding of the target side and decision (the two are highly correlated in expert mice). The Superficial SC-Deep SC shared subspace also exhibits marked prediction increases immediately following most stimulus onsets, pointing to early multisensory interactions between superficial and deep SC layers, and shared representation of stimulus direction. Together, these patterns reveal complementary temporal roles: private subspaces emphasize delay-period maintenance and response choice, while shared subspaces carry stimulus- and response response-proximal information that ramps as action nears.
0.7
private SupSC
private DeepSC
private ALM
0.6
Accuracy
shared DepSC-
DeepSC-ALM
shared SupSC-ALM
shared SupSC-DeepSC-AMM
0.0
1.0
2.0
3.0
4.0
Time (s)
Figure 18: Temeevesolved decoding accuracy (mean over 5-fold CV) for each latent subspace. Rows shared pairs and all-
correspond to private and shared subspaces (private SupSC/DeepSC/ALN. shared), columns are time. Dashed vertical lines mark cue onsets (every 0.5 s from 0.5-3.0 s), the solid line marks delay start (3.5 s), and the white dotted line marks the start of the decision making period (4.0 s).
Findings: Motor-Circuit dataset. Time-resolved reach-direction prediction accuracy shows pronounced increase following the Go-Cue (Fig. 19). This pattern is observed across latents obtained from CTAE, DLAG and DeepCCA. Shared latents from all three models peak in accuracy following the go-cue, indicating presence of higher directional information, with CTAE-shared consistently outperforming those from the other models. In contrast, PMd-specific latents show an increase in accuracy prior to the go-cue, while, M1-specific latents achieve peak- aackraccusscies post go-cue. These findings suggests that PMd encodes anticipatory directional information, while MI encodes directional information more strongly during movement. Overall, these results align with previous reports directional information representation in M1 and PMd (Lara et al. 2018).
28

---

Published as a conference paper at ICLR 2026
Reach direction prediction
1.0
CTAE shared
CTAE private M1
Accuracy
CTAE private PMd
DLAG shared
0.5
DLAG private M1
DLAG private PMd
DeepCCA shared
0.0
chance
-1
1
2
Time (ms)
Figure 19: Time-resolved reach-direction decoding accuracy (5-fold CV) for each subspace, across multiple models. Red dashed line represents go-cue. Shared latents from all models show sharp increase in accuracy following the Go-Cue, with CTAE-shared consistently outperforming others. PMd-specific latents exhibit a pre-Go-Cue rise, indicating anticipatory directional information, whereas M1 M1-specific latents peak post Go-Cue, reflecting irrection-specifie encoding after movement initiation. Shading represents +1 standard error across the different folds.
Encoder PMd (seltattention))
Encoder M1 (selt-attention)
0.
0.
0.0
Quary dme
002
0.2
0.0
Guinenyee iiu envi aid po wi aidr--o
Key Sime
Say time
Figure 20: Self-Attention Maps obtained from CTAE's encoder branches
G.3 ATTENTION MAPS
We note that as opposed DLAG, our model does not learn latent-specific delays between brain regions. However, observing the attention maps indicates interesting temporal information.
In Fig. 20 we visualize the attention maps for the causal self-attention of the encoders for M1 and PMd separately, averaged over all trials and conditions. The encoders demonstrate the latent were closely aligned in time both regions up the "go" cue, after which they diverged. A diagonal feature in both attention maps, shifted with respect to each other, suggests a lag between the two regions following the cue.
In Fig. 21 we visualize the attention maps for the cross- attentatontion in the decoder encoders for M1 and PMd separately, averaged over all trials and conditions. PMd the attention map shows strong separation between the time periods before and after the cue. The interpretation of the M1 decoder attention map is less clear, without clear separation before and after the cue.
H COMPARISON TO DEEPCCA
We applied DeepCCA the M1-PMd and SC-ALM dataset and extracted shared latents between the pairs of regions. We implemented DeepCCA using parallel multilayer perceptrons (MLPs) for each regions inputs. Each branch consisted of three fully connected layers with 1024 hidden units, sigmoid nonlinearities, and batch normalization (non-affine) applied after each layer. The final layer applicd batch normalization followed by linear projection into shared latent space that matched to
29

---

Published as a conference paper at ICLR 2026
Decoder PMd (cross-attn: query=PMd. key-latents)
Decoder2 MI (cross-attn: query-M1, keymlatents)
3.55
2.04
Quary
0.04
2.03
016
0.02
0.0
Oedton and Cimian we
Key time 00
Figure 21: Cross-Attention Maps obtained from CTAE's decoder branches
that of the CTAE's latent dimensionality. Canonical correlation analysis (CCA) was then performed between the two branches to maximize shared variance in this latent representation. We evaluated these latents on the downstream tasks described in Sec. [ and Sec. 5.2
H.1 M1-PMD DATA
The MI branch received 66-dimensional inputs and the PMd branch 52-dimensional inputs.
DeepCCA Shared
13 0.32
DeepCCA (0.34)
0.4
0.1
0.2
0.2
0.1
0.1
0.1
0.5
0.2
0.1
0.1
0.1
0.1
0.3
0.3
0.2
0.1
w=1011
0.2
0.4
0.1
03
-2
5
-2
Figure 22: DeepCCA Decoding Performance on M1-PMd dataset. (a) Scatter plot of predicted pondtiosisions obtained from linear decoding from DeepCCA shared latents. (b) Confusion matrix for reach-direction classification at single-trial level.
Continuous Decoding. As shown in Fig. 22 a). linear regression readout the hand position from the shared latents, DeepCCA attained an Rt value of 0.32. In comparison, the shared latents obtained from CTAE and DLAG have R" values 0.77 and 0.72. These results indicate the CTAE captures substantially more temporal variation than DeepCCA.
Discrete Decoding. Decoding the hand-reach condition for each trial using the shared latents, DeepCCA obtained an accuracy 34% at a single time-point level (Fig. 22 b)), compared to CTAE (69%) and DLAG (44%).
H.2 SC-ALM DATA
Since DeepCCA performs pairwise correlation maximization, applied it region pairs (i.e.) SupSC-DeepSC, DeepSC-ALM and SupSC-ALM pairs. As H..I the input dimensions for each branch were matched to the dimensions of the number of neurons in each region.
30

---

Published as a conference paper at ICLR 2026
Table 4: DeepCCA Pairwise Results on SC-ALM.
Data
DeepCCA
CTAE
SupSC-DeepSh
0.22
0.59
DeepSC-ALM
0.22
0.59
SupSC-ALM
0.20
0.39
These results, shown in Table. 4 highlight DeepCCA's inability to model temporal dynamics, which hinders time-resolved decoding and interpretability. On the other hand, CTAE combines strong continuous decoding, robust discrete classification a single-time point resolution and structured disentanglement between shared and private latents.
I COMPARISON TO CONCATENATED-DATT BASELINE
As a baseline, we used our transformer autoencoder backbone to analyze the concatenated data across all the regions, without enforcing latent disentangling into shared and private. As a result, we only use the orthogonality loss (Eq. [T) and the reconstruction loss (Eq. g across all regions) to train these models. The extracted latents were evaluated on both discrete and continuous behavioral decoding tasks. While this experiment provides baseline compare the amount of task-relevant information present in these latents, it is important to note that this baseline does not distinguish between computations that are performed in individual regions, or shared across different regions, as the recovered latents may be a mixture of both.
M1-PMd Data. The CTAE model consistently outperformed the concatenated baseline model on both the continuous position and discrete condition decoding task (Fig. 23 On the continuous position decoding task, the concatenated baseline model achieves an R² of 0.59, compared to the R of 0.77 and 0.72 obtained from CTAE and DLAG. The classification accuracy of the reach conditions obtained from the concatenated-data model is 57%, compared to accuracies of 69% and 44% obtained from CTAE and DLAG, respectively.
Concat Baseline
Concat Baseline (0.57)
0.59
S.T
01
0
0
D
0.2
0G
6:
6
0
N
02
m
0.
0.2
Figure 23: Concatenated-D Decoding Performance on M1-PMd dataset. (a) Scatter plot pre- dicted hand-positions obtained from matrix for reach-direction classification at a single-trial level. True and predicted reach conditions on the Y- and X- axis respectively.
linear decoding from Concatenated-Dait latents. (b) Confusion
This highlights that the shared latents between M1-PMd, extracted from CTAE contain more task- related information than the latents obtained from concatenating the data.
SC-ALM Data. The concatenated model outperforms CTAE in downstream behavioral decoding tasks of discrete modality classification (Fig. 24a) and continuous time decoding of target side (Fig. 24p). It is important note that the latents obtained in this baseline do not identify computations that are aegion-specific and shared across regions. This is evident in (Fig. (4p) where the continuous-
time decoding traces lacked the structure that is present in (Fig. 18
31

---

Published as a conference paper at ICLR 2026
a)
CTAE Concatenated Latents (0.68) b) Target Side Decoding Accuracy Time
struu nghe
0.3
0.7
3
83
txctle
Latents
-04
mutioesoryy
-02
muttienecry rgtt-
0.
0.2
0.E
tack rihh-
0.0
1.0
2.0
3.0
4.0
todde
Time (9)
Figure 24: Concatenated-Dare Data Decoding Performance SC-ALM dataset. (a) Confusion matrix for lask-modalities single-trial level. True and predicted modalities on the Y- and X- axis respectively. (b) Continuous-time target side decoding from the latents obtained from the Concatenated-data latents.
Thus, while the concatenated model can reflect dominant task information, it provides limited interpretability into the organization of region-specific versus shared computations. The concatenated approach collapses population activity into single latent space, capturing task-relevant information, but not distinguishing whether that variance shared across regions or specific to one region. In contrast, CTAE is designed explicitly separate shared and region-private components of the neural activity. Thus, the two approaches reflect different conceptual goals-task-focusee compression versus structured disentanglement-annd their outputs are directly interchangeable.
THE USE OF LARGE LANGUAGE MODELS
We used ChatGPT solely for language-related assistance, such as improving grammar, refining phrasing, and polishing the presentation of the text. All research ideas, methods, and results are entirely our own. We take full responsibility for the content of this work.
K REPRODUCIBILITY STATEMENT
We provide all details necessary reproduce our results.
Model and objective: Sec. 4 describes the architecture and all loss terms, with explicit equations for reconstruction, shared-only reconstruction, alignment, and orthogonality (Eqs. 8-12). Multi-region extension: App. B specifies the R > 2 formulation, including masked latent fusion (Eq. 13) and the generalized training losses (Eqs. 14-18).
Training and hyperparameters: App. C reports the full grid (Table 2), selected settings per dataset (Table 3), and the 10k-epoch training selection protocol/warm-up schedule.
Compute resources: (PyTorch/CUDA on Quadro RTX 8000 + HPC) are listed in App. C.
Data and preprocessing: App. details sources and processing (bin sizes, Gaussian smoothing), with task specifics and counts for M1-PMd and SC-ALM.
Evaluation protocols: Sec. 5.1-5.2 define the continuous (hand- and discrete (target/condditon) decoding tasks and present tross-validated metrics/visualizationn: App. E.2 details ablations and reports 5-fold statistics (Table 1).
Baselines: App. G documents the DeepCCA comparison and evaluation setup.
32


Reviews:
[{"review_id": "LHllUUDm3X", "reviewer": "Reviewer_ULYq", "summary": "This paper introduces a new transformer-based autoencoder for modeling neural data collected from multiple brain regions, called CTAE. It deals with the disentanglement of each region's private latent and the shared latent well, resulting in a more biologically interpretable latent subspace and dynamic results.", "questions": "* Why not directly require the shared latent subspace to be just one thing? What's the consideration of splitting them and then aligning them during training? Implementing such a model and other variants is easy and can be viewed as intermediate baselines.\n* Is there any theoretical conclusion about the uniqueness of the discovered latent?\n* For the reaching task dataset, what's the reason for assuming hand position is largely represented in the shared subspace?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The proposed method is intuitive and powerful.\n* The experiments and theoretical analysis are comprehensive, especially for the method extension to multiple regions and the real-world experiments.", "weaknesses": "* There seems to be no synthetic experiment in the main paper.\n* The reconstruction loss is MSE, which means a Gaussian distribution. This is a technically inappropriate distribution for Poisson spike data, as assumed by the authors in line 54. The real-world dataset is also Poisson spike count.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5nqVtdhpn1", "reviewer": "Reviewer_W2Xz", "summary": "This paper addresses the need to have a model that can effectively disentangle the shared components that mediate inter-area interactions from private signals unique to each region for mechanistic insight and designing causal experiments. The authors present previous work and claim that single-area tools fail when they are applied to concatenations of recordings (essentially making the data multi-area). Furthermore, they also mention that recent efforts show a subset of latent dimensions actively participate in inter-area communication. This subspace is believed to be orthogonal to region-specific dynamics. Other CCA-based approaches treat timepoints as iid samples and discard dynamics. This shows the need for CTAE. \nRecorded neural activity across 2 brain regions is modeled as a nonlinear function of underlying latent dynamics specific to each region. \nS = shared dynamics, \nP^1 = dynamics local to region 1, \nP^2 = dynamics local to region 2\n\nThe model is a coupled autoencoder built on transformer blocks that disentangles shared and private representations. The loss function recovers latents that maximally represent the neural activity. Each encoder is a Transformer stack with self-attention layers that capture long-range, nonlinear temporal dependencies within a region. Each decoder employs standard Transformer cross-attention to reconstruct the original firing rates from its region’s latents. \n\nOverall model (Figure 2):\n2 regions of the brain have separate data. These 2 regions are assumed to be sending signals to each other. The data is smoothed and passed into its individual encoder. The encoder brings the dimensionality down to some inferred latent representation. Each of these latent rep vectors is multiplied by a weight mask, and the result is added to produce a single fused latent. This fused latent contains signals that are local to region1/2 and also shared across regions. To disentangle these signals, 2 separate decoders are used. \n\nLoss: they use 4 loss functions. \nReconstruction loss: so that the autoencoder reproduces its own region’s activity.\nShared-only reconstruction loss such that each decoder reconstructs neural activity from its region using only the shared representation. The authors do this to prevent the decoder from shifting information into private subspaces, leading to inaccurate representations. \nAlignment loss:  The shared latents are meant to capture only dynamics common to both regions. To enforce this, they align each encoder’s shared output to their average, ensuring consistency across regions and preventing region-specific variance from leaking into the shared space. \nOrthogonality loss. To encourage each latent coordinate to capture distinct, non-redundant structure.\n\nThey apply CTAE on the dorsal premotor cortex (PMd) and the primary motor cortex (M1) in macaque monkeys performing a standard delayed center-out reaching task with eight outward targets. \n\nThey evaluate their inferred latents on 2 simple linear decoding tasks: predict hand position, and predict target (out of the 8 targets). \nThey hypothesize that the shared latent space would encode a majority of behaviorally relevant information, particularly target identity. M1 would be for finer temporal structure related to movement, and PMd is for higher-order planning signals. \nThey find: the shared latent subspace captured with CTAE showed dominant task-relevant signals. DLAG, in contrast, tended to assign this behavioral info to both PMd and M1 private subspaces. \n\nFor the 2nd test dataset, they use data from S,C which integrates visual and tactile information and contributes to orienting behavior and whereas ALM encodes preparatory and choice-\nrelated activity during decision-making tasks. The shared subspace captures task-relevant features such as stimulus type and target side. They confirm these findings with CTAE, but they do not contrast it with any other method.", "questions": "1. what is the meaning of different degrees of shared or private activity in each brain area, and should we be surprised when decoding a task-relevant variable succeeds in one case and fails in another?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "the method is simple and scalable to more than 2 regions. It is also potentially useful for other kinds of time series data outside of neuroscience. 1 experiment’s comparison with DLAG shows that CTAE is better. 2nd experiment compares CTAE to known region functions and proves that the output from the model matches expectations of region behavior.", "weaknesses": "1. comparison to other methods on only one dataset: For experiment 2 - the authors should have tested DLAG here as well. \n2. The interpretation of the results could be improved.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7lHYVtrH2e", "reviewer": "Reviewer_MYUp", "summary": "The authors propose Coupled Transformer Autoencoder (CTAE) for modeling simultaneous recordings from multiple brain areas using transformer based encoder and decoder priors. CTAE can extract the shared and private representations between regions thus making it a more interpretable model for neuroscience to understand the interactions between different brain regions. The authors demonstrate the utility of CTAE on two electrophysiology datasets.", "questions": "Please refer to the Weaknesses section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- CTAE uses a novel objective that explicitly encourages orthogonality between latent dimensions and also prevents region specific variance from leaking into the shared space. Compared to most of the other multi-region and multi-modal models in neuroscience, I like how principled the objective function has been constructed for a more clean way of ensuring the separation of shared and private latents.\n\n- The introduction of a transformer-based autoencoder for multi-region neural recordings is a unique contribution as far as I know. And this can act as a more flexible prior that can capture the necessary long range temporal dependencies that previous GP or RNN based models might not be able to.\n\n- CTAE is scalable to more than two regions, making it suitable for a variety of neuroscience datasets.\n\n- Clear organization and presentation.", "weaknesses": "Although these are [1, 2,3,4] multi-modal models, I find them to be closely related in terms of capturing the interpretable shared and private latent representations in neuroscientific data. Did the authors consider comparisons to multi-modal models in neuroscience (e.g. modifying them to handle multi-region data for comparisons)?\n\n1. Vahidi, P., Sani, O. G., & Shanechi, M. M. (2025). BRAID: input-driven nonlinear dynamical modeling of neural-behavioral data. arXiv preprint arXiv:2509.18627.\n2. Schneider, S., Lee, J. H., & Mathis, M. W. (2023). Learnable latent embeddings for joint behavioural and neural analysis. Nature, 617(7960), 360-368.\n3. Gondur, R., Sikandar, U. B., Schaffer, E., Aoi, M. C., & Keeley, S. L. Multi-modal Gaussian Process Variational Autoencoders for Neural and Behavioral Data. In The Twelfth International Conference on Learning Representations.\n4. Yi, D., Dong, H., Higley, M. J., Churchland, A., & Saxena, S. Shared-AE: Automatic Identification of Shared Subspaces in High-dimensional Neural and Behavioral Activity. In The Thirteenth International Conference on Learning Representations.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YXavLTQO2j", "reviewer": "Reviewer_TAyW", "summary": "The authors present an transformer-based autoencoder framework for modeling multi-region neural population data. The framework provides mechanisms for separately modeling shared and private variability across brain regions. The authors fit their model on two datasets - motor regions in monkey and multisensory regions in mouse - and analyze the information contained in the resulting subspaces.", "questions": "L85: is \"absent\" supposed to be here?\n\nThe authors claim one advantage of their framework is \"Generic downstream utility\", i.e. that the latent space is behavior agnostic. However, a robust line of work is developing that attempts to partition neural latents into \"behaviorally-relevant\" and \"behaviorally-irrelevant\". While directly addressing this is outside the scope of this work, I'm curious if the authors think there is room (and/or utility) in expanding their approach to encompass this desire for partitioned subspaces.\n\nAre the trials the same length in both datasets? If not, how do authors manage batches composed of different-length sequences during training?\n\nOne of the major benefits (in my opinion) of the transformer-based approach is that it automatically takes care of time delays between different regions, in a much more robust, adaptive, and elegant way than previous approaches. If this is indeed the case, the authors should highlight this advantage in the main text. How could one actually dive into this more deeply? If there is a way to infer trial-to-trial lags between regions that would be incredibly useful (and interesting to compare to the lags inferred by DLAG).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "CTAE represents a step forward in the literature of multi-region neural modeling. This framework is powerful and flexible, easily extending to more than 2 regions without an explosion of parameters that other models face. There is additional flexibility in defining subsets of regions that should have shared latent spaces, elegantly encoded in a single matrix. The transformer architecture also naturally takes into account temporal delays between regions, which previous models struggled to do in a scalable manner. The fitting procedure uses standard approaches (Adam) without too many bespoke elements (just a warm-up schedule for the orthogonality loss).\n\nI appreciate that the authors explored the use of this model in two very different datasets: monkey vs mouse, motor vs sensory/cognitive areas, 2 vs 3 regions, etc.\n\nThe writing and figures are very clear.", "weaknesses": "Results on real data are hard to interpret. For example, the authors state \"...the shared PMd–M1 subspace captured the dominant task-relevant signals. In contrast, DLAG tended to assign behaviorally relevant variance to both PMd and M1 private subspaces, potentially due to less effective separation between shared and private dynamics.\" While this feels intuitively true, it's impossible to know which model is \"more correct\". The example with the second dataset is even more complex, and hard to if these results are \"good\" or not. I'm wondering if there are other manipulations that can further highlight the benefits of this model. For example, what if you fit the model on data concatenated across the regions, such that there is just a single set of latents, and then use these latents to decode continuous hand position and target condition. How do these values compare to the private/shared subspaces? This sort of analysis would represent a \"baseline\" that is perhaps easier to compare to than DLAG or other existing models.\n\nFollowing the previous comment, this work could be strengthened considerably with simulated data. Can CTAE recover the ground truth when the model hyperparameters match those of the data generation process (e.g. number of latents)? How does performance change when the model hyperparameters are not matched (e.g. more/fewer latents)? How much data is actually needed to train the CTAE (under matched hyperparameters, say)? These questions should be explored more thoroughly with simulated data given the difficulty of interpreting the results on real data.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)