N EURONAL L EARNING A NALYSIS USING C YCLEC ONSISTENT A DVERSARIAL N ETWORKS
Anonymous authors
Paper under double-blind review
A BSTRACT
Understanding how activity in neural circuits reshapes following task learning could reveal fundamental mechanisms of learning.
Thanks to the recent advances in neural imaging technologies, high-quality recordings can be obtained from hundreds of neurons over multiple days or even weeks.
However, the complexity and dimensionality of population responses pose signiﬁcant challenges for analysis.
Existing methods of studying neuronal adaptation and learning often impose strong assumptions on the data or model, resulting in biased descriptions that do not generalize.
In this work, we use a variant of deep generative models called – cycle-consistent adversarial networks, to learn the unknown mapping between pre- and post-learning neuronal activities recorded in vivo .
To do so, we develop an end-to-end pipeline to preprocess, train and evaluate calcium ﬂuorescence signals, and a procedure to interpret the resulting deep learning models.
To assess the validity of our method, we ﬁrst test our framework on a synthetic dataset with known ground-truth transformation.
Subsequently, we applied our method to neuronal activities recorded from the primary visual cortex of behaving mice, where the mice transition from novice to expert-level performance in a visual-based virtual reality experiment.
We evaluate model performance on generated calcium imaging signals and their inferred spike trains.
To maximize performance, we derive a novel approach to pre-sort neurons such that convolutional-based networks can take advantage of the spatial information that exists in neuronal activities.
In addition, we incorporate visual explanation methods to improve the interpretability of our work and gain insights into the learning process as manifested in the cellular activities.
Together, our results demonstrate that analyzing neuronal learning processes with data-driven deep unsupervised methods holds the potential to unravel changes in an unbiased way.
I NTRODUCTION
One of the main objectives in computational neuroscience is to study the dynamics of neural processing and how neural activity reshapes in the course of learning.
A major hurdle was the difﬁculty in obtaining high-quality neural recordings of the same set of neurons across multiple experiments, though such limitation in recording techniques has seen tremendous improvements in recent years.
With the advent of modern neural imaging technologies, it is now possible to monitor a large population of neurons over days or even weeks (Williams et al., 2018a; Steinmetz et al., 2021), thus allowing experimentalists to obtain in vivo recordings from the same set of neurons across different learning stages.
Signiﬁcant efforts have been put into extracting interpretable and unbiased descriptions of how cortical responses change with experience.
Proposed approaches to model changes in neuronal activity include linear latent variable models such as PCA, TCA, GPFA, GPFADS and
PSID (Cunningham & Byron, 2014; Williams et al., 2018b; Sani et al., 2021; Yu et al., 2009; Rutten et al., 2020).
Methods employing deep learning models but with linear changes or mapping include
LFADS and PfLDS (Pandarinath et al., 2018; Gao et al., 2016).
While these methods enabled substantial progress in understanding the structure of neuronal activity, they do have strong assumptions inherent in the modelling technique or the analysis, such as the linearity assumption in linear latent variable models.
Therefore, making sense of the unknown mapping between pre- and post-learning neural activity in an unbiased manner remains a signiﬁcant challenge, and a data-driven method to interpret the circuit dynamics in learning is highly desirable.
Thanks to their ability to self-identity and self-learn features from complex data, deep neural networks (DNNs) have seen tremendous success in many biomedical applications (Cao et al., 2018;
Zemouri et al., 2019; Piccialli et al., 2021).
Speciﬁcally, deep generative networks have shown promising results in analyzing and synthesizing neuronal activities in recent years.
Pandarinath et al.
(2018) developed a variational autoencoder (VAE) to learn latent dynamics from single-trial spiking activities and Prince et al.
(2020) extended the framework to work with calcium imaging data.
Numerous work have demonstrated generative adversarial networks (GAN) are capable of synthesizing neuronal activities that capture the low-level statistics of recordings obtained from behaving animals (Molano-Mazon et al., 2018;
Ramesh et al., 2019; Li et al., 2020).
In this work, we explore the use of cycle-consistent adversarial networks (Zhu et al., 2017), or CycleGAN, to learn the mapping between pre- and post-learning neuronal activities in an unsupervised and data-driven manner.
In other words, given the neural recordings of a novice animal, can we translate the neuronal activities that correspond to the animal with expert-level performance, and vice versa?
The resulting transformation summarizes these changes in response characteristics in a compact form and is obtained in a fully data-driven way.
Such a transformation can be useful in follow-up studies to 1) identify neurons that are particularly important for describing the changes in the overall response statistics, not limited to ﬁrst or second order statistics; 2) detect response patterns relevant for changes from pre- to post-learning; and 3) determine what experimental details are of particular interest for learning.
To learn the transformation, we derive a standardized procedure to train, evaluate and interpret the CycleGAN framework.
To improve the explainability of our work, we incorporate a self-attention mechanism into our generator models and also employ a feature-importance visualization method into our pipeline so that we can visualize and identify the input that the networks deemed relevant in their decision making process.
In addition, we introduced a novel neuron ordering method to improve the learning performance of convolutional neural networks (CNN).
To quantify the capability of the proposed unsupervised learning method, we evaluate our method on two datasets: 1) an artiﬁcially constructed dataset with a handcrafted transformation, and 2) recordings obtained from the primary visual cortex of a behaving animal across multiple days.
We then compare several metrics and statistics between the recorded and translated calcium traces and their inferred spike trains.
M ETHODS 2.1 A NIMAL EXPERIMENT
To obtain neuronal activities that can demonstrate pre- and postlearning responses, we conducted a visual-based experiment which follows a similar procedure as
Pakan et al.
(2018) and Henschke et al.
Brieﬂy, a head-ﬁxed mouse was placed on a linear treadmill that allows it to move forward and backward.
A lick spout and two monitors were placed in front of the treadmill and a virtual corridor with deﬁned grating pattern was shown to the mouse.
A reward (water drop) would be made available if the mouse licked within the predeﬁned reward location (at 120-140 cm), in which a black screen is displayed as a visual clue.
Figure A.1 illustrates the experiment setup.
The mouse should learn to utilize both visual information and self-motion feedback to maximize reward.
The same set of neurons in the primary visual cortex were labelled with
GCaMP6 calcium indicator and monitored throughout 4 days of experiment, the relative changes in ﬂuorescence ( ∆ F/F 0 ) over time were used as a proxy for an action potential.
4 mice were used in the virtual-corridor experiment and all mice transitioned from novice to expert in the behaviour task within 4 days of training. Mouse 1 took on average 6.94s per trial on day 1 and 4.43s per trial on day 4, Table A.1 and A.2 shows the trial information of all the mice.
Hence, this dataset can provide excellent insights into how cortical responses reshape with experience.
2.2 C YCLE GAN
CycleGAN (Zhu et al., 2017) is a GAN-based unsupervised framework that learns the mapping between two unpaired distributions X and Y via the adversarial training and cycle-consistency optimization.
The framework has shown excellent results in a number of unsupervised translation tasks, including natural language translation (Gomez et al., 2018) and molecular optimization (Maziarka et al., 2020), to name a few. Let X and Y be two distributions with unknown mappings that correspond to (novice) pre- and (expert) post-learning neuronal activity, respectively.
CycleGAN consists of four DNNs: generator G : X → Y that maps novice activities to expert activities and generator F : Y → X that maps expert activities to novice activities; discriminator D X : X → [0 , 1] and discriminator D Y :
Y → [0 , 1] that learn to distinguish novice and expert neural activities, respectively. In a forward cycle step ( X → Y → X , illustrated in Figure B.1), we ﬁrst sample a novice recording x from distribution X and apply transformation G to obtain ˆ y = G ( x ) . We expect ˆ y to resembles data from the expert distribution Y , hence D Y learns to minimize (1) L D Y = − E y ∼ Y
[( D Y ( y ) − 1) 2 ] + E x ∼ X [ D Y ( G ( x )) 2 ] .
Similar to a typical GAN, generator G learns to deceive D Y with the objective of (2) L G = − E x ∼ X
[( D Y ( G ( x )) − 1) 2 ] .
Note that these are same objectives in LSGAN (Mao et al., 2017).
However, D Y can only verify if ˆ y ∈ Y , though cannot ensure that ˆ y is the corresponding expert activity of the novice recording x .
Moreover, X and Y are not paired hence we cannot directly compare ˆ y with samples in Y .
To tackle this issue, CycleGAN applies another transformation to reconstruct the novice recording ¯ x = F (ˆ y ) where the distance (cid:107) x − ¯ x (cid:107) or (cid:107) x − F ( G ( x )) (cid:107) should be minimal.
Therefore, the generators also optimize this cycle-consistent loss (3) L cycle =
E x ∼ X
[ (cid:107) x − F
( G ( x )) (cid:107) ]
+ E y ∼ Y
[ (cid:107) y − G ( F ( y ))
(cid:107) ] . Mean absolute error ( MAE ) was used as the distance function, though other distance functions can also be employed.
In addition, we would expect ˆ x = F ( x ) and ˆ y =
G ( y ) to be in distributions X and Y given that F : Y → X and G : X → Y , hence the identity loss objective (4) L G identity =
E y ∼ Y
[ (cid:107) y − G ( y ) (cid:107) ] .
Taken all together, G optimizes the following objectives: (5) L G total =
L G + λ cycle L cycle + λ identity L G identity where λ identity and λ cycle are hyper-parameters for identity and cycle loss coefﬁcients.
All four networks are trained jointly where L F total and L D X are similar to L G total and L D Y though in opposite directions.
In this work, we adapt the CycleGAN framework to learn the unknown mapping between pre- and post-learning neuronal activities recorded from the primary visual cortex of behaving mice.
In addition, we experiment with different GANs objective formulations on top of the original LSGAN objective, including GAN (Goodfellow et al., 2014), WGANGP (Arjovsky et al., 2017) and DRAGAN (Kodali et al., 2017). Table C.2 shows their exact formulations in CycleGAN.
2.3 M ODEL PIPELINE
We devise a consistent analysis framework, including data preprocessing and augmentation, networks interpretation, and evaluation of the generated calcium ﬂuorescence signals and their inferred spike trains.
Figure C.1 illustrates the complete pipeline of our work.
We denote the day 1 (pre-learning) and day 4 (post-learning) recording distributions to be X and Y .
With Mouse 1, W = 102 neurons from the primary visual cortex were monitored, as well as trial information such as the virtual distance, licks and rewards.
In total, 21471 and 21556 samples were recorded on day 1 and 4.
Since we want the generators and discriminators to identify patterns relevant to the animal experiment in a data-driven manner, we do not incorporate any trial information into the training data.
We ﬁrst segment the two datasets with a sliding window of size H =along the temporal dimension (around 85 s in wall-time), resulting in data with shape ( N, H, W ) for X and Y where N is the total number of segments.
We select a stride size that space out each segment evenly so that we obtained a sufﬁcient number of samples while keeping the correlations between samples reasonably low.
In order to take advantage of the spatiotemporal information in the neuronal activities in a 2D CNN, we further convert the two sets to have shape ( N, H, W, C ) where
C = 1 .
Finally, we normalize each set to the range [0 , 1] , and divide them into train, validation and test set with 3000, 200 and 200 samples respectively.
To evaluate the transformation results of G and F , we can compare the cycle-consistency
MAE ( X, F ( G ( X ))) and MAE ( Y, G ( F ( Y ))) , as well as the identity losses MAE ( X, F ( X )) and MAE ( Y, G ( Y ))
(e.g. we expect F to apply no transformation to a novice sample x ). We also evaluate the generated data in terms of spike activities in the following distribution combinations: novice against translated novice ( X | F ( Y ) ), novice against reconstructed novice ( X | F ( G ( X )) ), expert against translated expert ( Y | G ( X ) ) and expert against reconstructed expert ( Y | G ( F ( Y )) ).
We use Cascade (Rupprecht et al., 2021) to infer spike trains from the recorded and generated calcium signals to assess the credibility of the generated signals.
We measure the following commonly used spike train similarities and statistics: 1) mean ﬁring rate for evaluating single neuron statistics; 2) pairwise Pearson correlation for evaluating pairwise statistics; 3) pairwise van Rossum distance (Rossum, 2001) for evaluating general spike train similarity.
We evaluate these quantities across the whole population for each neuron or neuron pairs and compare the resulting distributions over these quantities obtained from the recorded and generated data.
We, therefore, validate the whole spatiotemporal ﬁrst and second-order statistics as well as general spike train similarities.
To improve the explainability of this work we introduce a number of recently proposed model interpretation methods into our pipeline.
We design a self-attention generator architecture which allows the network to learn a set of attention masks such that it encourages the network to better focus on speciﬁc areas of interest in the input and also enables us to visually inspect the learned attention maps.
In addition, we use GradCAM (Selvaraju et al., 2017), a method to visualize discriminative region(s) learned by a CNN classiﬁer w.r.t to the input, to extract localization maps from the generators and discriminators.
The self-attention mechanism and GradCAM visualization allow us to verify and interpret that the networks are learning meaningful features.
Moreover, these extracted attention maps can reveal neurons or activity patterns that are informative in the neuronal learning process.
A detail description of the model architectures are available in Section D.
2.3.1 N EURON ORDERING
CNNs with a smaller kernel can often perform as well or even better than models with larger kernels while consisting of fewer trainable parameters
(He et al., 2016a;
Li et al., 2021).
Nevertheless, a smaller kernel can also limit the receptive ﬁeld of the model, or the region in the input that the model is exposed to in each convolution step (Araujo et al., 2019).
In addition, the recordings obtained from the virtual-corridor experiment were annotated based on how visible the neurons were in the calcium image, rather than ordered in a particular manner (see
Figure A.1).
This could potentially restrict CNNs with small receptive ﬁeld to learn meaningful spatial-temporal information from the population responses.
To mitigate this issue, we derive a procedure to pre-sort X and Y , such that neurons that are highly correlated or relevant are nearby in their ordering.
A naive approach is to sort the neurons by their ﬁring rate or average pairwise correlation, where the neuron with the highest ﬁring rate or the neuron that, on average, is most correlated to other neurons is ranked ﬁrst in the data matrix.
However, it is possible that not all highﬁring neurons or most correlated neurons are the most inﬂuential in the learning process.
Therefore, we also explore a data-driven approach.
Deep autoencoders have shown excellent results in feature extraction and representation learning (Gondara, 2016; Wang et al., 2016; Tschannen et al., 2018), and we can take advantage of its unsupervised feature learning ability.
We employ a deep autoencoder AE which learns to reconstruct calcium signals in X and Y jointly.
AE consists of 3 convolution down-sampling blocks, followed by a bottleneck layer, then 3 transposed-convolution up-sampling blocks.
The down-sampling block consists of a convolution layer followed by Instance Normalization (Ulyanov et al., 2016), Leaky ReLU (LReLU) activation (Maas et al., 2013) and Spatial Dropout (Tompson et al., 2015), whereas a transpose convolution is used in the up-sampling block instead.
We optimize the mean-squared error ( MSE ) reconstruction loss on the training set of X and Y , then we use the per-neuron reconstruction error on the test set to sort the neurons (in ascending order): order = argsort (0 .
5 × [ MSE ( X, AE ( X ))
+ MSE ( Y, AE ( Y )) ]) .
The neuron sorting process is part of the data preprocessing step and is independent from the CycleGAN framework.
2.3.2 S YNTHETIC DATA
CycleGAN was originally introduced for image-to-image translation.
Albeit the two image distributions are not aligned hence cannot be directly compared easily, one could still visually inspect whether or not ˆ x = F ( y ) and ˆ y = G ( x ) are reasonable transformations.
However, it would be difﬁcult to visually inspect the two transformations with calcium signals.
To this end, we introduce an additional dataset Y =
Φ( X ) with a known transformation Φ , such that G : X → Y = Φ( X ) and F : Y =
Φ( X ) → X .
We can then verify G ( x ) =
ˆ y =
Φ( x ) and F ( y ) =
ˆ x = x .
We deﬁned the spatiotemporal transformation Φ that can be identiﬁed visually as follows: (6) Φ( x ) = m diagonal x + 0 .
5 η , where m diagonal is a diagonal mask to zero-out the lower left corners of the signals and η ∼ N ( µ x , σ 2 x ) .
µ x and σ x are the per-neuron mean and standard deviation of X .
Figure 1 shows an augmented example.
Importantly, we shufﬂe the train set after the augmentation procedure so that X and Y appears to be unpaired to the model.
Whereas the test set remains in its original paired arrangement so that we can compare (cid:107)
X − F ( Y ) (cid:107) and (cid:107) Y − G ( X ) (cid:107) .
R ESULTS
We assessed the CycleGAN framework on synthetic data with known ground truth and on experimental data where we recovered trial information.
We also experimented with different GAN objective formulations as well as different neuron ordering methods.
All models presented below were trained with the Adam optimizer (Kingma & Ba, 2014) for 200 epochs where all models converged.
We trained all CycleGAN models on a single NVIDIA A100 GPU which on average took 15 hours to complete. It took an additional hour to train the autoencoder in the case where we pre-sort neurons according to the AE reconstruction loss. Table C.1 details the hyper-parameters used.
3.1 S YNTHETIC DATA
To show that our method is capable of learning subtle differences in calcium traces, we ﬁrst ﬁt our model on the synthetic dataset.
Figure 2 shows calcium signals of the forward and backward cycle transformation of neuron 75 from a randomly selected test segment, where AGResNet generators were trained with LSGAN objectives (more examples in Figure F.1).
Without paired samples, F ( y ) made a reasonable attempt in reconstructing the augmented region in y , whereas G ( x ) was able to learn to mask out the appropriate regions in x .
Since Φ( X )
=
Y performed a systematic spatiotemporal transformation to X , one would expect the networks to learn features that focus on the augmented region of the data.
We, therefore, use GradCAM (Selvaraju et al., 2017) localization maps to visualize regions of interest learned by the discriminators.
The localization map of discriminator D Y ( Y ) when given an augmented sample y =
Φ( x ) , shown in Figure 3, demonstrates a high level of attention around the edge of the diagonal region.
This indicates that D Y learned to distinguish whether or not a given sample is from distribution Y =
Φ( X ) by predominantly focusing on the edge of the masking area.
On the other hand, since no augmentation was done on the input to discriminator D X , the localization map does not appear to have a particular structural area of focus at ﬁrst (c.f. Figure F.3).
Interestingly, once we overlay the reward zones on the input, we observe that the area of focus learned by D X is loosely aligned with the reward zones.
Note that reward zones are external task-relevant regions that are expected to shape the neural activity in the primary visual cortex as the visual patterns change when the mouse enters the reward zone.
Our ﬁndings therefore suggest that D X learned distinctive patterns from highly ranked neurons around the reward zones.
Figure 3 shows the AG sigmoid masks from G ( x ) .
Both attention masks ignored the augmentation region (i.e. bottom left corner), as information in that area is not relevant in the G : X → Φ( X ) transformation.
Similar, F which should learn Φ( X ) → X also allocated less focus in the masked region in its reconstruction process, as it contains no useful information.
(see Figure F.3).
Since X and Y =
Φ( X ) are paired in the test set, this allows us to compute MAE ( Y, G ( X )) and
MAE ( X, F ( Y )) hence providing a good testbed to compare different generator architectures, GAN objective formulations and neuron ordering methods.
We also added the identity models as baseline, which should have perfect cycle-consistent loss as F ( G ( x )) and G ( F ( y )) perform no operation on the data.
Nevertheless, despite the fact that Φ is a relatively simple augmentation, one would expect the difference between X and Φ( X ) to be small.
Table 1 shows the direct comparison results of different combinations of objective formulations, generator architectures and neuron ordering methods. Both ResNet and AGResNet achieved signiﬁcantly better results than the identity model. To mitigate the issues of vanishing gradient and mode collapse, we used gradient penalty regularization to enforce the 1-Lipschitz condition in the discriminator. We, therefore, tested 4 popular GAN objectives with the CycleGAN framework.
Interestingly, the LSGAN objectives achieved slightly better results than GAN objectives while both performed better than identity. The two objectives with gradient penalty obtained lower cycle-consistent errors than GAN and LSGAN, yet performed signiﬁcantly worse in the intermediate transformations F ( Y ) and G ( X ) . This suggests that the discriminators could be overpowered by the generators when trained with WGANGP and DRAGAN, in which D X ( F ( Y )) and D Y ( G ( X )) are neither informative nor impactful to the overall objective. This is likely because the gradient penalty regularization further complicates the already perplexing
CycleGAN objectives. We employed 3 different methods to pre-sort neurons in the data, including ﬁring rate, pairwise correlation and autoencoder reconstruction loss. In addition, to demonstrate that 2D convolution can indeed better learn the spatial structure in neuronal activities, we trained a 1D variant of AGResNet (denoted as 1D-AGResNet ) as baseline which disregards all spatial information. Overall, models trained on sorted neurons achieved better results compared to unordered neurons and in most cases, sorting neurons according to the autoencoder reconstruction loss performed the best.
Moreover, 1D-AGResNet performed signiﬁcantly worse than its 2D counterparts, suggesting that the spatial structure in the neural activities is indeed important. In the remaining work, we use the LSGAN objective to train the generators with the AGResNet architecture along with neurons ordered based on autoencoder reconstruction loss as this combination achieved the best overall results on the synthetic data. We signals More examples are shown in Figure F. shows the original input, where yellow and orange dotted lines mark the ordered reconstruction
3.2 R ECORDED DATA
As our proposed method has successfully learned the unpaired transformations in the synthetic dataset, we now move on to the recordings obtained from the virtual-corridor experiment where we attempt to learn the unknown mapping between pre- and post-learning neuronal activity. Figure 4 shows the cycle transformation of neuron 50 from a randomly selected test segment.
Visually, G and F seems to be able to reconstruct ¯ x
= F ( G ( x )) and ¯ y
= G ( F ( y )) , and that the two generators are not simply passing through x and y in intermediate step ˆ y =
G ( x ) and ˆ x = F ( y ) .
To better analyse the transformation performance, we ﬁrst compare the generated calcium ﬂorescence signals with the recorded test set data.
The cycle-consistent loss on the test set achieved a values of MAE ( X, F ( G ( X )))
= 0 .
0733 and MAE ( Y, G ( F ( Y )))
= 0 .
The identity losses for MAE ( Y, G ( Y )) and MAE ( Y, G ( Y )) are also minimal, with values of 0 . 0101 and 0 .
0069 , respectively.
For reference, MAE ( X, Y ) = 0 .
This suggest G and F are not simply passing through the data without any processing.
In addition, the low identity loss indicates that the generators can correctly identify whether or not the given input is already part of its target distribution.
Table G.1 reports the cycle-consistent and identity loss with different neuron ordering methods.
Since we lack paired data in the in vivo recordings, we cannot directly compare MAE ( X, F ( Y )) nor
MAE ( Y, F ( X )) , in contrast to Section 3.1.
In order to better analyse the two intermediate transformations ˆ y =
G
( x ) and ˆ x = F ( y ) , and show that G and F can indeed translate x and y into their respective distributions ˆ y ∼ Y and ˆ x ∼ X , we also compare a set of spike train statistics. Section H [Table] X −
[Table] Y ) [Table] |
[Table]
| [Table] [Figure] More examples are shown in Figure G.1 and Figure G. shows that per-neuron and per-segment comparison.
We ﬁrst compare the ﬁring rate distribution of each neuron between recorded and translated data (e.g. X vs F ( Y ) and X vs F ( G ( X )) ).
Examples of the distribution comparisons are available in Section H. Since we expect that the distribution of the generated data resemble of those from the recorded data, we can compare the KL divergence for each neuron to quantify the transformation performance.
The ﬁring rate distributions of F ( Y ) and G ( X ) closely matched the distributions of X and Y , with average KL divergence of 1 .and 1 . 0697 , respectively.
Similarly, we can compute the pairwise correlation of each neuron w.r.t the population and compare the distribution between translated and recorded data.
X | F ( Y ) and Y | G ( X ) achieved an average KL divergence value of 0 . 0479 and 0 .
0493 in the pairwise correlation comparison, both were signiﬁcantly better results than the baseline identity model.
In addition, we measure the van Rossum distance between X and F ( Y ) for each neuron across 200 test samples, and represent the results in the form of a heatmap.
We can observe a clear diagonal line of lowintensity values in the heatmaps for most neurons (e.g. Figure H.3 and H.4 for G and F ).
Hence, there exists a spike train in X and Y that corresponds to a translated spike train in F ( Y ) and G ( X ) .
Table G.2 summaries the average KL divergence of the 3 spike statistics in different distribution combinations, the results indicate that the generators can indeed learn the distribution translation from pre- to post-learning neuronal activities, and vice-versa.
We additionally trained separate models on the activities recorded from the other mice and obtained similar results, which are available in Section I, J and K.
In the previous section, we were able to identify and interpret the learned features in a relatively straightforward manner due to the systematic augmentation we introduced into the data.
However, visualizing and interpreting the attention maps on pre- and post-learning data could be more challenging as there would not be obvious patterns in the inputs to anticipate.
Nevertheless, we would expect a higher level of activities in the V1 neurons when the mouse is about to enter or inside the reward zone, where the grating pattern on the virtual walls turn to black.
Subsequently, the generators and discriminators should learn meaningful features from responses surrounding the reward zones.
We ﬁrst visualize the sigmoid masks in AGResNet .
Figure 5 shows the learned attention masks of
G superimposed on the latent inputs (see Figure G.3 for F ).
When the neurons were ordered, either by ﬁring rate or autoencoder, we observe that the generators allocate more attention toward neurons that rank higher.
This suggest that by grouping neurons in a meaningful manner, the convolutional layers in the generators can extract relevant features more effectively as compared to when neurons were randomly ordered.
The spike analysis showed that ordering neurons in a structured manner does indeed yield better results across the board.
In most cases, ordering the neurons based on the reconstruction error achieved the best results.
We then inspect the GradCAM localization maps of the discriminators.
Similar to D X in the synthetic dataset, we observed regions of high attention surrounding the reward zones in both D X and
D Y (see Figure G.3 and 5).
To better visualize the relationship between the area of focus learned by the model and the virtual-corridor, we generate positional attention maps as shown in Figure 6. We ﬁrst compute GradCAM maps for all test samples, then we average the activation value for each neuron at each virtual position (160 cm in total) and plot the average activation value against distance. Effectively, these maps should represent the average attention learned by the models w.r.t. the visual location of the animal.
Importantly, the only objective the discriminators had was to distinguish if a given sample is from a particular distribution.
Thus, the discriminators could have learned trivial features.
Instead, D X focused on a speciﬁc group of neurons at 100 - 130 cm in the virtual environment, which coincides with the beginning of the reward zone.
Moreover, D Y learned to focus on two groups of neurons with attention patterns that were also in alignment with the reward zone in the virtual-corridor experiment.
Similarly, we can extract these positional attention maps for G ( X ) and F ( Y ) following the same procedure, where we monitor the change in gradient in the last residual block RB 9 (bottom row in Figure 6).
Interestingly, both generators focused on the ﬁrst few neurons in their transformation operations.
G focused on activities at the beginning of the trial as well as activities in the reward zone; whereas with F , it paid higher level of attention to activities right before the reward zone.
This suggests that to learn the transformation from post- to pre-learning responses, the activities the mouse exhibit as it approaches the reward zone is deemed more important by the networks.
Note that no trial information was incorporated into the training data nor was it formulated in the objective function.
Hence, these interesting patterns we observe here were learned entirely by the networks themselves via the adversarial process. G 6b
D ISCUSSION
We demonstrated that the CycleGAN (Zhu et al., 2017) framework is a capable data-driven method to model the translation between pre- and post-learning responses recorded in vivo .
With selfattention and feature-importance visualization methods, we are able to visualize information that the networks deemed important in their translation and discrimination process.
Intriguingly, without providing trial information in the training process, the networks self-identiﬁed activities surrounding the reward zone in the virtual-corridor experiment to be highly inﬂuential, which aligns with our understanding that the responses in the visual cortex were shaped by the change of visual cues.
In addition, we introduced a novel and simple to implement neuron ordering method enabling more effective learning by convolutional-based networks.
A signiﬁcant portion of the neuronal activity validation in Section 3.2 was performed in spike trains inferred from the recorded and generated calcium ﬂuorescent signals using Cascade (Rupprecht et al., 2021), which is a recently introduced method that has outperformed existing model-based algorithms.
However, reliable spike inference from ﬂuorescent calcium indicators signals remains an active area of research (Theis et al., 2016).
For instance, Vanwalleghem et al.
(2020) demonstrated that spiking activities could be missed due to the implicit non-negativity assumption in calcium imaging data which exists in many deconvolution algorithms, including Cascade.
Nonetheless, we would like to emphasize that Cascade was used to deconvolve calcium signals for all distributions of data and therefore all inferred spike trains experienced the same bias.
Another notable constraint in our method is the fundamental one-to-one mapping limitation in the CycleGAN framework.
The generators learn a deterministic mapping between the two domains and only associate each input with a single output.
However, most cross-domain relationships consist of one-to-many or manyto-many mappings.
More recently proposed methods, such as Augmented CycleGAN
(Almahairi et al., 2018), aim to address such fundamental limitations by introducing auxiliary noise to the two distributions, and are thus able to generate outputs with variations.
Nevertheless, these methods are most effective when trained in a semi-supervised manner which is not possible with our unpaired neural activity.
All in all, as deep unsupervised methods have become more expressive and explainable, and neuronal activities in different learning phases from behaving animals have become more readily available, there is potential for novel insights into fundamental learning mechanisms.
Future directions include sorting neurons in 2D space, as they were recorded, such that the model can take advantage of both vertical and horizontal spatial information.
A PPENDIX A A NIMAL EXPERIMENT
[Table]
[Table]
[Table]
order of the 102 recorded neurons. i.e. neuron #1 here would be at index 0 in the data matrix, and neuron #65 would be at index 64. Neurons followed the same order across all experiments.
[Figure]
C M ETHODS
[Figure]
GAN
LSGAN
WGANGP
DRAGAN
[Figure]
[Table] [Table]
Note that the L G listed in the table are the generator loss and total generator loss remains
[Equation]
c y c l e - c o n s i s t
e n t l o ss
Autoencoder
Preprocessing segementation order neurons augmentation normalization
Generator G
Generator F
Discriminator D
[Figure]
D N ETWORKS ARCHITECTURE
The generator architecture used in this work, shown in Figure D.1, is based on the ResNet-like (He et al., 2016a) generator in CycleGAN with a number of modiﬁcations.
Generally, the model consists of 2 down-sampling blocks ( DS 1 and DS 2 ), followed by 9 residual blocks ( RB i for 1 ≤ i ≤ 9 ), then 2 up-sampling blocks ( US 1 and US 2 ).
Each down-sampling block uses a 2D strided convolution layer to reduce the spatiotemporal dimensions by factor of 2, which is then follows by Instance Normalization, LReLU activation and Spatial Dropout.
Each up-sampling block has the same structure as the down-sampling blocks but with a transposed convolution layer instead.
Each residual block consists of two convolution blocks with padding added to offset the dimensionality reduction and a skip connection that connect the input to the block with the output of the last convolution block via element-wise addition.
A convolution layer with a ﬁlter size of 1 then compresses the channel of the output from US 1 , followed by a sigmoid activation to scale the ﬁnal output to have range [0 , 1] .
Residual connections are known to improve gradient ﬂow in CNN, thus mitigating the issue of vanishing gradients and allowing deeper networks to be trained effectively (He et al., 2016a;b;
Huang et al., 2017).
Therefore, shortcut connections are added between the down-sampling and up-sampling blocks of the same level.
For instance, the output of down-sampling block DS 2 is concatenated with the output of residual block RB 9 , then passes the resultant vector to the next upsampling block US 1 , such level-wise residual connection was ﬁrst introduced in Ronneberger et al.
We denote the level-wise residual connected network as ResNet .
Furthermore, we adapted the Additive Attention Gate (AG) module in Oktay et al.
(2018) as a replacement for the concatenation operation in the residual connection described above.
The yellow block in Figure D.1 illustrates the AG structure.
AG takes two inputs q and a , both with height H AG and width W AG but varying channels, where q is the output of the previous processing block and a is a shortcut connection from the down-sampling block of the same level.
In AG 1 for instance, q and a are the output of RB 9 and DS 2 respectively.
Both q and a are processed by two separate 1 ×convolution layers followed by Instance Normalization.
The two vectors are then summed elementwise such that overlapping regions from the two vectors would have higher intensity.
We then apply
ReLU activation to eliminate negative values, followed by a 1 × 1 convolution layer with 1 ﬁlter and Instance Normalization, resulting in a vector with shape ( H AG , W AG , 1) .
Sigmoid activation is applied to obtain a [0 , 1] attention mask σ , where units closer to 1 indicate regions that are more relevant.
We apply the sigmoid mask to a , and concatenate it with q .
Since q is a set of high-level features processed by the stack of residual blocks, whereas a is the low-dimensional representation of the original input.
Therefore, the sigmoid attention mask should learn to eliminate information in the input that is less relevant to the output.
Moreover, as the attention mask is of the same dimension of the input q , we can later superimpose the attention mask onto q to visualize the region of interest learned by the model.
We denote the attention-gated ResNet as AGResNet .
We use a PatchGAN-based (Isola et al., 2017) discriminator architecture in this work, as it provides more ﬁne-grained discrimination information to the generators instead of the single value discrimination in the discriminator in vanilla GAN.
D X and D Y contain 3 down-sampling blocks where each block reduces the spatiotemporal dimension by a factor of 2, like the down-sampling blocks in the generators.
For an input sample with shape ( H = 2048 , W = 102 , C = 1) , the discriminator outputs a sigmoid activated vector with shape (256 , 13 , 1) .
Each element has range [0 , 1] where a value closer to 1 suggests that the corresponding patch is a real sample.
Padding
CONV Block ×
Input
CONV Block DS
CONV
InstanceNorm
LReLU 2D Dropout
CONV Block DS
Residual Block RB i
Attention Gate AG
CONV Block +
Padding
CONV Block
Padding
CONV Block US
Attention Gate AG
CONV Block CONV Block
Padding
Sigmoid
Output q +
InstanceNorm 1×
CONV
I n s t a n c e
N o r m 1 ×
C O
N V a
InstanceNorm
[Equation]
ReLU
Sigmoid ×
Concat
E N EURON ORDERING
11121416 1719  23 24       34  4042  495052   57          6970      82            96   101(a)
Original                                36    42    48   53    59                     82      90   95      (b) Firing rate      7                                    45 48          60           74     81  85     9294    100 (
[Figure]
Autoencoder
[Figure]
[Figure]
[Figure]
[Table]
F R ESULTS IN SYNTHETIC DATA
x
G(x) F(G(x))
∆ F / F    85 -0.
Time (s)
N e u r o n  (a) Forward cycle:
X → Y → X -0.
y
F(y)
G(F(y))
∆ F / F    85 -0.
Time (s)
N e u r o n
[Figure]
N e u r o n
Time
[Figure]
)     x
G ( x ) F ( G ( x )) (
[Figure] [Figure] cycle:
X → Y → X
N e u r o n
[Figure]
[Figure]
G R ESULTS IN RECORDED DATA
x
G(x) F(G(x))
∆ F / F    85 -0.
Time (s)
N e u r o n  (a) Forward cycle:
X → Y → X -0.
y
F(y)
G(F(y))
∆ F / F    85 -0.
Time (s)
N e u r o n
[Figure]
Note that, unlike the synthetic dataset, the traces presented here are not unpaired.
Hence, we cannot directly compare x with F ( y ) nor y with G ( x ) .
[Table]
[Equation]
[Table] [Equation] [Table] [Table]
N e u r o n
Time
[Figure]
)     x
G ( x )
F ( G ( x )) (
[Figure]
cycle:
[Figure]
→ Y → X
N e u r o n
[Figure]
[Figure] [Figure]
F ( y ) .
The learned sigmoid masks shown here did not exhibit strong patterns as compare to Figure 5 and Figure G.3. [Figure] [Figure] [Figure]
H S PIKE ANALYSIS
[Figure]
77 152 177 150 31 168 11 132 125 136 40 57 x = F(y) 10413710929146101x
Neuron      0.
1.1 KL divergence
C oun t van-Rossum distance KL
[Figure]
18735x
Neuron      104 22 134 157 45 148 127 74 121 94 101 65 x = F(G(x))
10413445127121101x
Neuron      0.
0.2 KL divergence
C oun t van-Rossum distance KL
[Figure]
60 93 91 127 25 122 55 31 177 116 y = G(x) 1333715484188145y
Neuron      0.
0.8 KL divergence
C oun t van-Rossum distance KL
[Figure]
78y
Neuron      3 101 66 125 194 112 84 131 188 78 145 40 y = G(F(y)) 1333715484188145y
Neuron      0.
0.4 KL divergence
C oun t van-Rossum distance KL
I M OUSE 2 RECORDED DATA
[Table]
[Table]
J M OUSE 3 RECORDED DATA
[Table]
[Table]
K M OUSE 4 RECORDED DATA
[Table] [Table]