Year: 2021

Abstract:
Understanding how activity in neural circuits reshapes following task learning could reveal fundamental mechanisms of learning. Thanks to the recent advances in neural imaging technologies, high-quality recordings can be obtained from hundreds of neurons over multiple days or even weeks. However, the complexity and dimensionality of population responses pose significant challenges for analysis. Existing methods of studying neuronal adaptation and learning often impose strong assumptions on the data or model, resulting in biased descriptions that do not generalize. In this work, we use a variant of deep generative models called – cycle-consistent adversarial networks, to learn the unknown mapping between preand post-learning neuronal activities recorded in vivo. To do so, we develop an end-to-end pipeline to preprocess, train and evaluate calcium fluorescence signals, and a procedure to interpret the resulting deep learning models. To assess the validity of our method, we first test our framework on a synthetic dataset with known ground-truth transformation. Subsequently, we applied our method to neuronal activities recorded from the primary visual cortex of behaving mice, where the mice transition from novice to expert-level performance in a visual-based virtual reality experiment. We evaluate model performance on generated calcium imaging signals and their inferred spike trains. To maximize performance, we derive a novel approach to pre-sort neurons such that convolutional-based networks can take advantage of the spatial information that exists in neuronal activities. In addition, we incorporate visual explanation methods to improve the interpretability of our work and gain insights into the learning process as manifested in the cellular activities. Together, our results demonstrate that analyzing neuronal learning processes with data-driven deep unsupervised methods holds the potential to unravel changes in an unbiased way.

text: Understanding how activity in neural circuits reshapes following task learning could reveal fundamental mechanisms of learning. Thanks to the recent advances in neural imaging technologies, high-quality recordings can be obtained from hundreds of neurons over multiple days or even weeks. However, the complexity and dimensionality of population responses pose significant challenges for analysis. Existing methods of studying neuronal adaptation and learning often impose strong assumptions on the data or model, resulting in biased descriptions that do not generalize. In this work, we use a variant of deep generative models called – cycle-consistent adversarial networks, to learn the unknown mapping between pre- and post-learning neuronal activities recorded in vivo. To do so, we develop an end-to-end pipeline to preprocess, train and evaluate calcium fluorescence signals, and a procedure to interpret the resulting deep learning models. To assess the validity of our method, we first test our framework on a synthetic dataset with known ground-truth transformation. Subsequently, we applied our method to neuronal activities recorded from the primary visual cortex of behaving mice, where the mice transition from novice to expert-level performance in a visual-based virtual reality experiment. We evaluate model performance on generated calcium imaging signals and their inferred spike trains. To maximize performance, we derive a novel approach to pre-sort neurons such that convolutional-based networks can take advantage of the spatial information that exists in neuronal activities. In addition, we incorporate visual explanation methods to improve the interpretability of our work and gain insights into the learning process as manifested in the cellular activities. Together, our results demonstrate that analyzing neuronal learning processes with data-driven deep unsupervised methods holds the potential to unravel changes in an unbiased way.

heading: 1 INTRODUCTION
text: One of the main objectives in computational neuroscience is to study the dynamics of neural processing and how neural activity reshapes in the course of learning. A major hurdle was the difficulty in obtaining high-quality neural recordings of the same set of neurons across multiple experiments, though such limitation in recording techniques has seen tremendous improvements in recent years. With the advent of modern neural imaging technologies, it is now possible to monitor a large population of neurons over days or even weeks (Williams et al., 2018a; Steinmetz et al., 2021), thus allowing experimentalists to obtain in vivo recordings from the same set of neurons across different learning stages. Significant efforts have been put into extracting interpretable and unbiased descriptions of how cortical responses change with experience. Proposed approaches to model changes in neuronal activity include linear latent variable models such as PCA, TCA, GPFA, GPFADS and PSID (Cunningham & Byron, 2014; Williams et al., 2018b; Sani et al., 2021; Yu et al., 2009; Rutten et al., 2020). Methods employing deep learning models but with linear changes or mapping include LFADS and PfLDS (Pandarinath et al., 2018; Gao et al., 2016). While these methods enabled substantial progress in understanding the structure of neuronal activity, they do have strong assumptions inherent in the modelling technique or the analysis, such as the linearity assumption in linear latent variable models. Therefore, making sense of the unknown mapping between pre- and post-learning neural activity in an unbiased manner remains a significant challenge, and a data-driven method to interpret the circuit dynamics in learning is highly desirable.
Thanks to their ability to self-identity and self-learn features from complex data, deep neural networks (DNNs) have seen tremendous success in many biomedical applications (Cao et al., 2018; Zemouri et al., 2019; Piccialli et al., 2021). Specifically, deep generative networks have shown promising results in analyzing and synthesizing neuronal activities in recent years. Pandarinath et al. (2018) developed a variational autoencoder (VAE) to learn latent dynamics from single-trial spiking activities and Prince et al. (2020) extended the framework to work with calcium imaging data. Numerous work have demonstrated generative adversarial networks (GAN) are capable of synthesizing neuronal activities that capture the low-level statistics of recordings obtained from behaving animals (Molano-Mazon et al., 2018; Ramesh et al., 2019; Li et al., 2020).
In this work, we explore the use of cycle-consistent adversarial networks (Zhu et al., 2017), or CycleGAN, to learn the mapping between pre- and post-learning neuronal activities in an unsupervised and data-driven manner. In other words, given the neural recordings of a novice animal, can we translate the neuronal activities that correspond to the animal with expert-level performance, and vice versa? The resulting transformation summarizes these changes in response characteristics in a compact form and is obtained in a fully data-driven way. Such a transformation can be useful in follow-up studies to 1) identify neurons that are particularly important for describing the changes in the overall response statistics, not limited to first or second order statistics; 2) detect response patterns relevant for changes from pre- to post-learning; and 3) determine what experimental details are of particular interest for learning.
To learn the transformation, we derive a standardized procedure to train, evaluate and interpret the CycleGAN framework. To improve the explainability of our work, we incorporate a self-attention mechanism into our generator models and also employ a feature-importance visualization method into our pipeline so that we can visualize and identify the input that the networks deemed relevant in their decision making process. In addition, we introduced a novel neuron ordering method to improve the learning performance of convolutional neural networks (CNN). To quantify the capability of the proposed unsupervised learning method, we evaluate our method on two datasets: 1) an artificially constructed dataset with a handcrafted transformation, and 2) recordings obtained from the primary visual cortex of a behaving animal across multiple days. We then compare several metrics and statistics between the recorded and translated calcium traces and their inferred spike trains.

heading: 2 METHODS
text: 2.1 ANIMAL EXPERIMENT To obtain neuronal activities that can demonstrate pre- and postlearning responses, we conducted a visual-based experiment which follows a similar procedure as Pakan et al. (2018) and Henschke et al. (2020). Briefly, a head-fixed mouse was placed on a linear treadmill that allows it to move forward and backward. A lick spout and two monitors were placed in front of the treadmill and a virtual corridor with defined grating pattern was shown to the mouse. A reward (water drop) would be made available if the mouse licked within the predefined reward location (at 120-140 cm), in which a black screen is displayed as a visual clue. Figure A.1 illustrates the experiment setup. The mouse should learn to utilize both visual information and self-motion feedback to maximize reward. The same set of neurons in the primary visual cortex were labelled with GCaMP6 calcium indicator and monitored throughout 4 days of experiment, the relative changes in fluorescence (∆F/F0) over time were used as a proxy for an action potential. 4 mice were used in the virtual-corridor experiment and all mice transitioned from novice to expert in the behaviour task within 4 days of training. Mouse 1 took on average 6.94s per trial on day 1 and 4.43s per trial on day 4, Table A.1 and A.2 shows the trial information of all the mice. Hence, this dataset can provide excellent insights into how cortical responses reshape with experience.
2.2 CYCLEGAN CycleGAN (Zhu et al., 2017) is a GAN-based unsupervised framework that learns the mapping between two unpaired distributions X and Y via the adversarial training and cycle-consistency optimization. The framework has shown excellent results in a number of unsupervised translation tasks, including natural language translation (Gomez et al., 2018) and molecular optimization (Maziarka et al., 2020), to name a few.
Let X and Y be two distributions with unknown mappings that correspond to (novice) pre- and (expert) post-learning neuronal activity, respectively. CycleGAN consists of four DNNs: generator G : X → Y that maps novice activities to expert activities and generator F : Y → X that maps expert activities to novice activities; discriminator DX : X → [0, 1] and discriminator DY : Y → [0, 1] that learn to distinguish novice and expert neural activities, respectively. In a forward
cycle step (X → Y → X , illustrated in Figure B.1), we first sample a novice recording x from distribution X and apply transformation G to obtain ŷ = G(x). We expect ŷ to resembles data from the expert distribution Y , hence DY learns to minimize (1) LDY = −Ey∼Y [(DY (y)− 1)2] + Ex∼X [DY (G(x))2]. Similar to a typical GAN, generator G learns to deceive DY with the objective of (2) LG = −Ex∼X [(DY (G(x))−1)2]. Note that these are same objectives in LSGAN (Mao et al., 2017). However, DY can only verify if ŷ ∈ Y , though cannot ensure that ŷ is the corresponding expert activity of the novice recording x. Moreover,X and Y are not paired hence we cannot directly compare ŷ with samples in Y . To tackle this issue, CycleGAN applies another transformation to reconstruct the novice recording x̄ = F (ŷ) where the distance ‖ x − x̄ ‖ or ‖ x − F (G(x)) ‖ should be minimal. Therefore, the generators also optimize this cycle-consistent loss (3) Lcycle = Ex∼X [‖ x − F (G(x)) ‖] + Ey∼Y [‖ y − G(F (y)) ‖]. Mean absolute error (MAE) was used as the distance function, though other distance functions can also be employed. In addition, we would expect x̂ = F (x) and ŷ = G(y) to be in distributions X and Y given that F : Y → X and G : X → Y , hence the identity loss objective (4) LGidentity = Ey∼Y [‖ y −G(y) ‖].
Taken all together, G optimizes the following objectives: (5) LGtotal = LG + λcycleLcycle + λidentityLGidentity where λidentity and λcycle are hyper-parameters for identity and cycle loss coefficients. All four networks are trained jointly where LFtotal and LDX are similar to LGtotal and LDY though in opposite directions. In this work, we adapt the CycleGAN framework to learn the unknown mapping between pre- and post-learning neuronal activities recorded from the primary visual cortex of behaving mice. In addition, we experiment with different GANs objective formulations on top of the original LSGAN objective, including GAN (Goodfellow et al., 2014), WGANGP (Arjovsky et al., 2017) and DRAGAN (Kodali et al., 2017). Table C.2 shows their exact formulations in CycleGAN.
2.3 MODEL PIPELINE We devise a consistent analysis framework, including data preprocessing and augmentation, networks interpretation, and evaluation of the generated calcium fluorescence signals and their inferred spike trains. Figure C.1 illustrates the complete pipeline of our work.1
We denote the day 1 (pre-learning) and day 4 (post-learning) recording distributions to be X and Y . With Mouse 1, W = 102 neurons from the primary visual cortex were monitored, as well as trial information such as the virtual distance, licks and rewards. In total, 21471 and 21556 samples were recorded on day 1 and 4. Since we want the generators and discriminators to identify patterns relevant to the animal experiment in a data-driven manner, we do not incorporate any trial information into the training data. We first segment the two datasets with a sliding window of size H = 2048 along the temporal dimension (around 85 s in wall-time), resulting in data with shape (N,H,W ) for X and Y where N is the total number of segments. We select a stride size that space out each segment evenly so that we obtained a sufficient number of samples while keeping the correlations between samples reasonably low. In order to take advantage of the spatiotemporal information in the neuronal activities in a 2D CNN, we further convert the two sets to have shape (N,H,W,C) where C = 1. Finally, we normalize each set to the range [0, 1], and divide them into train, validation and test set with 3000, 200 and 200 samples respectively.
To evaluate the transformation results of G and F , we can compare the cycle-consistency MAE(X,F (G(X))) and MAE(Y,G(F (Y ))), as well as the identity losses MAE(X,F (X)) and MAE(Y,G(Y )) (e.g. we expect F to apply no transformation to a novice sample x). We also evaluate the generated data in terms of spike activities in the following distribution combinations: novice against translated novice (X | F (Y )), novice against reconstructed novice (X | F (G(X))), expert against translated expert (Y | G(X)) and expert against reconstructed expert (Y | G(F (Y ))). We use Cascade (Rupprecht et al., 2021) to infer spike trains from the recorded and generated calcium signals to assess the credibility of the generated signals. We measure the following commonly used spike train similarities and statistics: 1) mean firing rate for evaluating single neuron statistics; 2) pairwise Pearson correlation for evaluating pairwise statistics; 3) pairwise van Rossum distance (Rossum, 2001) for evaluating general spike train similarity. We evaluate these quantities across the whole population for each neuron or neuron pairs and compare the resulting distributions over these quantities obtained from the recorded and generated data. We, therefore, validate the whole spatiotemporal first and second-order statistics as well as general spike train similarities.
To improve the explainability of this work we introduce a number of recently proposed model interpretation methods into our pipeline. We design a self-attention generator architecture which allows
1The software codebase will be made publicly available upon acceptance.
the network to learn a set of attention masks such that it encourages the network to better focus on specific areas of interest in the input and also enables us to visually inspect the learned attention maps. In addition, we use GradCAM (Selvaraju et al., 2017), a method to visualize discriminative region(s) learned by a CNN classifier w.r.t to the input, to extract localization maps from the generators and discriminators. The self-attention mechanism and GradCAM visualization allow us to verify and interpret that the networks are learning meaningful features. Moreover, these extracted attention maps can reveal neurons or activity patterns that are informative in the neuronal learning process. A detail description of the model architectures are available in Section D.
2.3.1 NEURON ORDERING CNNs with a smaller kernel can often perform as well or even better than models with larger kernels while consisting of fewer trainable parameters (He et al., 2016a; Li et al., 2021). Nevertheless, a smaller kernel can also limit the receptive field of the model, or the region in the input that the model is exposed to in each convolution step (Araujo et al., 2019). In addition, the recordings obtained from the virtual-corridor experiment were annotated based on how visible the neurons were in the calcium image, rather than ordered in a particular manner (see Figure A.1). This could potentially restrict CNNs with small receptive field to learn meaningful spatial-temporal information from the population responses. To mitigate this issue, we derive a procedure to pre-sort X and Y , such that neurons that are highly correlated or relevant are nearby in their ordering. A naive approach is to sort the neurons by their firing rate or average pairwise correlation, where the neuron with the highest firing rate or the neuron that, on average, is most correlated to other neurons is ranked first in the data matrix. However, it is possible that not all highfiring neurons or most correlated neurons are the most influential in the learning process. Therefore, we also explore a data-driven approach. Deep autoencoders have shown excellent results in feature extraction and representation learning (Gondara, 2016; Wang et al., 2016; Tschannen et al., 2018), and we can take advantage of its unsupervised feature learning ability.
We employ a deep autoencoder AE which learns to reconstruct calcium signals in X and Y jointly. AE consists of 3 convolution down-sampling blocks, followed by a bottleneck layer, then 3 transposed-convolution up-sampling blocks. The down-sampling block consists of a convolution layer followed by Instance Normalization (Ulyanov et al., 2016), Leaky ReLU (LReLU) activation (Maas et al., 2013) and Spatial Dropout (Tompson et al., 2015), whereas a transpose convolution is used in the up-sampling block instead. We optimize the mean-squared error (MSE) reconstruction loss on the training set of X and Y , then we use the per-neuron reconstruction error on the test set to sort the neurons (in ascending order): order = argsort(0.5×[ MSE(X,AE(X))+MSE(Y,AE(Y )) ]). The neuron sorting process is part of the data preprocessing step and is independent from the CycleGAN framework.
2.3.2 SYNTHETIC DATA CycleGAN was originally introduced for image-to-image translation. Albeit the two image distributions are not aligned hence cannot be directly compared easily, one could still visually inspect whether or not x̂ = F (y) and ŷ = G(x) are reasonable transformations. However, it would be difficult to visually inspect the two transformations with calcium signals. To this end, we introduce an additional dataset Y = Φ(X) with a known transformation Φ, such that G : X → Y = Φ(X) and F : Y = Φ(X) → X . We can then verify G(x) = ŷ = Φ(x) and F (y) = x̂ = x. We defined the spatiotemporal transformation Φ that can be identified visually as follows: (6) Φ(x) = mdiagonalx + 0.5η, where mdiagonal is a diagonal mask to zero-out the lower left corners of the signals and η ∼ N (µx, σ2x). µx and σx are the per-neuron mean and standard deviation ofX . Figure 1 shows an augmented example. Importantly, we shuffle the train set after the augmentation procedure so that X and Y appears to be unpaired to the model. Whereas the test set remains in its original paired arrangement so that we can compare ‖ X−F (Y ) ‖ and ‖ Y −G(X) ‖.

heading: 3 RESULTS
text: We assessed the CycleGAN framework on synthetic data with known ground truth and on experimental data where we recovered trial information. We also experimented with different GAN objective formulations as well as different neuron ordering methods. All models presented below were trained with the Adam optimizer (Kingma & Ba, 2014) for 200 epochs where all models converged. We trained all CycleGAN models on a single NVIDIA A100 GPU which on average took 15 hours to complete. It took an additional hour to train the autoencoder in the case where we pre-sort neurons according to the AE reconstruction loss. Table C.1 details the hyper-parameters used.
3.1 SYNTHETIC DATA To show that our method is capable of learning subtle differences in calcium traces, we first fit our model on the synthetic dataset. Figure 2 shows calcium signals of the forward and backward cycle transformation of neuron 75 from a randomly selected test segment, where AGResNet generators were trained with LSGAN objectives (more examples in Figure F.1). Without paired samples, F (y) made a reasonable attempt in reconstructing the augmented region in y, whereas G(x) was able to learn to mask out the appropriate regions in x. Since Φ(X) = Y performed a systematic spatiotemporal transformation to X , one would expect the networks to learn features that focus on the augmented region of the data. We, therefore, use GradCAM (Selvaraju et al., 2017) localization maps to visualize regions of interest learned by the discriminators. The localization map of discriminator DY (Y ) when given an augmented sample y = Φ(x), shown in Figure 3, demonstrates a high level of attention around the edge of the diagonal region. This indicates that DY learned to distinguish whether or not a given sample is from distribution Y = Φ(X) by predominantly focusing on the edge of the masking area. On the other hand, since no augmentation was done on the input to discriminator DX , the localization map does not appear to have a particular structural area of focus at first (c.f. Figure F.3). Interestingly, once we overlay the reward zones on the input, we observe that the area of focus learned by DX is loosely aligned with the reward zones. Note that reward zones are external task-relevant regions that are expected to shape the neural activity in the primary visual cortex as the visual patterns change when the mouse enters the reward zone. Our findings therefore suggest that DX learned distinctive patterns from highly ranked neurons around the reward zones. Figure 3 shows the AG sigmoid masks from G(x). Both attention masks ignored the augmentation region (i.e. bottom left corner), as information in that area is not relevant in the G : X → Φ(X) transformation. Similar, F which should learn Φ(X) → X also allocated less focus in the masked region in its reconstruction process, as it contains no useful information. (see Figure F.3).
Since X and Y = Φ(X) are paired in the test set, this allows us to compute MAE(Y,G(X)) and MAE(X,F (Y )) hence providing a good testbed to compare different generator architectures, GAN objective formulations and neuron ordering methods. We also added the identity models as baseline, which should have perfect cycle-consistent loss as F (G(x)) and G(F (y)) perform no operation on the data. Nevertheless, despite the fact that Φ is a relatively simple augmentation, one would expect the difference between X and Φ(X) to be small. Table 1 shows the direct comparison results of different combinations of objective formulations, generator architectures and neuron ordering methods. Both ResNet and AGResNet achieved significantly better results than the identity model. To mitigate the issues of vanishing gradient and mode collapse, we used gradient penalty regularization to enforce the 1-Lipschitz condition in the discriminator. We, therefore, tested 4 popular GAN objectives with the CycleGAN framework. Interestingly, the LSGAN objectives achieved slightly better results than GAN objectives while both performed better than identity. The two objectives with gradient penalty obtained lower cycle-consistent errors than GAN and LSGAN, yet performed significantly worse in the intermediate transformations F (Y ) and G(X). This suggests that the discriminators could be overpowered by the generators when trained with WGANGP and DRAGAN, in which DX(F (Y )) and DY (G(X)) are neither informative nor impactful to the overall objective. This is likely because the gradient penalty regularization further complicates the already perplexing CycleGAN objectives. We employed 3 different methods to pre-sort neurons in the data, including firing rate, pairwise correlation and autoencoder reconstruction loss. In addition, to demonstrate that 2D convolution can indeed better learn the spatial structure in neuronal activities, we trained a 1D variant of AGResNet (denoted as 1D-AGResNet) as baseline which disregards all spatial information. Overall, models trained on sorted neurons achieved better results compared to unordered neurons and in most cases, sorting neurons according to the autoencoder reconstruction loss performed the best. Moreover, 1D-AGResNet performed significantly worse than its 2D counterparts, suggesting that the spatial structure in the neural activities is indeed important. In the
remaining work, we use the LSGAN objective to train the generators with the AGResNet architecture along with neurons ordered based on autoencoder reconstruction loss as this combination achieved the best overall results on the synthetic data.
3.2 RECORDED DATA As our proposed method has successfully learned the unpaired transformations in the synthetic dataset, we now move on to the recordings obtained from the virtual-corridor experiment where we attempt to learn the unknown mapping between pre- and post-learning neuronal activity. Figure 4 shows the cycle transformation of neuron 50 from a randomly selected test segment. Visually, G and F seems to be able to reconstruct x̄ = F (G(x)) and ȳ = G(F (y)), and that the two generators are not simply passing through x and y in intermediate step ŷ = G(x) and x̂ = F (y). To better analyse the transformation performance, we first compare the generated calcium florescence signals with the recorded test set data. The cycle-consistent loss on the test set achieved a values of MAE(X,F (G(X))) = 0.0733 and MAE(Y,G(F (Y ))) = 0.0737. The identity losses for MAE(Y,G(Y )) and MAE(Y,G(Y )) are also minimal, with values of 0.0101 and 0.0069, respectively. For reference, MAE(X,Y ) = 0.3674. This suggest G and F are not simply passing through the data without any processing. In addition, the low identity loss indicates that the generators can correctly identify whether or not the given input is already part of its target distribution. Table G.1 reports the cycle-consistent and identity loss with different neuron ordering methods.
Since we lack paired data in the in vivo recordings, we cannot directly compare MAE(X,F (Y )) nor MAE(Y, F (X)), in contrast to Section 3.1. In order to better analyse the two intermediate transformations ŷ = G(x) and x̂ = F (y), and show that G and F can indeed translate x and y into their respective distributions ŷ ∼ Y and x̂ ∼ X , we also compare a set of spike train statistics. Section H
shows that per-neuron and per-segment comparison. We first compare the firing rate distribution of each neuron between recorded and translated data (e.g. X vs F (Y ) and X vs F (G(X))). Examples of the distribution comparisons are available in Section H. Since we expect that the distribution of the generated data resemble of those from the recorded data, we can compare the KL divergence for each neuron to quantify the transformation performance. The firing rate distributions of F (Y ) and G(X) closely matched the distributions of X and Y , with average KL divergence of 1.1648 and 1.0697, respectively. Similarly, we can compute the pairwise correlation of each neuron w.r.t the population and compare the distribution between translated and recorded data. X | F (Y ) and Y |G(X) achieved an average KL divergence value of 0.0479 and 0.0493 in the pairwise correlation comparison, both were significantly better results than the baseline identity model. In addition, we measure the van Rossum distance between X and F (Y ) for each neuron across 200 test samples, and represent the results in the form of a heatmap. We can observe a clear diagonal line of lowintensity values in the heatmaps for most neurons (e.g. Figure H.3 and H.4 for G and F ). Hence, there exists a spike train in X and Y that corresponds to a translated spike train in F (Y ) and G(X). Table G.2 summaries the average KL divergence of the 3 spike statistics in different distribution combinations, the results indicate that the generators can indeed learn the distribution translation from pre- to post-learning neuronal activities, and vice-versa. We additionally trained separate models on the activities recorded from the other mice and obtained similar results, which are available in Section I, J and K.
In the previous section, we were able to identify and interpret the learned features in a relatively straightforward manner due to the systematic augmentation we introduced into the data. However, visualizing and interpreting the attention maps on pre- and post-learning data could be more challenging as there would not be obvious patterns in the inputs to anticipate. Nevertheless, we would expect a higher level of activities in the V1 neurons when the mouse is about to enter or inside the reward zone, where the grating pattern on the virtual walls turn to black. Subsequently, the generators and discriminators should learn meaningful features from responses surrounding the reward zones. We first visualize the sigmoid masks in AGResNet. Figure 5 shows the learned attention masks of G superimposed on the latent inputs (see Figure G.3 for F ). When the neurons were ordered, either by firing rate or autoencoder, we observe that the generators allocate more attention toward neurons that rank higher. This suggest that by grouping neurons in a meaningful manner, the convolutional layers in the generators can extract relevant features more effectively as compared to when neurons were randomly ordered. The spike analysis showed that ordering neurons in a structured manner does indeed yield better results across the board. In most cases, ordering the neurons based on the reconstruction error achieved the best results.
We then inspect the GradCAM localization maps of the discriminators. Similar to DX in the synthetic dataset, we observed regions of high attention surrounding the reward zones in both DX and DY (see Figure G.3 and 5). To better visualize the relationship between the area of focus learned by the model and the virtual-corridor, we generate positional attention maps as shown in Figure 6. We first compute GradCAM maps for all test samples, then we average the activation value for each neuron at each virtual position (160 cm in total) and plot the average activation value against distance. Effectively, these maps should represent the average attention learned by the models w.r.t. the visual location of the animal. Importantly, the only objective the discriminators had was to distinguish if a given sample is from a particular distribution. Thus, the discriminators could have learned trivial features. Instead, DX focused on a specific group of neurons at 100 - 130 cm in the virtual environment, which coincides with the beginning of the reward zone. Moreover, DY learned to focus on two groups of neurons with attention patterns that were also in alignment with the reward zone in the virtual-corridor experiment. Similarly, we can extract these positional attention maps for G(X) and F (Y ) following the same procedure, where we monitor the change in gradient in the last residual block RB9 (bottom row in Figure 6). Interestingly, both generators focused on the first few neurons in their transformation operations. G focused on activities at the beginning of the trial as well as activities in the reward zone; whereas with F , it paid higher level of attention to activities right before the reward zone. This suggests that to learn the transformation from post- to pre-learning responses, the activities the mouse exhibit as it approaches the reward zone is deemed more important by the networks. Note that no trial information was incorporated into the training data nor was it formulated in the objective function. Hence, these interesting patterns we observe here were learned entirely by the networks themselves via the adversarial process.

heading: 4 DISCUSSION
text: We demonstrated that the CycleGAN (Zhu et al., 2017) framework is a capable data-driven method to model the translation between pre- and post-learning responses recorded in vivo. With selfattention and feature-importance visualization methods, we are able to visualize information that the networks deemed important in their translation and discrimination process. Intriguingly, without providing trial information in the training process, the networks self-identified activities surrounding the reward zone in the virtual-corridor experiment to be highly influential, which aligns with our understanding that the responses in the visual cortex were shaped by the change of visual cues. In addition, we introduced a novel and simple to implement neuron ordering method enabling more effective learning by convolutional-based networks.
A significant portion of the neuronal activity validation in Section 3.2 was performed in spike trains inferred from the recorded and generated calcium fluorescent signals using Cascade (Rupprecht et al., 2021), which is a recently introduced method that has outperformed existing model-based algorithms. However, reliable spike inference from fluorescent calcium indicators signals remains an active area of research (Theis et al., 2016). For instance, Vanwalleghem et al. (2020) demonstrated that spiking activities could be missed due to the implicit non-negativity assumption in calcium imaging data which exists in many deconvolution algorithms, including Cascade. Nonetheless, we would like to emphasize that Cascade was used to deconvolve calcium signals for all distributions of data and therefore all inferred spike trains experienced the same bias. Another notable constraint in our method is the fundamental one-to-one mapping limitation in the CycleGAN framework. The generators learn a deterministic mapping between the two domains and only associate each input with a single output. However, most cross-domain relationships consist of one-to-many or manyto-many mappings. More recently proposed methods, such as Augmented CycleGAN (Almahairi et al., 2018), aim to address such fundamental limitations by introducing auxiliary noise to the two distributions, and are thus able to generate outputs with variations. Nevertheless, these methods are most effective when trained in a semi-supervised manner which is not possible with our unpaired neural activity.
All in all, as deep unsupervised methods have become more expressive and explainable, and neuronal activities in different learning phases from behaving animals have become more readily available, there is potential for novel insights into fundamental learning mechanisms. Future directions include sorting neurons in 2D space, as they were recorded, such that the model can take advantage of both vertical and horizontal spatial information.

heading: A ANIMAL EXPERIMENT
text: B CYCLEGAN

heading: C METHODS

heading: D NETWORKS ARCHITECTURE
text: The generator architecture used in this work, shown in Figure D.1, is based on the ResNet-like (He et al., 2016a) generator in CycleGAN with a number of modifications. Generally, the model consists of 2 down-sampling blocks (DS1 and DS2), followed by 9 residual blocks (RBi for 1 ≤ i ≤ 9), then 2 up-sampling blocks (US1 and US2). Each down-sampling block uses a 2D strided convolution layer to reduce the spatiotemporal dimensions by factor of 2, which is then follows by Instance Normalization, LReLU activation and Spatial Dropout. Each up-sampling block has the same structure as the down-sampling blocks but with a transposed convolution layer instead. Each residual block consists of two convolution blocks with padding added to offset the dimensionality reduction and a skip connection that connect the input to the block with the output of the last convolution block via element-wise addition. A convolution layer with a filter size of 1 then compresses the channel of the output from US1, followed by a sigmoid activation to scale the final output to have range [0, 1].
Residual connections are known to improve gradient flow in CNN, thus mitigating the issue of vanishing gradients and allowing deeper networks to be trained effectively (He et al., 2016a;b; Huang et al., 2017). Therefore, shortcut connections are added between the down-sampling and up-sampling blocks of the same level. For instance, the output of down-sampling block DS2 is concatenated with the output of residual block RB9, then passes the resultant vector to the next upsampling block US1, such level-wise residual connection was first introduced in Ronneberger et al. (2015). We denote the level-wise residual connected network as ResNet.
Furthermore, we adapted the Additive Attention Gate (AG) module in Oktay et al. (2018) as a replacement for the concatenation operation in the residual connection described above. The yellow block in Figure D.1 illustrates the AG structure. AG takes two inputs q and a, both with height HAG and width WAG but varying channels, where q is the output of the previous processing block and a is a shortcut connection from the down-sampling block of the same level. In AG1 for instance, q and a are the output of RB9 and DS2 respectively. Both q and a are processed by two separate 1 × 1 convolution layers followed by Instance Normalization. The two vectors are then summed elementwise such that overlapping regions from the two vectors would have higher intensity. We then apply ReLU activation to eliminate negative values, followed by a 1 × 1 convolution layer with 1 filter and Instance Normalization, resulting in a vector with shape (HAG,WAG, 1). Sigmoid activation is applied to obtain a [0, 1] attention mask σ, where units closer to 1 indicate regions that are more relevant. We apply the sigmoid mask to a, and concatenate it with q. Since q is a set of high-level features processed by the stack of residual blocks, whereas a is the low-dimensional representation of the original input. Therefore, the sigmoid attention mask should learn to eliminate information in the input that is less relevant to the output. Moreover, as the attention mask is of the same dimension of the input q, we can later superimpose the attention mask onto q to visualize the region of interest learned by the model. We denote the attention-gated ResNet as AGResNet.
We use a PatchGAN-based (Isola et al., 2017) discriminator architecture in this work, as it provides more fine-grained discrimination information to the generators instead of the single value discrimination in the discriminator in vanilla GAN. DX and DY contain 3 down-sampling blocks where each block reduces the spatiotemporal dimension by a factor of 2, like the down-sampling blocks in the generators. For an input sample with shape (H = 2048,W = 102, C = 1), the discriminator outputs a sigmoid activated vector with shape (256, 13, 1). Each element has range [0, 1] where a value closer to 1 suggests that the corresponding patch is a real sample.
E NEURON ORDERING
1 2 3
456 78
9
10 11
12
13 14
15
16 17 18
19 20
21
22
2324
25
26
27
28
29
30
31 32 33
34 35 36
37 38
39
4041 42
43
4445
464748
4950
51
5253
54
55
56
57 58
59
60
61
62 63
64
65
66
67
68
69 70
71
72
73
74 75 76 77
78
79
80
81
82 83 84
85
86
87
88
89
90 91
92
93
94
95 96 97
98
99
100
101 102
(a) Original
1
2
3
4
5
6
7
8
9
10
11
12
1314
15
16
17
18
19 20
21
22
23
24 25
26
27
28
29
30
31
32
33 34
35
36 37
38
39
40
41
4243
44
45
46
47
48 49
50 51
52
53 54
55
56
57
58
5960
61
62 63
64
65
66
67 68
69
70
71
72
73
74
75
76
77
78 79
80
81
82 83
84
85
86
87
88
89
90 91
92
93
94
9596
97
98
99
100 101 102
(b) Firing rate
1
2
3
4
5
6
78 9
10
11
12
13
14
15
16
17 18
19
20
21
22
23
24 25
26
27
28 29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45 46
47
4849
50
51
52 53
54
55 56
57
58
59
6061
62
63
64 65
66
67
68
9
70
71
72
73
74
75
76
77
78
79
80
81 82
83
84
85 86
87
88
89
90
91
92 93
94 95
96
97
98
99
100
101
102
(c) Autoencoder
Figure E.1: Neuron ordering based on (a) original annotation, (b) firing rate and (c) autoencoder reconstruction loss. The original order was based on how visible the neuron were in the calcium imaging data, hence not sorted in a particular manner. One naive approach is to sort neurons base on their overall firing rate, such that active neurons can be closer in space thus allow more efficient learning by convolutional-based networks. We proposed to train an autoencoder AE which learns to reconstruct X and Y jointly, and sort neurons base on the average reconstruction error on the test set. See Section 2.3.1 for detail regarding different neuron ordering methods and their motivations.
METHOD ORDER
(A) N/A 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102
(B) FIRING RATE 18, 14, 12, 30, 8, 15, 36, 4, 21, 19, 3, 7, 43, 33, 20, 42, 13, 6, 11, 39, 2, 22, 75, 28, 55, 100, 31, 62, 10, 67, 63, 54, 17, 40, 52, 46, 99, 88, 61, 77, 57, 34, 85, 41, 27, 98, 84, 47, 65, 73, 5, 1, 44, 101, 58, 80, 16, 29, 87, 9, 26, 83, 92, 74, 24, 45, 49, 23, 97, 48, 68, 60, 71, 76, 59, 53, 70, 89, 25, 93, 32, 56, 66, 81, 72, 94, 38, 64, 79, 82, 50, 51, 96, 90, 37, 86, 95, 91, 102, 69, 35, 78
(C) CORRELATION 36, 27, 46, 28, 39, 30, 42, 20, 92, 10, 18, 11, 67, 14, 4, 33, 19, 77, 75, 13, 24, 99, 8, 43, 65, 101, 63, 7, 25, 44, 12, 76, 80, 9, 47, 3, 34, 71, 87, 52, 22, 1, 85, 61, 84, 29, 45, 31, 93, 100, 5, 58, 57, 17, 74, 21, 96, 55, 82, 91, 2, 48, 6, 56, 83, 62, 49, 16, 26, 81, 97, 53, 73, 94, 89, 59, 40, 95, 23, 32, 54, 66, 98, 72, 35, 88, 15, 41, 50, 60, 90, 70, 78, 68, 69, 86, 38, 51, 64, 79, 37, 102
(D) AUTOENCODER 89, 52, 100, 97, 83, 59, 64, 51, 93, 37, 96, 50, 99, 38, 61, 81, 87, 60, 53, 62, 55, 35, 40, 94, 21, 86, 95, 17, 32, 23, 72, 69, 3, 54, 41, 58, 49, 22, 91, 84, 90, 36, 92, 82, 39, 68, 66, 8, 73, 88, 71, 4, 46, 18, 11, 44, 70, 78, 25, 85, 29, 56, 20, 80, 28, 9, 26, 101, 65, 24, 5, 98, 1, 57, 43, 10, 12, 31, 63, 33, 75, 77, 19, 47, 45, 76, 27, 74, 42, 102, 30, 48, 7, 34, 13, 67, 16, 79, 2, 15, 14, 6
Table E.1: Neuron ordering based on (a) original annotation, (b) firing rate, (c) average pairwise correlation and (d) autoencoder reconstruction loss with respect to the original annotation order recorded on Day 4 data. The physical location of each neuron is available in Figure E.1.
F RESULTS IN SYNTHETIC DATA
-0.24
1.56
3.36 x G(x) F(G(x))
-0.15
0.31
0.78
∆ F / F
0 21 42 63 85 -0.29
1.29
2.87
0 21 42 63 85
Time (s)
0 21 42 63 85
25 N
eu ron 50 75
(a) Forward cycle: X → Y → X
-0.25
1.71
3.67 y F(y) G(F(y))
-0.11
0.3
0.7
∆ F /F
0 21 42 63 85 -0.33
1.35
3.04
0 21 42 63 85
Time (s)
0 21 42 63 85
25 N
eu ron 50 75
(b) Backward cycle: Y → X → Y
Figure F.1: (a) forward and (b) backward cycle of neuron 25, 50 and 75 from AGResNet trained with LSGAN objectives. Since X and Y in the test set are paired (see Section 2.3.2), we expect x ≈ F (y) and y ≈ G(x). Notice that neurons with a higher index (e.g. neuron 75 in y) would have more units being masked out and replaced by noise. We can see that generator F was able to reconstruct the masked out regions (e.g. neuron 50 and 75 in F (y)) that resemble the traces in X .
1
26
51
76
102 1
26
51
76
102
N eu
ro n
0 21 42 63 85
Time (s)
1
26
51
76
102
x G (x )
F (G (x ))
(a) Forward cycle: X → Y → X 1
26
51
76
102 1
26
51
76
102
N eu
ro n
0 21 42 63 85
Time (s)
1
26
51
76
102
y F (y )
G (F (y ))
(b) Backward cycle: Y → X → Y
Figure F.2: (a) forward and (b) backward cycle of the entire 102 neurons from a randomly selected test segment. Model was trained with AGResNet using the LSGAN objectives on the synthetic dataset where Y = Φ(X), see Section 2.3.2 for detail.
Figure F.3: (Left) Self-learned attention masks in AG1 and AG2 in AGResNet G given a random test segment y = Φ(x) ∼ X . AG1 and AG2 were less focused in the masked region as it is filled with Gaussian noise and it not informative in the reconstruction process. Note that AG1 and AG2 are at 4 and 2 times lower-dimensional than the original input dimension (see Section D). (Right) GradCAM localization maps of DX given a randomly select test segment x ∼ X . The top panel shows the original input, where yellow and orange dotted lines mark the start and end of each reward zones. The second panel shows the GradCAM localization map superimposed on the input. We observe that DX focused on neuronal activities surrounding the reward zone areas in its discrimination process, which is expected since the activities in the visual cortex are shaped by the visual-clues in the virtual-corridor experiment. Note that trial information such as reward zone locations were not provided to the networks, the pattern observed here was learned by the models themselves.
G RESULTS IN RECORDED DATA
-0.24
0.48
1.21 x G(x) F(G(x))
-0.21
0.49
1.19
∆ F / F
0 21 42 63 85 -0.25
0.54
1.32
0 21 42 63 85
Time (s)
0 21 42 63 85
25 N
eu ron 50 75
(a) Forward cycle: X → Y → X
-0.24
0.47
1.17 y F(y) G(F(y))
-0.28
0.39
1.06
∆ F /F
0 21 42 63 85 -0.25
0.61
1.47
0 21 42 63 85
Time (s)
0 21 42 63 85
25 N
eu ron 50 75
(b) Backward cycle: Y → X → Y
Figure G.1: (a) forward and (b) backward cycle of neuron 6, 27 and 75 from a randomly selected segment. Model was trained with AGResNet using the LSGAN objective on the recorded dataset. Note that, unlike the synthetic dataset, the traces presented here are not unpaired. Hence, we cannot directly compare x with F (y) nor y with G(x).
ORDER |X − F (G(X))| |X − F (X)| |Y −G(F (Y ))| |Y −G(Y )|
1D-AGRESNET 0.1806± 0.0077 0.1502± 0.0064 0.1811± 0.0163 0.1463± 0.0149 ORIGINAL 0.0874± 0.0037 0.0123± 0.0015 0.0766± 0.0025 0.0101± 0.0010
FIRING RATE 0.0760± 0.0030 0.0108± 0.0013 0.0752± 0.0028 0.0070± 0.0005 CORRELATION 0.0778± 0.0028 0.0111± 0.0012 0.0757± 0.0024 0.0089± 0.0022 AUTOENCODER 0.0733± 0.0025 0.0101± 0.0012 0.0737± 0.0027 0.0069± 0.0007
Table G.1: Cycle-consistent and identity loss in the test set of Mouse 1 recordings, where neurons were ordered by 1) original annotation, 2) firing rate 3) pairwise correlation and 4) autoencoder reconstruction loss. We also trained a 1D variant of the model as an additional baseline (1D-AGResNet in the table) such that all spatial information of the neurons is disregarded. The AGResNet generator architecture was used for G and F , and were optimized with LSGAN objectives. The lowest loss in each category is marked in bold. For reference, |X−Y | = 0.3674±0.0236 in the test set.
KL(X,F (Y )) KL(X,F (G(X))) KL(Y,G(X)) KL(Y,G(F (Y )))
(A) PAIRWISE CORRELATION
IDENTITY 0.0875± 0.0549 0 0.0821± 0.0471 0 1D-AGRESNET 0.2027± 0.1040 0.4715± 0.2051 0.1901± 0.1003 0.4149± 0.2194
ORIGINAL 0.0552± 0.0419 0.0754± 0.0353 0.0583± 0.0553 0.0174± 0.0110 FIRING RATE 0.0507± 0.0358 0.0266± 0.0146 0.0504± 0.0438 0.0267± 0.0176 CORRELATION 0.0539± 0.0329 0.0339± 0.0176 0.0534± 0.0474 0.0205± 0.0133 AUTOENCODER 0.0479± 0.0372 0.0329± 0.0163 0.0493± 0.0448 0.0283± 0.0206
(B) FIRING RATE
IDENTITY 8.0705± 6.5500 0 7.7781± 6.7338 0 1D-AGRESNET 3.5688± 3.8895 7.9101± 5.3517 3.0572± 3.1114 8.3185± 5.5950
ORIGINAL 1.5401± 1.2491 2.0442± 2.0936 1.8527± 1.3563 1.4697± 1.1412 FIRING RATE 1.3402± 1.0450 1.2658± 1.0784 1.6994± 1.4170 1.4152± 1.2221 CORRELATION 1.4006± 1.1079 1.5450± 1.0786 1.4088± 1.0828 1.4674± 1.3505 AUTOENCODER 1.1648± 0.7934 1.4022± 1.2734 1.0697± 0.7689 1.2705± 1.1148
(C) PAIRWISE VAN ROSSUM DISTANCE
IDENTITY 0.5510± 0.2960 0 0.3053± 0.1211 0 1D-AGRESNET 0.3613± 0.1597 0.8045± 0.1846 0.3764± 0.1565 1.3897± 0.8256
ORIGINAL 0.2790± 0.2186 0.1878± 0.0477 0.3216± 0.1352 0.1581± 0.0664 FIRING RATE 0.2539± 0.1708 0.1003± 0.0514 0.3080± 0.1173 0.1536± 0.0663 CORRELATION 0.2629± 0.1877 0.1905± 0.0485 0.2953± 0.1230 0.1797± 0.0696 AUTOENCODER 0.2387± 0.1488 0.1041± 0.0376 0.3031± 0.1138 0.1328± 0.0592
Table G.2: The average KL divergence between generated and recorded distributions of Mouse 1 in (a) pairwise correlation, (b) firing rate and (c) pairwise van Rossum distance. We trained AGResNet with neurons ordered according to the following methods: 1) original annotation, 2) firing rate, 3) pairwise correlation and 4) autoencoder reconstruction loss. We also trained a 1D variant of the model (denoted as 1D-AGResNet) such that all spatial information of the neurons is disregarded. Note that we added the identity model (first row of each sub-table) as a baseline where we should obtain perfect cycle reconstruction. Entries with the lowest value are marked in bold.
1
26
51
76
102 1
26
51
76
102
N eu
ro n
0 21 42 63 85
Time (s)
1
26
51
76
102
x G (x )
F (G (x ))
(a) Forward cycle: X → Y → X 1
26
51
76
102 1
26
51
76
102
N eu
ro n
0 21 42 63 85
Time (s)
1
26
51
76
102
y F (y )
G (F (y ))
(b) Backward cycle: Y → X → Y
Figure G.2: (a) forward and (b) backward cycle of the entire 102 neurons from a randomly selected segment. Model was trained with AGResNet generators using the LSGAN objective on the recorded dataset.
Figure G.3: (Left) self-learned sigmoid attention masks AG1 and AG2 in AGResNet F given a random test segment y ∼ Y , where neurons were sorted by the autoencoder AE reconstruction loss. (Right) GradCAM localization map of DX given a randomly select test segment x ∼ X . The top panel shows the original input, where yellow and orange dotted lines mark the start and end of each reward zones. The second panel shows the localization map superimposed on the input. Again, we observe attention patterns that loosely align with the reward zones. Note that trial information such as reward zone locations were not provided to the networks, the pattern observed here was learned by the models themselves.
0 511 1023 1535 2047
1 26 51 76
102
In pu
t
0 127 255 383 511
1 7
13 19 26
AG 1
0 255 511 767 1023
1 13 26 38 51 AG 2
0 511 1023 1535 2047 Time-step
1 26 51 76
102
Ou tp
ut
(a) G(x)
0 511 1023 1535 2047
1 26 51 76
102
In pu
t
0 127 255 383 511
1 7
13 19 26
AG 1
0 255 511 767 1023
1 13 26 38 51 AG 2
0 511 1023 1535 2047 Time-step
1 26 51 76
102
Ou tp
ut
(b) F (y)
Figure G.4: Self-learned sigmoid attention masks AG1 and AG2 in AGResNet (a) G given a random test segment x ∼ X and (b) F for a given random test segment y ∼ Y . Top panels shows the original input, where yellow and orange dotted lines mark the start and end of each reward zones. Bottom panels show the generated outputs of G(x) and F (y). The learned sigmoid masks shown here did not exhibit strong patterns as compare to Figure 5 and Figure G.3.
0.05
0.10
0.1
0.2
0.3
0.05
0.10
0.05
0.10
0.05
0.10
0.05
0.10
0.15
0.05
0.10
0.05
0.10
0.1
0.2
0 21 42 64 85
Time (s)
0.05
0.10
N 6 4
N 5 1
N 9 3
N 3 7
N 9 6
N 5 0
N 9 9
N 3 8
N 6 1
N 8 1
∆ F /F
(a) Pre-learning
0.25
0.50
0.2
0.4
0.05
0.10
0.15
0.05
0.10
0.1
0.2
0.05
0.10
0.05
0.10
0.05
0.10
0.1
0.2
0 21 42 64 85
Time (s)
0.1
0.2
N 6 4
N 5 1
N 9 3
N 3 7
N 9 6
N 5 0
N 9 9
N 3 8
N 6 1
N 8 1
∆ F /F
(b) Post-learning
Figure G.5: The (a) pre-learning and (b) post-learning traces of the top 10 neurons that DX paid the most attention to. The positional attention maps of DX is available in Figure 6 (Top Left). Note that the pre-learning and post-learning activities are not paired.
0.05
0.10
0.05
0.10
0.15
0.05
0.10
0.05
0.10
0.1
0.2
0.05
0.10
0.050
0.075
0.25
0.50
0.75
0.1
0.2
0.3
0 21 42 64 85
Time (s)
0.05
0.10
N 9 6
N 5 0
N 9 9
N 3 8
N 8 6
N 9 5
N 1 7
N 3 2
N 2 3
N 7 2
∆ F /F
(a) Pre-learning
0.1
0.2
0.05
0.10
0.05
0.10
0.05
0.10
0.05
0.10
0.05
0.10
0.1
0.2
0.050
0.075
0.05
0.10
0 21 42 64 85
Time (s)
0.2
0.4
N 9 6
N 5 0
N 9 9
N 3 8
N 8 6
N 9 5
N 1 7
N 3 2
N 2 3
N 7 2
∆ F /F
(b) Post-learning
Figure G.6: The (a) pre-learning and (b) post-learning traces of the top 10 neurons that DY paid the most attention to. The positional attention maps of DY is available in Figure 6 (Top Right). Note that the pre-learning and post-learning activities are not paired.

heading: H SPIKE ANALYSIS
text: -0.4 -0.0 0.3 0.7 1.0 Pair-wise correlation
0
588
1176
1764
2353
C ou
nt Trial 0
x x = F(y)
-0.5 -0.1 0.2 0.6 1.0 Pair-wise correlation
0
747
1495
2243
2991
C ou
nt
Trial 5
x x = F(y)
-0.5 -0.1 0.3 0.6 1.0 Pair-wise correlation
0
744
1488
2232
2976
C ou
nt
Trial 10
x x = F(y)
0.0 0.1 0.2 0.3 0.3 KL divergence
0
9
18
27
37
C ou
nt
Correlation KL
0.1 0.6 1.1 1.7 2.2 Hz
0
7
14
21
28
C ou
nt
Neuron 25
x x = F(y)
0.0 0.2 0.4 0.6 0.8 Hz
0
15
30
45
61
C ou
nt Neuron 50
x x = F(y)
0.1 0.6 1.1 1.6 2.0 Hz
0
5
11
16
22
C ou
nt
Neuron 75
x x = F(y)
0.0 1.0 2.0 3.0 4.0 KL divergence
0
3
7
11
15
C ou
nt
Firing Rate
132 92 17 14 64 177 7 147 102 196 180 98 x = F(y)
153 56 168 188 106 30 148 162 159 150 71 51 x
Neuron 25
8
18
28
38
48
58
1 88 13 40 44 142 51 166 143 199 141 178 x = F(y)
99 63 106 153 112 51 180 29 187 38 35 20 x
Neuron 50
0
6
12
18
24
31
77 152 177 150 31 168 11 132 125 136 40 57 x = F(y)
104 106 137 157 109 85 29 60 146 51 101 65 x
Neuron 75
11
20
30
40
50
60
0.1 0.3 0.6 0.8 1.1 KL divergence
0
9
19
28
38
C ou
nt
van-Rossum distance KL
Figure H.1: Spike statistics of (top) firing rate of 3 randomly selected neurons, (middle) pairwise correlation of 3 randomly selected segments and (bottom) van Rossum distance of 3 randomly selected segments between X and F (Y ) where X was ordered by autoencoder reconstruction loss. The right columns show the KL divergence of each metrics and Table G.2 shows the mean and standard deviation of the KL divergence comparisons.
-0.3 -0.0 0.3 0.7 1.0 Pair-wise correlation
0
853
1706
2559
3412
C ou
nt
Trial 0
x x = F(G(x))
-0.5 -0.1 0.2 0.6 1.0 Pair-wise correlation
0
847
1694
2541
3388
C ou
nt
Trial 5
x x = F(G(x))
-0.3 0.0 0.4 0.7 1.0 Pair-wise correlation
0
834
1668
2502
3337
C ou
nt
Trial 10
x x = F(G(x))
0.0 0.0 0.1 0.1 0.1 KL divergence
0
5
11
17
23
C ou
nt
Correlation KL
0.1 0.6 1.2 1.7 2.3 Hz
0
6
12
18
24
C ou
nt
Neuron 25
x x = F(G(x))
0.0 0.2 0.3 0.5 0.7 Hz
0
15
31
46
62
C ou
nt
Neuron 50
x x = F(G(x))
0.1 0.6 1.0 1.4 1.9 Hz
0
5
11
16
22
C ou
nt
Neuron 75
x x = F(G(x))
0.0 1.4 2.8 4.3 5.7 KL divergence
0
4
9
13
18
C ou
nt
Firing Rate
153 160 168 27 39 192 194 182 145 150 172 51 x = F(G(x))
153 160 168 27 39 192 194 182 145 150 172 51 x
Neuron 25
1
12
24
36
47
59
1 137 169 2 112 51 180 29 187 38 35 20 x = F(G(x))
99 63 106 153 112 51 180 29 187 38 35 20 x
Neuron 50
0
5
11
17
23
29
104 22 134 157 45 148 127 74 121 94 101 65 x = F(G(x))
104 22 134 157 45 148 127 74 121 94 101 65 x
Neuron 75
1
11
22
33
43
54
0.0 0.1 0.1 0.2 0.2 KL divergence
0
4
8
12
17
C ou
nt
van-Rossum distance KL
Figure H.2: Spike statistics of (top) firing rate of 3 randomly selected neurons, (middle) pairwise correlation of 3 randomly selected segments and (bottom) van Rossum distance of 3 randomly selected segments between X and F (G(X)) where X was ordered by autoencoder reconstruction loss. The right columns show the KL divergence of each metrics and Table G.2 shows the mean and standard deviation of the KL divergence comparisons.
-0.5 -0.1 0.3 0.6 1.0 Pair-wise correlation
0
895
1790
2685
3581
C ou
nt
Trial 0
y y = G(x)
-0.4 -0.1 0.3 0.6 1.0 Pair-wise correlation
0
938
1877
2816
3755
C ou
nt
Trial 5
y y = G(x)
-0.4 -0.0 0.3 0.7 1.0 Pair-wise correlation
0
820
1641
2461
3282
C ou
nt
Trial 10
y y = G(x)
0.0 0.1 0.1 0.2 0.3 KL divergence
0
10
21
31
42
C ou
nt
Correlation KL
0.0 0.2 0.3 0.5 0.6 Hz
0
7
15
23
31
C ou
nt
Neuron 25
y y = G(x)
0.0 0.5 1.0 1.5 2.1 Hz
0
22
45
68
91
C ou
nt
Neuron 50
y y = G(x)
0.0 0.2 0.3 0.5 0.6 Hz
0
16
32
48
64
C ou
nt
Neuron 75
y y = G(x)
0.0 1.9 3.7 5.6 7.5 KL divergence
0
4
8
12
17
C ou
nt
Firing Rate
18 15 194 47 86 155 169 36 49 128 184 197 y = G(x)
186 185 193 49 130
9 57 149 170 124 39 85 y
Neuron 25
0
7
13
20
26
33
0 161 33 90 9 199 99 45 186 129 114 108 y = G(x)
199 48 120 184 163 57 157 131 27 116 78 134 y
Neuron 50
0
13
26
39
52
65
1 35 60 93 91 127 25 122 55 31 177 116 y = G(x)
133 110 37 161 154 112 84 41 188 78 145 40 y
Neuron 75
0
6
13
20
27
34
0.1 0.3 0.4 0.6 0.8 KL divergence
0
5
11
17
23
C ou
nt
van-Rossum distance KL
Figure H.3: Spike statistics of (top) firing rate of 3 randomly selected neurons, (middle) pairwise correlation of 3 randomly selected segments and (bottom) van Rossum distance of 3 randomly selected segments between Y and G(X) where Y was ordered by autoencoder reconstruction loss. The right columns show the KL divergence of each metrics and Table G.2 shows the mean and standard deviation of the KL divergence comparisons.
-0.5 -0.1 0.3 0.6 1.0 Pair-wise correlation
0
892
1784
2676
3569
C ou
nt
Trial 0
y y = G(F(y))
-0.4 -0.1 0.3 0.6 1.0 Pair-wise correlation
0
1017
2034
3051
4069
C ou
nt
Trial 5
y y = G(F(y))
-0.4 -0.1 0.3 0.6 1.0 Pair-wise correlation
0
957
1914
2871
3828
C ou
nt
Trial 10
y y = G(F(y))
0.0 0.0 0.1 0.1 0.1 KL divergence
0
10
21
31
42
C ou
nt
Correlation KL
0.0 0.2 0.3 0.5 0.7 Hz
0
6
12
18
25
C ou
nt
Neuron 25
y y = G(F(y))
0.0 0.5 1.0 1.5 2.1 Hz
0
22
45
67
90
C ou
nt
Neuron 50
y y = G(F(y))
0.0 0.2 0.4 0.6 0.8 Hz
0
21
42
63
84
C ou
nt
Neuron 75
y y = G(F(y))
0.0 1.6 3.3 4.9 6.5 KL divergence
0
3
6
9
13
C ou
nt
Firing Rate
98 90 115 59 133 99 18 120 157 55 191 85 y = G(F(y))
98 185 115 59 133 99 18 120 157 55 191 85 y
Neuron 25
1
7
14
21
28
35
2 56 95 128 22 57 157 131 27 116 78 134 y = G(F(y))
199 48 120 184 163 57 157 131 27 116 78 134 y
Neuron 50
0
14
28
42
56
70
3 101 66 125 194 112 84 131 188 78 145 40 y = G(F(y))
133 110 37 161 154 112 84 41 188 78 145 40 y
Neuron 75
0
7
15
22
30
37
0.0 0.1 0.2 0.3 0.4 KL divergence
0
5
11
16
22
C ou
nt
van-Rossum distance KL
Figure H.4: Spike statistics of (top) firing rate of 3 randomly selected neurons, (middle) pairwise correlation of 3 randomly selected segments and (bottom) van Rossum distance of 3 randomly selected segments between Y andG(F (Y )) where Y was ordered by autoencoder reconstruction loss. The right columns show the KL divergence of each metrics and Table G.2 shows the mean and standard deviation of the KL divergence comparisons.

heading: I MOUSE 2 RECORDED DATA

heading: J MOUSE 3 RECORDED DATA
text: ORDER |X − F (G(X))| |X − F (X)| |Y −G(F (Y ))| |Y −G(Y )|
NONE 0.2684± 0.0290 0.0656± 0.0037 0.3229± 0.0476 0.0796± 0.0047 FIRING RATE 0.2679± 0.0309 0.0585± 0.0034 0.3192± 0.0477 0.0777± 0.0043
AUTOENCODER 0.2677± 0.0282 0.0554± 0.0023 0.3199± 0.0487 0.0672± 0.0034
Table J.1: Cycle-consistent and identity loss of AGResNet on Mouse 3 recordings, where neurons were ordered by 1) original annotation, 2) firing rate and 3) autoencoder reconstruction loss. For reference, |X − Y | = 0.4764± 0.1520 in the test set. The lowest loss in each category marked in bold.
KL(X,F (Y )) KL(X,F (G(X))) KL(Y,G(X)) KL(Y,G(F (Y )))
(A) PAIRWISE CORRELATION
IDENTITY 1.0188± 0.5731 0 0.7363± 0.3732 0 N/A 0.5361± 0.2817 0.5678± 0.3145 0.6975± 0.2202 0.7381± 0.2977 FIRING RATE 0.5021± 0.2596 0.5184± 0.2536 0.6281± 0.2830 0.6616± 0.2850 AUTOENCODER 0.5140± 0.2538 0.4751± 0.2421 0.6137± 0.2997 0.4625± 0.2443
(B) FIRING RATE
IDENTITY 12.2077± 7.3556 0 12.4075± 7.3156 0 N/A 1.0164± 0.7129 1.8203± 1.9280 1.2904± 0.9448 1.4786± 1.4374 FIRING RATE 0.9371± 0.6735 1.7893± 2.5419 1.0712± 0.7793 1.2805± 1.5136 AUTOENCODER 0.8936± 0.5655 1.1152± 0.6797 1.2114± 0.7281 0.6928± 0.4643
(C) PAIRWISE VAN ROSSUM DISTANCE
IDENTITY 4.2704± 2.0834 0 4.9623± 1.4393 0 N/A 3.0412± 1.8467 2.0246± 1.3422 4.6059± 2.0664 3.0293± 1.5854 FIRING RATE 2.9009± 1.7587 1.6458± 1.2375 4.1910± 1.7950 2.8613± 1.7788 AUTOENCODER 2.8383± 1.5942 1.4747± 1.1150 3.9709± 1.7732 1.4767± 1.0195
Table J.2: The average KL divergence between generated and recorded distributions of Mouse 3 in (a) pairwise correlation, (b) firing rate and (c) population pairwise van Rossum distance. We compare AGResNet results with different neuron ordering including 1) original annotation, 2) firing rate and 3) autoencoder reconstruction loss. Note that we added the identity model (first row of each sub-table) as a baseline comparison and should obtain perfect cycle reconstruction. Entries with the lowest value are marked in bold.

heading: K MOUSE 4 RECORDED DATA
text: ORDER |X − F (G(X))| |X − F (X)| |Y −G(F (Y ))| |Y −G(Y )|
NONE 0.2538± 0.0399 0.0443± 0.0015 0.2403± 0.0395 0.0808± 0.0061 FIRING RATE 0.2511± 0.0389 0.0376± 0.0015 0.2388± 0.0406 0.0764± 0.0067
AUTOENCODER 0.2489± 0.0381 0.0382± 0.0012 0.2367± 0.0396 0.0764± 0.0053
Table K.1: Cycle-consistent and identity loss of AGResNet on Mouse 4 recordings, where neurons were ordered by 1) original annotation, 2) firing rate and 3) autoencoder reconstruction loss. For reference, |X − Y | = 0.4383± 0.2354 in the test set. The lowest loss in each category marked in bold.
KL(X,F (Y )) KL(X,F (G(X))) KL(Y,G(X)) KL(Y,G(F (Y )))
(A) PAIRWISE CORRELATION
IDENTITY 0.3724± 0.2169 0 0.5124± 0.3238 0 N/A 0.2849± 0.1552 0.1735± 0.0918 0.3536± 0.2541 0.5750± 0.2883 FIRING RATE 0.2482± 0.1502 0.1478± 0.0848 0.3482± 0.2561 0.5471± 0.2577 AUTOENCODER 0.2096± 0.1155 0.1587± 0.0867 0.3460± 0.2568 0.4795± 0.2457
(B) FIRING RATE
IDENTITY 5.8031± 4.8030 0 5.1383± 5.4684 0 N/A 1.3062± 1.0097 0.6034± 0.6294 1.4253± 1.5599 2.9196± 3.1077 FIRING RATE 1.0818± 0.9274 0.5480± 0.5043 1.2120± 1.2971 2.8206± 2.7266 AUTOENCODER 1.0564± 1.1415 0.5474± 0.5223 1.1570± 1.0830 2.1015± 2.2399
(C) PAIRWISE VAN ROSSUM DISTANCE
IDENTITY 2.2670± 1.2707 0 2.8134± 1.5536 0 N/A 1.8698± 1.1525 0.5625± 0.4399 2.4011± 1.4879 3.3849± 1.7608 FIRING RATE 1.5416± 0.9327 0.3931± 0.2821 2.1379± 1.4338 3.3865± 1.9320 AUTOENCODER 1.3246± 0.8537 0.4578± 0.3639 2.2134± 1.3838 2.6537± 1.6526
Table K.2: The average KL divergence between generated and recorded distributions of Mouse 4 in (a) pairwise correlation, (b) firing rate and (c) population pairwise van Rossum distance. We compare AGResNet results with different neuron ordering including 1) original annotation, 2) firing rate and 3) autoencoder reconstruction loss. Note that we added the identity model (first row of each sub-table) as a baseline comparison and should obtain perfect cycle reconstruction. Entries with the lowest value are marked in bold.

Review summaries:
This paper uses CycleGAN to map neuronal activities of mice (as measured by Calcium traces) pre- and post-learning. The main contributions are (1) empirical results of using CycleGAN to learn the pre- and post-learning mapping look promising. (2) using both attention mask (which is for gating residual concatenations) and Grad-CAM to help with interpretation. (3) sorting neurons with an autoencoder’s reconstruction error, essentially sorting them based on their importance.

This paper presents a new method for learning the transformation in neural population activity that takes place during task learning. The method is based on CycleGAN, but includes additional modifications related to neural data and the manner in which it is collected. The paper also presents visual interrogations of the learned model to better understand details of the learning process.