Neuronal Learning Analysis using Cycle-Consistent Adversarial Networks

Understanding how activity in neural circuits reshapes following task learning could reveal fundamental mechanisms of learning . Thanks to the recent advances in neural imaging technologies , high-quality recordings can be obtained from hundreds of neurons over multiple days or even weeks . However , the complexity and dimensionality of population responses pose significant challenges for analysis . Existing methods of studying neuronal adaptation and learning often impose strong assumptions on the data or model , resulting in biased descriptions that do not generalize . In this work , we use a variant of deep generative models called – cycle-consistent adversarial networks , to learn the unknown mapping between pre- and post-learning neuronal activities recorded in vivo . To do so , we develop an end-to-end pipeline to preprocess , train and evaluate calcium fluorescence signals , and a procedure to interpret the resulting deep learning models . To assess the validity of our method , we first test our framework on a synthetic dataset with known ground-truth transformation . Subsequently , we applied our method to neuronal activities recorded from the primary visual cortex of behaving mice , where the mice transition from novice to expert-level performance in a visual-based virtual reality experiment . We evaluate model performance on generated calcium imaging signals and their inferred spike trains . To maximize performance , we derive a novel approach to pre-sort neurons such that convolutional-based networks can take advantage of the spatial information that exists in neuronal activities . In addition , we incorporate visual explanation methods to improve the interpretability of our work and gain insights into the learning process as manifested in the cellular activities . Together , our results demonstrate that analyzing neuronal learning processes with data-driven deep unsupervised methods holds the potential to unravel changes in an unbiased way . 1 INTRODUCTION . One of the main objectives in computational neuroscience is to study the dynamics of neural processing and how neural activity reshapes in the course of learning . A major hurdle was the difficulty in obtaining high-quality neural recordings of the same set of neurons across multiple experiments , though such limitation in recording techniques has seen tremendous improvements in recent years . With the advent of modern neural imaging technologies , it is now possible to monitor a large population of neurons over days or even weeks ( Williams et al. , 2018a ; Steinmetz et al. , 2021 ) , thus allowing experimentalists to obtain in vivo recordings from the same set of neurons across different learning stages . Significant efforts have been put into extracting interpretable and unbiased descriptions of how cortical responses change with experience . Proposed approaches to model changes in neuronal activity include linear latent variable models such as PCA , TCA , GPFA , GPFADS and PSID ( Cunningham & Byron , 2014 ; Williams et al. , 2018b ; Sani et al. , 2021 ; Yu et al. , 2009 ; Rutten et al. , 2020 ) . Methods employing deep learning models but with linear changes or mapping include LFADS and PfLDS ( Pandarinath et al. , 2018 ; Gao et al. , 2016 ) . While these methods enabled substantial progress in understanding the structure of neuronal activity , they do have strong assumptions inherent in the modelling technique or the analysis , such as the linearity assumption in linear latent variable models . Therefore , making sense of the unknown mapping between pre- and post-learning neural activity in an unbiased manner remains a significant challenge , and a data-driven method to interpret the circuit dynamics in learning is highly desirable . Thanks to their ability to self-identity and self-learn features from complex data , deep neural networks ( DNNs ) have seen tremendous success in many biomedical applications ( Cao et al. , 2018 ; Zemouri et al. , 2019 ; Piccialli et al. , 2021 ) . Specifically , deep generative networks have shown promising results in analyzing and synthesizing neuronal activities in recent years . Pandarinath et al . ( 2018 ) developed a variational autoencoder ( VAE ) to learn latent dynamics from single-trial spiking activities and Prince et al . ( 2020 ) extended the framework to work with calcium imaging data . Numerous work have demonstrated generative adversarial networks ( GAN ) are capable of synthesizing neuronal activities that capture the low-level statistics of recordings obtained from behaving animals ( Molano-Mazon et al. , 2018 ; Ramesh et al. , 2019 ; Li et al. , 2020 ) . In this work , we explore the use of cycle-consistent adversarial networks ( Zhu et al. , 2017 ) , or CycleGAN , to learn the mapping between pre- and post-learning neuronal activities in an unsupervised and data-driven manner . In other words , given the neural recordings of a novice animal , can we translate the neuronal activities that correspond to the animal with expert-level performance , and vice versa ? The resulting transformation summarizes these changes in response characteristics in a compact form and is obtained in a fully data-driven way . Such a transformation can be useful in follow-up studies to 1 ) identify neurons that are particularly important for describing the changes in the overall response statistics , not limited to first or second order statistics ; 2 ) detect response patterns relevant for changes from pre- to post-learning ; and 3 ) determine what experimental details are of particular interest for learning . To learn the transformation , we derive a standardized procedure to train , evaluate and interpret the CycleGAN framework . To improve the explainability of our work , we incorporate a self-attention mechanism into our generator models and also employ a feature-importance visualization method into our pipeline so that we can visualize and identify the input that the networks deemed relevant in their decision making process . In addition , we introduced a novel neuron ordering method to improve the learning performance of convolutional neural networks ( CNN ) . To quantify the capability of the proposed unsupervised learning method , we evaluate our method on two datasets : 1 ) an artificially constructed dataset with a handcrafted transformation , and 2 ) recordings obtained from the primary visual cortex of a behaving animal across multiple days . We then compare several metrics and statistics between the recorded and translated calcium traces and their inferred spike trains . 2 METHODS . 2.1 ANIMAL EXPERIMENT To obtain neuronal activities that can demonstrate pre- and postlearning responses , we conducted a visual-based experiment which follows a similar procedure as Pakan et al . ( 2018 ) and Henschke et al . ( 2020 ) . Briefly , a head-fixed mouse was placed on a linear treadmill that allows it to move forward and backward . A lick spout and two monitors were placed in front of the treadmill and a virtual corridor with defined grating pattern was shown to the mouse . A reward ( water drop ) would be made available if the mouse licked within the predefined reward location ( at 120-140 cm ) , in which a black screen is displayed as a visual clue . Figure A.1 illustrates the experiment setup . The mouse should learn to utilize both visual information and self-motion feedback to maximize reward . The same set of neurons in the primary visual cortex were labelled with GCaMP6 calcium indicator and monitored throughout 4 days of experiment , the relative changes in fluorescence ( ∆F/F0 ) over time were used as a proxy for an action potential . 4 mice were used in the virtual-corridor experiment and all mice transitioned from novice to expert in the behaviour task within 4 days of training . Mouse 1 took on average 6.94s per trial on day 1 and 4.43s per trial on day 4 , Table A.1 and A.2 shows the trial information of all the mice . Hence , this dataset can provide excellent insights into how cortical responses reshape with experience . 2.2 CYCLEGAN CycleGAN ( Zhu et al. , 2017 ) is a GAN-based unsupervised framework that learns the mapping between two unpaired distributions X and Y via the adversarial training and cycle-consistency optimization . The framework has shown excellent results in a number of unsupervised translation tasks , including natural language translation ( Gomez et al. , 2018 ) and molecular optimization ( Maziarka et al. , 2020 ) , to name a few . Let X and Y be two distributions with unknown mappings that correspond to ( novice ) pre- and ( expert ) post-learning neuronal activity , respectively . CycleGAN consists of four DNNs : generator G : X → Y that maps novice activities to expert activities and generator F : Y → X that maps expert activities to novice activities ; discriminator DX : X → [ 0 , 1 ] and discriminator DY : Y → [ 0 , 1 ] that learn to distinguish novice and expert neural activities , respectively . In a forward cycle step ( X → Y → X , illustrated in Figure B.1 ) , we first sample a novice recording x from distribution X and apply transformation G to obtain ŷ = G ( x ) . We expect ŷ to resembles data from the expert distribution Y , hence DY learns to minimize ( 1 ) LDY = −Ey∼Y [ ( DY ( y ) − 1 ) 2 ] + Ex∼X [ DY ( G ( x ) ) 2 ] . Similar to a typical GAN , generator G learns to deceive DY with the objective of ( 2 ) LG = −Ex∼X [ ( DY ( G ( x ) ) −1 ) 2 ] . Note that these are same objectives in LSGAN ( Mao et al. , 2017 ) . However , DY can only verify if ŷ ∈ Y , though can not ensure that ŷ is the corresponding expert activity of the novice recording x. Moreover , X and Y are not paired hence we can not directly compare ŷ with samples in Y . To tackle this issue , CycleGAN applies another transformation to reconstruct the novice recording x̄ = F ( ŷ ) where the distance ‖ x − x̄ ‖ or ‖ x − F ( G ( x ) ) ‖ should be minimal . Therefore , the generators also optimize this cycle-consistent loss ( 3 ) Lcycle = Ex∼X [ ‖ x − F ( G ( x ) ) ‖ ] + Ey∼Y [ ‖ y − G ( F ( y ) ) ‖ ] . Mean absolute error ( MAE ) was used as the distance function , though other distance functions can also be employed . In addition , we would expect x̂ = F ( x ) and ŷ = G ( y ) to be in distributions X and Y given that F : Y → X and G : X → Y , hence the identity loss objective ( 4 ) LGidentity = Ey∼Y [ ‖ y −G ( y ) ‖ ] . Taken all together , G optimizes the following objectives : ( 5 ) LGtotal = LG + λcycleLcycle + λidentityLGidentity where λidentity and λcycle are hyper-parameters for identity and cycle loss coefficients . All four networks are trained jointly where LFtotal and LDX are similar to LGtotal and LDY though in opposite directions . In this work , we adapt the CycleGAN framework to learn the unknown mapping between pre- and post-learning neuronal activities recorded from the primary visual cortex of behaving mice . In addition , we experiment with different GANs objective formulations on top of the original LSGAN objective , including GAN ( Goodfellow et al. , 2014 ) , WGANGP ( Arjovsky et al. , 2017 ) and DRAGAN ( Kodali et al. , 2017 ) . Table C.2 shows their exact formulations in CycleGAN . 2.3 MODEL PIPELINE We devise a consistent analysis framework , including data preprocessing and augmentation , networks interpretation , and evaluation of the generated calcium fluorescence signals and their inferred spike trains . Figure C.1 illustrates the complete pipeline of our work.1 We denote the day 1 ( pre-learning ) and day 4 ( post-learning ) recording distributions to be X and Y . With Mouse 1 , W = 102 neurons from the primary visual cortex were monitored , as well as trial information such as the virtual distance , licks and rewards . In total , 21471 and 21556 samples were recorded on day 1 and 4 . Since we want the generators and discriminators to identify patterns relevant to the animal experiment in a data-driven manner , we do not incorporate any trial information into the training data . We first segment the two datasets with a sliding window of size H = 2048 along the temporal dimension ( around 85 s in wall-time ) , resulting in data with shape ( N , H , W ) for X and Y where N is the total number of segments . We select a stride size that space out each segment evenly so that we obtained a sufficient number of samples while keeping the correlations between samples reasonably low . In order to take advantage of the spatiotemporal information in the neuronal activities in a 2D CNN , we further convert the two sets to have shape ( N , H , W , C ) where C = 1 . Finally , we normalize each set to the range [ 0 , 1 ] , and divide them into train , validation and test set with 3000 , 200 and 200 samples respectively . To evaluate the transformation results of G and F , we can compare the cycle-consistency MAE ( X , F ( G ( X ) ) ) and MAE ( Y , G ( F ( Y ) ) ) , as well as the identity losses MAE ( X , F ( X ) ) and MAE ( Y , G ( Y ) ) ( e.g . we expect F to apply no transformation to a novice sample x ) . We also evaluate the generated data in terms of spike activities in the following distribution combinations : novice against translated novice ( X | F ( Y ) ) , novice against reconstructed novice ( X | F ( G ( X ) ) ) , expert against translated expert ( Y | G ( X ) ) and expert against reconstructed expert ( Y | G ( F ( Y ) ) ) . We use Cascade ( Rupprecht et al. , 2021 ) to infer spike trains from the recorded and generated calcium signals to assess the credibility of the generated signals . We measure the following commonly used spike train similarities and statistics : 1 ) mean firing rate for evaluating single neuron statistics ; 2 ) pairwise Pearson correlation for evaluating pairwise statistics ; 3 ) pairwise van Rossum distance ( Rossum , 2001 ) for evaluating general spike train similarity . We evaluate these quantities across the whole population for each neuron or neuron pairs and compare the resulting distributions over these quantities obtained from the recorded and generated data . We , therefore , validate the whole spatiotemporal first and second-order statistics as well as general spike train similarities . To improve the explainability of this work we introduce a number of recently proposed model interpretation methods into our pipeline . We design a self-attention generator architecture which allows 1The software codebase will be made publicly available upon acceptance . the network to learn a set of attention masks such that it encourages the network to better focus on specific areas of interest in the input and also enables us to visually inspect the learned attention maps . In addition , we use GradCAM ( Selvaraju et al. , 2017 ) , a method to visualize discriminative region ( s ) learned by a CNN classifier w.r.t to the input , to extract localization maps from the generators and discriminators . The self-attention mechanism and GradCAM visualization allow us to verify and interpret that the networks are learning meaningful features . Moreover , these extracted attention maps can reveal neurons or activity patterns that are informative in the neuronal learning process . A detail description of the model architectures are available in Section D. 2.3.1 NEURON ORDERING CNNs with a smaller kernel can often perform as well or even better than models with larger kernels while consisting of fewer trainable parameters ( He et al. , 2016a ; Li et al. , 2021 ) . Nevertheless , a smaller kernel can also limit the receptive field of the model , or the region in the input that the model is exposed to in each convolution step ( Araujo et al. , 2019 ) . In addition , the recordings obtained from the virtual-corridor experiment were annotated based on how visible the neurons were in the calcium image , rather than ordered in a particular manner ( see Figure A.1 ) . This could potentially restrict CNNs with small receptive field to learn meaningful spatial-temporal information from the population responses . To mitigate this issue , we derive a procedure to pre-sort X and Y , such that neurons that are highly correlated or relevant are nearby in their ordering . A naive approach is to sort the neurons by their firing rate or average pairwise correlation , where the neuron with the highest firing rate or the neuron that , on average , is most correlated to other neurons is ranked first in the data matrix . However , it is possible that not all highfiring neurons or most correlated neurons are the most influential in the learning process . Therefore , we also explore a data-driven approach . Deep autoencoders have shown excellent results in feature extraction and representation learning ( Gondara , 2016 ; Wang et al. , 2016 ; Tschannen et al. , 2018 ) , and we can take advantage of its unsupervised feature learning ability . We employ a deep autoencoder AE which learns to reconstruct calcium signals in X and Y jointly . AE consists of 3 convolution down-sampling blocks , followed by a bottleneck layer , then 3 transposed-convolution up-sampling blocks . The down-sampling block consists of a convolution layer followed by Instance Normalization ( Ulyanov et al. , 2016 ) , Leaky ReLU ( LReLU ) activation ( Maas et al. , 2013 ) and Spatial Dropout ( Tompson et al. , 2015 ) , whereas a transpose convolution is used in the up-sampling block instead . We optimize the mean-squared error ( MSE ) reconstruction loss on the training set of X and Y , then we use the per-neuron reconstruction error on the test set to sort the neurons ( in ascending order ) : order = argsort ( 0.5× [ MSE ( X , AE ( X ) ) +MSE ( Y , AE ( Y ) ) ] ) . The neuron sorting process is part of the data preprocessing step and is independent from the CycleGAN framework . 2.3.2 SYNTHETIC DATA CycleGAN was originally introduced for image-to-image translation . Albeit the two image distributions are not aligned hence can not be directly compared easily , one could still visually inspect whether or not x̂ = F ( y ) and ŷ = G ( x ) are reasonable transformations . However , it would be difficult to visually inspect the two transformations with calcium signals . To this end , we introduce an additional dataset Y = Φ ( X ) with a known transformation Φ , such that G : X → Y = Φ ( X ) and F : Y = Φ ( X ) → X . We can then verify G ( x ) = ŷ = Φ ( x ) and F ( y ) = x̂ = x . We defined the spatiotemporal transformation Φ that can be identified visually as follows : ( 6 ) Φ ( x ) = mdiagonalx + 0.5η , where mdiagonal is a diagonal mask to zero-out the lower left corners of the signals and η ∼ N ( µx , σ2x ) . µx and σx are the per-neuron mean and standard deviation ofX . Figure 1 shows an augmented example . Importantly , we shuffle the train set after the augmentation procedure so that X and Y appears to be unpaired to the model . Whereas the test set remains in its original paired arrangement so that we can compare ‖ X−F ( Y ) ‖ and ‖ Y −G ( X ) ‖ .

Summary: This paper uses CycleGAN to map neuronal activities of mice (as measured by Calcium traces) pre- and post-learning. The main contributions are (1) empirical results of using CycleGAN to learn the pre- and post-learning mapping look promising. (2) using both attention mask (which is for gating residual concatenations) and Grad-CAM to help with interpretation. (3) sorting neurons with an autoencoder’s reconstruction error, essentially sorting them based on their importance.