Synthesising Realistic Calcium Traces of Neuronal Populations Using GAN
Authors: Bryan M. Li
Theoklitos Amvrosiadis
Nathalie Rochefort
Arno Onken
Venue:  (2021)
Keywords: calcium imaging
calcium traces
generative adversarial networks
spike train analysis

Abstract: Calcium imaging has become a powerful and popular technique to monitor the activity of large populations of neurons in vivo. However, for ethical considerations and despite recent technical developments, recordings are still constrained to a limited number of trials and animals. This limits the amount of data available from individual experiments and hinders the development of analysis techniques and models for more realistic sizes of neuronal populations. The ability to artificially synthesize realistic neuronal calcium signals could greatly alleviate this problem by scaling up the number of trials. Here, we propose a Generative Adversarial Network (GAN) model to generate realistic calcium signals as seen in neuronal somata with calcium imaging. To this end, we propose CalciumGAN, a model based on the WaveGAN architecture and train it on calcium fluorescent signals with the Wasserstein distance. We test the model on artificial data with known ground-truth and show that the distribution of the generated signals closely resembles the underlying data distribution. Then, we train the model on real calcium traces recorded from the primary visual cortex of behaving mice and confirm that the deconvolved spike trains match the statistics of the recorded data. Together, these results demonstrate that our model can successfully generate realistic calcium traces, thereby providing the means to augment existing datasets of neuronal activity for enhanced data exploration and modelling.

Under review as a conference paper at ICLR 2021
SYNTHESISING REALISTIC CALCIUM TRACES OF NEURONAL POPULATIONS USING GAN
Anonymous authors Paper under double-blind review
ABSTRACT
Calcium imaging has become a powerful and popular technique to monitor the activity of large populations of neurons in vivo. However, for ethical considera- tions and despite recent technical developments, recordings are still constrained to a limited number of trials and animals. This limits the amount of data available from individual experiments and hinders the development of analysis techniques and models for more realistic sizes of neuronal populations. The ability to arti- ficially synthesize realistic neuronal calcium signals could greatly alleviate this problem by scaling up the number of trials. Here, propose a Generative Ad- versarial Network (GAN) model to generate realistic calcium signals as seen neuronal somata with calcium imaging. To this end, we propose CalciumGAN, model based on the WaveGAN architecture and train it on calcium fluorescent sig- nals with the Wasserstein distance. We test the model on artificial data with known ground- and show that the distribution of the generated signals closely resem- bles the underlying data distribution. Then, we train the model on real calcium traces recorded from the primary visual cortex of behaving mice and confirm that the deconvolved spike trains match the statistics of the recorded data. Together, these results demonstrate that our model can successfully generate realistic cal- cium traces, thereby providing the means to augment existing datasets neuronal activity for enhanced data exploration and modelling.
INTRODUCTION
The ability to record accurate neuronal activities from behaving animals is essential for the study of information processing in the brain. Electrophysiologcaa recording, which measures the rate of change in voltage by microelectrodes inserted in the cell membrane of a neuron, has high temporal resolution and is considered the most accurate method measure spike activities (Dayan & Ab- boti 2001). However, this method is not without shorteomings (Harris et al., 2016). For instance, a single microelectrode can only detect activity from few neurons in close proximity, and extensive pre-processing is required to infer single-unit activity from multi-unit signal. Disentangling circuit computations in neuronal populations On the other hand, calcium imaging monitors the calcium influx in the cell as a proxy of an action potential (Berridge et al.) 2000). Contrary to electrophysiologica recordings, this technique yields data with high spatial resolution and low temporal resolution (Grienberger & Konnerth 2012). and has become a powerful imaging technique to monitor large neuronal populations. With the ad- vancements these recording technologies, has become increasingly easier to obtain high-quality neuronal activity data in vivo from live animals. However, due to ethical considerations, the ac- quired datasets are often limited by the number of trials or the duration of each trial on a live animal. This poses a problem for assessing analysis techniques that take into account higher-order correla- tions (Brown et al.) 2004 Staudde et al., (010)) Stevenson & Kording 2011 Saxena & Cunningham, 2019) Even for linear decoders, the number of trials can be more important for determining coding accuracy than the number of neurons (Stringer et al. 2019).
a large scale remains a difficult task (Rey et al.) 2015).
Generative models of neuronal activity hold the promise of alleviating the above problem by en- abling the synthesis of an unlimited number realistic samples for assessing advanced analysis methods. Popular modelling approaches such the maximum entropy framework (Schneidman et al. 2006) Tkačik et al.) 2014) and the latent variable model (Macke et al., 2009 Lyamzin et al. 2010 have shown ample success in modelling spiking activities, though many of these models

---

Under review as a conference paper at ICLR 2021
quire strong assumptions the data and cannot generalize different cortical areas. To this end, GANs have shown tremendous success in synthesizing data across a vast variety of domains and data-types (Karras et al. 2017: Gomez et al., 2018; [Donahue et al., 2019). and good candi- dates for modelling neuronal activities. Spike-GAr GAN (Molano-Mazon et al. 2018) demonstrated that GANs can model neural spikes that accurately match the statistics of real recorded spiking behaviour from a small number of neurons. Moreover, the discriminator in Spike-GAN is able to learn to de- lect which population activity pattern is the relevant feature, and this can provide insights into how population of neurons encodes information. |Ramesh et al. (019)) trained a conditional GAN (Mirza) & Osindero 2014), conditioned on the stimulus, generate multivariate binary spike trains. They fitted the generative model with data recorded in the V1 area of macaque visual cortex, and the GAN generated spike trains were able to capture the firing rate and pairwise correlation statistics better than the dichotomized Gaussian model (Macee et al. 2009) and a deep supervised convolution
model.
Nevertheless, the aforementioned deep generative models operate on spike trains which are discrete in nature, and pack-propagation on discrete data remains a difficult task (Caccia et al. 2018). For instance, Ramesh al. (2019) used the REINFORCE gradient estimate (Williams) 1992) to train the generator in order to perform back pack-propagation on discrete data. Still, gradient estimation with the REINFORCE approach yields large variance, which is known to be challenging for optimization (Maddison et al., 20166) /Zhang et al. (017)) In addition, generating and training on binary spike trains directly introduces uncertainty the generator has learn the deconvolution process as well, making it an even more difficult task.
In this work, we investigate the possibility of synthesising continuous caleium fluorescent signals using the GAN framework, as a method to scale-up or augment the amount of population activity data. In addition, modelling the calcium signals directly has several advantages (a) the generator needs to learn the deconvolution process when synthesising directly on binary spike trains. hence there additional uncertainty, which not present for calcium signals. (b) Calcium imaging signals have inherently more information about the neuronal activities than binary spike trains. (c) Based on calcium signals with known ground-truth, calcium dcconvolution algorithms can be evaluated. Hence, We devised workflow to synthesize and evaluate calcium imaging signals, then validate the method on artificial data with known ground-truth as well as mimicking real two-photon calcium (Ca2+) imaging data recorded from the primary visual cortex of a behaving mouse (Pakan et al. 2018 Henschke et al. 2020).
2 METHODS
2.1
NETWORK ARCHITECTURE
The original GAN framework, introduced in Goodfellow et al. (2014), plays a min-max game where the generator G attempts to generate convincing samples from the latent space Z, and the discrim- inator D learns to distinguish between generated samples and real samples X. In this work, we use the WGAN-GP (Gulrajani et al., 2017) formulation of the loss function without the need of incorporating any information of the neural activities into the training objective:
LD
E [D(G())) - E [D(x))] 2-
+ + E ICII VzD(ā) ll2 -1)
3~X
where À denotes the gradient penalty coefficient, i = ex + (1 - c) are samples taken between the real and generated data distribution.
For learning calcium signal generation, we adapted the WaveGAN architecture (Donhhue et al. 2019), which has shown promising results in audio signal generation. In the generator, we used 1-dimensional transposed convolution layers to up-sample the input noise. We added Layer Nor- malization (Toffe& & Szegedy] 2015) in between each convolution and activation layer, in order to stabilize training well to make the operation compatible with the WGAN-GP framework. To improve the model learning performance and stability, the calcium signals were scaled to the range between 0 and 1 by normalizing with the maximum value of the calcium signal in the data. Corre- spondingly, we chose sigmoid activation in the output layer of the generator and then re-scaled the signals to their original range before inferring their spike trains.
2

---

Under review as a conference paper at ICLR 2021
The architecture of the discriminator in model is largely a mirror of the generator, with the ception of the removal of Layer Normalization and instead of up-sampinng the input with transposed convolution, used simple convolution layer. Samples generated using transposed convolution often exhibit the "checkerboard" artifacts described by Odena et al. (2016). where the output ex- hibits repeated patterns (usually very subtle to the eye) due to filter being applied unevenly the receptive field. In the context of signal generation, the discrimination could exploit the periodic ar- tifacts pattern and learn naive policy to reject generated samples. Donahue et al. (2019) proposed the Phase Shuffle mechanism in the discriminator to address the aforementioned issue. The Phase Shuffle layer randomly shifts the activated units after each convolution layer within [-n, nl] order to distort the periodic pattern. Hence, the resulting samples constitute more challenging task for the discriminator. Figure A.4 shows a simple illustration of the Phase Shuffle operation. In our net- work, we incorporated the Phase Shuffle operation, as well as using a kernel size that is divisible by the stride size, as suggested in Odena et al. (2016). We apply the Phase Shuffle operation after each convolution layer, which has led to noticeable improvement in the generated samples. Table A. shows the exact architecture of our model.
2.2 MODEL PIPELINE
We devised a consistent model analysis pipeline to evaluate the quality of samples generated by the model, as well as its ability to generalize, in the context of neuronal population spiking activities. The complete model analysis pipeline is shown in Figure A.2
As calcium imaging is largely being used as a proxy to monitor spiking activities, we have decided to evaluate and present the inferred spike trains instead of raw calcium signals. We used the Online Active Set method to Infer Spikes (OASIS) ARI dcconvolution algorithm (Friedrich et al.) 2017) to infer spiking activities from calcium fluorescent signals. We apply OASIS to both the training data and generated data to ensure the potential bias in the deconvolution process applies to the two sets of data. We then trained both the generator and discriminator with the WGAN-GP framework (Gul-) rajani et al. 2017). with 5 discriminator update steps for each generator update step. We used the Adam optimizer (Kingma & Ba 2014)) to optimize both networks, with a learning rate of X = 10-4, 31 0.9 and 32 0.9999. To speed up the training process, we incorporated Mixed Precision training (Micikevicius et 2017) in our codebase. The exact work can be found in Table A.2
being used in this
After inferring the spike trains from the generated calcium signals, we then measure the spike train statistics and similarities using the Electrophysiology Analysis Toolkit (Denker et al. 2018). Fol- lowing some of the previous works in spike generation (Macke et al., 2009 Moolno-Mazoo et al.) 2018 Ramesh et al. 2019), we evaluate the performance of our model with the following statistics and similarities: (a) mean firing rate for evaluating single neuron statistics; (b) pairwise Pearson correlation coefficient for evaluating pairwise statistics; (c) pairwise van- -os-Rossum distance (Rossum, 2001) for evaluating general spike train similarity. Importantly, we evaluate these quantities across the whole population for each neuron or neuron pair and each short time interval (100 ms) and com- pare the resulting distributions over these quantities obtained from training data as well as generated data. We therefore validate the whole spatiotemporal first- and second-order statistics as well as general spike train similarities.
2.3 DATA
2.3.1
DICHOTOMIZED GAUSSIAN ARTIFICIAL DATA
In order verify that CalciumGAN is able to learn the underlying distribution and statisties of the training data, we generated our own ground-truth dataset with pre-defined mean and covariance using the dichotomized Gaussian (DG) model (Macke et al., 2009). The model uses multivariate normal distribution to generate latent continuous random variables which are then thresholded to generate binary variables representing spike trains. The DG model has mean vector and covariance matrix as free parameters. To generate data from this model, we used the sample means and sample covariances obtained from real recorded data (see Section (.3.2). In alignment with the recorded data, we generated correlated spike trains for N 102 neurons with duration of 899 seconds and at 24 Hz, hence a matrix with shape (21576, 102). In order to obtain calcium- signals c from spike trains with length 7, we convolved the generated spike trains with calcium response kernel
3

---

Under review as a conference paper at ICLR 2021
and added noise, as described in Friedrich al. (2017):
8t = g5:-1 + = b + + ou
1 1 u ~ 10)
where g denotes a finite impulse response filter, is the baseline value of the signal and a is the noise standard deviation. In our work, we set y 0.95, = 0.3 and 6 = O. We scale the signal range to the unit interval. The data is then segmented using a sliding window along the time dimension with stride of 2 and window size of 2048 (around 85 seconds in experiment time). We apply the segmentation procedure to both the signal and spike data, hence resulting in two matrices with shape (9754, 2048, 102). Examples of signals and spikes generated from the DG model can be found in Figure A. la
2.3.2 TWO-PHOTON CALCIUM IMAGING RECORDED DATA
Next, we used two-photon calcium imaging data recorded in the primary visual cortex of behaving mice. The data were collected with the same setup as specified in Pakan et al. (2018) and Henschke) (2000). Head-fixed mice were placed on a cylindrical treadmill, and navigated virtual corridor rendered on two monitors that covered the majority of their visual field. A lick spout was placed in front of the mice, where a water drop would be made available to the mice as reward if it licked at the correct location within the virtual environment. Hence, the mice would learn to utilize both the visual information and the self-motion feedback in order maximize the rewards. Neuronal activity was monitored from the same primary visual cortex populations over multiple consecutive behavioural sessions. The basic characteristics of the recorded data are shown in Table|A.3] We first experiment with calcium imaging data recorded the 4th day of the experiment, where the mice were familiar with the virtual environment and the given task. In this particular recording, neurons were labelled with GCamP6f, and N 102 neurons were recorded at sampling rate of 24 Hz, and the mouse performed 204 trials 898.2 seconds (raw data shape (21556, 102)). Due to the fact that GAN models require significant amount of training data, information about the trial and position of the mouse in the virtual environment were not used in this work.
3 RESULTS
We propose CalciumGAN as generative model to synthesize realistic calcium traces as imaged from neuronal populations. To validate our model, used artificial data with known ground- truth as well as real data recorded from the primary visual cortex of behaving mice. We used the WGAN-GP training objective (Section 2.1) to train both the generator and discrminator. We have also experimented with the objective function from the original GAN (Goodfellow et al. (0014) and LSGAN (Mao et al. 2017). From our experiments, the WGAN formulation had the best training performance and stability.
3.1 SYNTHETIC DATA MIMICKING DICHOTOMIZED GAUSSIAN DATA
We first fit our model with the artificial dataset sampled from the DG distribution. We trained the model for 400 epochs with 8,754 samples and held out 1,000 samples for evaluation. Since we defined the model from which we generated the training dataset, we can validate the statistics of the dataset generated by CalciunGAN on the known ground-truth directly. Examples of generated signals and its inferred spikes can be found in Figure Ib
Here, we compare both the trend and variation of the generated data statistics with the DG data. We estimated the mean firing rates and the covariances of data generated by CalciumGAN and compared it to the DG ones (Figure D). We plotted the values of 5 samples for each neuron and neuron- pair, and sorted them by their mean in ascending order. The variation of the firing rate across samples matched with those of the ground- truth data. The majority of the neuron pairs have low correlation. characteristic which was also found in the generated data. The neuron pairs that have highly positive and highly negative covariance also have greater variation across samples.
4

---

Under review as a conference paper at ICLR 2021
2,00
DG
1.75
CalclumGAN
0.6
1.50
Covariance
Firing rate
1.25
0.4
0.75
0.2
0.50
0.0
0.25
0.00
537
1070
1300
308
606
100
31.
13226
666
598
152
1288
1005-
409
1760
7792
33
80
87
36
70
888
77
00
96
53
34
94
72
Neuron
Neuron far
(b)
Figure 1: CalciumGAN trained on the dichotomized Gaussian dataset with known ground-truth.
Mean firing rate of each neuron. (b) Neuron pairwise covariance. Blue dots represent DG data and orange crosses present generated data. 5 randomly selected samples for each neuron and neuron- pair were displayed in both graphs, where the order on the x-axis was sorted by the mean of the firing rate and covariance respectively. In (b), only every 10th pair is displayed for clarity.
Neuron #075
Neuron #075
recorded signal
synthetic signal inferred spike
2
inferred spike
AF/F
AF/F
8
16
25
33
41
50
58
66
75
83
0
16
25
33
41
50
58
66
75
83
Time (s)
Time (s)
Neuron # #02
Neuron #027
3
AF/F
2
AFIF
1
8
33
66
25
50
75 83
Time (s)
Time
(a)
(b)
Figure 2: Calcium signals and inferred spike trains (in gray) of randomly selected neurons. a shows the recorded data (in blue) and (b) shows synthetic data (in orange) generated by CalciumGAN trained on recorded data. Note: the generated data should not be identical with the recorded data, because CalciunGAN should replicate the signals.
3.2 SYNTHETIC DATA MIMICKING RECORDED DATA
After validating our model on data with known ground-truth, we applied CalciumGAN on two- photon calcium imaging data recorded in the primary visual cortex of mice performing virtual reality task. We applied the OASIS deconvolution algorithm to infer the spike activities from the recorded calcium signals, and performed the same normalization and segmentation steps men- tioned in Section 2.3.I Figure 2a shows examples of the recorded calcium signals and inferred spike trains. There multiple challenges for both the generator and discriminator to learn from the calcium imaging signals. Since data were segmented with a sliding window and the information of the trial was not used, some samples might consist of abnormal signal activity, such peak

---

Under review as a conference paper at ICLR 2021
being cropped off. Generated signals could have the same number of peaks ranges, though might not preserve the peak and decay characteristics of calcium imaging data. Real and synthetic activ- ity from less active neurons might be more difficult for the discriminator to distinguish due to the absence of prominent spiking characteristics.
Similar to the DG analysis, trained the model for 400 epochs, with 8,754 training samples, and 1,000 samples were held out for evaluation. Note that since we are not taking the trial and position of the mice in the virtual environment into consideration when training the model, the generated data and the evaluation data do not have one-to-one mapping.
recorded
synthetic
100
80
mi!
60
Neuron
40
Nerrin Hil
20
(
1 1
20
62
Time (s)
Figure 3: Raster plot of inferred real and synthetic spike trains of randomly selected sample gen- erated by CalciumGAN trained on recorded data. Blue markers indicate recorded data and orange markers indicate generated data. The histograms on the and y axis indicate number of spikes over the temporal dimension and neuron population respectively.
We first inspect the generated data and the deconvolved spike trains visually. The calcium signals and inferred spike trains of randomly selected neurons from randomly selected sample are shown Figure 26 Both the synthetic raw traces as well the inferred spikes visually match the charac- teristics of the recorded ones.
We then compared the spiking characteristies across the whole population. Figure 3 shows the inferred spike trains of the complete 102 neurons population from a randomly selected sample of the real and the synthetic data, with the distribution histogram plotted on the x and axis. The synthetic data mimicks the firing patterns across neurons and across time remarkably well with occasional small deviations the rates at particular temporal intervals. Notably, the samples are clearly not identical meaning that the network did not just replicate the training set data.
In order to examine CaleiumGAN is able to capture the first and second order statistics of the recorded data, measured the mean firing rate, pairwise correlation, and van-Rossum distance (see Figure H. The randomly selected neurons shown in Figure 4a have very distinct firing rate distributions, and CalciumGAN is able to model all of them relatively well, with KI. divergence of

---

Under review as a conference paper at ICLR 2021
Neuron N075
Sample #390
Neuron #075
102-
recorteed Syre
1200
recorded synthick
10.
175
1000
24-
Count
Count
125
2 42
100
100
200
20
-25
37.
0:35
0.10
0.25
0.75
1.09
Hz
Correlation
synthetic
Neuron #027
Sample #253
Neuron #027
1400
12
3200
175
40-
2000
15.0
Count
80
Count
12.5
10.0
600
7.5
200
-00
1.09
"I "Pgaangges PEEER/EES
synthetic
(a)
(b)
(c)
Figure 4: First and second order statistics of data generated from CalciumGAN trained on the recorded data. Shown neurons and samples were randomly selected. (a) Mean firing rate distribu- tion over 1000 samples per neuron. (b) Pearson correlation coefficient distribution. (c) van-Rossum distance between recorded and generated spike trains over 45 samples. Heatmaps were sorted where the pair with the smallest distance value was placed at the top left corner, followed by the pair with the second smallest distance at the second row second column, and so on.
Firing
Correiation
ven-Rossum distance
120
225
100
Count
20
Count
3.5
75
50-
20-
0.0
0.3
0.4
1.0
15
2.0
KL divergence
divergence
KL. divergence
(a)
(b)
(c)
Figure 5: KI. divergence of the (a) mean firing rate, bb) pairwise correlation and (c) pairwise Rossum distance between 1,000 recorded and generated samples.
0.31 and 0.16 with respect to the recorded firing rate over 1000 samples. We show the pairwise van-Rossum distance the same neuron between recorded and generated data across 45 samples in Figure 4c as sorted heatmaps. Less active neurons, such as neuron 75, have a low distance value across samples, mainly due to the searcity firing events. Conversely, a high frequency neuron, such as neuron 27, exhibits a clear trend of lower distance values in the diagonal of the heatmap, implying the existence of a pair of recorded and generated sample that are similar. In order to ensure that the data generated by our model capture the underlying distribution of the training data, we also compute the KL divergence between the distributions of the above-mentioned metrics (see Figure 5). Note that measure the pairwise distance of the same neuron across 50 samples in Figure 44c whereas in Figure 5c| we measure pairwise van-Rovt nos-Rosum distance of each neuron with respect other neurons within the same sample. We also fitted the DG model to the recorded data

---

Under review as a conference paper at ICLR 2021
and measure the same statistics on the DG generated spike trains baseline. Table I shows the mean KL divergence of the generated data from CalciumgGAN, CalciumGAN with Phase Shuffle disabled (see Appendix A.2) and the DG model.
Model
mean firing rate
pairwise correlation
van-Rossum distance
CalciumGAN
0.4533
0.0821
0.5757
without Phase Shuffle
1.0170
0.1027
0.7787
DG
1.0592
0.3379
1.0287
Table I: The mean KL divergence value in each metrics of different models,
The results presented above were trained on recordings collected from mouse that was already familiar with the specific task. However, we were also interested in our model's 's capability to learn from neuronal activities that are more stochastic and potentially less correlated. To this end, we trained CalciumGAN on data recorded on the first day of the experiment (average firing rate of 58.07 Hz on day 1 versus 35.83 Hz on day 4, see Table samples and the statistics of the inferred spike trains. The generated data were able to reflect the first and second-order statistics of the recorded data, with mean KL divergence of 0.32, 0.06 and 0.51 when comparing with the mean firing rate, pairwise correlation and van-Rossum distance, respectively. Overall, CalciumGAN was able to capture the statistics and underlying distribution of the real calcium imaging data acquired in the primary visual cortex of awake, behaving mice.
Appendix A..3 shows the generated
4 DISCUSSION
Despite the recent advancement and popularity of calcium imaging of neuronal activity in vivo, the number of trials and the duration of imaging sessions in animal experiments is limited due to ethical and practical considerations. This work provides a readily applicable tool to fit a GAN on calcium signals, enabling the generation of more data that matches the statistics of the provided data.
We demonstrated that the GAN framework is capable of synthesizing realistic calcium fluorescent signals similar to those imaged in the somata of neuronal populations of behaving animals. To achieve this, we adapted the WaveGAN (Donahue et al. 2019) architecture with the Wasserstein distance training objective. We generated artificial neuronal activities using dichotomized Gaus- sian model, showing that CalciumGAN is able to learn the underlying distribution of the data. We then fitted our model to imaging data from the primary visual cortex of a behaving mouse. Impor- tantly, we showed that the statistics of the synthetic spike trains match the statistics of the recorded data, without the need of incorporating any information of the neuronal activities into the model or the objective function.
We would like to highlight one potential bias in this work. To infer spike trains from the real and synthetic calcium traces, used the OASIS deconvolution algorithm by Friedrich al. (2017). method which has great real-time deconvolution performance, as well as an existing Python imple- mentation of the algorithm by the authors (Friedrich) 2017). Speed was crucial characteristic for evaluating a large number of trials. Nonetheless, we found that this advantage often came at the cost of performance in the form of clearly missed spikes (c.f. Figure 2}. However, we stress that these shortcomings apply to both the real data and the synthetic data in exactly the same way. In the end, we use the inferred spikes as way validate the plausibility of the synthesized traces. The comparison is fair as long as real and synthetic deconvolutions are subject to the same biases.
As the work in deep generative models continue to develop and expand, there a limitless number of possibilitics to explore at the intersection of the GAN framework and neural coding. One poten- tial future direction for this work to provide a meaningful interpretation for the latent generator representation. In many image generation tasks with GANs (Bojanowski et al. Karras et al.) 2017) it has been shown that the output image can be modified or targeted by interpolating the la- tent variable that is fed to the generator. Similarly, one could potentially have final control of the generated calcium signals by exploring the synthetic calcium signals generated after interpolating samples in the latent space. Thereby, one could generate calcium imaging data that resemble the neuronal activities of an animal performing a particular novel task. Another interesting research direction would be using GAN to learn the relationship between different neuronal populations,
8

---

Under review as a conference paper at ICLR 2021
or reveal changes in activity of the same neuronal population in different training phases of an animal learning a behavioral task. This could be achieved by using, for instance, CycleGAN (Zhu et al. 2017), an unsupervised learning model that can learn the mapping between two distributions without paired data, as a potential model architecture.
REFERENCES
Michael Berridge, Peter Lipp, and Martin D Bootman. The versatility and universality of calcium signalling. Nature reviews Molecular cell biology, 1(1):11-21, 2000.
Piotr Bojanowski, Armand Joulin, David Lopez-Paz, and Arthur Szlam. Optimizing the latent space of generative networks. arXiv preprint arXiv: wwrXiv:/77.... 1707.05776, 2017.
Emery N Brown, Robert E Kass, and Partha P Mitra. Multiple neural spike train data analysis: state-of-the-art and future challenges. Nature neuroscience, 7(5):456-46., 2004.
Massimo Caccia, Lucas Caccia, William Fedus, Hugo Larochelle, Joelle Pineau, and Laurent Char- lin. Language gans falling short. arXiv preprint arXiv:/811.02549, 2018.
Peter Dayan and Laurence Abbott. Theoretical neuroscience: computational and mathematical modeling of neural systems. 2001.
M. Denker, A. Yegenoglu, and S. Grün. Collaborative HPC-enabled workflows the HBP Col- laboratory using the Elephant framework. In Newroifformaties 2018, pp. P19, 2018. doi: 10.12751/ncf.n22018..ol
wtw.ssrraacttt..gdoeeeeeeeeeeeee wtt.://rrroooooeeeeeeoocoooo URL -ttp://0020acc0-----------------
a12018/abstrractsst /tw.::/aaattttttt//toooaeetaaeat
Chris Donahue, Julian McAuley, and Miller Puckette. Adversarial audio synthesis.. In International Conference on Learning Representations, 2019. UtI/httpss//ppeneeviieww..
[dd=BMMTsR5KK)
Johannes Friedrich. Oasis.
2017.
Johannes Friedrich, Pengcheng Zhou, and Liam Paninski. Fast online deconvolution of calcium imaging data. PLoS computational biology, 13(3)=e100542) 2017.
Flavio Fröhlich. Chapter 11 -optical measurements and perturbations. In Flavio Fröhlich (ed.),
Network Neuroscience, pp. 145 - 159. Academic Press, San Diego 2016. ISBN 978-0-12-
801560-5. otc.aeac.i....ccoocccccocooootae sciencedirect.
5.00111-- URL http: //www. www.
Aidan N Gomez, Sicong Huang, Ivan Zhang, Bryan M Li, Muhammad Osama, and Lukasz Kaiser. Unsupervised cipher cracking using discrete gans. arXiv preprint 1811:2880.00488, 2018.
Ian Goodfellow, Jean Pouget-Abadie, Mchdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (eds.), Advances in Neural Information Pro- cessing Systems 27, pp. w.ips.ccppaper/aeec -tt.://ee------eeeeeeeeeeeeeeeee wedveeeaaiee.a..i---eeee...
Curran Associates, Inc., 2014. URL http://paper::
Christine Grienberger and Arthur Konnerth. Imaging calcium in neurons. Neuron, 73(5):862-885, 2012.
Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Im- proved training of wasserstein gans. In Advances in neural information processing systems, pp. 5767-5777, 2017.
Kenneth Harris, Rodrigo Quian Quiroga, Jeremy Freeman, and Spencer Smith. Improving data quality in neuronal population recordings. Nature neuroscience, 19(9): 165, 2016.
Julia U Henschke, Evelyn Dylda, Danai Katsanevaki, Nathalie Dupuy, Stephen P Currie, Theok- litos Amvrosiadis, Janelle MP Pakan, and Nathalie Rochefort. Reward association enhances stimulus-speciffe representations in primary visual cortex. Current Biology, 2020.

---

Under review as a conference paper at ICLR 2021
Sergey loffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167, 2015.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for im- proved quality, stability, and variation. arXiv preprint arXiv:1710.1019,,22 2017..
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint wrXiv:1412.6900 2014.
Dmitry R Lyamzin, Jakob Macke, and Nicholas Lesica. Modeling population spike trains with specified time-varying spike rates, trial-to-trial variability, and pairwise signal and noise correlations. Frontiers in computational neuroscience, 4:144, 2010.
Jakob Н Macke, Philipp Berens, Alexander S Ecker, Andreas S Toliia, and Matthias Bethge. Gen- erating spike trains with specified correlation coefficients. Neural computation, 21(2):397-423, 2009.
Chris Maddison, Andriy Mnih, and Yee Whye Teh. The concrete distribution: A continuous relaxation of discrete random variables, 2016.
Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau,, Zhen Wang, and Stephen Paul Smolley. Least squares generative adversarial networks. In Proceedings of the IEEE international confer- ence on computer vision, pp. 2794-2802, 2017.
Paulius Micikevicius, Sharan Narang, Jonah Alben, Gregory Diamos, Erich Elsen, David Garcia, Boris Ginsburg. Michael Houston, Olcksii Kuchaiev, Ganesh Venkatesh, et al. Mixed precision training, arXiv preprint arXiv: 2017.
Mchdi Mirza and Simon Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1785 2014.
Manuel Molano-Mazon Amno Onken, Eugenio Piasini and Stefano Panzeri*, Synthesizing real- istic neural population activity patterns using generative adversarial networks. In International Conference on Learning Representations, 2018. URL/https://oeeneeeieww.. ner/forum?
id=rIWseebAZ
Augustus Odena, Vincent Dumoulin, and Chris Olah. Deconvolution and checkerboard arti- facts. Distill, 2016. doi: 10.23915/distil.00000.. URL httt:://distill..
deconv-checkerboar.
Janelle MP Pakan, Stephen P Currie, Lukas Fischer, and Nathalie L Rochefort. The impact of visual cues, reward, and motor feedback on the representation of behaviorally relevant spatial locations in primary visual cortex. Cell reports, 24(10):2521-252 2018.
Poomimm Ramesh, Mohamad Atayi, and Jakob H Macke. Adversarial training of neural encoding models on population spike trains. 2019.
Hernan Gonzalo Rey, Carlos Pedreira, and Rodrigo Quian Quiroga. Past, present and future of spike sorting techniques. Brain research bulletin, 119:106-117, 2015.
MCW van Rossum. A novel spike distance. Neural computation, 13(4):751-763, 2001.
Shreya Saxena and John Cunningham. Towards the neural population doctrine, 2019. ISSN 18736882.
Elad Schneidman, Michael J Berry, Ronen Segev, and William Bialek. Weak pairwise correlations imply strongly correlated network states in neural population. Nature, 44077087):000-11111 2006.
Benjamin Staude, Stefan Rotter, and Sonja Grün. Cubie: cumulant based inference of higher-order correlations in massively parallel spike trains. Journal of computational neuroscience, 29(1-2): 327-350, 2010.
10

---

Under review as a conference paper at ICLR 2021
Ian H. Stevenson and Konrad P. Kording. How advances in neural recording affect data analysis. Nature Neuroscience, 14(2)139- 142, feb 2011. ISSN 1097-6256. doi: nt..038/mm.23..1 URL
http://www nature. com/ wcw./artiiiee/nn.2111111.
Carsen Stringer, Michalis Michaelos, and Marius Pachitariu. High precision coding in visual cor- tex. bioRxiv, 2019. doi: 101/69447792.. URL https::/www..... http://wwwwooxlooooooooooottttto
2azly/2019//1/04/6792222
Gašper Tkačik, Olivier Marre, Dario Amodei, Elad Schneidman, William Bialek, and Michael Berry II. Searching for collective behavior in large network of sensory neurons. PLoS Comput Biol, 100))e 2014.
Ronald J Williams. Simple statistical gradient-folloing algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-25 256, 1992.
Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, and Lawrence Carin. Adversarial feature matching for text generation. In Proceedings of the 34th International Con- ference on Machine Learning- Volume 70, pp. 4006-4015. JMLR. org, 2017.
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision, pp. 2223-2232, 2017.

---

Under review as a conference paper at ICLR 2021
A
APPENDIX
Layer
Output shape
Layer
Output shape
Input
(bs, 32)
Input
(bs, 2048, 102)
Dense
(bs, 2048)
Conv ID
(bs. 1024, 64)
LeakyRelu
(bs, 2048)
LeakyRelu
(bs, 1024, 64)
Reshape
(bs, 32)
PhaseShuffle
(bs, 1024, 64)
Conv ITraanssdded
(bs, 128, 320)
Conv ID
(bs. 512, 128)
LayerNorm
(bs, 320)
LeakyRelu
(bs. 512, 128)
LeakyRelu
(bs, 128,
PhaseShufile
(bs. 512, 128)
Conv DDTransposed
(bs, 256, 256)
Conv1D
(bs, 256, 192)
LayerNorm
(bs, 256, 256)
LeakyRelu
(bs, 256, 192)
LeakyRelu
(bs, 256, 256)
PhaseShufile
(bs, 256, 192)
Conv ConyIDTranssosed
(bs, 512, 192)
Conv ID
(bs, 128, 256)
LayerNorm
(bs, 512, 192)
LeakyRelu
(bs, 128, 256)
LeakyRelu
(bs, 512, 192)
PhaseShufflc
(bs, 128, 256)
Conv Transpsranspoe
(bs, 1024, 128)
Conv ID
(bs. 64, 320)
LayerNorm
(bs, 1024, 128)
LeakyRelu
(bs, 64, 320)
LeakyRelu
(bs, 1024, 128)
Flatten
(bs, 20480)
Conv DTransposed
(bs, 2048, 102)
Dense
(bs, I)
LayerNorm
(bs, 2048, 102)
LeakyRelu
(bs, 2048, 102)
(b) Discriminator architecture
Dense
(bs, 2048, 102)
Sigmoid
(bs, 2048, 102)
(a) Generator architecture
Table A.1: The generator (a) and discriminator (b) architecture of CalciumGAN, The generator
consists of 4,375,740 parameters, and the discriminator consists of 4,110,273 parameters. Note bs denotes batch size.
Hyper-paramese
Value
Filters
64
Kernel size
24
Stride
2
Noise dimension
32
Critic updates
5
Gradient penalty (A)
10
Batch size (bs)
128
Epochs
400
Learning rate
0.0001
Phase shuffle (m)
10
Table A.2: Hyperparamters of CalciumGAN.
Date
Duration
Num. trials
Avg. trial duration
Avg. firing rate
Day 894.73
129
6.94s
58.07 Hz
Day 4
898.45
203
4.43
35.83 Hz
Table A.3:
Information about the neuron population of 102 neurons recorded at 24 Hz from the primary visual cortex of behaving mouse the 1st day and 4th day of the experiment.
12

---

Under review as a conference paper at ICLR 2021
Neuron #075
Neuron #075
recorded signal Inferred spike
synth signal
1
Inferred spike
AF/F
AF/F
0
16
25
33
41
50
58
75
83
8
25
33
41
50
58
66
83
Time (s)
Time (s)
Neuron #027
Neuron #027
AF/F
AF/F
0-
0
16
25
33
41
50
58
75
83
8
16
25
33
41
50
58
66
83
Time (s)
Time
(s)
(a)
(b)
Figure A.1: Calcium signals and inferred spike trains (in gray) of randomly selected neurons. shows the DG data (in blue) and (b) shows synthetic data (in orange) generated by CalciumGAN trained on the DG data. Notice that artificial signal data transformed from DG spike data do not have the peak and decay characteristics of typical calcium imaging data. Note: the generated data do not incorporate the trial information, hence the generated traces do not correspond to the recorded signal in the plotted example.
ap
A.1 CALCIUMGAN PIPELINE
In order to train and evaluate our GAN model, we have to first pre-process the calcium signals so that they have standardized format. For calcium imaging data of N neurons with recorded length of L,, we would receive a raw data shape of (I, N). We then used a slicing window of size to segment the data along the time dimension into M segments (see Figure A.3), resulting in matrix with shape (M, T, .N.. To improve the network training performance, we scale the raw calcium signals a to the range [0, 1] before we train our generative model:
T. Z'min
[0001]
(4)
Tmax
Umin
We use ([oo1] to denote datum that has range of [0, 1].
After the above pre-processing step, train CalciumGAN in mini-batches and store 1,000 samples for evaluation. Since we evaluate our model performance terms of spike activities, we needed a deconvolution algorithm infer the spike trains from caleium signals. In this work, we used the OA- SIS deconvolution algorithm (Friedrich et al., 2017) for its fast online deconvolution performance. Prior to inferring the spiking activities from the generated signals á we first have scale the signal back to the same range as the raw calcium signals:
â 100 (Tmox xmiin) + Imin
(5)
We inferred the spike trains from the generated signals as well as the real recorded data with OASIS in order ensure the possible biases of the deconvolution algorithm are the same for both data.
13

---

Under review as a conference paper at ICLR 2021
recorded signals
Whan my myrramamnn
OASIS
segmentation
pre-processing
input X(0.1]
noise
CalciumGAN
synthetic signals
Aamaa Ma
post-processing
OASIS
recorded spikes
synthetic spikes
spike analysis
Figure A.2: Pipeline diagram of CalciumGAN analysis. White boxes illustrate data in different processing stages. Blue boxes illustrate analysis steps and techniques.
14

---

Under review as a conference paper at ICLR 2021
Window size T
Stride
Neuron 1
Neuron 2
Neuron
Neuron 102
Time steps
Figure A.3: Illustration of the sliding window process. The light green and green boxes represent the window with sequence length 7' along the temporal dimension of the calcium signals. We create our training and validation dataset using a window size T 2048 and stride size of s = 2.
A.2 PIASE SHUFFLE
Figure A.4: Illustation of the 1-dimensional Phase Shuffle mechanism. Each box denotes acti- vated unit after convolution layer, and the units are mirrored along the grey dash lines. The large box in light green represents the output of the Phase Shuffle layer with -2
In order to reduce the effect of the "checkerboard" artifact, we adapted the Phase Shuffle mechanism (see 2.1) in the discriminator. In this section examine the effectiveness of Phase Shuffle in terms of the visual quality of the generated traces as well as the effect it had on the inferred spike trains. A common characteristic of the calcium indicators when action potential occur is sharp
15

---

Under review as a conference paper at ICLR 2021
that such characteristic in the calcium traces were more prominent when Phase Shuffle was enabled. We believe that such differences in the generation quality exist mainly because of the repetitive patterns in the transposed convolution layer Odena et al. (2016), since the discriminator can simply distinguish generated samples from real samples by learning if such patterns exists. As the Phase Shuffle mechanism shifts the temporal dimension (by 10 units in our experiment) randomly, it forces the discriminator learn from other features the data instead of the "shortcut" provided by the (undesired) nature of transposed convolution.
onset followed by slow decay in the signal (Fröhlich 2016). In Figure A.5
Moreover, not only did Phase Shuffle affect the visual quality of the generated samples, it also impacted the spike train statistics. The traces generated without Phase Shuffle lack the spiking characteristics, which made more difficult for the deconvolution algorithm register spike the data, thus increasing the inaccuracy of the inferred spike trains. When comparing the KI. divergence of the spike train statistics, the samples generated without Phase Shuffle suffer worse results across the 3 statistics (see Table D especially with mean firing rate.
2.0
2.0
3.0
3.0
2.5
2.5
1.0
2.0
0.5
AFF
1.5
0.0
1.0
0.5
0.5
0.0
0.0-
41
50
58
66
16
25
33
41
66
75
83
Time (s)
(s)
(a)
(b)
Figure A.5: Generated traces of Neuron 6 from a randomly selected sample with (a) PhaseShuffle
10 and (b) PhaseShuffle = O. The sharp rise to peak followed by tail of decaying signal is less observable in when Phase Shuffle disabled.
A.3 DAY RECORDINGS
The following figures are the generated data and spike train statistics of CalciumGAN trained on the calcium imaging recordings collected the first day of the mice experiment.
16

---

Under review as a conference paper at ICLR 2021
Neuron #075
Neuron #075
3
recorded signal inferred spike
synth signal
2
Inferred spike
AF/F
AFIF
1
11
8
16
25
33
41
50
58
66
75
16
25
33
41
50
58
75
83
Time (s)
Time (s)
Neuron #027
Neuron #027
2
2-
AF/F
1
1
0
8
16
33
41
50
58
66
75
16 25
33
41
50
58
66
75
83
Time (s)
Time (s)
(a)
(b)
Figure A.6: Calcium signals and inferred spike trains (in gray) of randomly selected neurons. shows the recorded data (in blue) and (b) shows synthetic data (in orange) generated by Calcium- GAN trained on recorded data. Note: the generated data do incorporate the trial information, hence the generated traces do not correspond to the recorded signal in the plotted example.
a
recorded
synthetic
100
10
H6
80
Neuron
I (nduatt
20
41
62
83
Time (s)
Figure A.7: Raster plot of inferred real and synthetic spike trains of randomly selected sample generated by CalciumGAN trained on recorded data from the first day of the mice experiment.
17

---

Under review as a conference paper at ICLR 2021
Neuron #075
Sample #253
Neuron 077
28-
25-
18
60
500
12-
26-
Count
50
20
3
40-
300-
4.4
30
18
200
22
0.0
Correlation
NAV -ee sererr PT Pirt synthetic PoPtin
Neuron
Sample
20-
1200
13-
30
100
1000
18
Count
Count
60
40-
10
200
10
0.0
0.2
0.4
0.6
0.8
-0.50
-0.25
0.00 0.25
5.5 0.75 3.60
H2
Correlation
synthelic trial
Nouran 8:066
Sample #642
Neuron 2006
roco ded synthetic
recorded synthesic
10
80
200
Count
Count
20
40-
400
1.65
RAW
Aot Ruie "2" ARO synthelic
Correlation
(a)
(b)
(c)
Figure A.8: First and second order statisties of data generated from CalciumGAN trained on the recorded data. Shown neurons and samples were randomly selected. (a) Mean firing rate distribution over 1000 samples per neuron. (b) Pearson correlation coefficient distribution. (c) nan-Rossum distance between recorded and generated spike trains over 45 samples.
200
120
175-
25
200
150-
20
Count
80-
40-
5.
20
6.0
55
1.0 1.5
2.0
0.50
0.05
0.10
6.15
0.25
055
2.0
KL divergennce
KL.
divergence
KL. divergnnce
(a)
(b)
(c)
Figure A.9: KL divergence of recorded data and generated data distributions. mean firing rate of each neuron, (b) pairwise Pearson correlation coefficient and (c) pairwise Ros-uo distance. The mean KL divergence of each statistics are 0.3240, 0.0590 and 0.5106 respectively.
18


Reviews:
[{"review_id": "vcTIi86Qlas", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors construct a GAN network for generating artificial calcium imaging activity from large neural populations. This is an interesting but relatively straightforward application of GANs in a neuroscience context. Unfortunately, I don't see the pressing need for such methods, and the analysis in the paper leaves me unconvinced that the author's method works really convincingly.\n\n* Need: The authors motivate their method with experimental hurdles to recording large ensembles of neurons, but recent years have seen a dramatic improvement in this respect (see e.g. work of Steinmetz, Pachitariu, Stringer and colleagues) who routinely record from 1000s of neurons. The authors should carefully think about what their method allows one to do that is not possible with real data.\n\n* Evaluation: I don't understand why the authors chose to evaluate their method on the deconvolved spikes. Surely this is interesting as well, but I would have liked to see actually more of the generated data and more evaluation on the generated data directly. From Fig. 2 it seems e.g. that the noise characteristics of the generated data are quite different than for the true data. Is this the case? The DG generated data is very hard to judge, since neither the artificial or generated data are shown, just summary statistics in Fig. 1.\n\n* Clarity: I would have liked to see a clearer explanation of what goes into the network and what comes out. Somewhere, they mention a sliding window approach - a figure would go a long way.\n\n* What was the role of the Wasserstein objective function? Where others tried? What was their outcome?\n\n* Table 1: How was the DG model for the experimental data estimated? Combining a normal DG with the forward model of Friedrich? \n\n\ntypo: p8: ref to Donahue is having () in wrong place  ", "extra_scores": {}, "extra_text": {}}, {"review_id": "Rl6PAu_iAYT", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\nThe paper proposes to use a GAN framework to generate the realistic neuronal calcium signals, enabling to scale-up the neuronal population activity data. The solution is based on WAVEGAN architecture with Wasserstein distance to train on calcium fluorescent signals. The experiments are performed in comparison to artificial calcium signals with known ground-truth closely resembles the underlying data distribution. The accuracy of the approach, robustness of generated signals from the model are evaluated.\n\nStrengths:\n- The application is very relevant as an increase in calcium imaging samples helps analyze large populations of neurons.\n\nWeaknesses:\n- This paper lacks contribution, and the novelty of the work is limited in terms of methods.\n- Fig.2 does not actually show significant activation locality in the corresponding neurons; the peaks are not present in the generated signal for Neuron 27, similarly for Neuron 75 (completely mismatched with recorded data).\n- This is further confirmed by Fig.3, where the generated markers are scattered across all the neurons in time, the neuronal activity irrelevant to recorded activity.\n- Figures 3, 4, and 5 are not very informative as no legend is included. \n- Here, the author aims to increase more realistic calcium signals for analysis of neuronal population, but how it will be useful for the healthcare/research community in terms of applications is not clear.\n- In Fig.2, the generated recording does not have an inspired spike for the Neuron 27. Please clarify this.\nDetails for reproducing the experiments of dichotomized gaussian data are left unclear.\n\nMajor Comments:\n- Why are the Pearson correlation values low (i.e., between -0.25 and 0.25)?.\n- Please clarify whether the Pearson correlation is calculated between the recorded signal and the generated signal?.\n- What is the performance when the model trained on real calcium signals recorded from the primary visual cortex of behaving mice and tested on artificial data?\n- Suppose, if authors perturb the real calcium signals data, is it affecting the model performance?.\n- Typos:\nAbstract: Here we propose (Here, we propose)\nIntroduction: paragraph-1, high quality (high-quality)\n\nAddressed Concerns:\n- corrected the typos\n- Some figures are updated\n \nNot Addressed:\n- Figure 1b is not uniform as 1a (legend information is missing)\n- As mentioned in weaknesses: the novelty aspect of this work is missing.\n- Further, the authors have not answered question 1 in major comments.\n- Based on Fig 2 & 3, Calcium GAN generated signals are mismatching with recorded data. How one can deploy this model in real-time when model results are not identical at all?.\n ", "extra_scores": {}, "extra_text": {}}, {"review_id": "9djLv1bTn5H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This manuscript proposes a GAN-based approach to generate calcium traces of neurons by using adapting methods from Wasserstein GANs and WaveGAN to produce the traces.  Using OASIS to extract spikes gives similar patterns to real data on a variety of metrics. This is confirmed over a relatively large number of neurons.\n\nThe big pitfall of this manuscript is that the authors have not demonstrated the scientific utility of their method.  The purported rationale, \"Generative models of neuronal activity hold the promise of alleviating the above problem by enabling the synthesis of an unlimited number of realistic samples for assessing advanced analysis methods,\" is not shown.  The motivation about how generative approaches can provide additional data to find higher-order relationships is fraught: perhaps the advanced analysis techniques would find more patterns by augmenting fake data, but it doesn't really increase the scientific confidence nor address the fundamental uncertainty due to limited data.  While there are some cases where research groups have used GANs to help advanced analysis techniques, such as in domain adaptation and semi-supervised learning, those cases are not shown here, nor is it clear how to build them in.  This paper needs to be revised with a clear use case to motivate it.\n\nPros:\n\nThe manuscript is clearly written, and shows an effective use of GANs to mimic scientific data.\n\nSignificant exploration of the results are shown.\n\nCons:\n\nThe metrics shown compare derived properties of the traces, not the traces themselves.  This is limiting.  In Figure 2, the real data and synthetic data have clearly different noise levels and background patterns.  If the results are visually different in a relatively small figure, it points that more exploration of the traces themselves and the generative quality needs to be explored.\n\nThe title suggests that \"calcium imaging data\" is being generated, but that's not really true.  Instead, the calcium traces on individual neurons are being generated, which is a derived product from calcium imaging data.  Perhaps this is just semantics, but it's misleading and should be clarified up front.\n\nAs in the major point, no clearly elucidate use case.\n\nThe lack of conditioning (on trials, behavior, unlying spike train) is a major limitation, since for most scientific applications we care about recovering these properties.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The approach proposed here have raised major concerns from multiple reviewers especially concerning the novelty and the experimental validation procedure. Authors did not succeed in convincing reviewers of the value of their work for ML or calcium imaging processing.
Decision: Reject