Title: LEARNING SPATIO-TEMPORAL REPRESENTATIONS USING SPIKE-BASED BACKPROPAGATION

Abstract: Spiking neural networks (SNNs) offer a promising alternative to current artificial neural networks to enable low-power event-driven neuromorphic hardware. However, training SNNs remains a challenge primarily because of the complex non-differentiable neuronal behavior arising from their spike-based computation. In this paper, we propose an algorithm to train spiking autoencoders on regenerative learning tasks. A sigmoid approximation is used in place of the Leaky Integrate-and-Fire neuron's threshold based activation during backpropagation to enable differentiability. The loss is computed on the membrane potential of the output layer, which is then backpropagated through the network at each time step. These spiking autoencoders learn meaningful spatio-temporal representations of the data, across two modalities -audio and visual. We demonstrate audio to image synthesis in a spike-based environment by sharing these spatio-temporal representations between the two modalities. These models achieve very low reconstruction loss, comparable to ANNs, on MNIST and Fashion-MNIST datasets, and while converting TI-46 digits audio samples to MNIST images.

1. INTRODUCTION:

In recent years, Artificial Neural Networks (ANNs have become powerful computation tools for complex tasks such as pattern recognition, classification and function estimation problems (LeCun et al., 2015) . They have an "activation" function as their neuron (compute unit). These functions are mostly sigmoid, tanh, or ReLU (Nair & Hinton, 2010) and are very different from a biological neuron. Spiking neural networks (SNNs), on the other hand, are recognized as the "third generation of neural networks" (Maass, 1997) , with their "spiking" neuron model (or compute unit) much closely mimicking a biological neuron. They have a biologically plausible architecture that can potentially achieve high computational power and efficient neural implementation (Ghosh-Dastidar & Adeli, 2009; Maass, 2015) . However, training methods for these spiking neural networks (Jin et al., 2018; Sengupta et al., 2018) are still in an early development stage, where each method comes with its own advantages and challenges.

For any neural network, the first step of learning is the ability to encode the input into meaningful representations. We investigate how input spike trains can be processed and encoded into meaningful hidden representations in a spatio-temporal format of output spike trains which can be used to recognize and regenerate the original input. Autoencoders are very effective tools to learn underlying representations of data, especially visual data (Vincent et al., 2008) . Their simple two-layer structure makes them easy to train as well. In the domain of SNNs, autoencoders provide an exciting opportunity for implementing unsupervised feature learning.

One way to train spiking autoencoders is by using Spike Timing Dependent Plasticity (STDP) (Sjöström & Gerstner, 2010) , an unsupervised local learning rule in which the weight update of a synapse is dependent on when the pre-neuron and post-neuron of the synapse spike with respect to each other. Burbank (2015) and Tavanaei et al. (2018) use STDP based learning rules to train autoencoders on MNIST and natural images. However, STDP, being unsupervised and localized, still fails to train SNNs to perform at par with ANNs. Another approach is derived from ANN backpropagation; the average firing rate of the output neurons is used to compute the global loss (Bohte et al., 2002; Lee et al., 2016) . Rate-coded loss fails to include spatio-temporal information of the network, as the network response is accumulated over time to compute the loss. Wu et al. (2018) applied backpropagation through time (BPTT) (Werbos, 1990) , while Jin et al. (2018) proposed a hybrid backpropagation technique to incorporate the temporal effects. However, it continues to be a challenge to accurately map the time-dependent neuronal behavior with a time-averaged rate coded loss function.

Apart from firing rate, spiking neurons can be characterized by their internal state, referred to as the membrane potential (V mem ). The V mem changes over time depending on the input to the neuron, and whenever it exceeds a threshold, the neuron generates a spike. Thus the firing rate of the spiking neuron is regulated by its membrane potential. Panda & Roy (2016) first presented a backpropagation algorithm for spiking autoencoders that uses V mem of the output neurons to compute the loss of the network. They proposed an approximate gradient descent based algorithm to learn hierarchical representations in stacked convolutional autoencoders. In this work, we compute the loss of the network using V mem of the output neurons, and we incorporate BPTT (Werbos, 1990) to compute the gradients, as membrane potential integrates over time, and use a sigmoid function approximation for the discontinuous step function relation between neuron output and it's internal state (V mem ).

Generally, an autoencoder can learn the hidden representations of data belonging to one modality only. However, the information surrounding us presents itself in multiple modalities -visual, audio, sensory perception. We learn to associate sound, visuals and other sensory stimuli to one object. For example, "apple" when shown as an image, or as text or heard as an audio holds the same meaning for us. A learning system should be capable of learning shared representation of multimodal data (Srivastava & Salakhutdinov, 2012) . Wysoski et al. (2010) proposed a bimodal SNN model that performs person authentication using speech and visual (face) signals. In this work, we explore the possibility of two sensory inputs -audio and visual, of the same object, learning a shared representation using multiple autoencoders, and then use this shared representation to synthesize images from audio samples.

The main contributions of this work are 1. We propose an algorithm to train spiking neural networks that computes loss based on membrane potential of the output neurons. This loss is then backpropagated through the layers, with sigmoid approximation for the derivative of the neuron activation with respect to its membrane potential. The algorithm is verified by training autoencoders on MNIST and Fashion-MNIST and benchmarked against ANNs trained with mini-batch stochastic gradient descent (SGD). 2. We demonstrate that in a spike-based environment, inputs can be transformed into compressed spatio-temporal spike maps, which can be then be utilized to reconstruct the input later, or can be transferred across network models, and data modalities. A spiking autoencoder is used to generate compressed spatio-temporal spike maps of images (MNIST). For audio-to-image synthesis, a spiking audiocoder learns to map audio samples to compressed spike map representations, which are then converted back to images with high fidelity using the spiking autoencoder. This is the first work to perform audio to image synthesis in a spike-based environment. 3. We observe that in presence of sparse/quantized data, a spiking autoencoder can potentially outperform an ANN in reconstruction task, as seen by training spiking audiocoders on single/two bit spatio-temporal compressed spike maps of MNIST images. This could be a result of the ability of SNNs to learn with sparse data because of the temporal nature of their computation that is carried out over several time steps.

The paper is organized in the following manner: In Section 2, the neuron model, the network structure and notations are introduced. The backpropagation algorithm is explained in detail. This followed by the Section 3 where the performance of these spiking autoencoders is evaluated against several benchmarks. We illustrate our Audio to Image synthesis model and evaluate it for converting TI-46 digits audio samples to MNIST images. Finally, in Section 4, we conclude the paper with discussion on this work and its future prospects.

2. TRAINING SPIKING AUTOENCODERS:



2.1. INPUT AND NEURON:

A spiking neural network differs from a conventional ANN in two main aspects -inputs and activation functions. For an image classification task, for example, an ANN would typically take the raw pixel values as input. However, in SNNs, inputs are binary spike events that happen over time.

There are several methods for input encoding in SNNs currently in use, such as rate encoding, rank order coding and temporal coding (Wu et al., 2007) . One of the most common methods is rate encoding, where each pixel is mapped to a neuron that produces a Poisson spike train, and its firing rate is proportional to the pixel value. In this work, every pixel value of 0 -255 is scaled to a value between [0, 1] and a corresponding Poisson spike train of fixed duration, with a pre-set maximum firing rate, is generated (Fig. 1 ).

Figure 1 : The input image is converted into a spike map over time. At each time step neurons spike with a probability proportional to the corresponding pixel value at their location. These spike maps, when summed over several time steps, resemble the original input

The neuron model is that of a leaky integrate-and-fire (LIF) neuron. The membrane potential (V mem ) is the internal state of the neuron that gets updated at each time step based on the input of the neuron, Z [t] (eq. 1).The output activation (A [t] ) of the neuron depends on whether V mem reaches a threshold (V th ) or not. At the time instant when V mem ≥ V th , the neuron spikes (eq. 2). At any time instant, the output of the neuron is 0 if it has not spiked, or 1 if it has spiked. The leak factor is determined by a constant α. After a neuron spikes, it's membrane potential is reset to 0. Fig. 2b illustrates a typical neuron's behavior over time in an SNN.

V [t] mem = (1 -α)V [t-1] mem + Z [t]

(1)

A [t] = 0, V [t] mem < V th 1, V [t] mem ≥ V th (2)

The activation function ( eq. 2) is non-differentiable with respect to V mem , and hence we cannot take its derivative during backpropagation. For backpropagation, the derivative of A [t] is approximated as the derivative of a sigmoid, (A

[t]

apx ), which is centered around V th (eq. 3, 4).

A [t] apx = 1 1 + exp(-(V [t] mem -V th ))

(3)

∂A [t] ∂V [t] mem ≈ ∂A [t] apx ∂V [t] mem = exp(-(V [t] mem -V th )) (1 + exp(-(V [t] mem -V th ))) 2 (4) 2.2 NETWORK MODEL

The autoencoder used here is a two layer fully connected feed forward network. To evaluate our proposed training algorithm, we have used two datasets -MNIST (LeCun et al., 1998) and Fashion MNIST (Xiao et al., 2017) . For both the datasets the input and the output layers have 784 neurons each, which is the input size of the datasets. The number of layer (1) neurons varies with dataset. The input neurons (layer (0) ) are mapped to the image pixels in a one-to-one manner and generate ). The input value governs the firing rate of the neuron, i.e. number of times the neuron output is 1 in a given duration. (b) The membrane potential of the neuron integrates over time (with leak). As soon as it crosses V th , the neuron output changes to 1, and V mem is reset to 0. For taking derivative during backpropagation, a sigmoid approximation is used for the neuron activation the Poisson spike trains. These autoencoders later form the building blocks of the audio-to-image synthesis network. The description of the network and the notation used throughout the paper is given in Fig. 2a .

2.3. BACKPROPAGATION USING MEMBRANE POTENTIAL:

In this work, loss is computed using the membrane potential of output neurons (Panda & Roy, 2016) at every time step and then it's gradient with respect to weights is backpropagated for weight update. The input image is provided to the network as 784×1 binary vector over T time steps, represented as X (t) spike . At each time step the desired membrane potential of the output layer is calculated (eq. 5). The loss is the difference between the desired membrane potential and the actual membrane potential of the output neurons. Additionally a masking function can be used that helps us focus on specific neurons at a time. The mask used here is bitwise XOR between expected spikes (X

[t]

spike ) and output spikes (A (2) [t] ) at a given time instant. The mask only preserves the error of those neurons that either were supposed to spike but did not spike, or were not supposed to spike, but spiked. It sets the loss to be zero for all other neurons. We observed that masking is essential for training in spiking autoencoder as shown in Fig. 3b .

O [t] = V th .X [t] spike (5) mask = bitXOR(X [t] spike , A (2)[t] ) (6) Error = E = mask.(O [t] -V (2)[t] mem ) (7) Loss = L = 1 2 |E| 2 (8)

The weight gradients, ∂W ∂L , are computed by back-propagating loss in the two layer network as depicted in Fig. 2a . We derive the weight gradients below.

∂L ∂V:

(2)[t] mem = -E (9) From eq. 1, ∂V (2)[t] mem ∂W (2) = (1 -α) ∂V (2)[t-1] mem ∂W (2) + A (1)[t] T (10)

From eq. 9 -10 and applying the chain rule.

∂L ∂W (2) = ∂L ∂V (2)[t] mem ∂V (2)[t] mem ∂W (2) = -E (1 -α) ∂V (2)[t-1] mem ∂W (2) + A (1)[t] T (11) From eq. 1, ∂V (2)[t] mem ∂Z (2)[t] = I (12)

From 9 and 12, we obtain the local error of layer (2) with respect to the overall loss which is backpropagated to layer (1) .

δ 2 = ∂L ∂Z (2)[t] = I(-E) = -E (13)

Next, the gradients for layer (1) are calculated.

EQUATION

From 13 -16,

EQUATION

Thus, equations 11 and 17 show how gradients of the loss function with respect to weights are calculated. For weight update, we use mini-batch gradient descent and a weight decay value of 1e-5. We implement Adam optimization (Kingma & Ba, 2014), but the first and second moments of the weight gradients are averaged over time steps per batch (and not averaged over batches). We store

∂V (l)[t]

mem ∂W (l) of the current time step for use in next time step. The initial condition is,

∂V (l)[0]

mem ∂W (l) = 0. If a neuron spikes, it's membrane potential is reset and therefore we reset

∂V (l,m)[t]

mem ∂W (l) to 0 as well, where l is the layer number and m is the neuron number.

3. EXPERIMENTS:



3.1. REGENERATIVE LEARNING WITH SPIKING AUTOENCODERS:

For MNIST, a 784-196-784 fully connected network is used. The spiking autoencoder (AE-SNN) is trained for 1 epoch with a batch size of 100, learning rate 5e-4, and a weight decay of 1e-4. The threshold, V th , is set to 1. We use define two metrics for network performance. Spike-MSE is the mean square error between the input spike map and the output spike map, both summed over the entire duration. MSE is the mean square error between the input image and output spike map summed over the entire duration. Both are normalized, zero mean and unit variance, and then the mean square error is computed. The duration of inference is kept same as the training duration of the network. It is observed in Fig. 3a that a leaky neuron (LIF) performs better than a neuron without any leak (α = 0), i.e an Integrate-and-Fire (IF) neuron. We use Spike-MSE as the comparison metric, to observe how well the autoencoder can recreate the input spike train. Going forth, we set the leak coefficient at 0.1 for all subsequent simulations. Fig. 3b shows that using a mask function is essential for training this type of network. Without a masking function, all of the 784 neurons are being forced to have membrane potential of 0 or V th , which makes training difficult. With a masking For comparison with ANNs, a network (AE-ANN) of same size (784×196×784) is trained with SGD, both with and without Adam optimizer (Kingma & Ba, 2014) on MNIST for 1 epoch with a learning rate of 0.1, batch size of 100, and weight decay of 1e-4. When training the AE-SNN, the first and second moments of the gradients are computed over sequential time steps within a batch (and not across batches). Thus it is not analogous to the AE-ANN trained with Adam, where the moments are computed over batches. Hence, we compare our network with both variants of the AE-ANNs. The AE-SNN achieves better performance than the AE-ANN trained without Adam; however it lacks behind AE-ANN optimized Adam as shown in Fig. 4a . Some of the reconstructed MNIST images are depicted in Fig. 4b . One important thing to note is that the AE-SNN is trained at every time step, hence there are 15× more backpropagation steps as compared to an AE-ANN. However at every backpropagation step, the AE-SNN only backpropagates the error vector of a single spike map, which is very sparse, and carries less information than the error vector of the AE-ANN.

Next, the spiking autoencoder is next evaluated on Fashion-MNIST dataset (Xiao et al., 2017) . It is similar to MNIST, and is composed of 28×28 gray-scale images (60,000 training, 10,000 testing) of clothing items belonging to 10 distinct classes. We test our algorithm on two network sizes: 784-512-784 (AE-SNN-512) and 784-1024-784 (AE-SNN-1024). The AE-SNNs are compared against AE-ANNs of the same sizes (AE-ANN-512, AE-ANN-1024) in Fig. 5a . For the AE-SNNs, the duration of input spike train is 60, leak coefficient is 0.1, and learning rate is set at 5e-4. The networks are trained for 1 epoch, with a batch size of 100. The longer the spike duration, the better 

3.2. AUDIO TO IMAGE SYNTHESIS USING SPIKING AUTO-ENCODERS:



3.2.1. DATASET:

For audio samples, the 0-9 digits subset of TI-46 speech corpus (Liberman et al., 1993) is used. The dataset is composed of read utterances of 16 speakers. It has total 4136 audio samples. The dataset is divided into 3500 train samples and 636 test samples, maintaining an 85%/15% train/test ratio. The audio clips were preprocessed using Auditory Toolbox (Slaney, 1998) . They were converted to spectrograms having 39 frequency channels over 1500 time steps. The spectrogram is then converted into a 58500×1 vector of length 58500. This vector is then mapped to the input neurons (layer (0) ) of the audiocoder, which then generate Poisson spike trains over the given training interval. Images are taken from the MNIST, (LeCun et al., 1998) , a dataset of handwritten digits. Two multimodal datasets are prepared as described below. The testing set is kept same for both datasets, composed of 636 audio samples.

1. Dataset A: 10 unique images of the 10 digits is manually selected (1 image per class) and audio samples are paired with the image belonging to their respective classes (one-imageper-audio-class) 2. Dataset B: Each audio sample of the training set is paired with a randomly selected image (of the same label) from the MNIST dataset (one-image-per-audio-sample).

3.2.2. NETWORK MODEL:

The principle of stacked autoencoders is used to perform audio-to-image synthesis. An autoencoder is built of two sets of weights; the layer (1) weights (W (1) ) encodes the information into a "hidden state" of a different dimension, and the second layer (W (2) ) decodes it back to it's original representation. We first train a spiking autoencoder on MNIST dataset. We use the AE-SNN as trained in Fig. 4a . Using layer (1) weights (W [1] ) of this AE-SNN, we generate "hidden-state" representations of the images in the training set of the multimodal dataset. These hidden-state representations are spike trains of a fixed duration. Then we construct an audiocoder: a two layer spiking network that converts spectrograms to this hidden state representation. The audiocoder is trained with membrane potential based backpropagation as described in Section 2.3. The generated representation, when fed to the "decoder" part of the autoencoder, gives us the corresponding image. The network model is illustrated in Fig. 6 3.

2.3. RESULTS:

The AE-SNN used for audio-to-image synthesis task is trained using the following parameters: batch size of 100, learning rate 5e-4, leak coefficient 0.1, weight decay 1e-4, input spike train duration 15, and number of epochs 1. We use Dataset A and Dataset B (as described in section 3.2.1) to train and evaluate our audio-to-image synthesis model. The images that were paired with the training audio samples are converted to Poisson spike trains (duration 15 time steps) and fed to the AE-SNN, which generates a 196×15 corresponding bitmap as the output of layer (1) (Fig. 2a ). This spatio temporal representation is stored. Instead of storing the entire duration of 15 time steps, one can choose to store a subset, such as first 5 or 10 time steps. We use T h to denote the saved hidden state's duration. This stored spike map serves as the target spike map for training the audiocoder, which is a 58500×2048×196 fully connected network. The spectrogram (39×1500) of the audio samples was converted to 58500×1 vector which is mapped one-to-one to the input neurons(layer (0) ). These input neurons then generate Poisson spike trains for 60 time steps. The target map was of T h time steps was shown repeatedly over this duration. The audiocoder is trained over 20 epochs, with a learning rate of 5e-5 and leak coefficient of 0.1. Weight decay is set at 1e-4 and the batch size is 50. Once trained, the audiocoder is then merged with W (2) of AE-SNN to create the audio-to-image synthesis model (Fig. 6 ).

For The duration (T h ) of stored "hidden state" spike train was varied from 15 to 10, 5, 2, and 1. A spike map at a single time step is a 1-bit representation. The AE-SNN compresses an 784×8 bit representation into 196×T h -bit representation. For T h = 15, 10, 5, 2, and 1, the compression is 2.1×, 3.2×, 6.4×, 16× and 32× respectively. Even when the AC-SNN is trained with a much smaller "hidden state", the AE-SNN is able to reconstruct the images without much loss. In Fig. 7a we observe the reconstruction loss (test set) over epochs for training using different lengths of hidden state.

For comparison, we initialize an ANN audiocoder (AC-ANN) of size 58500×2048×196. The AE-ANN trained over MNIST in section 3.1 is used to convert the images of the multimodal dataset (A/B) to 196×1 "hidden state" vectors. Each element of this vector is 16 bit full precision number. In case of AE-SNN, the "hidden state" is represented 196×T h bit map. For comparison, we quantize the equivalent hidden state vector into 2 T h levels. The AC-ANN is trained using these quantized hidden state representations with the following learning parameters: learning rate 1e-4, weight decay batch size 100, epochs 20. Once trained, the ANN audio-to-image synthesis model is built by combining AC-ANN and layer (2) weights (W (2) ) of AE-ANN. The AC-ANN is trained with/without Adam optimizer is paired with the AE-ANN trained with/without Adam optimizer respectively. In Fig. 7b , we see that our spiking model achieves a performance in between the two ANN models, a trend we have observed earlier while training autoencoders on MNIST. In this case, the AC-SNN is trained with T h as 15, while AC-ANNs are trained without any output quantization; both are trained on Dataset A. In Fig. 7c , we observe the impact of quantization for the ANN model and the corresponding impact of lower T h for SNN. For higher hidden state bit precision, the ANN model outperforms the SNN one. However for extreme low precision case, number of bits = 2, and 1, the SNN performs better. This could possibly be attributed to the temporal nature of SNN, where the computation is event-driven and spread out over several time steps. In this work, we proposed an algorithm to train spiking networks, and in Table 1 , we have summarized the results of this work 12 . The proposed algorithm brings SNN performance at par with ANNs for the given tasks. We demonstrate that spiking autoencoders can be used to generate reducedduration spike maps ("hidden state") of an input spike train, which are a highly compressed version of the input, and they can be utilized across applications. This is also the first work to demonstrate audio to image synthesis in spiking domain. While training these autoencoders, we made a few important and interesting observations; the first one is the importance of bit masking of the output layer. Trying to steer the membrane potentials of all the neurons is extremely hard to optimize, and selectively correcting only incorrectly spiked neurons makes training easier. This could be applicable to any spiking neural network with a large output layer. Second, while the AE-SNN is trained with spike durations of 15 time steps, we can use hidden state representations of much lower duration to train our audiocoder with negligible loss in reconstruction of images for the audio-to-image synthesis task. In this task, the ANN model trained with Adam outperformed the SNN one when trained with full precision "hidden state". However, at ultra-low precision, the hidden state loses it's meaning in ANN domain, but in SNN domain, the network can still learn from it. This observation raises important questions on the ability of SNNs to possibly compute with less data. While sparsity during inference has always been an important aspect of SNNs, in this work, we explored how SNNs can be used to compress information into compact spatio-temporal representations and then reconstruct that information back from it. Another interesting observation is that we can potentially train autoencoders and stack them to create deeper spiking networks with greater functionalities. This could be an alternative approach to training deep spiking networks. Thus, this work sheds light on the interesting behavior of spiking neural networks, their ability to generate compact spatio-temporal representations of data, and offers a new training paradigm for learning meaningful representations of complex data.

:

Table 1: Audio-to-Image A: SNN: T h = 15, ANN : no quantization for hidden state

Table 1: Audio-to-Image B: SNN: T h = 10, ANN : no quantization for hidden state

Figures and Tables:

Figure2: The operation of a Spiking Neural Network:(a) The input vector is mapped one-to-one to the input neurons(layer (0) ). The input value governs the firing rate of the neuron, i.e. number of times the neuron output is 1 in a given duration. (b) The membrane potential of the neuron integrates over time (with leak). As soon as it crosses V th , the neuron output changes to 1, and V mem is reset to 0. For taking derivative during backpropagation, a sigmoid approximation is used for the neuron activation

Figure 3: The AE-SNN (784-196-784) is trained over MNIST (60,000 training samples, batch size = 100) and we study the impact of (a) leak, (b) mask, and (c) input spike train duration.

Figure 5: AE-SNN trained on Fashion-MNIST (training examples = 60,000, batch size = 100)

Figure 6: Audio to Image synthesis model using an Autoencoder trained on MNIST images, and an Audiocoder trained to convert TI-46 digits audio samples into corresponding hidden state of the MNIST images.

Dataset A, we compare the images generated by audio samples of a class against the MNIST image of that class to compute the MSE. In case of Dataset B, each audio sample of the train set is paired with an unique image. For calculating training set MSE, we compare the paired image and the generated image. For testing set, the generated image of an audio sample is compared with all the training images in the dataset, and the lowest MSE is recorded. The output spike map is normalized and compared with the normalized MNIST images, as was done previously. Our model gives lower MSE for Dataset A compared to Dataset B (Fig8a), as it is easier to learn just one representative image for a class. The network trained with Dataset A generates very good identical images for audio samples belonging to a class. In comparison the network trained on Dataset B generates a blurry image, thus indicating that it has learned to associate the underlying shape and structure of the digits, but has not been able to learn finer details better. This is because the network is trained over multiple different images of the same class, and it learns what common among them all. Fig.8bdisplays the generated output spike map for the two models trained over Dataset A and B for 50 different test audio samples (5 of each class).

(a) Reconstruction loss of the audio-to-image synthesis model for varying T h (b) Audiocoder (AC-SNN) (T h = 15) vs AC-ANN (16 bit full precision) (c) Effect of training with reduced hidden state representation on AC-SNN and AC-ANN model

Figure 7: The audiocoder (AC-SNN/AC-ANN) is trained over Dataset A, while the autoencoder (AE-SNN/AE-ANN) is fixed. MSE is reported on the overall audio-to-image synthesis model composed of AC-SNN/ANN and AE-SNN/ANN

Summary of results obtained for the 3 tasks -Autoencoder on MNIST, Autoencoder on Fashion-MNIST, and Audio to Image conversion (T = input duration for SNN)

Table content: <table><tr><td>Dataset</td><td>Network Size</td><td colspan="2">Epochs T</td><td colspan="2">Loss (MSE) (test) SNN ANN ANN (with Adam)</td></tr><tr><td>MNIST</td><td>784-196-784</td><td>1</td><td colspan="2">15 0.357 0.226</td><td>0.122</td></tr><tr><td>Fashion-MNIST</td><td>784-512-784 784-1024-784</td><td>1 1</td><td colspan="2">60 0.178 0.416 60 0.140 0.418</td><td>0.300 0.387</td></tr><tr><td colspan="2">Audio-to-Image A 58500-2048-196/196-784</td><td>20</td><td colspan="2">30 0.254 0.408</td><td>0.144</td></tr><tr><td colspan="2">Audio-to-Image B 58500-2048-196/196-784</td><td>20</td><td colspan="2">30 0.543 0.611</td><td>0.556</td></tr><tr><td colspan="2">4 DISCUSSION AND CONCLUSION</td><td/><td/><td/></tr></table>

Peer review (iclr2019):
 In this work, the authors investigate the training of spiking (leaky integrate and fire) neural networks, for which the normally used threshold function is replaced with the sigmoid function for backpropagation. They demonstrate the proposed method for an autoencoder network on MNIST and Fashion-MNIST, as well as an audio to image synthesis task. The reconstruction loss of the ANN and SNN models is compared, typically with an advantage for the ANN model, but on Fashion-MNIST the SNN results look better. The paper addresses learning of deep SNNs, which is a difficult topic with relevance to both the machine learning and computational neuroscience communities. Since the paper does not aim for biological plausibility, the target audience is ML and neuromorphic engineering. For ICLR this is a paper of medium relevance. The originality of the work is low, because approaches that propose to replace the non-differentiable threshold function with a differentiable proxy, in this case membrane potentials, have been known for several years, e.g. "Spiking Deep Networks with LIF Neurons", Eric Hunsberger and Chris Eliasmith, 2015, and "Training Deep Spiking Neural Networks Using Backpropagation", Jun-Haeng Lee et al. 2016. The main novelty is therefore the application to autoencoders, but overall this reduces originality and hence relevance for ICLR. A thorough discussion and differentiation to previous work is missing. Another criticism is that the relevance of the multi-modal experiments for the idea of replacing the threshold function with a sigmoid function is not clear. Instead, the authors could provide additional experiments on classification tasks or investigate the scalability of the idea for deep (more layers) networks (e.g., on CIFAR10 classification). In general, the experimental section could be more tailored towards the central idea of the paper. While some of the parameters are explored in detail, the choice of others is not motivated. For example, the network sizes are just given without further justification, similarly the parameters for ANN training, which might have required more fine-tuning (e.g. learning rates). Figure 3(b) remains strange, because at some batch number the unmasked version simply ceases to work, but this could have been explained without a graph. Especially the comparison to the ANN without ANN would have required more fine tuning of learning rates, thus I do not see much value in the AE-SNN outperforming this method. The resulting reconstructions in Figures 4c look like they have some completely dark and some very strongly firing pixels, which indicates a far-from-perfect reconstruction. Actually the final reconstruction MSE of 0.2 on the binary MNIST images looks rather high. I find it surprising that the SNN outperforms the ANN on Fashion-MNIST, and I don*t really see a reason why this should happen, and the argument in the article is not convincing, neither are the differences explored in detail. I therefore assume this is a bad parameter setting for the ANNs, and would encourage the authors to test and evaluate this in more detail. Pros: + Overall the paper is well written and has a transparent and meaningful structure. The central ideas are laid out in a comprehensive manner. + In general, the training of spiking neural networks remains a challenging task and investigation of potential solutions is relevant to ICLR. Cons: - Low originality and missing comparison to related work - Unconvincing experimental section Minor remarks: * The title is misleading in the sense, that the spatial part of the spatio-temporal representation is not related to actual space, and further, that the backpropagation is not spike-based * The figures appear blurry, and in the case of Fig. 2 is very hard to read. * In Figure 3c the unit of the spike train duration is not given in figure or caption. * There are a number of grammatical errors.