Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: The submission describes an adaptive spiking neuron model that is based on the Laplace transform of the model output.
This reformulation allows to train a recurrent neural network of spiking neural network with good performances.
Three tasks are proposed to assess the recurrent net, on synthetic data and real signal.

I think this contribution could not be accepted for a methodological problem: the main idea is to use neuromorphic chips that use spiking neural networks with an approximation that is basically similar to an artificial neural network. The authors need to introduce a complex rescaling of the time step, repercussion on the computation. This point should be shown experimentally. The authors introduced a modification of the aI&F and are applying low pass filtering the spike trains. 

As all the results are obtained on a python simulator, it is difficult to assess the interest and the applicability to real neuromorphic chips.
As pointed out by the authors, the main interest of these architectures are the lower energy consumption than CPU/GPU-based architecture.
Unfortunately, it is not possible to assess if the proposed neuron model is working on energy efficient architecture.


[1] Nair, M. V. and Indiveri, G. (2019). An ultra-low power sigma-delta neuron circuit. In 2019 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1–5.

review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: The paper promises efficient training of spiking neuron models using back-propagation. The authors say that this is important because spiking networks offer significant energy savings, yet they typically perform poorly compared to prevalent artificial neural networks (ANNs). They aim to support this with several synthetic and one real-world task.

I selected to reject the paper based on the support for the main claim of the paper being weak and inadequately demonstrated. No comparisons are to baselines are presented in the text. Reproducibility is hampered by a great deal of the experimental details missing.

On the experimental results:
* In section 4.1, some justification is needed for why these experiments are relevant. 
* In sections 4.4 and 4.5, experiments are discussed, but results are not presented.
* If space is a limitation, I would strongly prefer a table of actual results for sections 4.4 and 4.5 rather than Figure 2.
* Since the encoding of the task in the spiking and non-spiking cases is so drastically different, making that crystal clear would be great.

Other comments:
* The goal of energy savings is very appealing, so it would help to give *some* quantitative measure of savings promised.
* Similarly, the claim about the poor performance of spiking neurons would benefit from some quantitative measures.
* A figure showing the operation of the novel SigmaDelta neuron, compared to the standard aI&F neuron, would be welcome.

Overall I think the paper would benefit from focussing much more on the results, making the claims as quantitatively apparent as possible. 


review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper deals with neuromorphic computing architectures for solving deep learning problems. Given that the energy-efficiency is a critical need for modern machine learning tools, this is an important study in my opinion. Until now, most of the spiking neuron based works don't really apply for realistic deep model learning. Therefore, advancements such as these would be very useful. 

However, my biggest concern is that the paper may be a bit difficult to grasp for the general ML audience. I myself am not an expert in this topic and had quite a bit of difficulty as an ML researcher in understanding parts of the paper. As a matter of fact, some of the supplementary material was quite helpful to understand the entire context. So, I suggest to move some of the basics from there to the main paper. Also, some of the performance comparison results in the supplementary section are more convincing and clear compared to the results in the main paper. 

Based on my understanding, the algorithmic approach seems logical and the empirical results are convincing. However, I am not in a position to assess the level of novelty. So, I am giving the benefit of the doubt to the authors, given that this is a critical topic for the ML community.   
review_title: Official Blind Review #1

Meta-review: This paper presents a new mechanism to train spiking neural networks that is more suitable for neuromorphic chips.
While the text is well written and the experiments provide an interesting analysis, the relevance of the proposed neuron models to the ICLR/ML community seems small at this point. My recommendation is that this paper should be submitted to a more specialised conference/workshop dedicated to hardware methods.
Decision: Reject