Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 3: Clear rejection
review_content: This manuscript explores a mathematical framework and theory for a signal encoding/decoding scheme that shares many similarities to sparse deconvolution algorithms.  They demonstrate that they can learn a compressed representation of this signal, and there are several interesting theoretical properties.

Pros:
Some theoretical properties show intriguing properties
Seems to work well empirically

Cons:
The paper needs a significant revision for clarity.
I am not confident in all of the theoretical analysis.

I have some concerns about the theory.  First, there is a mismatch between the model definition in (1) and the theoretical analysis in Section 4.  The theory only holds if delta_j->0, which isn't discussed at all, and isn't what was used in the experiments. Or at least this is what is shown by the definitions given in (3) and (4).  This must be discussed at a minimum.

The "Perfect Reconstruction Theorem" is achieved by limiting the class to signals produced by the decomposition.  It is unsurprising that the system can decompose signals produced by it.  Equation 5 is fairly well-known from deconvolutional models.  The relationship to existing work needs to be more clearly defined, so that the contributions can be considered in context.

I'm confused by the claimed innovation in Lemma 3.  $X^*$ is defined at the minimum MSE for a number of spikes N.  Thus, it is the minimum energy and any other solution has equal or greater error.

Shouldn't equation 2 look at reconstruction error and not simply the minimum?
review_title: Interesting ideas but needs to clarify theory and contributions

final_score: rating: 5: Marginally below acceptance threshold
review_content: In this paper, the authors describe a framework for stimulus encoding and reconstruction using spike trains. They characterise a class of signals where the proposed technique is guaranteed to yield perfect reconstruction. Finally, they provide examples of application to audio signals, and they compare their technique with existing sparse coding approaches. 

Overall, the paper is convincing - the authors provide detailed theoretical grounding for their technique, and I wasn't able to spot any weakness from the formal standpoint. I only have one technical remark, and a few minor ones. Therefore, my initial recommendation is for the paper to be accepted.

### Technical remark: 
From Lemma 1, stimulus reconstruction requires the values of the threshold functions at the time of spiking; spike times only are not sufficient. This information is typically not considered available in classic applications of spike coding, as a spike train is by definition just a set of spike times. This is not an obstacle to the practical application of the framework presented here, as one could always imagine transmitting the values of the threshold function along with the spike time when needed. On the other hand, it does make the claim that the framework operates with spike trains misleading, strictly speaking. I suggest that the authors highlight this in the introduction, title and/or abstract, for instance by talking about "marked" spike trains (by analogy to marked point processes), which carry additional information beyond the spike times. 

###  Minor remarks: 

1. Since this is a framework for stimulus reconstruction from spikes, it would have been good to acknowledge in the introduction the existing work on the same topic (though with different tools) done in the neuromorphic computing community. I encourage the authors to take a look at Gallego et al 2020 for a recent review. 
2. At the beginning of Section 6: "the computational resources necessary to code and reconstruct (three dimensional) video signals is estimated to be an order of magnitude higher than (one dimensional) audio signals". Estimated by who? What exactly is meant here by "resources"? please provide references. 
3. The annotations and text in the figures (labels, legends, etc) is so small as to be almost unreadable. Please make it larger. 


###  References 
G. Gallego et al., "Event-based Vision: A Survey," in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2020.3008413. 


-----
Post feedback edit:

The authors did not address the (admittedly, small) terminology issue I raised in my review. More specifically, I argued that their method in general requires transmission of the value of the threshold function, to which they replied that this is not true if the threshold function has a particular form (not required by their general theory), and then added some discussion about removal of spikes which was not what I was getting at. My point still stands: in general, for arbitrary threshold functions, transmission of the value of the function is needed for succesful decoding, and this constitutes more information than what is generally meant when discussing spike trains (which are just sets of time stamps).

Moreover, I was not convinced by the authors' reply to the points raised by Rev 4. This is actually more important than the point above, since Rev 4's objections are on the substance of the method (unlike my own points, which are little more than presentation and terminology matters).

Therefore, I am lowering my score.

I am not convinced by the authors' reply to Ref 4's comments. Moreover, the smaller terminology issue I raised in my own comment (about the
review_title: Signal Coding and Reconstruction using Spike Trains 

final_score: rating: 7: Good paper, accept
review_content: PROS:
* new
* both theoretical and experimental results
* an alternative to matching pursuit, more accurate in certain regimes, which could have a broad range of applications

CONS:
* fully deterministic (see below)

The authors propose a new way to encode a broad class of continuous signals (those with finite rate of innovation, which include bandlimited signals) into spike trains, and to reconstruct those signals from the spikes. The reconstruction is exact under certain hypotheses: the signal should be a weighted sum of the kernels used by the neurons with some temporal shifts. In general several signals are compatible with a given spike train, and the algorithm finds the one with minimal energy.

This algorithm is an alternative to matching pursuit, and is shown to be more accurate in some cases (Fig 4).

In my opinion these results are new and worth sharing. I have only a few minor suggestions to improve the paper:
* How would the reconstruction degrade with noise in the spike trains, e.g. temporal jitter or extra/missing spikes?
* Noise in the (filtered) signal can be beneficial in some cases, because subthreshold signals can then cause spikes, with a higher rate if the denoised signal is near threshold. So the subthreshold signal can be estimated from the spike rates - something that would be impossible in the absence of noise. This is called stochastic resonance, and I wonder if the theory presented here could shed light on it.
* The author uses a soft refractory period, i.e. an adapting threshold, that is increased after a spike, and then goes back to the baseline linearly. From a biological point of view, an exponential decay would be more realistic. Could the theory cope with such an exponential decay?

MINOR POINTS:
* The abstract says "the transformation from external stimuli to
spike trains is essentially deterministic". This is highly debated!
* Eq 3 = Eq 4


review_title: A new algorithm to encode continuous signals into spike trains, and to reconstruct the original signals from those spike trains.

final_score: rating: 3: Clear rejection
review_content: The authors describe a method for representing a continuous signal by a pulse code, in a manner inspired by auditory processing in the brain. The resulting framework is somewhat like matching pursuit except that filters are run a single time in a causal manner to find the spike times (which would be faster than MP), and then a N*N least squares problem is solved (which makes it slower). The authors claim that their method will perfectly reconstruct signals of finite innovation rate, however there appear to be mathematical errors in the proof.

First, while any function in the class G defined in section 4 has finite innovation rate, not all finite innovation rate functions are of this form. For example, suppose there was only one kernel K, which was band-limited, having no power above a frequency f. Then no signal with power above f can be represented as a finite sum of shifted copies of K (or even an infinite sum). Yet these signals can still have finite innovation rate (for example if band-limited to 2f).

Second, the use of the representer theorem appears to be incorrect. The L^2 norm defines a Hilbert space, but not a reproducing kernel Hilbert space as required for the representer theorem.  The representer theorem does not concern convolutions of L^2 functions with a kernel K, but an optimization over members of an RKHS whose norm is defined by K, whose objective function is defined by values at a finite set of points.

As a counter-example to the claimed perfect reconstruction theorem, consider the single boxcar kernel K(x) = {0 if x<0; 1 if 0<=x<=1; 0 if x>1}. Let the signal X(x) = K(x), which belongs to the class G with N=1, t_1 = 1, and alpha_1 = 1. Let the threshold T=0.5. Then the first spike will come at time 0.5; depending on the refractory period there may be any other number of spikes, but there will in general not be a spike at time 1. Thus the original signal is not in the class defined by equation (3).

In summary, this paper describes a potentially interesting biologically-inspired algorithm, but at least some of the claims appear to be incorrect.  

review_title: SIGNAL CODING AND RECONSTRUCTION USING SPIKE TRAINS

Meta-review: The paper presents a mathematical framework for encoding and decoding continuous valued signals from spiking neurons, proving both mathematical theory and simulation results. Two reviewers had serious concerns about the mathematical correctness and exactness of some of the presented mathematical results, and the AC raised questions about relationship to (un-cited) prior work. The authors aimed to address these shortcomings in the discussion phase, but neither me nor the reviewers were convinced that it provides a correct, and substantial, advance over prior approaches. I do hope that the feedback from the process will useful to the reviewers, and will help them in clarifying their contributions.
Decision: Reject