Signal Coding and Reconstruction using Spike Trains
Authors: Anik Chattopadhyay
Arunava Banerjee
Venue:  (2021)
Keywords: spike trains
signal encoding
reconstruction
kernel
representer theorem
compression
convolutional matching pursuit
COMP

Abstract: In many animal sensory pathways, the transformation from external stimuli to spike trains is essentially deterministic. In this context, a new mathematical framework for coding and reconstruction, based on a biologically plausible model of the spiking neuron, is presented. The framework considers encoding of a signal through spike trains generated by an ensemble of neurons via a standard convolve-then-threshold mechanism, albeit with a wide variety of convolution kernels. Neurons are distinguished by their convolution kernels and threshold values. Reconstruction is posited as a convex optimization minimizing energy. Formal conditions under which perfect reconstruction of the signal from the spike trains is possible are then identified. Coding experiments on a large audio dataset are presented to demonstrate the strength of the framework.

Under review as a conference paper at ICLR 2021
SIGNAL CODING AND RECONSTRUCTION USING SPIKE TRAINS
Anonymous authors Paper under double-blind review
ABSTRACT
In many animal sensory pathways, the transformation from external stimuli to spike trains is essentially deterministic. In this context, a new mathematical framework for coding and reconstruction, based on a biologically plausible model of the spiking neuron, is presented. The framework considers encoding of signal through spike trains generated by an ensemble of neurons via a standard convolve-then- threshold mechanism, albeit with wide variety of convolution kernels. Neurons are distinguished by their convolution kernels and threshold values. Reconstruction is posited convex optimization minimizing energy. Formal conditions under which perfect and approximate reconstruction of the signal from the spike trains is possible are then identified. Coding experiments on large audio dataset are presented to demonstrate the strength of the framework.
INTRODUCTION
In biological systems, sensory stimuli is communicated to the brain primarily via ensembles of discrete events that are spatiotemporally compact electrical disturbances generated by neurons, otherwise known as spikes. Spike train representation of signals, when sparse, are only intrinsically energy efficient, but can also facilitate downstream computation I0). In their seminal work, Olshausen and Field (13) showed how efficient codes can arise from learning sparse representations of natural stimulus statistics, resulting in striking similarities with observed biological receptive fields. (19) developed biophysically motivated spiking neural network which for the first time predicted the full diversity of V1 simple cell receptive field shapes when trained on natural images. Although these results signify substantial progress, an effective end to end signal processing framework that deterministically represents signals via spike train ensembles is yet to be laid out. Here we present new framework for coding and reconstruction leveraging a biologically plausible coding mechanism which is superset of the standard leaky integrate-and and-fire neuron model (5).
Our proposed framework identifies reconstruction guarantees for very general class of signals those with finite of innovation (18)as shown in perfect and approximate reconstruction theorems. Most other classes, e.g. bandlimited signals, are subsets of this class. The proposed technique first formulates reconstruction as an optimization that minimizes the energy of the reconstructed signal subject to consistency with the spike train, and then solves it in closed form. We then identify general class of signals for which reconstruction is provably perfect under certain ideal conditions. Subsequently, we present mathematical bound on the error of an approximate reconstruction when the model deviates from those ideal conditions. Finally, we present simulation experiments coding for large dataset of audio signals that demonstrate the efficacy of the framework. In a separate set of experiments on a smaller subset of audio signals we compare our framework with existing sparse coding algorithms viz matching pursuit and orthogonal matching pursuit, establishing the strength of our technique.
The remainder of the paper is structured follows. In Sections 2 and 3 we introduce the coding and decoding frameworks. Section identifies the class of signals for which perfect reconstruction is achievable if certain ideal conditions met. In Section 5 we discuss how in practice those ideal conditions can be approached and provide a mathematical bound for approximate reconstruction. Simulation results presented in Section 5 We conclude in Section g

---

Under review as a conference paper at ICLR 2021
2 CODING
The general class of deterministic mappings (i.e., the set of all nonlinear operators) from continuous time signals to spike trains is difficult to characterize because the space of all spike trains does not lend itself to a natural topology that is universally embraced. The result is that simple characterizations, such as the set of all continuous operators, can not be posited in manner that has general consensus. To resolve this issue, we take a cue from biological systems. In most animal sensory pathways. external stimulus passes through series of transformations before being turned into spike trains(17). For example, visual signal in the retina is processed by multiple layers of non-spiking horizontal, amacrine and bipolar cells, before being converted into spike trains by the retinal ganglion cells. Accordingly, we can consider the set of transformations that pass via an intermediate continuous time signal which is then transformed into a spike train through a stereotyped mapping where spikes mark threshold crossings, The complexity of the operator now lies in the mapping from the continuous time input signal the continuous time intermediate signal. Since any time invariant, continuous, nonlinear operator with fading memory can be approximated by a finite Volterra series operator()). this general class of nonlinear operators from continuous time signals to spike trains can be modeled as the composition of finite Volterra series operator and neuronal thresholding operation to generate a spike train. Here, the simplest subclass of these transformations is considered: the case where the Volterra series operator has a single causal, bounded-timg linear term, the output of which is composed with a thresholding operation of a potentially time varying threshold. The overall operator from the input signal to the spike train remains nonlinear due the thresholding operation. The code generated by an ensemble of such transformations, corresponding to an ensemble of spike trains, is explored.
Formally, we assume the input signal X X()) to be a bounded square integrable function over the compact interval [0, T for some T RR+ i.e., we interested in the class of input signals F = (X) E L T)). Since the framework involves signal snippets of arbitrary length, this choice of T' is without loss of generalization. We assume an ensemble of convolution kernels K = [K*Ij E Z+ S n). consisting of n kernels K3, 1....,P. n. We assume that (² is a continuous function on bounded time interval [0, T), i.e. Vj E (1.....,) .n. K (1) E T),
T E Finally, we assume that K3 has a time varying threshold denoted by (t).
The ensemble of convolution kernels K encodes a given input signal X(I into sequence of spikes {1, K*)), where the 7th spike is produced by the jth kernel K3. at time 4 if and only if: [() - -T)dr- = TT) In our experiments a specific threshold function assumed in which the time varying threshold I²(t) (t) of the jth kernel remains constant at C³ until that kernel produces spike, at which time affer-hyperpolaricatin potential (ahp) is introduced to raise the threshold to a high value M >> C³, which then drops back linearly to its original value within a refractory period 85. Stated formally,
t 8- > 8- (1) M (-C(0))--C) by
T'())
(I)
t ô- - (t)
Where (1) denotes the time of the last spike generated by K3 prior to time L.
3 DECODING
How rich is the coding mechanism just described? We can investigate this question formally by positing a decoding module. The objective of the decoding module is reconstruct the original signal from the encoded ensemble of spike trains. It is worthwhile to mention that to be able to communicate signals properly by our proposed framework, the decoding module needs to be designed in manner so that it can operate solely the spike train data handed over by the encoding module, without explicit access to the input signal itself. Considering the prospect of the invertibility of the coding scheme, we seek a signal that satisfies the same set of constraints the original signal when generating all spikes apropos the set of kernels in ensemble K. Recognizing that such a signal might not be unique, choose the reconstructed signal the one with minimum L2-norm
Formally, the reconstruction (denoted by X (t)) of the input signal X(t) is formulated to be the solution to the optimization problem:
2

---

Under review as a conference paper at ICLR 2021
(1) argmin [
X s.t. J ((T) -) T
(2)
where {(4, Kū)li E N}} is the set of all spikes generated by the encoder. The choice the linchpin of our
of L2 minimization the objective of the reconstruction problem-which framework, as demonstrated in the theorems- can only be weakly justified at the current juncture. The perfect reconstruction theorem that follows provides the strong justification. As it stands, the I,² minimization objective is in congruence with the dictum of energy efficiency in biological systems. The assumption is that, of all signals, the one with the minimum energy that is consistent with the spike trains is desirable. Additionally, an L minimization in the objective of ( reduces the convex optimization problem to solvable linear system of equations as shown Lemmas I and 3, Later we shall show that L Lz-minimization has the surprising benefit of recovering the original signal perfectly under certain conditions.
4 SIGNAL CLASS FOR PERFECT RECONSTRUCTION
To establish the effectiveness of the described coding-decoding model, we have to evaluate the accuracy of reconstruction over class of input signals. We observe that in general the encoding of square integrable signals into spike trains not a one-to-one map; the same set of spikes can be generated by different signals as to result in the same convolved values at the spike times. Naturally, with a finite and fixed ensemble of kernels K, one cannot achieve perfect reconstruction for the general class of signals F as defined in Section 2 We now restrict ourselves to a subset G of the original O K5r Ex-
t). E (1,.... ) Op R. lp E RR N E Z+} } and address the question of reconstruction accuracy. Essentially G consists of all linear combinations of arbitrarily shifted kernel functions. N is bounded above by the total number of spikes that the ensemble K can generate over [0, T]. In the parlance of signal processing, constitutes Finite rafe of Innovation signals (18). For the class G the perfect reconstruction theorem is presented below. The theorem is proved with the help of three lemmas.
class F defined g (X()) F()(
Perfect Reconstruction Theorem:
Let X (t) E & be an input signal. Then for appropriately chosen lime-varying thresholds of the kernels, the reconstruction. X (t), resulting from the proposed coding-
decoding framework is accurate with respect to the L metric, i.e., (I) X X()ll2 O.
Lemma 1: The solution X" (t) to the reconstruction problem given by 2 can be written as:
N
x*(1= = Σ a. Ki
1=1
where the coefficients a; R can be solved from system of linear equations. Proof: An approach analogous to the Representer Theorem (15), splitting a putative solution to 2
into its within the span of the kernels component and a remnant orthogonal component, results in
equation (3). In essence, the reconstructed signal (t)() becomes a summation of the kernels, shifted
to their respective times of generation of spikes, scaled by appropriate coefficients. Plugging (3) into
the constraints 2 gives:
VISieNi
- - t)dr= (t,)
Σ
W=1
Setting b; = (t,) and Pa
f Kit - Ki r)dT results in:
N
VISieN:
Σ
PaROr = b;
(4)
Equation 44) defines system of N equations in N unknowns of the form:
Pa T
(5)
where a = (a........' T= ( TéN (tx))T and P is an x N matrix with elements
Pie = f Kit (/x K3, rddT. Clearly P is the Gramian Matrix of the shifted kernels
3

---

Under review as a conference paper at ICLR 2021
(K" -ta- 1.,, in the Hilbert space with the standard inner product. It is well known that P is invertible if and only if {K" - t)l E 1, ..... . is a linearly independent set. If P is invertible has a unique solution. If, on the other hand, P is not invertible, has multiple solutions. However, as the next lemma shows, every such solution leads to the same reconstruction (t), and hence any value of a that satisfies/5 be chosen. We note in passing that in our experiments have used the least square solution.
D Import: The goal of the optimization problem is find the best object in the feasible set. However, the application of the Representer Theorem converts the constraints into a determined system of unknowns and equations, turning the focus onto the feasible set, effectively changing the optimization problem into a solvable system that results in a closed form solution for the o; s. This implies that instead of solving ( we can solve for the reconstruction from (t) N a; t), where a; is the i-th element of a = P-1T, Here, p-1 represents either the inverse or the Moore- inverse, as the case may be. Lemma 2: Let equation reeultinn from the optimization problem 2 have multiple solutions. Con- sider any two different solutions for a, namely a and az and hence the corresponding reconstruc- tions are given by (t) = Ex= oj,k" -) and X²(t) = -x 02 (t: t), respectively. Then X1 = X2-
Proof: The proof of this lemma follows from the existence of unique function in the Hilbert Space spanned by {K> (4; - -1)li E 2, ...: N) that satisfies the constraint of equation 2 proof is furnished in the appendix A
The details of the
Import: Lemma 2 essentially establishes the uniqueness of solution to the optimization problem formulated in 2 as any solution to equation 5 The proof follows from the fact that the reconstruction is in the span of the shifted kernels { (K* (t, - t)li E 1,2, N} and the inner products of the reconstruction with each of K (t, t) is given (by the spike constraints of 2 Such reconstruction must be unique in the subspace S.
Lemma 3: Let (*) be the reconstruction of an input signal X()) and {(4,, K be the set of spikes generated. Then, for any arbitrary signal (t) within the span of {K*(tt (t:
t)li E (1,2...,,N)] i.e., the set of shifted kernels respective spike times, given by X(t) =
EN a,K5i(,, - 1) the following inequality holds: (Ix - X= s [[x(t) - ((Il
Proof:
[|x(t)) - x(till = ((t(t) -
x())ll
B
First, (i, f)) (X(,), K* 1)) (x (t) ()), 21, N)
= TTh((t,) -Thh Tt,(t)) =0 =0
(Using the constraints in ② & 2
N
N
Second, (A, = (A, Σ (a: a.) k.) - ())
(By Lemma 1 (x) = Σ a, Ks t))
t=1
N
Σ (a: (A) K3 ((-5(411))) -1)) =
==1
[|X(t)
2( B1 ||||| + +IIBI 2 ||411 = [|x(t) Xt))
[X(1)= - ((II 2 (x())-X'())
The implication of the above lemma is quite remarkable. The objective defined in chooses signal with minimum energy satisfying the constraints, deemed the reconstructed signal. However as the lemma demonstrates. this signal also has the minimum error with respect to the input signal in the span of the shifted kernels. This signifies that our choice of the objective in the decoding module not only draws from biologically motivated energy optimization principles, but also performs optimally in terms of reconstructing the original input signal within the span of the appropriately shifted spike generating kernels.
Import:
Corollary: An important consequence of Lemma 3 is that additional spikes in the system do not worsen the reconstruction. For a given input signal X(t) if S1 and S₂ are two sets of spike trains where S₁ S2, the second a superset of the first, then Lemma 3 implies that the reconstruction due to S2 is at least as good as the reconstruction due to S, because the reconstruction due to S is in the span of the shifted kernel functions of S2 S₁ C S2. This immediately leads to the conclusion that
4

---

Under review as a conference paper at ICLR 2021
for given input signal the more kernels we add to the ensemble the better the reconstruction.
Proof of the Theorem: The proof of the theorem follows directly from Lemma 3. Since the input
signal (t) E G, let X be given by: (t) = - ap (tp -t) (ap E tp E RRt,, E z
Assume that the time varying thresholds of the kernels in our kernel ensemble K are set in such a
manner that the following conditions are satisfied: (X(:); (t, - 1)) T (4p) Vip E {1,..., N)
i.e., each of the kernels K3y the very least produces spike time lp against (()) (regardless of
other spikes at other times). Clearly then (t) lies in the span of the appropriately shifted response
functions of the spike generating kernels. Applying Lemma 3 follows that: (t) - x (f)ll2 S
(tx - X (t)]l2 = 0
D
demonstrating the potency of the coding-decoding scheme, this theorem that the coding strategy of sensory neurons be adapted to the statistics of the stimuli- in mathematically concrete terms. Going by the theorem, the spike based encoding necessitates the signals to be in the span of the encoding kernels for perfect reconstruction. Inverting the argument, kernels must learn to adapt to the basis elements that generate the signal corpora for superior reconstruction.
Import: In addition
frames Barlow's 's efficient coding hypothesis that
APPROXIMATE RECONSTRUCTION AND THE EFFECT OF AHP
The perfect reconstruction theorem stipulates the conditions under which exact recovery of signal is feasible in the proposed framework. At first glance, it may seem challenging to meet these conditions for an arbitrary class of natural signals. The concern stems from two difficulties: firstly, given a fixed set of kernels, the input signal may not lie in the span of their arbitrary shifts, and secondly, we may not be able to generate spikes at the desired locations as postulated in the proof of the theorem.
To address these issues, we observe that our decoding model is a continuous transformation from the space of spike trains to I.2 ff-functios,, in the sense that small changes in spike times or slight mismatch of the spiking kernels from the components of the signal, bring about only small changes in the reconstruction. In what follows, we furnish an Approximate Reconstruction Theorem (c) that provides bound on the reconstruction eтror under such deviations. To address the first problem, it is important to choose kernel functions appropriately so that they can represent the input signals reasonably well. One can leverage biological knowledge; for example, it is well-known that auditory filters are effectively modeled using gammatones (14). Hence our experiments in Section 6 on auditory signals were coded using gammatone kernels. Not surprisingly, the reconstructions were excellent. To alleviate the second problem, we observe that spikes can be produced reasonably close to the desired locations by setting a low baseline threshold and small refractory period of the potential (ahp) for each kernel, a technique that is guaranteed to give good
after-hype ceeer--yyepoorraaattiii results as is confirmed by our experiments in Section [6] The following lemma formalizes the notion of how lowering the threshold and the refractory period of a kernel helps in generating spikes at the desired locations. The lemma followed by the Approximate Reconstruction Theorem.
Lemma 4: Let (t) be an input signal. Let KP be a kernel for which we want to generate spike at time tp. Let the inner product (X (t), (ttt, +)) = IP. Then, if the bascline threshold of the kernel KP is C < T and the absolute refractory period is 8 as modeled in Equation I the kernel K must produce a spike in the interval [tp 5, tpl according to the threshold model defined in Equation [ Proof: The proof of this lemma follows directly from the intermediate value theorem and is detailed in appendix B
Approximate Reconstruction Theorem:
Let the following assumptions be true:
X(t), the input signal to the proposed framework, can be written as linear combination of some
component functions as- X(t)= = Σn aufp. (t: - t) where ag are bounded real coefficients, the component functions fp. (t) are chosen from a possibly infinite set 7/ = (f)(t) E Z+, (t) = 1} of functions of unit I,² norm with compact support, and the corresponding 4, E R+ are chosen to be bounded arbitrary time shifts of the component functions so that the overall signal has compact support in [0, T' for some T e R and thus the input signal still belongs to the same class of signals
F, as defined in section 2
There is at least one kernel K from the bag of encoding kernels K, such that the --d-iitance of
fn, (t) from Kd: (t) is bounded. Formally, 3 8 E R+ s.t. Il5 (t) k*t(t)l2 < 8 V E {1,..., N).
When X (t) is encoded by the proposed framework, each one of these kemels K3: produce spike
at time f/ at threshold T. such that |t, - t'] < A Vi, for some A E Rt.
Each kernel K E K satisfies Lipschitz type condition follows:
5

---

Under review as a conference paper at ICLR 2021
3 s.t. (t) Kt At)l22 S ClAt), V R, Vj.
And lastly the shifted component functions satisfy a frame bound type of condition as follows: Exzi (t t.), fr. (t tx)) I 17 Vi N1
Then, reconstruction X* (t), resulting from the proposed framework, has a bounded noise signal
ratio. Specifically, the following inequality is satisfied:
[ (0011/11x(011 < (6 + CA)(+tmarr//11
where Imaz is positive number E N 1] that depends on the maximum overlap of the support of component functions fp. (t t.).
Proof: A detailed proof of this theorem is provided in the appendix C
EXPERIMENTS ON REAL SIGNALS
The proposed framework is general enough apply any class of signals. However, since the computational resources necessary to code and reconstruct video signals (function of three variables- r, y. t) would be sufficiently larger than audio signals (function of only one variable f), to demonstrate that the proposed framework can indeed be adopted in real engineering applications as novel encoding scheme, we ran experiments on a repository of audio signals.
6.1 DATASET
We chose the Freesound Dataset Kaggle 2018 8 (or FSDKaggle2018 for short), an audio dataset of natural sounds posted on Kaggle referred in (7,, containing 18,873 audio files annotated with labels from Google's AudioSet Ontology (9) For the purpose the experiments, we ignored the labels and only focused the sound data, since were only interested in encoding and decoding the input signals. All audio samples in this dataset are provided as uncompressed PCM 16bit, 44. 1Hz,, mono audio files, with each file consisting of sound snippets of duration ranging between 300ms to 30s. In the experiment, we ran our proposed methodology over at least 1000 randomly chosen sound snippets from the samples in the dataset. For ease of computation, we kept the length of the input audio snippets to be relatively small (ideally of size less than 50ms), splitting longer signals. This choice of considering small snippets as input made the computation feasible on limited resource machines within reasonable time bounds by reducing the size of the P-matrix referred to in Equation This choice is without loss of generalisation since for encoding signals of greater length, reconstruction using this framework can be done piece-wise: splitting longer signal into smaller pieces, reconstructing wise and finally stitching the reconstructed pieces together.
6.2
SET OF KERNELS
The proposed encoding technique is operational on a set of kernels, as stated in Equation 2 The first order of business therefore the choice of suitable set of kernels for our experiments. Since gammatone filters are widely used as a reasonable model of cochlear filters in auditory systems (14), fairly simple to --
and mathematically, we chose set of gammatone filters as our kernels (Figure D. The implementation of the filterbank is similar to (16), and we used up to 2000 gammatone kernels whose center frequencies were uniformly spaced on the ERB scale between 20 Hz to 20 kHz. In all experiments, the kernels were normalized, and the baseline thresholds and the ahp parameters were kept the same across all kernels.
cos(2 f'1 d)-in our experiments
6.3 RESULTS
Following the assertion of Lemma 4, in all experiments, the baseline threshold and the absolute refractory period were kept low enough so that for each sound snippet near perfect reconstructions could be obtained at high spike rate. A typical value of the refractory period was &s 5ms and the baseline threshold value was kept as low as 10-3 As consequence of the corollary to Lemma 3, additional spikes did not hurt reconstruction. Experiments were conducted with varying number of kernels. Once reconstruction at high spike rate was attained, greedy technique that removed spikes in order their impact on the reconstruction was instituted to get a compressed code for each snippet. Reconstructions were then recomputed with the fewer spikes constraints. We

---

Under review as a conference paper at ICLR 2021
600d KEEP HeS
6x
Kis os
Figure I: Five sample gammatone filters used as kernels with center frequencies located at approxi- mately (a) 82 Hz, (b) 118 Hz, (c) 158 Hz, (d) 203 and (c) 253 Hz, respectively.
(4)
16)
6c1
16)
8
1g1
Time mm
m
Amolitude
ADC
483
Indionss
Karmal
Figure 2: Reconstruction of sample snippet in an experiment with 2000 kernels. (a) the input snippet, extended with zero padding to accommodate future spikes; (b) Spike trains of kernels obtained at low threshold and refractory period displayed raster plot with time (y-axis) and index the gammatone kernels in increasing order center frequencies (x-axis), and (c) the resulting reconstruction. This is almost perfect reconstruction with 32.7DB SNR at 1146 kHz spike rate the ensemble. Subsequently spikes were deleted greedily (see text) to obtain reconstructions lower spike rates. (d) Resulting spike pattern of the ensemble 88.4KH and (e) resulting reconstruction with SNR 19.7DB. Likewise, (f) spike pattern of the ensemble at 17 77.44Hz and (g) resulting reconstruction with SNR 9DB. Time scale on left apply all plots. noteworthy that in (f)&(f) spikes of the higher frequency kernels ended up deleted in the culling process.
should emphasize here that soon as spikes are removed to get a compressed representation of the signal, signals are longer encoded via simple spike train representation which idcally should communicate only spike times and not their corresponding threshold values. In other words, compressed signal representation in this scheme needs to communicate both the spike times and the threshold values because once spikes are culled the decoder cannot infer the threshold values from the spike times and the given threshold function the neurons equation] [ In that sense a compressed representation of signal in this approach can be realized through marked spike trains that carry both time as well as threshold information rather than true spike train based representation. Figure 2 demonstrates this process applied to sample sound snippet through several stages of removal of spikes. This process was repeated over 1000 randomly chosen sound snippets from the dataset. Figure displays the complete results of the experiment with z 2000 gammatone kernels. As the
7

---

Under review as a conference paper at ICLR 2021
70
70
SMR DB
30
30
20
spike
(a)
(b)
Figure 3: Comprehensive result of an experiment with 2000 kernels reconstructing 1000 sound snippets. The graphs show scatter plots of reconstructions (each dot represents reconstruction) with spike-rate of the ensemble (x-axis) against corresponding SNR value of the reconstructions (y-axis) (a) Scatter plot reconstructions up to 441 kHz spike rate and (b) zoom in the same graph for up to 89kHz. Solid line represents the average SNR of all reconstructions at given spike rate.
figure demonstrates. at high spike rates nearly perfect reconstructions were obtained consistently, and though lowering the spike rate gradually increased noise, reasonable reconstructions could be obtained at low spike rates ( 15DB at 25kHz on average). Since each reconstruction is calculated by solving system of linear equations involving a P-matrix whose dimension is O(N2) where N is the number of spikes under consideration, computation is fairly time consuming, and the choice of parameters, such as the length of the input snippets, the number of kernels the threshold parameters were made to ensure feasibility of computation with available resources while maintaining efficacy of the overall reconstruction process.
Since the proposed framework approximates signal with sparse linear combination of shifted kernels (as shown by signal class G in and therefore has similarities to compressed sensing, another of experiments were designed to compare the proposed framework with existing sparse coding techniques viz, Convolutional Matching Pursuit and Convolutional Orthogonal Matching Pursuit. Since the sparse coding techniques are computationally intensive over continuous signals, this set of experiments were restricted to only 10 gammatone kernels (for CMP and COMP all possible shifts of these 10 kernels were considered) and the experiments were run over 30 sound snippets. Our technique was applied as before, starting at high spike rate with spikes culled gradually to achieve better compression. The results of comparison of average SNR values obtained by the techniques shown in figure 4 As is evident, our technique in its simplest form does slightly better than COMP up until zs 50kH beyond which COMP performs better. Our technique was = 1.2 times faster than COMP on an i7 hexa-core processor for these experiments and should naturally scale much better than COMP since the proposed technique does not involve repeated computation of inner products. The implementation details of the experiment be found in simulation code available at: /ttt:::bbbbbbbceee wwwpe/uwww...tt.o..ooooeoeeooeee crtstacystaaaaaaa www.ysstaooniiioo
ng.git.
7 RELATION TO PRIOR WORK
The problem of representing continuous time signals using ensembles of spike trains has rich history both the neuromorphic computing community as well as computational neuroscience. Most such work rely classical Nyquist-Shannon sampling theory wherein signals are assumed to be band- band-limited and reconstruction is realized through sinc filters, albeit via the spike trains. Among existing spike based coding techniques, (4) has explored the spike generating mechanism of the neuron as an oversampling, noise shaping analog-to-dighh converter, and (IT) represents signals via time encoding machines. Likewise, an image encoding technique has been discussed in (8) using
8

---

Under review as a conference paper at ICLR 2021
20
our framework
MP
15
SNR DB
10
60
80
sampling kHz
Figure 4: Average SNR values plotted functions of sampling rate (spike rate for our framework and number of atoms chosen per second of the snippet duration in case of COMP/CMP).
integrate- therstd--threshol-rsssttt framework that results in spike trains, which leverages differential pulse-code modulation (DPCM) at its core. In our case the input signals considered are elements of I,2 (RR with finite rate of innovation, the reconstruction error of which tends to zero as the signal approaches the span of appropriately chosen kernel functions which are again a generic class of continuous functions. Our work differs from existing approaches in that using our scheme signal reconstruction is realized via a sparse set of idealized spikes, whereas the former case signals need be sampled at a rate higher than the Nyquist rate and reconstruction implicitly relies sinc interpolation. Since the class of signals & considered in our analysis takes the form X(t) Σp a ",,K''(t, (tp t), our problem formulation is comparable to that of convolution sparse coding compressive sensing deconvolution, which, in general, is hard problem and hence is solved under certain relaxed criteria (3) or by using certain greedy heuristics (12). Our framework provides corresponding approximate solution the general problem leveraging biological thresholding scheme to produce spikes simultancously at high rate and then gradually removing unimportant spikes. The proposed technique, therefore, is novel alternative to existing solutions.
8 CONCLUSION
We have proposed a framework that codes for continuous time signals using ensemble of spike trains in a manner that is very different from the pulse-density paradigm. The framework applies to all finite rate of innovation signals, which is very large class that includes bandlimited signals. Although approximate reconstruction is computationally more expensive than interpolation with a sinc kernel (as in Nyquist Shannon), it is feasible, unlike in the case of compressed sensing where the generic case is NP-hard. Fortuitously, the system of linear equations is best solved using the conjugate gradient method since a symmetrie positive semidefinite matrix. The excellent reconstruction results we have obtained with 2000 kernels-we no parameter tuning- is testament to the potential of the technique. The human cochlear nerve, in comparison, contains axons of z 50, 000 spiral ganglion cells (corresponding, therefore, to 50,000 kernels). As our theorems show, reconstruction with such a large set of kernels is guaranteed to even better, albeit a higher computational cost.
REFERENCES
[1] Horace B Barlow. Possible principles underlying the transformations of sensory messages. Sensory Communication, pp. 217-234, 1961.
[2] Stephen Boyd and Leon Chua. Fading memory and the problem of approximating nonlinear operators with volterra series. IEEE Transactions on circuits and systems, 32(11) (11: 1100-111111

---

Under review as a conference paper at ICLR 2021
1985.
[3] E. J. Candes, Romberg, and T. Tao. Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trunsactions on Information Theory, 52 (0:4899509 2006. doi: 10. 1211099711120055822000
[4] Dmitri Chklovskii and Daniel Soudry. Neuronal spike generation mechanism as an over- sampling, noise-shaping a-to-d converter. In F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinherger (eds.), Advances in Neural Information Processing Systems 25, pp. 503-511. 2012.
[5] Peter Dayan and L. F. Abbott. Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. The MIT Press, 2005. ISBN 0262541858.
[6] P. Földiák. Forming sparse representations by local anti-hebbian learning. Biological Cybernet- ics, 64(2):165-170, Dec 1990.
[7] Eduardo Fonseca, Manoj Plakal, Frederic Font, Daniel W. Ellis, Xavier Favory, Jordi Pons, and Xavier Serra. General-purpon tagging of freesound audio with audioset labels: Task description, dataset, and baseline, 2018.
[8] G. Gallego, T. Delbruck, G. Orchard, C. Bartolozzi, B. Taba, A. Censi, S. Leutenegger, A. Davi- son, J. Conradt, K. Daniilidis, and D. Scaramuzza. Event-based vision: survey, IEEE Transactions on Pattern Analysis & Machine Intelligence, (01):1-1, jul 5555. ISSN 1939-3539. doi: 10.1 00.100//TAMM222000000000
[9] F. Gemmeke, D. P. W. Ellis, D. Freedman, A. Jansen, W. Lawrence, R. C. Moore, M. Plakal, and M. Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 776-780, 2017.
[10] Daniel Graham and David Field. Sparse coding the neocortex. Evolution of Nervous Systems, 3:181-187, 2007.
[11] A. A. Lazar and L. T. Toth. Time encoding and perfect recovery of bandlimited signals. In 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)., volume 6, pp. VI-709, April 2003.
[12] S. Mallat and Z. Zhang. Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process., 41:3397-3415, 1993.
[13] David J. Olshausen, Bruno A.and Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 381 381:607-609, Jun 1996.
[14] R. Patterson, Ian Nimmo-Smith, J. Holdsworth, and Rice. An efficient auditory filterbank based on the gammatone function. 01 1988.
[15] Bernhard Schölkopf, Ralf Herbrich, and Alex Smola. A generalized representer theorem. In International Conference on Computational Learning Theory, pp. 416-426. Springer, 2001.
[16] Malcolm Slaney. An efficient implementation of the patterson-hollsworhh auditory filter bank.
11 2000.
[17] L.R. Squire. Fundamental Neuroscience. Academic Press/Elsevier, 2008 9780123740199.
ISBN
[18] M. Vetterli, P. Marziliano, and T. Blu. Sampling signals with finite rate of innovation. IEEE Transactions on Signal Processing, 5066):1417-1425 2002.
[19] Joel Zylberberg, Jason Timothy Murphy, and Michael Robert Weeeese. sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of vl simple cell receptive fields. PLOS Computational Biology. 7(10):1-1,, 10 2011.
10

---

Under review as a conference paper at ICLR 2021
A PROOF LEMMA 2:
Lemma 2: Let equation 5 resulting from the optimization problem have multiple solutions.
Consider any two different solutions for a, namely on and 02 and hence the corresponding Ei= t) and X2(() = 02 K (4: t),
reconstructionn are given by (t) = E (n respectiveds, Then X X2. Proof: Let S' be the subspace of Li-functions spanned by {K> (4+ - t)li E 1, NN with the standard inner product (by assumption each of (K i) 1,2,.... NN. are L²-functions and hence S is subspace of the larger space of all L L'nnttotinss Clearly S is a Hilbert space with (Sm(()) Σ N. Hence there exists fei...., eas }. an orthonormal basis of S (where M S N). Assume that the hypothesis is false, i.e. X1 * X2. This implies that Dass and bis such that (t())= EM a and X²(t)= EM bie; where not all a,s are same as the corresponding b,s. = k such that O% o by = (Xi(t), ex) = ag * by (X2(t), ex)
G, Kh (K:hh(44+ (1:-t) But ex span spann((Ki (1, (1;-t)i E 1,, NJ) = 3 fc -. CN) such that = & - -))= =
= (x)(t),es) = = a (t),
- (t))
Now, since X, (1), are both solutions to the optimization problem
(X) ex) = = (X2(t)) (t - t) = (t) ex)
ex) (x)(1),,x)
-which contradicts the hypothesis.
B PROOF OF LEMMA
Lemma 4: Let (t) be an input signal. Let K2 be a kernel for which we want to generate a spike at time lp. Let the inner product (X),, kP - ()) = IP. Then, if the baseline threshold of the kernel KP is OP < IP and the absolute refractory period is ō as modeled in Equation | the kernll K" must produce a spike in the interval [tp-
- 6, tp] according to the threshold model defined in Equation
The lemma is casily proved by contradiction. Assume that prior to and including time tp. the last spike produced by kernel K" was at time t4- Also assume that t tp 5 so that there is no spike in the interval [t. - 5, tpl. Then by Equation the threshold of kernel K at time ip continuous
Proof:
1)) IP 2 CP. Furthermore, since (P(()) function, the convolution C(I) =f ( -T)T varies continuously. By Equation 1 as the ahp kicks up the threshold to an arbitrarily high value M at time f and falls linearly to the value CP before time tp. by the intermediate value theorem, the threshold must be crossed by the convolution C(t) after time l and before time lp- Since 4 was the last spike of kernel RTD before time lp. this is a c contradiction. Hence, & X ip 5, implying that there must spike in the interval [t. 5, tp)- O
is (1,) - CP. But, (X(1), KP k1
C PROOF OF APPROXIMATE RECONSTRUCTION THEOREM
Approximate Reconstruction Theorem: Let the following assumptions be true:
X(),, the input signal to the proposed framework, can be written as linear combination of some component functions as below:
componee
x(t) Σ
aaff (t: -)
where o are bounded real coefficients, the component functions fp. (t) are chosen from possibly infinite set H. (f((t)i i E compact support, and the corresponding & E R are chosen to be bounded arbitrary time shifts of the component functions that overall signal has compact support in [0, T] for some come? T E R+ and thus the input auputtt signal still belongs to the same class of signals F, as defined in section 2
= 1} of functions of unit L2 norm with
11

---

Under review as a conference paper at ICLR 2021
There is at least one kernel K from the bag of encoding kernels K, such that the L distance of fp. (t) from K (t) is bounded. Formally,
3 3 E R+ s.t. (t) - K* (t)ll2 <óvic E [I,....
When (t) is encoded by the proposed framework, each one of these kernels K produce
spike at time i threshold T, such that It 1 < Vi, for some E R*
Each kernel KJ K satisfies Lipschitz type of condition as follows:
3'E R s.t. (t) - Kt - At)ll2 - Cl v At E R. Vj. And lastly the shifted component functions satisfy frame bound type of condition as follows:
Ex -m)), (t tx)) Sive (1.
Then, reconstruction X (t), resulting from the proposed framework, has a bounded noise signal ratio. Specifically, the following inequality is satisfid::
[|x() (000-xx0011/)xxxx0 < (6 (//01+/mm//111
where Tmoz is positive number E N 1] that depends on the maximum overlap of the support of component functions fp. (1 tz).
Proof of the Theorem: By hypothesis each kernel K produces spike time E Vi E {1, ..., N) Let us call these spikes as fitring spikes. But the coding model might generate some other spikes against (t) too. Other than the set of fitting spikes {tt, K>))l [V) let (TT,, Kk)] E (1,..,,M) denote those extra set of spikes that the coding model produces for input (L) against the kernel bag K and call these extra spikes spurious spikes. Here, M is the number of spurious spikes. By Lemmal (t) can be represented as below:
(- = EM ( En ou Kin -))
where and di are real coefficients whose values can be formulated again from Lemmal. Let Tj be the thresholds which kernel K produced the spike at time tę as given in the hypothesis. Hence for generation of the fitting spikes the following condition must be satisfied:
(t)t) (1,- f)) Ti e {1, (1.,,...,NN N}
(6)
Consider hypothetical signal X hap (t) defined by the equations below:
(ty = Σ K (1) t), R
s.t. (X) K (1,
- TT,
(7)
Clearly this hypothetical signal Xnyp(t) (t) can be deemed as it is the reconstructed signal where we only considering the fitting spikes and ignoring all spurious spikes. Since, X (t) lies in the span of the shifted kernels used in reconstruction of ((t) using Lemma 3 we may now write:
(t) - (t)l 2 (x(t) - Xt)ll
(8)
[I((1) - (nyn(1)) = (X(1) - (1) X(1) - Xnyp (!))
=(((1) XX (()) - (Xx(I) (happ (t)) -(((1) - X (1), X(1)) En"ia;;
1a (X(1) - X (1), K2t (())
= =I|x(00113- - (ttp (t))
(Since by construction (X (h (1)p(1), K3t ()) = TiVi E (1...N))
DN
NN 2 aua (Si(1 [4, (x))
EN
E 1 14), K3x (5-(1-(())) -(,)))
12

---

Under review as a conference paper at ICLR 2021
=a Fa a F
(denote a [a1, 022 ... anl", a -aa-l1,,0 02, aN]"
F - [FaINNNN an N x N matrix, where Fix = (fi(t Selt -t4) and Fx = [[FF))t N where (Fx)a (si(l -1), K (1-1))) But using the results of Lemmal can be written as:
a = P- where P = [PalNXNN Psk = (K* (t - 1) KJ -(4)
And, T= [1 where T; = (X)()))))))))))))))))))
b)
E k== 0z (2), (5 (4) FFaa
7 a= Pa FFa
Plugging this expression of a in equations
9 we get,
[I((1) - (1np = a" Fa a² Fx P F
(10)
But((Fk)t ) (fi(t t2), (t-))))) t(,)
-K-i( 4), (r (1 11))
(K5'(L (K-((-11) (6)) - -Sell - 1.), Kit (4))
(P)ix (Ex)
(11)
(denoting & [[E))an]
where (Ek) = (K (t. 11) - [i(t- (6--t) tt), K (t) (t-)))) -111))
Also, (F)i (Ji(l [x), - -4))
=ff((t t.) - (t (--))+K((tt
falt (x) - -K* (t (t (1) + K (t (22)
tx)-
= (E)x - (EK)sk (Ex) )xi + (P)
(12)
Combining 10 and 12 we get,
(IX(1) - Xnun(1)1 = a" Fa- - a Fx p- 1pprk F
=a Eax -a" EKE - a² Ea a² Pa
aa "'Exa + a² a= &K a² &EE P -
aa a P
S Ea
(Since, P is an SPD matrix, a at ExP-11T & > 0)
(13)
We seek for a bound for the above expression. For that we observe the following:
[( = 1( 4)- KR (k) (x) - K (1 (1)) = [I - t4) - (t- - tx) K (t - t 2.
fallt
(where Fik [0, 1]. We also note that Fa is close to 0 when there is not much overlap in the support of the two components and their corresponding fitting kernels.)
s (ll(fitt - t4) Ki - ++]l+ - - -tt -t (t- (1l - (x) - -K* (1-44)) twll
+ + (t tx) - (t - f))))
(E)a S 15 + CA)²
(14)
Using Gershgorin circle theorem, the maximum eigen value of E: Amar (E) mar; ((E) + ExpI)(())K))
13

---

Under review as a conference paper at ICLR 2021
I (6 C (2 max 1)
(Using 14
(15)
(where Tmas E [0, N - 1] is a positive number that depends on the maximum overlap of the supports of the component signals and their fitting kernels))
Similarly, the minimum eigen value of Fi::: Amoi (F) = min,((F)) EienKfp. (t t.), Sve (t - tx))))
21-1
(16)
(By assumption Ligh| < fp. (t f.), (t tx) > >>> mnn
Combining the results from
13 15 and 16 get:
[|x(0) ( (110//1X0000 a TTTTa Ea/aaFFa Fa
$ Amar (E)/Amsn (F)
I (6 + CA) maa 1)/(1 -n)
(17)
Finally using we conclude,
[ (01 /110///1111111-11 S (X(t)-Xnvz(t))) (/1/0////1/// (
S (6 + ( + +1)/(111
14


Reviews:
[{"review_id": "tFpJKTs8lJ9", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors describe a method for representing a continuous signal by a pulse code, in a manner inspired by auditory processing in the brain. The resulting framework is somewhat like matching pursuit except that filters are run a single time in a causal manner to find the spike times (which would be faster than MP), and then a N*N least squares problem is solved (which makes it slower). The authors claim that their method will perfectly reconstruct signals of finite innovation rate, however there appear to be mathematical errors in the proof.\n\nFirst, while any function in the class G defined in section 4 has finite innovation rate, not all finite innovation rate functions are of this form. For example, suppose there was only one kernel K, which was band-limited, having no power above a frequency f. Then no signal with power above f can be represented as a finite sum of shifted copies of K (or even an infinite sum). Yet these signals can still have finite innovation rate (for example if band-limited to 2f).\n\nSecond, the use of the representer theorem appears to be incorrect. The L^2 norm defines a Hilbert space, but not a reproducing kernel Hilbert space as required for the representer theorem.  The representer theorem does not concern convolutions of L^2 functions with a kernel K, but an optimization over members of an RKHS whose norm is defined by K, whose objective function is defined by values at a finite set of points.\n\nAs a counter-example to the claimed perfect reconstruction theorem, consider the single boxcar kernel K(x) = {0 if x<0; 1 if 0<=x<=1; 0 if x>1}. Let the signal X(x) = K(x), which belongs to the class G with N=1, t_1 = 1, and alpha_1 = 1. Let the threshold T=0.5. Then the first spike will come at time 0.5; depending on the refractory period there may be any other number of spikes, but there will in general not be a spike at time 1. Thus the original signal is not in the class defined by equation (3).\n\nIn summary, this paper describes a potentially interesting biologically-inspired algorithm, but at least some of the claims appear to be incorrect.  \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "i_Vx4D5V7E0", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors describe a framework for stimulus encoding and reconstruction using spike trains. They characterise a class of signals where the proposed technique is guaranteed to yield perfect reconstruction. Finally, they provide examples of application to audio signals, and they compare their technique with existing sparse coding approaches. \n\nOverall, the paper is convincing - the authors provide detailed theoretical grounding for their technique, and I wasn't able to spot any weakness from the formal standpoint. I only have one technical remark, and a few minor ones. Therefore, my initial recommendation is for the paper to be accepted.\n\n### Technical remark: \nFrom Lemma 1, stimulus reconstruction requires the values of the threshold functions at the time of spiking; spike times only are not sufficient. This information is typically not considered available in classic applications of spike coding, as a spike train is by definition just a set of spike times. This is not an obstacle to the practical application of the framework presented here, as one could always imagine transmitting the values of the threshold function along with the spike time when needed. On the other hand, it does make the claim that the framework operates with spike trains misleading, strictly speaking. I suggest that the authors highlight this in the introduction, title and/or abstract, for instance by talking about \"marked\" spike trains (by analogy to marked point processes), which carry additional information beyond the spike times. \n\n###  Minor remarks: \n\n1. Since this is a framework for stimulus reconstruction from spikes, it would have been good to acknowledge in the introduction the existing work on the same topic (though with different tools) done in the neuromorphic computing community. I encourage the authors to take a look at Gallego et al 2020 for a recent review. \n2. At the beginning of Section 6: \"the computational resources necessary to code and reconstruct (three dimensional) video signals is estimated to be an order of magnitude higher than (one dimensional) audio signals\". Estimated by who? What exactly is meant here by \"resources\"? please provide references. \n3. The annotations and text in the figures (labels, legends, etc) is so small as to be almost unreadable. Please make it larger. \n\n\n###  References \nG. Gallego et al., \"Event-based Vision: A Survey,\" in IEEE Transactions on Pattern Analysis and Machine Intelligence, doi: 10.1109/TPAMI.2020.3008413. \n\n\n-----\nPost feedback edit:\n\nThe authors did not address the (admittedly, small) terminology issue I raised in my review. More specifically, I argued that their method in general requires transmission of the value of the threshold function, to which they replied that this is not true if the threshold function has a particular form (not required by their general theory), and then added some discussion about removal of spikes which was not what I was getting at. My point still stands: in general, for arbitrary threshold functions, transmission of the value of the function is needed for succesful decoding, and this constitutes more information than what is generally meant when discussing spike trains (which are just sets of time stamps).\n\nMoreover, I was not convinced by the authors' reply to the points raised by Rev 4. This is actually more important than the point above, since Rev 4's objections are on the substance of the method (unlike my own points, which are little more than presentation and terminology matters).\n\nTherefore, I am lowering my score.\n\nI am not convinced by the authors' reply to Ref 4's comments. Moreover, the smaller terminology issue I raised in my own comment (about the", "extra_scores": {}, "extra_text": {}}, {"review_id": "_HwOXf0d2GK", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This manuscript explores a mathematical framework and theory for a signal encoding/decoding scheme that shares many similarities to sparse deconvolution algorithms.  They demonstrate that they can learn a compressed representation of this signal, and there are several interesting theoretical properties.\n\nPros:\nSome theoretical properties show intriguing properties\nSeems to work well empirically\n\nCons:\nThe paper needs a significant revision for clarity.\nI am not confident in all of the theoretical analysis.\n\nI have some concerns about the theory.  First, there is a mismatch between the model definition in (1) and the theoretical analysis in Section 4.  The theory only holds if delta_j->0, which isn't discussed at all, and isn't what was used in the experiments. Or at least this is what is shown by the definitions given in (3) and (4).  This must be discussed at a minimum.\n\nThe \"Perfect Reconstruction Theorem\" is achieved by limiting the class to signals produced by the decomposition.  It is unsurprising that the system can decompose signals produced by it.  Equation 5 is fairly well-known from deconvolutional models.  The relationship to existing work needs to be more clearly defined, so that the contributions can be considered in context.\n\nI'm confused by the claimed innovation in Lemma 3.  $X^*$ is defined at the minimum MSE for a number of spikes N.  Thus, it is the minimum energy and any other solution has equal or greater error.\n\nShouldn't equation 2 look at reconstruction error and not simply the minimum?", "extra_scores": {}, "extra_text": {}}, {"review_id": "V4kUxuNS4Te", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "PROS:\n* new\n* both theoretical and experimental results\n* an alternative to matching pursuit, more accurate in certain regimes, which could have a broad range of applications\n\nCONS:\n* fully deterministic (see below)\n\nThe authors propose a new way to encode a broad class of continuous signals (those with finite rate of innovation, which include bandlimited signals) into spike trains, and to reconstruct those signals from the spikes. The reconstruction is exact under certain hypotheses: the signal should be a weighted sum of the kernels used by the neurons with some temporal shifts. In general several signals are compatible with a given spike train, and the algorithm finds the one with minimal energy.\n\nThis algorithm is an alternative to matching pursuit, and is shown to be more accurate in some cases (Fig 4).\n\nIn my opinion these results are new and worth sharing. I have only a few minor suggestions to improve the paper:\n* How would the reconstruction degrade with noise in the spike trains, e.g. temporal jitter or extra/missing spikes?\n* Noise in the (filtered) signal can be beneficial in some cases, because subthreshold signals can then cause spikes, with a higher rate if the denoised signal is near threshold. So the subthreshold signal can be estimated from the spike rates - something that would be impossible in the absence of noise. This is called stochastic resonance, and I wonder if the theory presented here could shed light on it.\n* The author uses a soft refractory period, i.e. an adapting threshold, that is increased after a spike, and then goes back to the baseline linearly. From a biological point of view, an exponential decay would be more realistic. Could the theory cope with such an exponential decay?\n\nMINOR POINTS:\n* The abstract says \"the transformation from external stimuli to\nspike trains is essentially deterministic\". This is highly debated!\n* Eq 3 = Eq 4\n\n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper presents a mathematical framework for encoding and decoding continuous valued signals from spiking neurons, proving both mathematical theory and simulation results. Two reviewers had serious concerns about the mathematical correctness and exactness of some of the presented mathematical results, and the AC raised questions about relationship to (un-cited) prior work. The authors aimed to address these shortcomings in the discussion phase, but neither me nor the reviewers were convinced that it provides a correct, and substantial, advance over prior approaches. I do hope that the feedback from the process will useful to the reviewers, and will help them in clarifying their contributions.
Decision: Reject