Signal Coding and Reconstruction using Spike Trains

In many animal sensory pathways , the transformation from external stimuli to spike trains is essentially deterministic . In this context , a new mathematical framework for coding and reconstruction , based on a biologically plausible model of the spiking neuron , is presented . The framework considers encoding of a signal through spike trains generated by an ensemble of neurons via a standard convolve-thenthreshold mechanism , albeit with a wide variety of convolution kernels . Neurons are distinguished by their convolution kernels and threshold values . Reconstruction is posited as a convex optimization minimizing energy . Formal conditions under which perfect and approximate reconstruction of the signal from the spike trains is possible are then identified . Coding experiments on a large audio dataset are presented to demonstrate the strength of the framework . 1 INTRODUCTION . In biological systems , sensory stimuli is communicated to the brain primarily via ensembles of discrete events that are spatiotemporally compact electrical disturbances generated by neurons , otherwise known as spikes . Spike train representation of signals , when sparse , are not only intrinsically energy efficient , but can also facilitate downstream computation ( 6 ; 10 ) . In their seminal work , Olshausen and Field ( 13 ) showed how efficient codes can arise from learning sparse representations of natural stimulus statistics , resulting in striking similarities with observed biological receptive fields . ( 19 ) developed a biophysically motivated spiking neural network which for the first time predicted the full diversity of V1 simple cell receptive field shapes when trained on natural images . Although these results signify substantial progress , an effective end to end signal processing framework that deterministically represents signals via spike train ensembles is yet to be laid out . Here we present a new framework for coding and reconstruction leveraging a biologically plausible coding mechanism which is a superset of the standard leaky integrate-and-fire neuron model ( 5 ) . Our proposed framework identifies reconstruction guarantees for a very general class of signals—those with finite rate of innovation ( 18 ) —as shown in our perfect and approximate reconstruction theorems . Most other classes , e.g . bandlimited signals , are subsets of this class . The proposed technique first formulates reconstruction as an optimization that minimizes the energy of the reconstructed signal subject to consistency with the spike train , and then solves it in closed form . We then identify a general class of signals for which reconstruction is provably perfect under certain ideal conditions . Subsequently , we present a mathematical bound on the error of an approximate reconstruction when the model deviates from those ideal conditions . Finally , we present simulation experiments coding for a large dataset of audio signals that demonstrate the efficacy of the framework . In a separate set of experiments on a smaller subset of audio signals we compare our framework with existing sparse coding algorithms viz matching pursuit and orthogonal matching pursuit , establishing the strength of our technique . The remainder of the paper is structured as follows . In Sections 2 and 3 we introduce the coding and decoding frameworks . Section 4 identifies the class of signals for which perfect reconstruction is achievable if certain ideal conditions are met . In Section 5 we discuss how in practice those ideal conditions can be approached and provide a mathematical bound for approximate reconstruction . Simulation results are presented in Section 6 . We conclude in Section 8 . 2 CODING . The general class of deterministic mappings ( i.e. , the set of all nonlinear operators ) from continuous time signals to spike trains is difficult to characterize because the space of all spike trains does not lend itself to a natural topology that is universally embraced . The result is that simple characterizations , such as the set of all continuous operators , can not be posited in a manner that has general consensus . To resolve this issue , we take a cue from biological systems . In most animal sensory pathways , external stimulus passes through a series of transformations before being turned into spike trains ( 17 ) . For example , visual signal in the retina is processed by multiple layers of non-spiking horizontal , amacrine and bipolar cells , before being converted into spike trains by the retinal ganglion cells . Accordingly , we can consider the set of transformations that pass via an intermediate continuous time signal which is then transformed into a spike train through a stereotyped mapping where spikes mark threshold crossings . The complexity of the operator now lies in the mapping from the continuous time input signal to the continuous time intermediate signal . Since any time invariant , continuous , nonlinear operator with fading memory can be approximated by a finite Volterra series operator ( 2 ) , this general class of nonlinear operators from continuous time signals to spike trains can be modeled as the composition of a finite Volterra series operator and a neuronal thresholding operation to generate a spike train . Here , the simplest subclass of these transformations is considered : the case where the Volterra series operator has a single causal , bounded-time , linear term , the output of which is composed with a thresholding operation of a potentially time varying threshold . The overall operator from the input signal to the spike train remains nonlinear due to the thresholding operation . The code generated by an ensemble of such transformations , corresponding to an ensemble of spike trains , is explored . Formally , we assume the input signal X ( t ) to be a bounded square integrable function over the compact interval [ 0 , T ] for some T ∈ R+ , i.e. , we are interested in the class of input signals F = { X ( t ) |X ( t ) ∈ L2 [ 0 , T ] } . Since the framework involves signal snippets of arbitrary length , this choice of T is without loss of generalization . We assume an ensemble of convolution kernels K = { Kj |j ∈ Z+ , j ≤ n } , consisting of n kernels Kj , j = 1 , . . . , n. We assume that Kj ( t ) is a continuous function on a bounded time interval [ 0 , T ] , i.e . ∀j ∈ { 1 , . . . , n } , Kj ( t ) ∈ C [ 0 , T ] , T ∈ R+ . Finally , we assume that Kj has a time varying threshold denoted by T j ( t ) . The ensemble of convolution kernels K encodes a given input signal X ( t ) into a sequence of spikes { ( ti , Kji ) } , where the ith spike is produced by the jthi kernel Kji at time ti if and only if : ∫ X ( τ ) Kji ( ti − τ ) dτ = T ji ( ti ) In our experiments a specific threshold function is assumed in which the time varying threshold T j ( t ) of the jth kernel remains constant at Cj until that kernel produces a spike , at which time an after-hyperpolarization potential ( ahp ) is introduced to raise the threshold to a high value M j Cj , which then drops back linearly to its original value within a refractory period δj . Stated formally , T j ( t ) = C j , t− δj > tjl ( t ) M j − ( t−t j l ( t ) ) ( M j−Cj ) δj , t− δj ≤ tjl ( t ) ( 1 ) Where tjl ( t ) denotes the time of the last spike generated by K j prior to time t . 3 DECODING . How rich is the coding mechanism just described ? We can investigate this question formally by positing a decoding module . The objective of the decoding module is to reconstruct the original signal from the encoded ensemble of spike trains . It is worthwhile to mention that to be able to communicate signals properly by our proposed framework , the decoding module needs to be designed in a manner so that it can operate solely on the spike train data handed over by the encoding module , without explicit access to the input signal itself . Considering the prospect of the invertibility of the coding scheme , we seek a signal that satisfies the same set of constraints as the original signal when generating all spikes apropos the set of kernels in ensemble K. Recognizing that such a signal might not be unique , we choose the reconstructed signal as the one with minimum L2-norm . Formally , the reconstruction ( denoted by X∗ ( t ) ) of the input signal X ( t ) is formulated to be the solution to the optimization problem : X∗ ( t ) = argmin X̃ ||X̃ ( t ) ||22 s.t . ∫ X̃ ( τ ) Kji ( ti − τ ) dτ = T ji ( ti ) ; 1 ≤ i ≤ N ( 2 ) where { ( ti , Kji ) |i ∈ { 1 , ... , N } } is the set of all spikes generated by the encoder . The choice of L2 minimization as the objective of the reconstruction problem—which is the linchpin of our framework , as demonstrated in the theorems—can only be weakly justified at the current juncture . The perfect reconstruction theorem that follows provides the strong justification . As it stands , the L2 minimization objective is in congruence with the dictum of energy efficiency in biological systems . The assumption is that , of all signals , the one with the minimum energy that is consistent with the spike trains is desirable . Additionally , an L2 minimization in the objective of ( 2 ) reduces the convex optimization problem to a solvable linear system of equations as shown in Lemmas 1 and 3 . Later we shall show that L2-minimization has the surprising benefit of recovering the original signal perfectly under certain conditions . 4 SIGNAL CLASS FOR PERFECT RECONSTRUCTION . To establish the effectiveness of the described coding-decoding model , we have to evaluate the accuracy of reconstruction over a class of input signals . We observe that in general the encoding of square integrable signals into spike trains is not a one-to-one map ; the same set of spikes can be generated by different signals so as to result in the same convolved values at the spike times . Naturally , with a finite and fixed ensemble of kernelsK , one can not achieve perfect reconstruction for the general class of signals F as defined in Section 2 . We now restrict ourselves to a subset G of the original class F defined as G = { X ( t ) |X ( t ) ∈ F , X ( t ) = ∑N p=1 αpK jp ( tp − t ) , jp ∈ { 1 , ... , n } , αp ∈ R , tp ∈ R+ , N ∈ Z+ } and address the question of reconstruction accuracy . Essentially G consists of all linear combinations of arbitrarily shifted kernel functions . N is bounded above by the total number of spikes that the ensemble K can generate over [ 0 , T ] . In the parlance of signal processing , G constitutes Finite rate of Innovation signals ( 18 ) . For the class G the perfect reconstruction theorem is presented below . The theorem is proved with the help of three lemmas . Perfect Reconstruction Theorem : Let X ( t ) ∈ G be an input signal . Then for appropriately chosen time-varying thresholds of the kernels , the reconstruction , X∗ ( t ) , resulting from the proposed codingdecoding framework is accurate with respect to the L2 metric , i.e. , ||X∗ ( t ) −X ( t ) ||2 = 0 . Lemma 1 : The solution X∗ ( t ) to the reconstruction problem given by ( 2 ) can be written as : X∗ ( t ) = N∑ i=1 αiK ji ( ti − t ) ( 3 ) where the coefficients αi ∈ R can be solved from a system of linear equations . Proof : An approach analogous to the Representer Theorem ( 15 ) , splitting a putative solution to ( 2 ) into its within the span of the kernels component and a remnant orthogonal component , results in equation ( 3 ) . In essence , the reconstructed signal X∗ ( t ) becomes a summation of the kernels , shifted to their respective times of generation of spikes , scaled by appropriate coefficients . Plugging ( 3 ) into the constraints ( 2 ) gives : ∀1≤i≤N ; ∫ N∑ k=1 αkK jk ( tk − t ) Kji ( ti − t ) dτ = T ji ( ti ) Setting bi = T ji ( ti ) and Pik = ∫ Kjk ( tk − τ ) Kji ( ti − τ ) dτ results in : ∀1≤i≤N ; N∑ k=1 Pikαk = bi ( 4 ) Equation ( 4 ) defines a system of N equations in N unknowns of the form : Pα = T ( 5 ) where α = 〈α1 , ... , αN 〉T , T = 〈T j1 ( t1 ) , ... , T jN ( tN ) 〉T and P is an N ×N matrix with elements Pik = ∫ Kjk ( tk − τ ) Kji ( ti − τ ) dτ . Clearly P is the Gramian Matrix of the shifted kernels { Kji ( ti − t ) |i ∈ 1 , 2 , ... , N } in the Hilbert space with the standard inner product . It is well known that P is invertible if and only if { Kji ( ti − t ) |i ∈ 1 , 2 , ... , N } is a linearly independent set . If P is invertible α has a unique solution . If , on the other hand , P is not invertible , α has multiple solutions . However , as the next lemma shows , every such solution leads to the same reconstruction X∗ ( t ) , and hence any value of α that satisfies 5 can be chosen . We note in passing that in our experiments we have used the least square solution . 2 Import : The goal of the optimization problem is to find the best object in the feasible set . However , the application of the Representer Theorem converts the constraints into a determined system of unknowns and equations , turning the focus onto the feasible set , effectively changing the optimization problem into a solvable system that results in a closed form solution for the αi ’ s . This implies that instead of solving ( 2 ) , we can solve for the reconstruction from X∗ ( t ) = ∑N i=1 αiK ji ( ti− t ) , where αi is the i-th element of α = P−1T . Here , P−1 represents either the inverse or the Moore-Penrose inverse , as the case may be . Lemma 2 : Let equation 5 resulting from the optimization problem 2 have multiple solutions . Consider any two different solutions for α , namely α1 and α2 , and hence the corresponding reconstructions are given by X1 ( t ) = ∑N i=1 α1iK ji ( ti − t ) and X2 ( t ) = ∑N i=1 α2iK ji ( ti − t ) , respectively . Then X1 = X2 . Proof : The proof of this lemma follows from the existence of a unique function in the Hilbert Space spanned by { Kji ( ti − t ) |i ∈ 1 , 2 , ... , N } that satisfies the constraint of equation 2 . The details of the proof is furnished in the appendix A . Import : Lemma 2 essentially establishes the uniqueness of solution to the optimization problem formulated in 2 as any solution to equation 5 . The proof follows from the fact that the reconstruction is in the span of the shifted kernels { Kji ( ti − t ) |i ∈ 1 , 2 , ... , N } and the inner products of the reconstruction with each of Kji ( ti − t ) is given ( by the spike constraints of 2 ) . Such a reconstruction must be unique in the subspace S. Lemma 3 : Let X∗ ( t ) be the reconstruction of an input signal X ( t ) and { ( ti , Kji ) } Ni=1 be the set of spikes generated . Then , for any arbitrary signal X̃ ( t ) within the span of { Kji ( ti − t ) |i ∈ { 1 , 2 , ... , N } } , i.e. , the set of shifted kernels at respective spike times , given by X̃ ( t ) =∑N i=1 aiK ji ( ti − t ) the following inequality holds : ||X ( t ) −X∗ ( t ) || ≤ ||X ( t ) − X̃ ( t ) || Proof : ||X ( t ) − X̃ ( t ) || = ||X ( t ) −X∗ ( t ) ︸ ︷︷ ︸ A +X∗ ( t ) − X̃ ( t ) ︸ ︷︷ ︸ B || First , 〈A , Kji ( ti − t ) 〉 = 〈X ( t ) , Kji ( ti − t ) 〉 − 〈X∗ ( t ) , Kji ( ti − t ) 〉 , ∀i ∈ { 1 , 2 , .. , N } = T ji ( ti ) − T ji ( ti ) = 0 ( Using the constraints in ( 2 ) & ( 2 ) ) Second , 〈A , B〉 = 〈A , N∑ i=1 ( αi − ai ) Kji ( ti − t ) 〉 ( By Lemma 1 X∗ ( t ) = N∑ i=1 αiK ji ( ti − t ) ) = N∑ i=1 ( αi − ai ) 〈A , Kji ( ti − t ) 〉 = 0 =⇒ ||X ( t ) − X̃ ( t ) ||2 = ||A+B||2 = ||A||2 + ||B||2 ≥ ||A||2 = ||X ( t ) −X∗ ( t ) ||2 =⇒ ||X ( t ) − X̃ ( t ) || ≥ ||X ( t ) −X∗ ( t ) || 2 Import : The implication of the above lemma is quite remarkable . The objective defined in ( 2 ) chooses a signal with minimum energy satisfying the constraints , deemed the reconstructed signal . However as the lemma demonstrates , this signal also has the minimum error with respect to the input signal in the span of the shifted kernels . This signifies that our choice of the objective in the decoding module not only draws from biologically motivated energy optimization principles , but also performs optimally in terms of reconstructing the original input signal within the span of the appropriately shifted spike generating kernels . Corollary : An important consequence of Lemma 3 is that additional spikes in the system do not worsen the reconstruction . For a given input signal X ( t ) if S1 and S2 are two sets of spike trains where S1 ⊂ S2 , the second a superset of the first , then Lemma 3 implies that the reconstruction due to S2 is at least as good as the reconstruction due to S1 because the reconstruction due to S1 is in the span of the shifted kernel functions of S2 as S1 ⊂ S2 . This immediately leads to the conclusion that for a given input signal the more kernels we add to the ensemble the better the reconstruction . Proof of the Theorem : The proof of the theorem follows directly from Lemma 3 . Since the input signalX ( t ) ∈ G , letX ( t ) be given by : X ( t ) = ∑N p=1 αpK jp ( tp−t ) ( αp ∈ R , tp ∈ R+ , N ∈ Z+ ) Assume that the time varying thresholds of the kernels in our kernel ensemble K are set in such a manner that the following conditions are satisfied : 〈X ( t ) , Kjp ( tp− t ) 〉 = T jp ( tp ) ∀p ∈ { 1 , ... , N } i.e. , each of the kernels Kjp at the very least produces a spike at time tp against X ( t ) ( regardless of other spikes at other times ) . Clearly then X ( t ) lies in the span of the appropriately shifted response functions of the spike generating kernels . Applying Lemma 3 it follows that : ||X ( t ) −X∗ ( t ) ||2 ≤ ||X ( t ) −X ( t ) ||2 = 0 2 Import : In addition to demonstrating the potency of the coding-decoding scheme , this theorem frames Barlow ’ s efficient coding hypothesis ( 1 ) —that the coding strategy of sensory neurons be adapted to the statistics of the stimuli—in mathematically concrete terms . Going by the theorem , the spike based encoding necessitates the signals to be in the span of the encoding kernels for perfect reconstruction . Inverting the argument , kernels must learn to adapt to the basis elements that generate the signal corpora for superior reconstruction .

Summary: The authors describe a method for representing a continuous signal by a pulse code, in a manner inspired by auditory processing in the brain. The resulting framework is somewhat like matching pursuit except that filters are run a single time in a causal manner to find the spike times (which would be faster than MP), and then a N*N least squares problem is solved (which makes it slower). The authors claim that their method will perfectly reconstruct signals of finite innovation rate, however there appear to be mathematical errors in the proof.