# Channel Identification Machines

Authors: Aurel A. Lazar; Yevgeniy B. Slutskiy
Journal: Computational Intelligence and Neuroscience
 (2012)

---

## Abstract

We present a formal methodology for identifying a channel in a system consisting of a communication channel in cascade with an asynchronous sampler. The channel is modeled as a multidimensional filter, while models of asynchronous samplers are taken from neuroscience and communications and include integrate-and-fire neurons, asynchronous sigma/delta modulators and general oscillators in cascade with zero-crossing detectors. We devise channel identification algorithms that recover a projection of the filter(s) onto a space of input signals loss-free for both scalar and vector-valued test signals. The test signals are modeled as elements of a reproducing kernel Hilbert space (RKHS) with a Dirichlet kernel. Under appropriate limiting conditions on the bandwidth and the order of the test signal space, the filter projection converges to the impulse response of the filter. We show that our results hold for a wide class of RKHSs, including the space of finite-energy bandlimited signals. We also extend our channel identification results to noisy circuits.


---

## Body

## 1. Introduction

Signal distortions introduced by a communication channel can severely affect the reliability of communication systems. If properly utilized, knowledge of the channel response can lead to a dramatic improvement in the performance of a communication link. In practice, however, information about the channel is rarely available a priori and the channel needs to be identified at the receiver. A number of channel identification methods [1] have been proposed for traditional clock-based systems that rely on the classical sampling theorem [2, 3]. However, there is a growing need to develop channel identification methods for asynchronous nonlinear systems, of which time encoding machines (TEMs) [4] are a prime example.TEMs naturally arise as models of early sensory systems in neuroscience [5, 6] as well as models of nonlinear samplers in signal processing and analog-to-discrete (A/D) converters in communication systems [4, 6]. Unlike traditional clock-based amplitude-domain devices, TEMs encode analog signals as a strictly increasing sequence of irregularly spaced times (tk)k∈ℤ. As such, they are closely related to irregular (amplitude) samplers [4, 7] and, due to their asynchronous nature, are inherently low-power devices [8]. TEMs are also readily amenable to massive parallelization [9]. Furthermore, under certain conditions, TEMs faithfully represent analog signals in the time domain; given the parameters of the TEM and the time sequence at its output, a time decoding machine (TDM) can recover the encoded signal loss-free [4, 5].A general TEM of interest is shown in Figure1. An analog multidimensional signal u is passed through a channel with memory that models physical communication links. We assume that the effect of this channel on the signal u can be described by a linear multidimensional filter. The output of the channel v is then mapped, or encoded, by a nonlinear asynchronous sampler into the time sequence (tk)k∈ℤ. A few examples of samplers include asynchronous A/D converters such as the one based on an asynchronous sigma/delta modulator (ASDM) [4], nonlinear oscillators such as the van der Pol oscillator in cascade with a zero-crossing detector (ZCD) [6], and spiking neurons such as the integrate-and-fire (IAF) or the threshold-and-fire (TAF) neurons [9]. The above-mentioned asynchronous samplers incorporate the temporal dynamics of spike (pulse) generation and allow one to consider, in particular for neuroscience applications, more biologically plausible nonlinear spike generation (sampling) mechanisms.Figure 1
Modeling the channel identification problem. A known multidimensional signalu(t), t∈ℝ, is first passed through a communication channel. A nonlinear sampler then maps the output v of the channel into an observable time sequence (tk)k∈ℤ.In this paper, we investigate the followingnonlinear identification problem: given both the input signal u and the time sequence (tk)k∈ℤ at the output of a TEM, what is the channel filter? System identification problems of this kind are key to understanding the nature of neural encoding and processing [10–14], process modeling and control [15], and, more generally, methods for constructing mathematical models of dynamical systems [16].Identification of the channel from a time sequence is to be contrasted with existing methods for rate-based models in neuroscience (see [10] for an extensive review). In such models the output of the system is taken to be its instantaneous response rate and the nonlinear generation of a time sequence is not explicitly modeled. Furthermore, in order to fit model parameters, identification methods for such models typically require the response rate to be known [17]. This is often difficult in practice since the same experiment needs to be repeated a large number of times to estimate the response rate. Moreover, the use of the same stimulus typically introduces a systematic bias during the identification procedure [10].The channel identification methodology presented in this paper employs test signals that are neither white nor have stationary statistics (e.g., Gaussian with a fixed mean/variance). This is a radical departure from the widely employed nonlinear system identification methods [10], including the spike-triggered average [18] and the spike-triggered covariance [19] methods. We carry out the channel identification using input signals that belong to reproducing kernel Hilbert spaces (RKHSs), and, in particular, spaces of bandlimited functions, that is, functions that have a finite support in the frequency domain. The latter signals are extensively used to describe sensory stimuli in biological systems and to model signals in communications. We show that for such signals the channel identification problem becomes mathematically tractable. Furthermore, we demonstrate that the choice of the input signal space profoundly effects the type of identification results that can be achieved.The paper is organized as follows. In Section2, we introduce three application-driven examples of the system in Figure 1 and formally state the channel identification problem. In Section 3, we present the single-input single-output (SISO) channel identification machine (CIM) for the finite-dimensional input signal space of trigonometric polynomials. Using analytical methods and simulations, we demonstrate that it is possible to identify the projection of the filter onto the input space loss-free and show that the SISO CIM algorithm can recover the original filter with arbitrary precision, provided that both the bandwidth and the order of the input space are sufficiently high. Then, in Section 4, we extend our methodology to multidimensional systems and present multi-input single-output (MISO) CIM algorithms for the identification of vector-valued filters modeling the channel. We generalize our methods to classes of RKHSs of input signals in Section 5.1 and work out in detail channel identification algorithms for infinite-dimensional Paley-Wiener spaces. In Section 5.2 we discuss extensions of our identification results to noisy systems, where additive noise is introduced either by the channel or the asynchronous sampler. Finally, Section 6 concludes our work.


## 2. The Channel Identification Problem

We investigate a general I/O system comprised of a filter or a bank of filters (i.e., a linear operator) in cascade with an asynchronous (nonlinear) sampler (Figure1). The I/O circuit belongs to the class of [Filter]-[Asynchronous Sampler] circuits. In general terms, the input to such a system is a vector-valued analog signal u=[u1(t),u2(t),…,uM(t)]T, t∈ℝ, M∈ℕ, and the output is a time sequence (tk)k∈ℤ generated by its asynchronous sampling mechanism. In the neural coding literature, such a system is called a time encoding machine (TEM) [4] as it encodes an unknown signal u into an observable time sequence (tk)k∈ℤ.


### 2.1. Examples of Asynchronous SISO and MISO Systems

An instance of the TEM in Figure1 is the SISO [Filter]-[Ideal IAF] neural circuit depicted in Figure 2(a). Here the filter is used to model the aggregate processing of a stimulus performed by the dendritic tree of a sensory neuron. The output of the filter v is encoded into the sequence of spike times (tk)k∈ℤ by an ideal integrate-and-fire neuron. Identification of dendritic processing in such a circuit is an important problem in systems neuroscience. It was first investigated in [20]. Another instance of the system in Figure 1 is the SISO [Filter]-[Nonlinear Oscillator-ZCD] circuit shown in Figure 2(b). In contrast to the first example, where the input was coupled additively, in this circuit the biased filter output v is coupled multiplicatively into a nonlinear oscillator. The zero-crossing detector then generates a time sequence (tk)k∈ℤ by extracting zeros from the observable modulated waveform at the output of the oscillator. Called a TEM with multiplicative coupling [6], this circuit is encountered in generalized frequency modulation [21].Examples of systems arising in neuroscience and communications. (a) Single-input single-output model of a sensory neuron. (b) Single-input single-output nonlinear oscillator in cascade with a zero-crossing detector. (c) Multi-input single-output analog-to-discrete converter implemented with an asynchronous sigma-delta modulator.M liner filters model M (different) communication links.An example of a MISO system is the [Filter]-[ASDM-ZCD] circuit shown in Figure2(c). Similar circuits arise practically in all modern-day A/D converters and constitute important front-end components of measurement and communication systems. Each signal um(t), t∈ℝ, m=1,2,…,M, is transmitted through a communication channel and the effect of the channel on each signal is modeled using a linear filter with an impulse response hm(t), t∈ℝ, m=1,2,…,M. The aggregate channel output v(t)=∑m=1Mvm(t)=∑m=1M(um*hm)(t), where um*hm denotes the convolution of um with hm, is additively coupled into an ASDM. Specifically, v(t) is passed through an integrator and a noninverting Schmitt trigger to produce a binary output z(t)∈{-b,b}, t∈ℝ. A zero-crossing detector is then used to extract the sequence of zero-crossing times (tk)k∈ℤ from z(t). Thus, the output of this [Filter]-[ASDM-ZCD] circuit is the time sequence (tk)k∈ℤ.


### 2.2. Modeling the Input Space

We model channel input signalsu=u(t), t∈ℝ, as elements of the space of trigonometric polynomials ℋ (see Section 5.1 for more general input spaces).Definition 1.
The space of trigonometric polynomialsℋ is a Hilbert space of complex-valued functions
(1)u(t)=1T∑l=-LLulexp(jlΩtL),t∈[0,T],
where ul∈ℂ, Ω is the bandwidth, L is the order and T=2πL/Ω, endowed with the inner product 〈·,·〉:ℋ×ℋ→ℂ(2)〈u,w〉=∫0Tu(t)w(t)¯dt.
Given the inner product in (2), the set of elements
(3)el(t)=1Texp(jlΩtL),l=-L,-L+1,…,L,
forms an orthonormal basis in ℋ. Thus, any element u∈ℋ and any inner product 〈u,w〉 can be compactly written as u=∑l=-LLulel and 〈u,w〉=∑l=-LLulwl¯. Moreover, ℋ is a reproducing kernel Hilbert space (RKHS) with a reproducing kernel (RK) given by
(4)K(s,t)=∑l=-LLel(s)el(t)¯=1T∑l=-LLexp(jlΩL(s-t)),
also known as a Dirichlet kernel [22].We note that a functionu∈ℋ satisfies u(0)=u(T). There is a natural connection between functions on an interval of length T that take on the same values at interval end-points and functions on ℝ that are T-periodic: both provide equivalent descriptions of the same mathematical object, namely a function on a circle. By abuse of notation, in what follows u will denote both a function defined on an interval of length T and a function defined on the entire real line. In the latter case, the function u is simultaneously periodic with period T and bandlimited with bandwidth Ω, that is, it has a finite spectral support supp(ℱu)⊆[-Ω,Ω], where ℱ denotes the Fourier transform. In what follows we will assume that ul≠0 for all l=-L,-L+1,…,L, that is, a signal u∈ℋ contains all 2L+1 frequency components.


### 2.3. Modeling the Channel and Channel Identification

The channel is modeled as a bank ofM filters with impulse responses hm, m=1,2,…,M. We assume that each filter is linear, causal, BIBO-stable and has a finite temporal support of length S≤T, that is, it belongs to the space H={h∈𝕃1(ℝ)∣supp(h)⊆[0,T]}. Since the length of the filter support is smaller than or equal to the period of an input signal, we effectively require that for a given S and a fixed input signal bandwidth Ω, the order L of the space ℋ satisfies L≥S·Ω/(2π). The aggregate channel output is given by v(t)=∑m=1M(um*hm)(t). The asynchronous sampler maps the input signal v into the output time sequence (tk)k=1n, where n denotes the total number of spikes produced on an interval t∈[0,T].Definition 2.
A signalu∈ℋM at the input to a [Filter]-[Asynchronous Sampler] circuit together with the resulting output 𝕋=(tk)k=1n of that circuit is called an input/output (I/O) pair and is denoted by (u,𝕋).We are now in a position to define the channel identification problem.Definition 3.
Let(ui), i=1,2,…,N, be a set of N signals from a test space ℋM. A channel identification machine implements an algorithm that estimates the impulse response of the filter from the I/O pairs (ui,𝕋i), i=1,2,…,N, of the [Filter]-[Asynchronous Sampler] circuit.Remark 4.
We note that a CIM recovers the impulse response of the filter based on the knowledge of I/O pairs(ui,𝕋i), i=1,2,…,N, and the sampler circuit. In contrast, a time decoding machine recovers an encoded signal u based on the knowledge of the entire TEM circuit (both the channel filter and the sampler) and the output time sequence 𝕋.


## 2.1. Examples of Asynchronous SISO and MISO Systems

An instance of the TEM in Figure1 is the SISO [Filter]-[Ideal IAF] neural circuit depicted in Figure 2(a). Here the filter is used to model the aggregate processing of a stimulus performed by the dendritic tree of a sensory neuron. The output of the filter v is encoded into the sequence of spike times (tk)k∈ℤ by an ideal integrate-and-fire neuron. Identification of dendritic processing in such a circuit is an important problem in systems neuroscience. It was first investigated in [20]. Another instance of the system in Figure 1 is the SISO [Filter]-[Nonlinear Oscillator-ZCD] circuit shown in Figure 2(b). In contrast to the first example, where the input was coupled additively, in this circuit the biased filter output v is coupled multiplicatively into a nonlinear oscillator. The zero-crossing detector then generates a time sequence (tk)k∈ℤ by extracting zeros from the observable modulated waveform at the output of the oscillator. Called a TEM with multiplicative coupling [6], this circuit is encountered in generalized frequency modulation [21].Examples of systems arising in neuroscience and communications. (a) Single-input single-output model of a sensory neuron. (b) Single-input single-output nonlinear oscillator in cascade with a zero-crossing detector. (c) Multi-input single-output analog-to-discrete converter implemented with an asynchronous sigma-delta modulator.M liner filters model M (different) communication links.An example of a MISO system is the [Filter]-[ASDM-ZCD] circuit shown in Figure2(c). Similar circuits arise practically in all modern-day A/D converters and constitute important front-end components of measurement and communication systems. Each signal um(t), t∈ℝ, m=1,2,…,M, is transmitted through a communication channel and the effect of the channel on each signal is modeled using a linear filter with an impulse response hm(t), t∈ℝ, m=1,2,…,M. The aggregate channel output v(t)=∑m=1Mvm(t)=∑m=1M(um*hm)(t), where um*hm denotes the convolution of um with hm, is additively coupled into an ASDM. Specifically, v(t) is passed through an integrator and a noninverting Schmitt trigger to produce a binary output z(t)∈{-b,b}, t∈ℝ. A zero-crossing detector is then used to extract the sequence of zero-crossing times (tk)k∈ℤ from z(t). Thus, the output of this [Filter]-[ASDM-ZCD] circuit is the time sequence (tk)k∈ℤ.


## 2.2. Modeling the Input Space

We model channel input signalsu=u(t), t∈ℝ, as elements of the space of trigonometric polynomials ℋ (see Section 5.1 for more general input spaces).Definition 1.
The space of trigonometric polynomialsℋ is a Hilbert space of complex-valued functions
(1)u(t)=1T∑l=-LLulexp(jlΩtL),t∈[0,T],
where ul∈ℂ, Ω is the bandwidth, L is the order and T=2πL/Ω, endowed with the inner product 〈·,·〉:ℋ×ℋ→ℂ(2)〈u,w〉=∫0Tu(t)w(t)¯dt.
Given the inner product in (2), the set of elements
(3)el(t)=1Texp(jlΩtL),l=-L,-L+1,…,L,
forms an orthonormal basis in ℋ. Thus, any element u∈ℋ and any inner product 〈u,w〉 can be compactly written as u=∑l=-LLulel and 〈u,w〉=∑l=-LLulwl¯. Moreover, ℋ is a reproducing kernel Hilbert space (RKHS) with a reproducing kernel (RK) given by
(4)K(s,t)=∑l=-LLel(s)el(t)¯=1T∑l=-LLexp(jlΩL(s-t)),
also known as a Dirichlet kernel [22].We note that a functionu∈ℋ satisfies u(0)=u(T). There is a natural connection between functions on an interval of length T that take on the same values at interval end-points and functions on ℝ that are T-periodic: both provide equivalent descriptions of the same mathematical object, namely a function on a circle. By abuse of notation, in what follows u will denote both a function defined on an interval of length T and a function defined on the entire real line. In the latter case, the function u is simultaneously periodic with period T and bandlimited with bandwidth Ω, that is, it has a finite spectral support supp(ℱu)⊆[-Ω,Ω], where ℱ denotes the Fourier transform. In what follows we will assume that ul≠0 for all l=-L,-L+1,…,L, that is, a signal u∈ℋ contains all 2L+1 frequency components.


## 2.3. Modeling the Channel and Channel Identification

The channel is modeled as a bank ofM filters with impulse responses hm, m=1,2,…,M. We assume that each filter is linear, causal, BIBO-stable and has a finite temporal support of length S≤T, that is, it belongs to the space H={h∈𝕃1(ℝ)∣supp(h)⊆[0,T]}. Since the length of the filter support is smaller than or equal to the period of an input signal, we effectively require that for a given S and a fixed input signal bandwidth Ω, the order L of the space ℋ satisfies L≥S·Ω/(2π). The aggregate channel output is given by v(t)=∑m=1M(um*hm)(t). The asynchronous sampler maps the input signal v into the output time sequence (tk)k=1n, where n denotes the total number of spikes produced on an interval t∈[0,T].Definition 2.
A signalu∈ℋM at the input to a [Filter]-[Asynchronous Sampler] circuit together with the resulting output 𝕋=(tk)k=1n of that circuit is called an input/output (I/O) pair and is denoted by (u,𝕋).We are now in a position to define the channel identification problem.Definition 3.
Let(ui), i=1,2,…,N, be a set of N signals from a test space ℋM. A channel identification machine implements an algorithm that estimates the impulse response of the filter from the I/O pairs (ui,𝕋i), i=1,2,…,N, of the [Filter]-[Asynchronous Sampler] circuit.Remark 4.
We note that a CIM recovers the impulse response of the filter based on the knowledge of I/O pairs(ui,𝕋i), i=1,2,…,N, and the sampler circuit. In contrast, a time decoding machine recovers an encoded signal u based on the knowledge of the entire TEM circuit (both the channel filter and the sampler) and the output time sequence 𝕋.


## 3. SISO Channel Identification Machines

As already mentioned, the circuits under investigation consist of a channel and an asynchronous sampler. Throughout this paper, we will assume that the structure and the parameters of the asynchronous sampler are known. We start by formally describing asynchronous channel measurements in Section3.1. Channel identification algorithms from asynchronous measurements are given in Section 3.2. Examples characterizing the performance of the identification algorithms are discussed in Section 3.3.


### 3.1. Asynchronous Measurements of the Channel Output

Consider the SISO [Filter]-[Ideal IAF] neural circuit in Figure2(a). In this circuit, an input signal u∈ℋ is passed through a filter with an impulse response (or kernel) h∈H and then encoded by an ideal IAF neuron with a bias b∈ℝ+, a capacitance C∈ℝ+, and a threshold δ∈ℝ+. The output of the circuit is a sequence of spike times (tk)k=1n on the time interval [0,T] that is available to an observer. This neural circuit is an instance of a TEM and its operation can be described by a set of equations
(5)∫tktk+1(u*h)(s)ds=qk,k=1,2,…,n-1,
where qk=Cδ-b(tk+1-tk). Intuitively, at every spike time tk+1 the ideal IAF neuron is providing a measurement qk of the signal v(t)=(u*h)(t) on the time interval [tk,tk+1).Definition 5.
The mapping of an analog signalu(t), t∈ℝ, into an increasing sequence of times (tk)k∈ℤ (as in (5)) is called the t-transform [4].Definition 6.
The operator𝒫:H→ℋ given by
(6)(𝒫h)(t)=∫0Th(s)K(s,t)¯ds
is called the projection operator.Proposition 7 (conditional duality).
For allu∈ℋ, a [Filter]-[Ideal IAF] TEM with a filter kernel h is I/O-equivalent to a [Filter]-[Ideal IAF] TEM with the filter kernel 𝒫h. Furthermore, the CIM algorithm for identifying the filter kernel 𝒫h is equivalent to the TDM algorithm for recovering the input signal 𝒫h encoded by a [Filter]-[Ideal IAF] TEM with the filter kernel u.Proof.
Sinceu∈ℋ, u(t)=〈u(·),K(·,t)〉 by the reproducing property of the kernel K(s,t). Hence, (u*h)(t)=(a)∫ℝ‍h(w)u(t-w)dw=(b)∫0T‍h(w)∫0T‍u(z)K(z,t-w)¯
d
z
d
w
=
(
c
)
∫
0
T
‍
u
(
z
)
∫
0
T
‍
h
(
w
)
K
(
w
,
t
-
z
)
¯
d
w
d
z
=
(
d
)
∫
0
T
‍
u
(
z
)
(
𝒫
h
)
(
t
-
z
)
d
z
=
(
e
)
(
u
*
𝒫
h
)
(
t
), where (a) follows from the commutativity of convolution, (b) from the reproducing property of the kernel K and the assumption that supp(h)⊆[0,T], (c) from the equality K(z,t-w)=K(w,t-z), (d) from the definition of 𝒫h in (6), and (e) from the definition of convolution for periodic functions [23]. It follows that on the interval t∈[0,T], (5) can be rewritten as
(7)∫tktk+1(u*𝒫h)(s)ds=(f)∫tktk+1(𝒫h*u)(s)ds=qk,
for all k=1,2,…,n-1, where (f) comes from the commutativity of convolution. The right-hand side of (7) is the t-transform of a [Filter]-[Ideal IAF] TEM with an input 𝒫h and a filter that has an impulse response u. Hence, a TDM can identify 𝒫h, given a filter-output pair (u,𝕋).The conditional duality between time encoding and channel identification is visualized in Figure3. First, we note the conditional I/O equivalence between the circuit in Figure 3(a) and the original circuit in Figure 2(a). The equivalence is conditional since 𝒫h is a projection onto a particular space ℋ and the two circuits are I/O-equivalent only for input signals in that space. Second, identifying the filter of the circuit in Figure 3(a) is the same as decoding the signal encoded with the circuit in Figure 3(b). Note that the filter projection 𝒫h is now treated as the input to the [Filter]-[Ideal IAF] circuit and the signal u appears as the impulse response of the filter. Effectively, we have transformed the channel identification problem into a time decoding problem and we can use the TDM machinery of [5] to identify the filter projection (𝒫h)(t) on t∈[0,T].Conditional duality between channel identification and time encoding. (a) For allu∈ℋ, the [Filter]-[Ideal IAF] circuit with an input-filter pair (u,h) is I/O equivalent to a [Filter]-[Ideal IAF] circuit with an input-filter pair (u,𝒫h). (b) The input-filter pair (u,𝒫h) in channel identification is dual to the (𝒫h,u) pair in time encoding.
(a)
Conditional I/O equivalence
(b)
Duality


### 3.2. Channel Identification from Asynchronous Measurements

Given the parameters of the asynchronous sampler, the measurementsqk of the channel output v can be readily computed from spike times (tk)k=1n using the definition of qk ((5) for the IAF neuron). Furthermore, as we will now show, for a known input signal, these measurements can be reinterpreted as measurements of the channel itself.Lemma 8.
There is a functionϕk(t)=∑l=-LL‍ϕl,kel(t)∈ℋ, such that the t-transform of the [Filter]-[Ideal IAF] neuron in (7) can be written as
(8)〈𝒫h,ϕk〉=qk,
and ϕl,k=T∫tktk+1‍u¯le¯l(t)dt for all l=-L,-L+1,…,L and k=1,2,…,n-1.Proof.
The linear functionalℒk:ℋ→ℝ defined by
(9)ℒk(w)=∫tktk+1(u*w)(s)ds,
where w∈ℋ, is bounded. Thus, by the Riesz representation theorem [22], there exists a function ϕk∈ℋ such that ℒk(w)=〈w,ϕk〉, k=1,2,…,n-1, and qk=ℒk(𝒫h)=∫tktk+1‍(u*𝒫h)(s)ds=〈𝒫h,ϕk〉. Since ϕk∈ℋ, we have ϕk(t)=∑l=-LL‍ϕl,kel for some ϕl,k∈ℂ, l=-L,-L+1,…,L. To find the latter coefficients, we note that ϕl,k=〈ϕk,el〉=〈el,ϕk〉¯=ℒk(el)¯. By definition of ℒk in (9), ℒk(el)=∫tktk+1‍(u*el)(t)dt=∫tktk+1‍∫0T‍∑i=-LL‍uiei(s)el(t-s)dsdt=T∫tktk+1‍ulel(t)dt.Sinceqk=∫tktk+1‍(u*𝒫h)(s)ds=〈v,𝒫1[tk,tk+1]〉, the measurements qk are projections of v=u*𝒫h onto 𝒫1[tk,tk+1], k=1,2,…,n-1. Assuming that u is known and there are enough measurements available, 𝒫h can be obtained by first recovering v from these projections and then deconvolving it with u. However, this two-step procedure does not work when the circuit is not producing enough measurements and one cannot recover v. A more direct route is suggested by Lemma 8, since the measurements (qk)k=1n-1 can also be interpreted as the projections of 𝒫h onto ϕk, that is, 〈𝒫h,ϕk〉, k=1,2,…,n-1. A natural question then is how to identify 𝒫h directly from the latter projections.Lemma 9.
Letu∈ℋ be the input to a [Filter]-[Ideal IAF] circuit with h∈H. If the number of spikes n generated by the neuron in a time interval of length T satisfies n≥2L+2, then the filter projection 𝒫h can be perfectly identified from the I/O pair (u,𝕋) as (𝒫h)(t)=∑l=-LL‍hlel(t), where h=Φ+q with [q]k=qk and Φ+ denotes the Moore-Penrose pseudoinverse of Φ. The matrix Φ is of size (n-1)×(2L+1) and its elements are given by
(10)[Φ]kl={ul(tk+1-tk),l=0,ulLT(el(tk+1)-el(tk))jlΩ,l≠0.Proof.
Since𝒫h∈ℋ, it can be written as (𝒫h)(t)=∑l=-LL‍hlel(t). Then from (8) we have
(11)qk=〈𝒫h,ϕk〉=∑l=-LLhlϕl,k¯.
Writing (11) for all k=1,2,…,n-1, we obtain q=Φh with [q]k=qk, [Φ]kl=ϕl,k¯ and [h]l=hl. This system of linear equations can be solved for h, provided that the rank r(Φ) of the matrix Φ satisfies r(Φ)=2L+1. A necessary condition for the latter is that the number of measurements qk is at least 2L+1, or, equivalently, the number of spikes n≥2L+2. Under this condition, the solution can be computed as h=Φ+q.Remark 10.
If the signalu is fed directly into the neuron, then ∫tktk+1‍(u*𝒫h)(t)dt=∫tktk+1‍u(t)dt, for k=1,2,…,n-1, that is, (𝒫h)(t)=K(t,0), t∈ℝ. In other words, if there is no processing on the input signal u, then the kernel K(t,0) in ℋ is identified as the filter projection. This is also illustrated in Figure 7.
In order to ensure that the neuron produces at least2L+1 measurements in a time interval of length T, it suffices to have tk+1-tk≤T/(2L+2). Since tk+1-tk≤Cδ/(b-c) for |v(t)|≤c<b, it suffices to have Cδ<(b-c)T/(2L+2). Using the definition of T=2πL/Ω and taking the limit as L→∞, we obtain the familiar Nyquist-type criterion Cδ<π(b-c)/Ω for a bandlimited stimulus u∈Ξ [4, 20] (see also Section 5.1).
Ideally, we would like to identify the impulse response of the filterh. Note that unlike h∈H, the projection 𝒫h belongs to the space ℋ. Nevertheless, under quite natural conditions on h (see Section 3.4), 𝒫h approximates h arbitrarily closely on t∈[0,T], provided that both the bandwidth and the order of the signal u are sufficiently large (see also Figure 9).The requirement of Lemma9 that the number of spikes n produced by the system in Figure 2(a) has to satisfy n≥2L+2 is quite stringent and may be hard to meet in practice, especially if the order L of the space ℋ is high. In that case we have the following result.Theorem 11 (SISO channel identification machine).
Let{ui∣ui∈ℋ}i=1N be a collection of N linearly independent stimuli at the input to a [Filter]-[Ideal IAF] circuit with h∈H. If the total number of spikes n=∑i=1N‍ni generated by the neuron satisfies n≥2L+N+1, then the filter projection 𝒫h can be perfectly identified from a collection of I/O pairs {(ui,𝕋i)}i=1N as
(12)(𝒫h)(t)=∑l=-LLhlel(t),
where h=Φ+q. Furthermore, Φ=[Φ1;Φ2;…;ΦN], q=[q1;q2;…;qN] and [qi]k=qki with each Φi of size (ni-1)×(2L+1) and qi of size (ni-1)×1. The elements of matrices Φi are given by
(13)[Φi]kl={uli(tk+1i-tki),l=0,uliLT(el(tk+1i)-el(tki))jlΩ,l≠0,
for all k=1,2,…,n-1, l=-L,-L+1,…,L, and i=1,2,…,N.Proof.
Since𝒫h∈ℋ, it can be written as (𝒫h)(t)=∑l=-LL‍hlel(t). Furthermore, since the stimuli are linearly independent, the measurements (qki)k=1ni-1 provided by the IAF neuron are distinct. Writing (5) for a stimulus ui, we obtain
(14)qki=〈𝒫h,ϕki〉=∑l=-LL‍hlϕl,ki¯,
or qi=Φih, with [qi]k=qki, [Φi]kl=ϕl,ki¯ and [h]l=hl. Repeating for all i=1,…,N, we get q=Φh with Φ=[Φ1;Φ2;…;ΦN] and q=[q1;q2;…;qN]. This system of linear equations can be solved for h, provided that the rank r(Φ) of matrix Φ satisfies r(Φ)=2L+1. A necessary condition for the latter is that the total number n=∑i=1N‍ni of spikes generated in response to all N signals satisfies n≥2L+N+1. Then, the solution can be computed as h=Φ+q. To find the coefficients ϕl,ki¯, we note that ϕl,ki=ℒki(el)¯ (see Lemma 8). Hence, the result follows.The time encoding interpretation of the channel identification problem for a SISO [Filter]-[Ideal IAF] circuit is shown in Figure4(a). The block diagram of the SISO CIM in Theorem 11 is shown in Figure 4(b). Note that the key idea behind the SISO CIM is the introduction of multiple linearly independent test signals ui∈ℋ, i=1,2,...,N. When the [Filter]-[Ideal IAF] circuit is producing very few measurements of 𝒫h in response to any given test signal ui, we use more signals to obtain additional measurements. We can do so and identify 𝒫h because 𝒫h∈ℋ is fixed. In contrast, identifying 𝒫h in a two-step deconvolving procedure requires reconstructing at least one vi. This is an ill-posed problem since each vi is signal-dependent and has a small number of associated measurements.SISO CIM algorithm for the [Filter]-[Ideal IAF] circuit. (a) Time encoding interpretation of the channel identification problem. (b) Block diagram of the SISO channel identification machine.
(a)
(b)


### 3.3. Examples

We now demonstrate the performance of the identification algorithms in Lemma9 and Theorem 11. First, we identify a filter in the SISO [Filter]-[Ideal IAF] circuit (Figure 2(a)) from a single I/O pair when this circuit produces a sufficient number of measurements in an interval of length T. Second, we identify the filter using multiple I/O pairs for the case when the number of measurements produced in response to any given input signal is small. Finally, we consider the SISO [Filter]-[Nonlinear Oscillator-ZCD] circuit with multiplicative coupling (Figure 2(b)) and identify its filter from multiple I/O pairs.


#### 3.3.1. SISO [Filter]-[Ideal IAF] Circuit, Single I/O Pair

We model the dendritic processing filter using the causal linear kernel(15)h(t)=ce-αt[(αt)33!-(αt)55!],t∈[0,0.1]s,
with c=3 and α=200. The general form of this kernel was suggested in [24] as a plausible approximation to the temporal structure of a visual receptive field. Since the length of the filter support S=0.1s, we will need to use a signal with a period T≥0.1s. In Figure 5(a), we apply a signal u that is bandlimited to 25Hz and has a period of T=0.2s, that is, the order of the space L=T·Ω/(2π)=5. The biased output of the filter v=(u*h)+b is then fed into an ideal integrate-and-fire neuron (Figure 5(b)). Here the bias b guarantees that the output of the integrator reaches the threshold value in finite time. Whenever the biased filter output is above zero (Figure 5(b)), the membrane potential is increasing (Figure 5(c)). If the membrane potential ∫tkt‍[(u*h)(s)+b]ds reaches a threshold δ, a spike is generated by the neuron at a time tk+1 and the potential is reset to zero (Figure 5(c)). The resulting spike train (tk)k=1n at the output of the [Filter]-[Ideal IAF] circuit is shown in Figure 5(d). Note that the circuit generated a total of n=13 spikes in an interval of length T=0.2s. According to Theorem 14, we need at least n=2L+2=12 spikes, corresponding to 2L+1=11 measurements, in order to identify the projection 𝒫h of the filter h loss-free. Hence, for this particular example, it will suffice to use a single I/O pair (u,𝕋).Figure 5
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using a single I/O pair.  (a) An input signalu is bandlimited to 25 Hz. The order of the space is L=5. (b) The corresponding biased output of the filter v(t)+b. (c) The filter output in (b) is integrated by the ideal IAF neuron. Whenever the membrane potential reaches a threshold δ, a spike is produced by the neuron and the potential is reset to 0. (d) The neuron generated a total of 13 spikes. (e) The identified impulse response of the filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is -77.5dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. Note that supp(ℱK)=supp(ℱ𝒫h*)=[-Ω,Ω] but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h*∈ℋ but h∉ℋ.In Figure5(e), we plot the original impulse response of the filter h, the filter projection 𝒫h, and the filter 𝒫h*. The latter filter was identified using the algorithm in Theorem 14. Notice that the identified impulse response 𝒫h* (red) is quite different from h (dashed black). In contrast, and as expected, the blue and red curves corresponding, respectively, to 𝒫h and 𝒫h* are indistinguishable. The mean squared error (MSE) between 𝒫h* and 𝒫h amounts to −77.5 dB.The difference between𝒫h and h is further evaluated in Figures 5(f)–5(h). By definition of 𝒫h in (6), 𝒫h=h*K(·,0)¯, or ℱ(𝒫h)=ℱ(h)ℱ(K(·,0)) since K¯=K. Hence both the projection 𝒫h and the identified filter 𝒫h* will contain frequencies that are present in the reproducing kernel K, or equivalently in the input signal u. In Figure 5(f) we show the double-sided Fourier amplitude spectrum of K(t,0). As expected, we see that the kernel is bandlimited to 25Hz and contains 2L+1=11 distinct frequencies. On the other hand, as shown in Figure 5(g), the original filter h is not bandlimited (since it has a finite temporal support). As a result, the input signal u explores h in a limited spectrum of [-Ω,Ω]rad/s, effectively projecting h onto the space ℋ with Ω=2π·25rad/s and L=5. The Fourier amplitude spectrum of the identified projection 𝒫h* is shown in Figure 5(h).


#### 3.3.2. SISO [Filter]-[Ideal IAF] Circuit, Multiple I/O Pairs

Next, we identify the projection ofh onto the space of functions that are bandlimited to 100 Hz and have the same period T=0.2s as in the first example. This means that the order L of the space of input signals ℋ is L=T·Ω/(2π)=20. In order to identify the projection 𝒫h loss-free, the neuron has to generate at least 2L+1=41 measurements. If the neuron produces about 13 spikes (12 measurements) on an interval of length T, as in the previous example, a single I/O pair will not suffice. However, we can still recover the projection 𝒫h if we use multiple I/O pairs.In Figure6 we illustrate identification of the filter using the algorithm in Theorem 11. A total of 48 spikes were produced by the neuron in response to four different signals u1,…,u4. Since 48>2L+N+1=45, the MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −73.3 dB.Figure 6
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using multiple I/O pairs.  (a) Input signalsu1,…,u4 are bandlimited to 100 Hz. The order of the space L=20.  (b) Biased output of the filter v1(t)+b in response to the stimulus u1.  (c) The filter output in (b) is integrated by an ideal IAF neuron.  (d) The neuron generated a total of 48 spikes in response to all 4 input signals.  (e) The identified impulse response 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −73.3 dB.  (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. Note that supp(ℱK)=[-Ω,Ω]=supp(ℱ𝒫h*) but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h*∈ℋ but h∉ℋ.Figure 7
Channel identification forh(t)=δ(t).  (a) Input signals u1,u2 are bandlimited to 50 Hz. The order of the space L=10. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) The filter output in (b) is integrated by an ideal IAF neuron. (d) The neuron generated a total of 28 spikes in response to 2 input signals. (e) The identified filter 𝒫h* (red) is exactly the kernel K(t,0) for ℋΩ,L1 with Ω=2π·10rad/s and L=10. Also shown is the original filter h=δ (dashed black) and its projection 𝒫h=δ*K(·,0)¯=K(·,0) (blue). The MSE between 𝒫h* and 𝒫h is −87.6 dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. As before, 𝒫h*∈ℋ but h∉ℋ.


#### 3.3.3. SISO [Filter]-[Ideal IAF] Circuit,h(t)=δ(t)

Now we consider a special case when the channel does not alter the input signal, that is, whenh(t)=δ(t), t∈ℝ, is the Dirac delta function. As explained in Remark 10, the CIM should identify the projection of δ(t) onto ℋ, that is, it should identify the kernel K(t,0). This is indeed the case as shown in Figure 7.


#### 3.3.4. SISO [Filter]-[Nonlinear Oscillator-ZCD] Circuit, Multiple I/O Pairs

Next we consider a SISO circuit consisting of a channel in cascade with a nonlinear dynamical system that has a stable limit cycle. We assume that the (positive) output of the channelv(t)+b is multiplicatively coupled to the dynamical system (Figure 2(b)) so that the circuit is governed by a set of equations
(16)dydt=(v(t)+b)f(y).
A system (16) followed by a zero-crossing detector is an example of a TEM with multiplicative coupling and has been previously investigated in [6]. It can be shown that such a TEM is input/output equivalent to an IAF neuron with a threshold δ that is equal to the period of the dynamical system on a stable limit cycle [6].As an example, we consider a [Filter]-[van der Pol - ZCD] TEM with the van der Pol oscillator described by a set of equations(17)dy1dt=(u*h+b)[μ(y1-13y13)-y2],dy2dt=(u*h+b)y1,
where μ is the damping coefficient [6]. We assume that y1 is the only observable state of the oscillator and without loss of generality we choose the zero phase of the limit cycle to be the peak of y1.In Figure8, we show the results of a simulation in which a SISO CIM was used to identify the channel. Input signals (Figure 8(a)) were bandlimited to 50 Hz and had a period T=0.5s, that is, L=25. In the absence of an input, that is, when u=0, a constant bias b=1 (Figure 8(b)) resulted a in period of 34.7 ms on a stable limit cycle (Figure 8(e)). As seen in Figures 8(b) and 8(c), downward/upward deviations of v1(t)+b in response to u1 resulted in the slowing-down/speeding-up of the oscillator. In order to identify the filter projection onto a space of order L=25 loss-free, we used a total of n=56 zeros at the output of the zero-crossing detector (Figure 8(d)). This is 1 more zero than the rank requirement of 2L+N+1=55 zeros, or equivalently of 2L+1=51 measurements. The MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −66.6 dB.Figure 8
Channel identification in a SISO [Filter]-[van der Pol-ZCD] circuit using multiple I/O pairs. (a) Input signalsu1,…,u4 are bandlimited to 50 Hz. The order of the space L=25. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) Downward and upward deviations of v1(t)+b from the bias b cause the oscillator to slow down and to speed up, respectively. The damping coefficient μ=20. (d) The oscillator produced a total of 56 spikes in response to 4 stimuli. Here spikes correspond to the peaks of the observable state variable y11. (e) A limit cycle of the van der Pol oscillator for μ=20 is shown in the phase plane. In the absence of channel output, the bias b resulted in a constant period of oscillation T(b)=34.7ms. The red dot denotes the zero phase (spike) of an oscillation. (f) The identified filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −66.6 dB. (g) Fourier amplitude spectra of h and 𝒫h*. As before, 𝒫h*∈ℋ but h∉ℋ.Comparison betweenh and 𝒫h in time and frequency domains.  (a) h (red) and its projection 𝒫h (blue) are shown for several values of Ω and L in the time domain. Ω=2π·20rad/s, 2π·50rad/s and 2π·100rad/s in the top, middle and bottom row, respectively. The period T is fixed at T=0.2s in the left column and T=0.5s in the right column. (b) Fourier amplitude spectra of h (red) and 𝒫h (blue) for the same values of Ω and L as in (a). Note that the differentiating filter h clearly removes the zero-frequency (dc) coefficient corresponding to l=0 in all cases.
(a)
Time domain
(b)
Frequency domain


### 3.4. Convergence of the SISO CIM Estimate

Recall, that the original problem of interest is that of recovering the impulse response of the filterh. The CIM lets us identify the projection 𝒫h of that filter onto the input space. A natural question to ask is whether 𝒫h converges to h and if so how and under what conditions. We formalize this below.Proposition 12.
If∫0T‍|h(t)|2dt<∞, then 𝒫h→h in the L2 norm and almost everywhere on t∈[0,T] with increasing Ω, L and fixed T.Proof.
LetT=2πL/Ω=const. Then K(t,0)=(1/T)∑l=-LLe(jΩl/L)t=(1/T)∑l=-LL‍e(j2πl/T)t and by definition of 𝒫h in (6), we have
(18)(𝒫h)(t)=∫0T[1T∑l=-LLe(j2πl/T)(t-s)]h(s)ds=∑l=-LL[1T∫0Th(s)e-(j2πl/T)sds]e(j2πl/T)t=∑l=-LLh^(l)e(j2πl/T)t=SLh(t),
where SLh is the Lth partial sum of the Fourier series of h and h^(l) is the lth Fourier coefficient. Hence the problem of convergence of 𝒫h to h is the same as that of the convergence of the Fourier series of h. We thus have convergence in the L2 norm and convergence almost everywhere follows from Carleson's theorem [23].Remark 13.
More generally, if∫0T‍|h(t)|pdt<∞, p∈(1,∞), then 𝒫h→h in the Lp norm and almost everywhere by Hunt's theorem [23].It follows from Proposition12 that 𝒫h approximates h arbitrarily closely (in the L2 norm, or MSE sense), given an appropriate choice of Ω and L. Since the number of measurements needed to identify the projection 𝒫h increases linearly with L, a single channel identification problem leads us to consider a countably infinite number of time encoding problems in order to identify the impulse response of the filter with arbitrary precision. To provide further intuition about the relationship between h and 𝒫h, we compare the two in time and frequency domains for multiple values of Ω and L in Figure 9.


## 3.1. Asynchronous Measurements of the Channel Output

Consider the SISO [Filter]-[Ideal IAF] neural circuit in Figure2(a). In this circuit, an input signal u∈ℋ is passed through a filter with an impulse response (or kernel) h∈H and then encoded by an ideal IAF neuron with a bias b∈ℝ+, a capacitance C∈ℝ+, and a threshold δ∈ℝ+. The output of the circuit is a sequence of spike times (tk)k=1n on the time interval [0,T] that is available to an observer. This neural circuit is an instance of a TEM and its operation can be described by a set of equations
(5)∫tktk+1(u*h)(s)ds=qk,k=1,2,…,n-1,
where qk=Cδ-b(tk+1-tk). Intuitively, at every spike time tk+1 the ideal IAF neuron is providing a measurement qk of the signal v(t)=(u*h)(t) on the time interval [tk,tk+1).Definition 5.
The mapping of an analog signalu(t), t∈ℝ, into an increasing sequence of times (tk)k∈ℤ (as in (5)) is called the t-transform [4].Definition 6.
The operator𝒫:H→ℋ given by
(6)(𝒫h)(t)=∫0Th(s)K(s,t)¯ds
is called the projection operator.Proposition 7 (conditional duality).
For allu∈ℋ, a [Filter]-[Ideal IAF] TEM with a filter kernel h is I/O-equivalent to a [Filter]-[Ideal IAF] TEM with the filter kernel 𝒫h. Furthermore, the CIM algorithm for identifying the filter kernel 𝒫h is equivalent to the TDM algorithm for recovering the input signal 𝒫h encoded by a [Filter]-[Ideal IAF] TEM with the filter kernel u.Proof.
Sinceu∈ℋ, u(t)=〈u(·),K(·,t)〉 by the reproducing property of the kernel K(s,t). Hence, (u*h)(t)=(a)∫ℝ‍h(w)u(t-w)dw=(b)∫0T‍h(w)∫0T‍u(z)K(z,t-w)¯
d
z
d
w
=
(
c
)
∫
0
T
‍
u
(
z
)
∫
0
T
‍
h
(
w
)
K
(
w
,
t
-
z
)
¯
d
w
d
z
=
(
d
)
∫
0
T
‍
u
(
z
)
(
𝒫
h
)
(
t
-
z
)
d
z
=
(
e
)
(
u
*
𝒫
h
)
(
t
), where (a) follows from the commutativity of convolution, (b) from the reproducing property of the kernel K and the assumption that supp(h)⊆[0,T], (c) from the equality K(z,t-w)=K(w,t-z), (d) from the definition of 𝒫h in (6), and (e) from the definition of convolution for periodic functions [23]. It follows that on the interval t∈[0,T], (5) can be rewritten as
(7)∫tktk+1(u*𝒫h)(s)ds=(f)∫tktk+1(𝒫h*u)(s)ds=qk,
for all k=1,2,…,n-1, where (f) comes from the commutativity of convolution. The right-hand side of (7) is the t-transform of a [Filter]-[Ideal IAF] TEM with an input 𝒫h and a filter that has an impulse response u. Hence, a TDM can identify 𝒫h, given a filter-output pair (u,𝕋).The conditional duality between time encoding and channel identification is visualized in Figure3. First, we note the conditional I/O equivalence between the circuit in Figure 3(a) and the original circuit in Figure 2(a). The equivalence is conditional since 𝒫h is a projection onto a particular space ℋ and the two circuits are I/O-equivalent only for input signals in that space. Second, identifying the filter of the circuit in Figure 3(a) is the same as decoding the signal encoded with the circuit in Figure 3(b). Note that the filter projection 𝒫h is now treated as the input to the [Filter]-[Ideal IAF] circuit and the signal u appears as the impulse response of the filter. Effectively, we have transformed the channel identification problem into a time decoding problem and we can use the TDM machinery of [5] to identify the filter projection (𝒫h)(t) on t∈[0,T].Conditional duality between channel identification and time encoding. (a) For allu∈ℋ, the [Filter]-[Ideal IAF] circuit with an input-filter pair (u,h) is I/O equivalent to a [Filter]-[Ideal IAF] circuit with an input-filter pair (u,𝒫h). (b) The input-filter pair (u,𝒫h) in channel identification is dual to the (𝒫h,u) pair in time encoding.
(a)
Conditional I/O equivalence
(b)
Duality


## 3.2. Channel Identification from Asynchronous Measurements

Given the parameters of the asynchronous sampler, the measurementsqk of the channel output v can be readily computed from spike times (tk)k=1n using the definition of qk ((5) for the IAF neuron). Furthermore, as we will now show, for a known input signal, these measurements can be reinterpreted as measurements of the channel itself.Lemma 8.
There is a functionϕk(t)=∑l=-LL‍ϕl,kel(t)∈ℋ, such that the t-transform of the [Filter]-[Ideal IAF] neuron in (7) can be written as
(8)〈𝒫h,ϕk〉=qk,
and ϕl,k=T∫tktk+1‍u¯le¯l(t)dt for all l=-L,-L+1,…,L and k=1,2,…,n-1.Proof.
The linear functionalℒk:ℋ→ℝ defined by
(9)ℒk(w)=∫tktk+1(u*w)(s)ds,
where w∈ℋ, is bounded. Thus, by the Riesz representation theorem [22], there exists a function ϕk∈ℋ such that ℒk(w)=〈w,ϕk〉, k=1,2,…,n-1, and qk=ℒk(𝒫h)=∫tktk+1‍(u*𝒫h)(s)ds=〈𝒫h,ϕk〉. Since ϕk∈ℋ, we have ϕk(t)=∑l=-LL‍ϕl,kel for some ϕl,k∈ℂ, l=-L,-L+1,…,L. To find the latter coefficients, we note that ϕl,k=〈ϕk,el〉=〈el,ϕk〉¯=ℒk(el)¯. By definition of ℒk in (9), ℒk(el)=∫tktk+1‍(u*el)(t)dt=∫tktk+1‍∫0T‍∑i=-LL‍uiei(s)el(t-s)dsdt=T∫tktk+1‍ulel(t)dt.Sinceqk=∫tktk+1‍(u*𝒫h)(s)ds=〈v,𝒫1[tk,tk+1]〉, the measurements qk are projections of v=u*𝒫h onto 𝒫1[tk,tk+1], k=1,2,…,n-1. Assuming that u is known and there are enough measurements available, 𝒫h can be obtained by first recovering v from these projections and then deconvolving it with u. However, this two-step procedure does not work when the circuit is not producing enough measurements and one cannot recover v. A more direct route is suggested by Lemma 8, since the measurements (qk)k=1n-1 can also be interpreted as the projections of 𝒫h onto ϕk, that is, 〈𝒫h,ϕk〉, k=1,2,…,n-1. A natural question then is how to identify 𝒫h directly from the latter projections.Lemma 9.
Letu∈ℋ be the input to a [Filter]-[Ideal IAF] circuit with h∈H. If the number of spikes n generated by the neuron in a time interval of length T satisfies n≥2L+2, then the filter projection 𝒫h can be perfectly identified from the I/O pair (u,𝕋) as (𝒫h)(t)=∑l=-LL‍hlel(t), where h=Φ+q with [q]k=qk and Φ+ denotes the Moore-Penrose pseudoinverse of Φ. The matrix Φ is of size (n-1)×(2L+1) and its elements are given by
(10)[Φ]kl={ul(tk+1-tk),l=0,ulLT(el(tk+1)-el(tk))jlΩ,l≠0.Proof.
Since𝒫h∈ℋ, it can be written as (𝒫h)(t)=∑l=-LL‍hlel(t). Then from (8) we have
(11)qk=〈𝒫h,ϕk〉=∑l=-LLhlϕl,k¯.
Writing (11) for all k=1,2,…,n-1, we obtain q=Φh with [q]k=qk, [Φ]kl=ϕl,k¯ and [h]l=hl. This system of linear equations can be solved for h, provided that the rank r(Φ) of the matrix Φ satisfies r(Φ)=2L+1. A necessary condition for the latter is that the number of measurements qk is at least 2L+1, or, equivalently, the number of spikes n≥2L+2. Under this condition, the solution can be computed as h=Φ+q.Remark 10.
If the signalu is fed directly into the neuron, then ∫tktk+1‍(u*𝒫h)(t)dt=∫tktk+1‍u(t)dt, for k=1,2,…,n-1, that is, (𝒫h)(t)=K(t,0), t∈ℝ. In other words, if there is no processing on the input signal u, then the kernel K(t,0) in ℋ is identified as the filter projection. This is also illustrated in Figure 7.
In order to ensure that the neuron produces at least2L+1 measurements in a time interval of length T, it suffices to have tk+1-tk≤T/(2L+2). Since tk+1-tk≤Cδ/(b-c) for |v(t)|≤c<b, it suffices to have Cδ<(b-c)T/(2L+2). Using the definition of T=2πL/Ω and taking the limit as L→∞, we obtain the familiar Nyquist-type criterion Cδ<π(b-c)/Ω for a bandlimited stimulus u∈Ξ [4, 20] (see also Section 5.1).
Ideally, we would like to identify the impulse response of the filterh. Note that unlike h∈H, the projection 𝒫h belongs to the space ℋ. Nevertheless, under quite natural conditions on h (see Section 3.4), 𝒫h approximates h arbitrarily closely on t∈[0,T], provided that both the bandwidth and the order of the signal u are sufficiently large (see also Figure 9).The requirement of Lemma9 that the number of spikes n produced by the system in Figure 2(a) has to satisfy n≥2L+2 is quite stringent and may be hard to meet in practice, especially if the order L of the space ℋ is high. In that case we have the following result.Theorem 11 (SISO channel identification machine).
Let{ui∣ui∈ℋ}i=1N be a collection of N linearly independent stimuli at the input to a [Filter]-[Ideal IAF] circuit with h∈H. If the total number of spikes n=∑i=1N‍ni generated by the neuron satisfies n≥2L+N+1, then the filter projection 𝒫h can be perfectly identified from a collection of I/O pairs {(ui,𝕋i)}i=1N as
(12)(𝒫h)(t)=∑l=-LLhlel(t),
where h=Φ+q. Furthermore, Φ=[Φ1;Φ2;…;ΦN], q=[q1;q2;…;qN] and [qi]k=qki with each Φi of size (ni-1)×(2L+1) and qi of size (ni-1)×1. The elements of matrices Φi are given by
(13)[Φi]kl={uli(tk+1i-tki),l=0,uliLT(el(tk+1i)-el(tki))jlΩ,l≠0,
for all k=1,2,…,n-1, l=-L,-L+1,…,L, and i=1,2,…,N.Proof.
Since𝒫h∈ℋ, it can be written as (𝒫h)(t)=∑l=-LL‍hlel(t). Furthermore, since the stimuli are linearly independent, the measurements (qki)k=1ni-1 provided by the IAF neuron are distinct. Writing (5) for a stimulus ui, we obtain
(14)qki=〈𝒫h,ϕki〉=∑l=-LL‍hlϕl,ki¯,
or qi=Φih, with [qi]k=qki, [Φi]kl=ϕl,ki¯ and [h]l=hl. Repeating for all i=1,…,N, we get q=Φh with Φ=[Φ1;Φ2;…;ΦN] and q=[q1;q2;…;qN]. This system of linear equations can be solved for h, provided that the rank r(Φ) of matrix Φ satisfies r(Φ)=2L+1. A necessary condition for the latter is that the total number n=∑i=1N‍ni of spikes generated in response to all N signals satisfies n≥2L+N+1. Then, the solution can be computed as h=Φ+q. To find the coefficients ϕl,ki¯, we note that ϕl,ki=ℒki(el)¯ (see Lemma 8). Hence, the result follows.The time encoding interpretation of the channel identification problem for a SISO [Filter]-[Ideal IAF] circuit is shown in Figure4(a). The block diagram of the SISO CIM in Theorem 11 is shown in Figure 4(b). Note that the key idea behind the SISO CIM is the introduction of multiple linearly independent test signals ui∈ℋ, i=1,2,...,N. When the [Filter]-[Ideal IAF] circuit is producing very few measurements of 𝒫h in response to any given test signal ui, we use more signals to obtain additional measurements. We can do so and identify 𝒫h because 𝒫h∈ℋ is fixed. In contrast, identifying 𝒫h in a two-step deconvolving procedure requires reconstructing at least one vi. This is an ill-posed problem since each vi is signal-dependent and has a small number of associated measurements.SISO CIM algorithm for the [Filter]-[Ideal IAF] circuit. (a) Time encoding interpretation of the channel identification problem. (b) Block diagram of the SISO channel identification machine.
(a)
(b)


## 3.3. Examples

We now demonstrate the performance of the identification algorithms in Lemma9 and Theorem 11. First, we identify a filter in the SISO [Filter]-[Ideal IAF] circuit (Figure 2(a)) from a single I/O pair when this circuit produces a sufficient number of measurements in an interval of length T. Second, we identify the filter using multiple I/O pairs for the case when the number of measurements produced in response to any given input signal is small. Finally, we consider the SISO [Filter]-[Nonlinear Oscillator-ZCD] circuit with multiplicative coupling (Figure 2(b)) and identify its filter from multiple I/O pairs.


### 3.3.1. SISO [Filter]-[Ideal IAF] Circuit, Single I/O Pair

We model the dendritic processing filter using the causal linear kernel(15)h(t)=ce-αt[(αt)33!-(αt)55!],t∈[0,0.1]s,
with c=3 and α=200. The general form of this kernel was suggested in [24] as a plausible approximation to the temporal structure of a visual receptive field. Since the length of the filter support S=0.1s, we will need to use a signal with a period T≥0.1s. In Figure 5(a), we apply a signal u that is bandlimited to 25Hz and has a period of T=0.2s, that is, the order of the space L=T·Ω/(2π)=5. The biased output of the filter v=(u*h)+b is then fed into an ideal integrate-and-fire neuron (Figure 5(b)). Here the bias b guarantees that the output of the integrator reaches the threshold value in finite time. Whenever the biased filter output is above zero (Figure 5(b)), the membrane potential is increasing (Figure 5(c)). If the membrane potential ∫tkt‍[(u*h)(s)+b]ds reaches a threshold δ, a spike is generated by the neuron at a time tk+1 and the potential is reset to zero (Figure 5(c)). The resulting spike train (tk)k=1n at the output of the [Filter]-[Ideal IAF] circuit is shown in Figure 5(d). Note that the circuit generated a total of n=13 spikes in an interval of length T=0.2s. According to Theorem 14, we need at least n=2L+2=12 spikes, corresponding to 2L+1=11 measurements, in order to identify the projection 𝒫h of the filter h loss-free. Hence, for this particular example, it will suffice to use a single I/O pair (u,𝕋).Figure 5
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using a single I/O pair.  (a) An input signalu is bandlimited to 25 Hz. The order of the space is L=5. (b) The corresponding biased output of the filter v(t)+b. (c) The filter output in (b) is integrated by the ideal IAF neuron. Whenever the membrane potential reaches a threshold δ, a spike is produced by the neuron and the potential is reset to 0. (d) The neuron generated a total of 13 spikes. (e) The identified impulse response of the filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is -77.5dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. Note that supp(ℱK)=supp(ℱ𝒫h*)=[-Ω,Ω] but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h*∈ℋ but h∉ℋ.In Figure5(e), we plot the original impulse response of the filter h, the filter projection 𝒫h, and the filter 𝒫h*. The latter filter was identified using the algorithm in Theorem 14. Notice that the identified impulse response 𝒫h* (red) is quite different from h (dashed black). In contrast, and as expected, the blue and red curves corresponding, respectively, to 𝒫h and 𝒫h* are indistinguishable. The mean squared error (MSE) between 𝒫h* and 𝒫h amounts to −77.5 dB.The difference between𝒫h and h is further evaluated in Figures 5(f)–5(h). By definition of 𝒫h in (6), 𝒫h=h*K(·,0)¯, or ℱ(𝒫h)=ℱ(h)ℱ(K(·,0)) since K¯=K. Hence both the projection 𝒫h and the identified filter 𝒫h* will contain frequencies that are present in the reproducing kernel K, or equivalently in the input signal u. In Figure 5(f) we show the double-sided Fourier amplitude spectrum of K(t,0). As expected, we see that the kernel is bandlimited to 25Hz and contains 2L+1=11 distinct frequencies. On the other hand, as shown in Figure 5(g), the original filter h is not bandlimited (since it has a finite temporal support). As a result, the input signal u explores h in a limited spectrum of [-Ω,Ω]rad/s, effectively projecting h onto the space ℋ with Ω=2π·25rad/s and L=5. The Fourier amplitude spectrum of the identified projection 𝒫h* is shown in Figure 5(h).


### 3.3.2. SISO [Filter]-[Ideal IAF] Circuit, Multiple I/O Pairs

Next, we identify the projection ofh onto the space of functions that are bandlimited to 100 Hz and have the same period T=0.2s as in the first example. This means that the order L of the space of input signals ℋ is L=T·Ω/(2π)=20. In order to identify the projection 𝒫h loss-free, the neuron has to generate at least 2L+1=41 measurements. If the neuron produces about 13 spikes (12 measurements) on an interval of length T, as in the previous example, a single I/O pair will not suffice. However, we can still recover the projection 𝒫h if we use multiple I/O pairs.In Figure6 we illustrate identification of the filter using the algorithm in Theorem 11. A total of 48 spikes were produced by the neuron in response to four different signals u1,…,u4. Since 48>2L+N+1=45, the MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −73.3 dB.Figure 6
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using multiple I/O pairs.  (a) Input signalsu1,…,u4 are bandlimited to 100 Hz. The order of the space L=20.  (b) Biased output of the filter v1(t)+b in response to the stimulus u1.  (c) The filter output in (b) is integrated by an ideal IAF neuron.  (d) The neuron generated a total of 48 spikes in response to all 4 input signals.  (e) The identified impulse response 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −73.3 dB.  (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. Note that supp(ℱK)=[-Ω,Ω]=supp(ℱ𝒫h*) but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h*∈ℋ but h∉ℋ.Figure 7
Channel identification forh(t)=δ(t).  (a) Input signals u1,u2 are bandlimited to 50 Hz. The order of the space L=10. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) The filter output in (b) is integrated by an ideal IAF neuron. (d) The neuron generated a total of 28 spikes in response to 2 input signals. (e) The identified filter 𝒫h* (red) is exactly the kernel K(t,0) for ℋΩ,L1 with Ω=2π·10rad/s and L=10. Also shown is the original filter h=δ (dashed black) and its projection 𝒫h=δ*K(·,0)¯=K(·,0) (blue). The MSE between 𝒫h* and 𝒫h is −87.6 dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. As before, 𝒫h*∈ℋ but h∉ℋ.


### 3.3.3. SISO [Filter]-[Ideal IAF] Circuit,h(t)=δ(t)

Now we consider a special case when the channel does not alter the input signal, that is, whenh(t)=δ(t), t∈ℝ, is the Dirac delta function. As explained in Remark 10, the CIM should identify the projection of δ(t) onto ℋ, that is, it should identify the kernel K(t,0). This is indeed the case as shown in Figure 7.


### 3.3.4. SISO [Filter]-[Nonlinear Oscillator-ZCD] Circuit, Multiple I/O Pairs

Next we consider a SISO circuit consisting of a channel in cascade with a nonlinear dynamical system that has a stable limit cycle. We assume that the (positive) output of the channelv(t)+b is multiplicatively coupled to the dynamical system (Figure 2(b)) so that the circuit is governed by a set of equations
(16)dydt=(v(t)+b)f(y).
A system (16) followed by a zero-crossing detector is an example of a TEM with multiplicative coupling and has been previously investigated in [6]. It can be shown that such a TEM is input/output equivalent to an IAF neuron with a threshold δ that is equal to the period of the dynamical system on a stable limit cycle [6].As an example, we consider a [Filter]-[van der Pol - ZCD] TEM with the van der Pol oscillator described by a set of equations(17)dy1dt=(u*h+b)[μ(y1-13y13)-y2],dy2dt=(u*h+b)y1,
where μ is the damping coefficient [6]. We assume that y1 is the only observable state of the oscillator and without loss of generality we choose the zero phase of the limit cycle to be the peak of y1.In Figure8, we show the results of a simulation in which a SISO CIM was used to identify the channel. Input signals (Figure 8(a)) were bandlimited to 50 Hz and had a period T=0.5s, that is, L=25. In the absence of an input, that is, when u=0, a constant bias b=1 (Figure 8(b)) resulted a in period of 34.7 ms on a stable limit cycle (Figure 8(e)). As seen in Figures 8(b) and 8(c), downward/upward deviations of v1(t)+b in response to u1 resulted in the slowing-down/speeding-up of the oscillator. In order to identify the filter projection onto a space of order L=25 loss-free, we used a total of n=56 zeros at the output of the zero-crossing detector (Figure 8(d)). This is 1 more zero than the rank requirement of 2L+N+1=55 zeros, or equivalently of 2L+1=51 measurements. The MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −66.6 dB.Figure 8
Channel identification in a SISO [Filter]-[van der Pol-ZCD] circuit using multiple I/O pairs. (a) Input signalsu1,…,u4 are bandlimited to 50 Hz. The order of the space L=25. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) Downward and upward deviations of v1(t)+b from the bias b cause the oscillator to slow down and to speed up, respectively. The damping coefficient μ=20. (d) The oscillator produced a total of 56 spikes in response to 4 stimuli. Here spikes correspond to the peaks of the observable state variable y11. (e) A limit cycle of the van der Pol oscillator for μ=20 is shown in the phase plane. In the absence of channel output, the bias b resulted in a constant period of oscillation T(b)=34.7ms. The red dot denotes the zero phase (spike) of an oscillation. (f) The identified filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −66.6 dB. (g) Fourier amplitude spectra of h and 𝒫h*. As before, 𝒫h*∈ℋ but h∉ℋ.Comparison betweenh and 𝒫h in time and frequency domains.  (a) h (red) and its projection 𝒫h (blue) are shown for several values of Ω and L in the time domain. Ω=2π·20rad/s, 2π·50rad/s and 2π·100rad/s in the top, middle and bottom row, respectively. The period T is fixed at T=0.2s in the left column and T=0.5s in the right column. (b) Fourier amplitude spectra of h (red) and 𝒫h (blue) for the same values of Ω and L as in (a). Note that the differentiating filter h clearly removes the zero-frequency (dc) coefficient corresponding to l=0 in all cases.
(a)
Time domain
(b)
Frequency domain


## 3.3.1. SISO [Filter]-[Ideal IAF] Circuit, Single I/O Pair

We model the dendritic processing filter using the causal linear kernel(15)h(t)=ce-αt[(αt)33!-(αt)55!],t∈[0,0.1]s,
with c=3 and α=200. The general form of this kernel was suggested in [24] as a plausible approximation to the temporal structure of a visual receptive field. Since the length of the filter support S=0.1s, we will need to use a signal with a period T≥0.1s. In Figure 5(a), we apply a signal u that is bandlimited to 25Hz and has a period of T=0.2s, that is, the order of the space L=T·Ω/(2π)=5. The biased output of the filter v=(u*h)+b is then fed into an ideal integrate-and-fire neuron (Figure 5(b)). Here the bias b guarantees that the output of the integrator reaches the threshold value in finite time. Whenever the biased filter output is above zero (Figure 5(b)), the membrane potential is increasing (Figure 5(c)). If the membrane potential ∫tkt‍[(u*h)(s)+b]ds reaches a threshold δ, a spike is generated by the neuron at a time tk+1 and the potential is reset to zero (Figure 5(c)). The resulting spike train (tk)k=1n at the output of the [Filter]-[Ideal IAF] circuit is shown in Figure 5(d). Note that the circuit generated a total of n=13 spikes in an interval of length T=0.2s. According to Theorem 14, we need at least n=2L+2=12 spikes, corresponding to 2L+1=11 measurements, in order to identify the projection 𝒫h of the filter h loss-free. Hence, for this particular example, it will suffice to use a single I/O pair (u,𝕋).Figure 5
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using a single I/O pair.  (a) An input signalu is bandlimited to 25 Hz. The order of the space is L=5. (b) The corresponding biased output of the filter v(t)+b. (c) The filter output in (b) is integrated by the ideal IAF neuron. Whenever the membrane potential reaches a threshold δ, a spike is produced by the neuron and the potential is reset to 0. (d) The neuron generated a total of 13 spikes. (e) The identified impulse response of the filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is -77.5dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. Note that supp(ℱK)=supp(ℱ𝒫h*)=[-Ω,Ω] but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h*∈ℋ but h∉ℋ.In Figure5(e), we plot the original impulse response of the filter h, the filter projection 𝒫h, and the filter 𝒫h*. The latter filter was identified using the algorithm in Theorem 14. Notice that the identified impulse response 𝒫h* (red) is quite different from h (dashed black). In contrast, and as expected, the blue and red curves corresponding, respectively, to 𝒫h and 𝒫h* are indistinguishable. The mean squared error (MSE) between 𝒫h* and 𝒫h amounts to −77.5 dB.The difference between𝒫h and h is further evaluated in Figures 5(f)–5(h). By definition of 𝒫h in (6), 𝒫h=h*K(·,0)¯, or ℱ(𝒫h)=ℱ(h)ℱ(K(·,0)) since K¯=K. Hence both the projection 𝒫h and the identified filter 𝒫h* will contain frequencies that are present in the reproducing kernel K, or equivalently in the input signal u. In Figure 5(f) we show the double-sided Fourier amplitude spectrum of K(t,0). As expected, we see that the kernel is bandlimited to 25Hz and contains 2L+1=11 distinct frequencies. On the other hand, as shown in Figure 5(g), the original filter h is not bandlimited (since it has a finite temporal support). As a result, the input signal u explores h in a limited spectrum of [-Ω,Ω]rad/s, effectively projecting h onto the space ℋ with Ω=2π·25rad/s and L=5. The Fourier amplitude spectrum of the identified projection 𝒫h* is shown in Figure 5(h).


## 3.3.2. SISO [Filter]-[Ideal IAF] Circuit, Multiple I/O Pairs

Next, we identify the projection ofh onto the space of functions that are bandlimited to 100 Hz and have the same period T=0.2s as in the first example. This means that the order L of the space of input signals ℋ is L=T·Ω/(2π)=20. In order to identify the projection 𝒫h loss-free, the neuron has to generate at least 2L+1=41 measurements. If the neuron produces about 13 spikes (12 measurements) on an interval of length T, as in the previous example, a single I/O pair will not suffice. However, we can still recover the projection 𝒫h if we use multiple I/O pairs.In Figure6 we illustrate identification of the filter using the algorithm in Theorem 11. A total of 48 spikes were produced by the neuron in response to four different signals u1,…,u4. Since 48>2L+N+1=45, the MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −73.3 dB.Figure 6
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using multiple I/O pairs.  (a) Input signalsu1,…,u4 are bandlimited to 100 Hz. The order of the space L=20.  (b) Biased output of the filter v1(t)+b in response to the stimulus u1.  (c) The filter output in (b) is integrated by an ideal IAF neuron.  (d) The neuron generated a total of 48 spikes in response to all 4 input signals.  (e) The identified impulse response 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −73.3 dB.  (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. Note that supp(ℱK)=[-Ω,Ω]=supp(ℱ𝒫h*) but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h*∈ℋ but h∉ℋ.Figure 7
Channel identification forh(t)=δ(t).  (a) Input signals u1,u2 are bandlimited to 50 Hz. The order of the space L=10. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) The filter output in (b) is integrated by an ideal IAF neuron. (d) The neuron generated a total of 28 spikes in response to 2 input signals. (e) The identified filter 𝒫h* (red) is exactly the kernel K(t,0) for ℋΩ,L1 with Ω=2π·10rad/s and L=10. Also shown is the original filter h=δ (dashed black) and its projection 𝒫h=δ*K(·,0)¯=K(·,0) (blue). The MSE between 𝒫h* and 𝒫h is −87.6 dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h*. As before, 𝒫h*∈ℋ but h∉ℋ.


## 3.3.3. SISO [Filter]-[Ideal IAF] Circuit,h(t)=δ(t)

Now we consider a special case when the channel does not alter the input signal, that is, whenh(t)=δ(t), t∈ℝ, is the Dirac delta function. As explained in Remark 10, the CIM should identify the projection of δ(t) onto ℋ, that is, it should identify the kernel K(t,0). This is indeed the case as shown in Figure 7.


## 3.3.4. SISO [Filter]-[Nonlinear Oscillator-ZCD] Circuit, Multiple I/O Pairs

Next we consider a SISO circuit consisting of a channel in cascade with a nonlinear dynamical system that has a stable limit cycle. We assume that the (positive) output of the channelv(t)+b is multiplicatively coupled to the dynamical system (Figure 2(b)) so that the circuit is governed by a set of equations
(16)dydt=(v(t)+b)f(y).
A system (16) followed by a zero-crossing detector is an example of a TEM with multiplicative coupling and has been previously investigated in [6]. It can be shown that such a TEM is input/output equivalent to an IAF neuron with a threshold δ that is equal to the period of the dynamical system on a stable limit cycle [6].As an example, we consider a [Filter]-[van der Pol - ZCD] TEM with the van der Pol oscillator described by a set of equations(17)dy1dt=(u*h+b)[μ(y1-13y13)-y2],dy2dt=(u*h+b)y1,
where μ is the damping coefficient [6]. We assume that y1 is the only observable state of the oscillator and without loss of generality we choose the zero phase of the limit cycle to be the peak of y1.In Figure8, we show the results of a simulation in which a SISO CIM was used to identify the channel. Input signals (Figure 8(a)) were bandlimited to 50 Hz and had a period T=0.5s, that is, L=25. In the absence of an input, that is, when u=0, a constant bias b=1 (Figure 8(b)) resulted a in period of 34.7 ms on a stable limit cycle (Figure 8(e)). As seen in Figures 8(b) and 8(c), downward/upward deviations of v1(t)+b in response to u1 resulted in the slowing-down/speeding-up of the oscillator. In order to identify the filter projection onto a space of order L=25 loss-free, we used a total of n=56 zeros at the output of the zero-crossing detector (Figure 8(d)). This is 1 more zero than the rank requirement of 2L+N+1=55 zeros, or equivalently of 2L+1=51 measurements. The MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −66.6 dB.Figure 8
Channel identification in a SISO [Filter]-[van der Pol-ZCD] circuit using multiple I/O pairs. (a) Input signalsu1,…,u4 are bandlimited to 50 Hz. The order of the space L=25. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) Downward and upward deviations of v1(t)+b from the bias b cause the oscillator to slow down and to speed up, respectively. The damping coefficient μ=20. (d) The oscillator produced a total of 56 spikes in response to 4 stimuli. Here spikes correspond to the peaks of the observable state variable y11. (e) A limit cycle of the van der Pol oscillator for μ=20 is shown in the phase plane. In the absence of channel output, the bias b resulted in a constant period of oscillation T(b)=34.7ms. The red dot denotes the zero phase (spike) of an oscillation. (f) The identified filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −66.6 dB. (g) Fourier amplitude spectra of h and 𝒫h*. As before, 𝒫h*∈ℋ but h∉ℋ.Comparison betweenh and 𝒫h in time and frequency domains.  (a) h (red) and its projection 𝒫h (blue) are shown for several values of Ω and L in the time domain. Ω=2π·20rad/s, 2π·50rad/s and 2π·100rad/s in the top, middle and bottom row, respectively. The period T is fixed at T=0.2s in the left column and T=0.5s in the right column. (b) Fourier amplitude spectra of h (red) and 𝒫h (blue) for the same values of Ω and L as in (a). Note that the differentiating filter h clearly removes the zero-frequency (dc) coefficient corresponding to l=0 in all cases.
(a)
Time domain
(b)
Frequency domain


## 3.4. Convergence of the SISO CIM Estimate

Recall, that the original problem of interest is that of recovering the impulse response of the filterh. The CIM lets us identify the projection 𝒫h of that filter onto the input space. A natural question to ask is whether 𝒫h converges to h and if so how and under what conditions. We formalize this below.Proposition 12.
If∫0T‍|h(t)|2dt<∞, then 𝒫h→h in the L2 norm and almost everywhere on t∈[0,T] with increasing Ω, L and fixed T.Proof.
LetT=2πL/Ω=const. Then K(t,0)=(1/T)∑l=-LLe(jΩl/L)t=(1/T)∑l=-LL‍e(j2πl/T)t and by definition of 𝒫h in (6), we have
(18)(𝒫h)(t)=∫0T[1T∑l=-LLe(j2πl/T)(t-s)]h(s)ds=∑l=-LL[1T∫0Th(s)e-(j2πl/T)sds]e(j2πl/T)t=∑l=-LLh^(l)e(j2πl/T)t=SLh(t),
where SLh is the Lth partial sum of the Fourier series of h and h^(l) is the lth Fourier coefficient. Hence the problem of convergence of 𝒫h to h is the same as that of the convergence of the Fourier series of h. We thus have convergence in the L2 norm and convergence almost everywhere follows from Carleson's theorem [23].Remark 13.
More generally, if∫0T‍|h(t)|pdt<∞, p∈(1,∞), then 𝒫h→h in the Lp norm and almost everywhere by Hunt's theorem [23].It follows from Proposition12 that 𝒫h approximates h arbitrarily closely (in the L2 norm, or MSE sense), given an appropriate choice of Ω and L. Since the number of measurements needed to identify the projection 𝒫h increases linearly with L, a single channel identification problem leads us to consider a countably infinite number of time encoding problems in order to identify the impulse response of the filter with arbitrary precision. To provide further intuition about the relationship between h and 𝒫h, we compare the two in time and frequency domains for multiple values of Ω and L in Figure 9.


## 4. MISO Channel Identification Machines

In this section we consider the identification of a bank ofM filters with impulse responses hm, m=1,2,…,M. We present a MISO CIM algorithm in Section 4.1, followed by an example demonstrating its performance in Section 4.2.


### 4.1. An Identification Algorithm for MISO Channels

Consider now the MISO ASDM-based circuit in Figure2(c), where the signal u=[u1(t),u2(t),…,uM(t)]T, t∈[0,T], M∈ℕ, is transformed into the time sequence (tk)k=1n. This circuit is also an instance of a TEM and (assuming z(t1)=b) its t-transform is given by
(19)∫tktk+1∑m=1M(um*hm)(s)ds=〈v,ϕk〉=qk,
where v=∑m‍(um*hm)(t), ϕk∈ℋ with ϕk=∑l‍ϕl,kel(t) and qk=(-1)k[2Cδ-b(tk+1-tk)]. One simple way to identify filters hm, m=1,2,…,M, is to identify them one by one as in Theorem 11. For instance, this can be achieved by applying signals of the form u=[0,…,0,um,0,…,0] when identifying the filter hm. In a number of applications, most notably in early olfaction [25], this model of system identification cannot be applied. An alternative procedure that allows to identify all filters at once is given below.Theorem 14 (MISO channel identification machine).
Let{ui∣ui∈ℋM}i=1N be a collection of N linearly-independent vector-valued signals at the input of a MISO [Filter]-[ASDM-ZCD] circuit with filters hm∈H, m=1,…,M. The filter projections 𝒫hm can be perfectly identified from a collection of I/O pairs {(ui,𝕋i)}i=1N as
(20)(𝒫hm)(t)=∑l=-LLhlmel(t),m=1,…,M. Here the coefficients hlm are given by h=Φ+q with q=[q1,q2,…,qN]T, [qi]k=qki and h=[h-L1,…,h-LM,h-L+11,…,h-L+1M,…,hL1,…,hLM]T, provided that the matrix Φ has rank r(Φ)=M(2L+1). The matrix Φ is given by
(21)Φ=[Φ10⋯00Φ2⋯0⋮⋮⋱⋮00⋯ΦN][U1U2⋮UN],withUi=[u-Li0⋯00u-L+1i⋯0⋮⋮⋱⋮00⋯uLi],
where uli=[uli1,uli2,…,uliM], i=1,2,…,N. Finally, the elements of matrix Φi are given by
(22)[Φi]kl={(tk+1i-tki),l=0,LT(el(tk+1i)-el(tki))jlΩ,l≠0.Proof.
Since𝒫hm∈ℋ for all m=1,…,M, it can be written as (𝒫hm)(t)=∑l=-LL‍hlmel(t). Hence, for the mth component of the stimulus ui we get (uim*hm)(t)=(uim*𝒫hm)(t)=T∑l=-LL‍hlmulimel(t) and
(23)vi(t)=∑m=1MT∑l=-LLhlmulimel(t).
Using the definition of ϕki=∑l=-LL‍ϕl,kiel(t) and substituting (23) into the t-transform (19), we obtain
(24)qki=〈vi,ϕki〉=∑m=1M∑l=-LLT‍hlmulimϕl,ki¯,
or qi=ΦiUih with [qi]k=qki, [Φi]kl=T·ϕl,ki¯, Ui=diag(u-Li,…,uLi), uli=[uli1,…,uliM] and h=[h-L1,…,h-LM,h-L+11,…,h-L+1M,…,hL1,…,hLM]T. Repeating for all stimuli ui, i=1,…,N, we obtain q=Φh with Φ as specified in (21). This system of linear equations can be solved for h, provided that the rank of Φ satisfies the condition r(Φ)=M(2L+1). To find the coefficients ϕl,ki¯, we note that ϕl,ki=ℒki(el)¯. Hence, the result follows.The MIMO time-encoding interpretation of the channel identification problem for a MISO [Filter]-[ASDM-ZCD] circuit is shown in Figure10(a). The block diagram of the MISO CIM in Theorem 14 is shown in Figure 10(b).MISO CIM algorithm for the [Filter]-[ASDM-ZCD] circuit. (a) Time encoding interpretation of the MISO channel identification problem. (b) Block diagram of the MISO channel identification machine.
(a)
(b)Remark 15.
From (23), we see that vi=∑l=-LL‍vliel(t) with vli=T∑m=1M‍hlmulim. Writing this for all i=1,…,N, we obtain vl=Ulhl, where [Ul]im=Tulim, hl=[hl1,hl2,…,hlM]T and vl=[vl1,vl2,…,vlN]T. In order to identify the multidimensional channel this system of equations must have a solution for every l. A necessary condition for the latter is that N≥M, that is, the number N of test signals ui is greater than the number of signal components M.Remark 16.
The rank conditionr(Φ)=M(2L+1) can be satisfied by increasing the number N of input signals ui. Specifically, if on average the system is providing ν measurements in a time interval t∈[0,T], then the minimum number of test signals is N=⌈M(2L+1)/ν⌉.


### 4.2. Example: MISO [Filter]-[ASDM-ZCD] Circuit

We now describe simulation results for identifying the channel in a MISO [Filter]-[ASDM - ZCD] circuit of Figure2(c). We use three different filters:
(25)h1(t)=ce-αt[(αt)33!-(αt)55!],h2(t)=h1(t-β),h3(t)=-h1(t),
with t∈[0,0.1] s, c=3 and α=200 and β=20ms. All N=5 signals are bandlimited to 100 Hz and have a period of T=0.2s, that is, the order of the space L=20. According to Theorem 14, the ASDM has to generate a total of at least M(2L+1)+N=128 trigger times in order to identify the projections 𝒫h1, 𝒫h2 and 𝒫h3 loss-free. We use all five triplets ui=[ui1,ui2,ui3], i=1,…,5, to produce 131 trigger times.A single such tripletu1 is shown in Figure 11(a). The corresponding biased aggregate channel output v1(t)-z1(t) is shown in Figure 11(b). Since the Schmitt trigger output z(t) switches between +b and -b (Figure 11(d)), the signal v1(t)-z1(t) is piece-wise continuous. Figure 11(c) shows the integrator output. Note that when z(t)=-b, the channel output is positively biased and the integrator output ∫tkt‍[v1(s)-z(s)]ds is compared against a threshold +δ. As soon as that threshold is reached, the Schmitt trigger output switches to z(t)=b and the negatively-biased channel output is compared to a threshold -δ. Passing the ASDM output z1(t) through a zero-crossing device (Figure 11(d)), we obtain a corresponding sequence of trigger times (tk1)k=122. The set of all 131 trigger times is shown in Figure 11(e). Three identified filters 𝒫h1*, 𝒫h2* and 𝒫h3* are plotted in Figures 11(f)–11(h). The MSE between filter projections and filters recovered by the algorithm in Theorem 14 is on the order of −60 dB.Figure 11
Channel identification in a MISO [FIlter]-[ASDM] circuit using multiple I/O pairs.  (a) An input triplet signalu1=[u11,u12,u13] is bandlimited to 100 Hz. The order of the space L=20. (b) Biased aggregate output of the channel v1(t)-z1(t) in response to the triplet u1. (c) Integrator output ∫tkt‍[v1(s)-z1(s)]ds (blue) is compared against two thresholds +δ and -δ (dashed red). Trigger times of the noninverting Schmitt trigger are indicated by red dots. (d) The ASDM output z1(t) (blue) is passed through a zero-crossing detector to produce a sequence of trigger times (tk1)k=122. (e) A total of 131 trigger times were generated by the ASDM in response to five input triplets. (f)–(h) Identified filters 𝒫h1* (red), 𝒫h2* (green) and 𝒫h3* (blue) are shown together with the original filters h1, h2, h3 (dashed black) and their projections 𝒫h1, 𝒫h2 and 𝒫h3 (black). The MSE achieved by the identification algorithm is less than −60 dB.


## 4.1. An Identification Algorithm for MISO Channels

Consider now the MISO ASDM-based circuit in Figure2(c), where the signal u=[u1(t),u2(t),…,uM(t)]T, t∈[0,T], M∈ℕ, is transformed into the time sequence (tk)k=1n. This circuit is also an instance of a TEM and (assuming z(t1)=b) its t-transform is given by
(19)∫tktk+1∑m=1M(um*hm)(s)ds=〈v,ϕk〉=qk,
where v=∑m‍(um*hm)(t), ϕk∈ℋ with ϕk=∑l‍ϕl,kel(t) and qk=(-1)k[2Cδ-b(tk+1-tk)]. One simple way to identify filters hm, m=1,2,…,M, is to identify them one by one as in Theorem 11. For instance, this can be achieved by applying signals of the form u=[0,…,0,um,0,…,0] when identifying the filter hm. In a number of applications, most notably in early olfaction [25], this model of system identification cannot be applied. An alternative procedure that allows to identify all filters at once is given below.Theorem 14 (MISO channel identification machine).
Let{ui∣ui∈ℋM}i=1N be a collection of N linearly-independent vector-valued signals at the input of a MISO [Filter]-[ASDM-ZCD] circuit with filters hm∈H, m=1,…,M. The filter projections 𝒫hm can be perfectly identified from a collection of I/O pairs {(ui,𝕋i)}i=1N as
(20)(𝒫hm)(t)=∑l=-LLhlmel(t),m=1,…,M. Here the coefficients hlm are given by h=Φ+q with q=[q1,q2,…,qN]T, [qi]k=qki and h=[h-L1,…,h-LM,h-L+11,…,h-L+1M,…,hL1,…,hLM]T, provided that the matrix Φ has rank r(Φ)=M(2L+1). The matrix Φ is given by
(21)Φ=[Φ10⋯00Φ2⋯0⋮⋮⋱⋮00⋯ΦN][U1U2⋮UN],withUi=[u-Li0⋯00u-L+1i⋯0⋮⋮⋱⋮00⋯uLi],
where uli=[uli1,uli2,…,uliM], i=1,2,…,N. Finally, the elements of matrix Φi are given by
(22)[Φi]kl={(tk+1i-tki),l=0,LT(el(tk+1i)-el(tki))jlΩ,l≠0.Proof.
Since𝒫hm∈ℋ for all m=1,…,M, it can be written as (𝒫hm)(t)=∑l=-LL‍hlmel(t). Hence, for the mth component of the stimulus ui we get (uim*hm)(t)=(uim*𝒫hm)(t)=T∑l=-LL‍hlmulimel(t) and
(23)vi(t)=∑m=1MT∑l=-LLhlmulimel(t).
Using the definition of ϕki=∑l=-LL‍ϕl,kiel(t) and substituting (23) into the t-transform (19), we obtain
(24)qki=〈vi,ϕki〉=∑m=1M∑l=-LLT‍hlmulimϕl,ki¯,
or qi=ΦiUih with [qi]k=qki, [Φi]kl=T·ϕl,ki¯, Ui=diag(u-Li,…,uLi), uli=[uli1,…,uliM] and h=[h-L1,…,h-LM,h-L+11,…,h-L+1M,…,hL1,…,hLM]T. Repeating for all stimuli ui, i=1,…,N, we obtain q=Φh with Φ as specified in (21). This system of linear equations can be solved for h, provided that the rank of Φ satisfies the condition r(Φ)=M(2L+1). To find the coefficients ϕl,ki¯, we note that ϕl,ki=ℒki(el)¯. Hence, the result follows.The MIMO time-encoding interpretation of the channel identification problem for a MISO [Filter]-[ASDM-ZCD] circuit is shown in Figure10(a). The block diagram of the MISO CIM in Theorem 14 is shown in Figure 10(b).MISO CIM algorithm for the [Filter]-[ASDM-ZCD] circuit. (a) Time encoding interpretation of the MISO channel identification problem. (b) Block diagram of the MISO channel identification machine.
(a)
(b)Remark 15.
From (23), we see that vi=∑l=-LL‍vliel(t) with vli=T∑m=1M‍hlmulim. Writing this for all i=1,…,N, we obtain vl=Ulhl, where [Ul]im=Tulim, hl=[hl1,hl2,…,hlM]T and vl=[vl1,vl2,…,vlN]T. In order to identify the multidimensional channel this system of equations must have a solution for every l. A necessary condition for the latter is that N≥M, that is, the number N of test signals ui is greater than the number of signal components M.Remark 16.
The rank conditionr(Φ)=M(2L+1) can be satisfied by increasing the number N of input signals ui. Specifically, if on average the system is providing ν measurements in a time interval t∈[0,T], then the minimum number of test signals is N=⌈M(2L+1)/ν⌉.


## 4.2. Example: MISO [Filter]-[ASDM-ZCD] Circuit

We now describe simulation results for identifying the channel in a MISO [Filter]-[ASDM - ZCD] circuit of Figure2(c). We use three different filters:
(25)h1(t)=ce-αt[(αt)33!-(αt)55!],h2(t)=h1(t-β),h3(t)=-h1(t),
with t∈[0,0.1] s, c=3 and α=200 and β=20ms. All N=5 signals are bandlimited to 100 Hz and have a period of T=0.2s, that is, the order of the space L=20. According to Theorem 14, the ASDM has to generate a total of at least M(2L+1)+N=128 trigger times in order to identify the projections 𝒫h1, 𝒫h2 and 𝒫h3 loss-free. We use all five triplets ui=[ui1,ui2,ui3], i=1,…,5, to produce 131 trigger times.A single such tripletu1 is shown in Figure 11(a). The corresponding biased aggregate channel output v1(t)-z1(t) is shown in Figure 11(b). Since the Schmitt trigger output z(t) switches between +b and -b (Figure 11(d)), the signal v1(t)-z1(t) is piece-wise continuous. Figure 11(c) shows the integrator output. Note that when z(t)=-b, the channel output is positively biased and the integrator output ∫tkt‍[v1(s)-z(s)]ds is compared against a threshold +δ. As soon as that threshold is reached, the Schmitt trigger output switches to z(t)=b and the negatively-biased channel output is compared to a threshold -δ. Passing the ASDM output z1(t) through a zero-crossing device (Figure 11(d)), we obtain a corresponding sequence of trigger times (tk1)k=122. The set of all 131 trigger times is shown in Figure 11(e). Three identified filters 𝒫h1*, 𝒫h2* and 𝒫h3* are plotted in Figures 11(f)–11(h). The MSE between filter projections and filters recovered by the algorithm in Theorem 14 is on the order of −60 dB.Figure 11
Channel identification in a MISO [FIlter]-[ASDM] circuit using multiple I/O pairs.  (a) An input triplet signalu1=[u11,u12,u13] is bandlimited to 100 Hz. The order of the space L=20. (b) Biased aggregate output of the channel v1(t)-z1(t) in response to the triplet u1. (c) Integrator output ∫tkt‍[v1(s)-z1(s)]ds (blue) is compared against two thresholds +δ and -δ (dashed red). Trigger times of the noninverting Schmitt trigger are indicated by red dots. (d) The ASDM output z1(t) (blue) is passed through a zero-crossing detector to produce a sequence of trigger times (tk1)k=122. (e) A total of 131 trigger times were generated by the ASDM in response to five input triplets. (f)–(h) Identified filters 𝒫h1* (red), 𝒫h2* (green) and 𝒫h3* (blue) are shown together with the original filters h1, h2, h3 (dashed black) and their projections 𝒫h1, 𝒫h2 and 𝒫h3 (black). The MSE achieved by the identification algorithm is less than −60 dB.


## 5. Generalizations

We shall briefly generalize the results presented in previous sections in two important directions. First, we consider a general class of signal spaces for test signals in Section5.1. Then we discus channel models with noisy observations in Section 5.2.


### 5.1. Hilbert Spaces and RKHSs for Input Signals

Until now we have presented channel identification results for a particular space of input signals, namely the space of trigonometric polynomials. The finite-dimensionality of this space and the simplicity of the associated inner product makes it an attractive space to work with when implementing a SISO or a MISO CIM algorithm. However, fundamentally the identification methodology relied on the the geometry of the Hilbert space of test signals [5, 26]; computational tractability was based on kernel representations in an RKHS.Theorem 17.
Let{ui∣ui∈ℋ(I)}i=1N be a collection of N linearly independent and bounded stimuli at the input of a [Filter]-[Asynchronous Sampler] circuit with a linear processing filter h∈H and the t-transform
(26)ℒki(𝒫h)=qki,
where ℒki:ℋ→ℝ is a bounded linear functional mapping 𝒫h into a measurement qki. Then there is a set of sampling functions {(ϕki)k∈ℤ}i=1N, in ℋ such that
(27)qki=〈𝒫h,ϕki〉,
for all k∈ℤ, i=1,2,…,N. Furthermore, if ℋ is an RKHS with a kernel K(s,t), s,t∈I, then ϕki(t)=ℒki(K(·,t))¯. Let the set of representation functions {(ψki)k∈ℤ}i=1N, span the Hilbert space ℋ. Then
(28)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t).
Finally, if {(ϕki)k∈ℤ}i=1N and {(ψki)k∈ℤ}i=1N are orthogonal basis or frames for ℋ, then the filter coefficients amount to h=Φ+q, where h=[h1,h2,…,hN]T with [hi]k=hki, [Φij]lk=〈ϕli,ψkj〉 and q=[q1,q2,…,qN]T with [qi]l=qki  for all i,j=1,2,…,N, and k,l∈ℤ.Proof.
By the Riesz representation theorem, since the linear functionalℒki:ℋ→ℝ is bounded, there is a set of sampling functions {(ϕki)k∈ℤ}i=1N in ℋ such that ℒki(𝒫h)=〈𝒫h,ϕki〉. If ℋ is an RKHS, a sampling function ϕki can be computed using the reproducing property of the kernel K as in
(29)ϕki(t)=〈ϕki,K(·,t)〉≡〈K(·,t),ϕki〉¯=ℒki(K(·,t))¯.
Finally, writing all inner products 〈ϕki,𝒫h〉=qki yields, with the notation above, a system of linear equations Φh=q and the fiter coefficients amount to h=Φ+q.


#### 5.1.1. Example: Paley-Wiener Space

As an example, we consider the Paley-Wiener space which is closely related to the space of trigonometric polynomials. Specifically, the finite-dimensional spaceℋ can be thought of as a discretized version of the infinite-dimensional Paley-Wiener space
(30)Ξ={u∈𝕃2(ℝ)∣supp(ℱu)⊆[-Ω,Ω]}
in the frequency domain. An element u∈ℋ has a line spectrum at frequencies lΩ/L, l=-L,-L+1,…,L. This spectrum becomes dense in [-Ω,Ω] as L→∞. The space Ξ with the inner product 〈·,·〉:Ξ×Ξ→ℝ given by
(31)〈u,w〉=∫ℝu(t)w(t)dt
is also an RKHS with an RK [22]
(32)K(s,t)=sin(Ω(t-s))π(t-s),
with t,s∈ℝ. Defining the projection of the filter h onto Ξ as (𝒫h)(t)=∫ℝ‍h(s)K(s,t)¯ds, we find that Lemma 8 still holds with ϕk∈Ξ and we can extend Theorem 11 to the following.Proposition 18.
Let{ui∣supp(ℱui)=[-Ω,Ω]}i=1N be a collection of N linearly independent and bounded stimuli at the input of a [Filter]-[Ideal IAF] neural circuit with a dendritic processing filter h∈H. If ∑j=1N‍(b/Cδ)>Ω/π, then (𝒫h)(t) can be perfectly identified from the collection of I/O pairs {(ui,𝕋i)}i=1N as
(33)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t),
where ψki(t)=K(t,tki), i=1,2,…,N, and k∈ℤ. Finally, h=Φ+q, where h=[h1,h2,…,hN]T with [hi]k=hki, [Φij]lk=∫tlitl+1i‍ui(s-tkj)ds and q=[q1,q2,…,qN]T with [qi]l=Cδ-b(tl+1i-tli) for all i,j=1,2,…,N, and k,l∈ℤ.Proof.
As before, the spikes(tki)k∈ℤ in response to each test signal ui, i=1,2,…,N, represent distinct measurements qki=〈ϕki,𝒫h〉 of (𝒫h)(t). Thus we can think of the {(qki)k∈ℤ}i=1N's as projections of 𝒫h onto {(ϕki)k∈ℤ}i=1N, where ϕki(t)=ℒki(K(·,t))=∫tkitk+1i‍∫ℝ‍ui(z)K(s-z,t)dzds=∫tkitk+1i‍ui(s-t)ds. Since the signals are linearly independent and bounded [5], it follows that, if ∑i=1N‍(b/Cδ)>Ω/π or equivalently if the number of test signals N>ΩCδ/πb, the set of functions {(ψki)k∈ℤ}i=1N with ψki(t)=K(t,tki), is a frame for Ξ [5, 26]. Hence
(34)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t).
If the set of functions {(ϕki)k∈ℤ}i=1N forms a frame for Ξ, we can find the coefficients hki,k∈ℤ, i=1,2,…,N, by taking the inner product of (34) with each element of {ϕli(t)}i=1N: 〈ϕli,𝒫h〉=∑k∈ℤ‍hk1〈ϕli,ψk1〉+∑k∈ℤ‍hk2〈ϕli,ψk2〉+⋯+∑k∈ℤ‍hkN〈ϕli,ψkN〉≡qli, for i=1,2,…,N, l∈ℤ. Letting [Φij]lk=〈ϕli,ψkj〉, we obtain
(35)qli=∑k∈ℤ‍[Φi1]lkhk1+∑k∈ℤ‍[Φi2]lkhk2+⋯+∑k∈ℤ‍[ΦiN]lkhkN,
for i=1,2,…,N, l∈ℤ. Writing (35) in matrix form, we have q=Φh with [Φij]lk=〈ϕli,ψkj〉=〈ϕli(·),K(·,tkj)〉=ϕli(tkj)=∫tlitl+1i‍ui(s-tkj)ds. Finally, the coefficients hki, i=1,2,…,N and k∈ℤ, amount to h=Φ+q.Simulation results of a SISO CIM for a Paley-Wiener space of test signals is shown in Figure12 Input signals u1,…,u5 were bandlimited to 100 Hz and the circuit generated a total of 38 spikes. The MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −71.1 dB.Figure 12
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using signals from the Paley-Wiener spaceΞ.  (a) In contrast to Figure 6, input signals ui∈Ξ, i=1,…,5. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) The filter output in (b) is integrated by an ideal IAF neuron.  (d) The neuron generated a total of 38 spikes in response to all 5 input signals.  (e) The identified impulse response of the filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −71.1 dB. (f)–(h) Fourier amplitude spectra of K, h, and 𝒫h*. In contrast to Figure 6, K and 𝒫h* do not exhibit a discrete (line) spectrum. Again, 𝒫h*∈Ξ but h∉Ξ.


### 5.2. Channels with Noisy Observations

In the derivations above we implicitly assumed that the I/O system was noiseless. In practice, noise is introduced either by the channel or the sampler itself. Here we revisit thet-transform in (5) and show that the analysis/methodology employed in the previous sections can be extended within an appropriate mathematical setting to I/O systems with noisy measurements.Recall thet-transform of an ideal IAF neuron is given by ∫tktk+1‍(u*h)(t)dt=〈𝒫h,ϕk〉=qk, k=1,2,…,n-1, where n is the number of spikes generated by the neuron in an interval of length T. The measurements qk were obtained by applying a piece-wise linear operator on the channel output v=u*h. If either the channel or the sampler introduce an error, we can model it by adding a noise term εk to the t-transform [9]:
(36)〈𝒫h,ϕk〉=qk+εk.
Here we will assume that εk~𝒩(0,σ2), k=1,2,…,n-1, are i.i.d.In the presence of noise it is not possible to identify the projection𝒫h loss-free. However, we can still identify an estimate 𝒫h^ of 𝒫h that is optimal for an appropriately defined cost function. For example, we can formulate a bi-criterion Tikhonov regularization problem
(37)min𝒫h^∈ℋ∑i=1N‍∑k=1n-1‍(〈𝒫h^,ϕki〉-qki)2+λ∥𝒫h^∥ℋ2,
where the scalar λ>0 provides a trade-off between the faithfulness of the identified filter projection 𝒫h^ to measurements (qk)k=1n-1 and its norm ∥𝒫h^∥ℋ.Theorem 19.
Problem (37) can be solved explicitly in analytical form. The optimal solution is achieved by
(38)(𝒫h^)(t)=∑l=-LLhlel(t),
with h=(ΦHΦ+λI)-1ΦHq, Φ=[Φ1;Φ2;…;ΦN] and Φi, i=1,2,…,N, as defined in (13).Proof.
Since the minimizer𝒫h^ is in ℋ, it is of the form given in (38). Substituting this into (37), we obtain
(39)minh∈ℂ2L+1∥Φh-q∥Rn-12+λ∥h∥ℂ2L+12,
where Φ=[Φ1;Φ2;…;ΦN] with Φi, i=1,2,…,N, as defined in (13). This quadratic optimization problem can be solved analytically by expressing the objective as a convex quadratic function J(h)=hHΦHΦh-2qHΦh+qHq+λhHh with H denoting the conjugate transpose. A vector h minimizes J if and only if ∇J=2(ΦHΦ+λI)h-2ΦHq=0, that is, h=(ΦHΦ+λI)-1ΦHq.Remark 20.
In Section3.2, identification of the projection (𝒫h)(t)=∑l=-LL‍hlel(t) amounted to finding 𝒫h∈ℋ such that the sum of the residuals (〈𝒫h,ϕk〉-qk)2 was minimized [9]. In other words, we were solving an unconstrained convex optimization problem of the form
(40)min𝒫h∈ℋ∑i=1N∑k=1n-1(〈𝒫h,ϕki〉-qki)2⟺minh∈ℂ2L+1∥Φh-q∥ℝn-12,
where h=[h-L,…,hL] and Φ=[Φ1;Φ2;…;ΦN] with Φi, i=1,2,…,N, as defined in (13).


#### 5.2.1. Example: Noisy SISO [Filter]-[Ideal IAF] Circuit

In the following example, we assume that noise is added to the measurements(qki)k=1n-1, i=1,2, by the neuron and we model that noise by introducing random thresholds that are normally distributed with a mean δ and a standard deviation 0.1δ, that is, δk~𝒩(δ,(0.1δ)2):∫tkitk+1i‍(ui*h)(t)dt=Cδk-b(tk+1i-tki)=[Cδ-b(tk+1i-tki)]+C(δk-δ)=qki+εki, where εki~𝒩(0,(0.1Cδ)2). Thus random thresholds result in additive noise εki~𝒩(0,(0.1Cδ)2), i=1,2.In Figure13(a) we show two stimuli that were used to probe the [Filter]-[Ideal IAF] circuit. Both stimuli are bandlimited to 25 Hz and have a period of T=0.2s, that is, the order of the space is L=5. The response of the neuron to a biased filter output v1(t)+b (Figure 13(b)) is shown in Figure 13(c). Note the significant deviations in thresholds δk around the mean value of δ=0.05. Although a significant amount of noise is introduced into the system, we can identify an optimal estimate 𝒫h^* that is still quite close to the true projection 𝒫h. The MSE of identification is −31.8 dB.Figure 13
Noisy channel identification in a SISO [Filter]-[Ideal IAF] circuit using multiple I/O pairs.  (a) Input signalsu1,u2 are bandlimited to 25 Hz. The order of the space L=5. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) Thresholds are random with δk~𝒩(δ,(0.1δ)2). (d) The neuron produced a total of 26 spikes in response to 2 stimuli. (e) The optimal estimate 𝒫h^* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). Note that the MSE between 𝒫h^* and 𝒫h is −31.8 dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h^*. As before, supp(ℱK)=[-Ω,Ω] = supp(ℱ𝒫h^*) but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h^*∈ℋ but h∉ℋ.


## 5.1. Hilbert Spaces and RKHSs for Input Signals

Until now we have presented channel identification results for a particular space of input signals, namely the space of trigonometric polynomials. The finite-dimensionality of this space and the simplicity of the associated inner product makes it an attractive space to work with when implementing a SISO or a MISO CIM algorithm. However, fundamentally the identification methodology relied on the the geometry of the Hilbert space of test signals [5, 26]; computational tractability was based on kernel representations in an RKHS.Theorem 17.
Let{ui∣ui∈ℋ(I)}i=1N be a collection of N linearly independent and bounded stimuli at the input of a [Filter]-[Asynchronous Sampler] circuit with a linear processing filter h∈H and the t-transform
(26)ℒki(𝒫h)=qki,
where ℒki:ℋ→ℝ is a bounded linear functional mapping 𝒫h into a measurement qki. Then there is a set of sampling functions {(ϕki)k∈ℤ}i=1N, in ℋ such that
(27)qki=〈𝒫h,ϕki〉,
for all k∈ℤ, i=1,2,…,N. Furthermore, if ℋ is an RKHS with a kernel K(s,t), s,t∈I, then ϕki(t)=ℒki(K(·,t))¯. Let the set of representation functions {(ψki)k∈ℤ}i=1N, span the Hilbert space ℋ. Then
(28)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t).
Finally, if {(ϕki)k∈ℤ}i=1N and {(ψki)k∈ℤ}i=1N are orthogonal basis or frames for ℋ, then the filter coefficients amount to h=Φ+q, where h=[h1,h2,…,hN]T with [hi]k=hki, [Φij]lk=〈ϕli,ψkj〉 and q=[q1,q2,…,qN]T with [qi]l=qki  for all i,j=1,2,…,N, and k,l∈ℤ.Proof.
By the Riesz representation theorem, since the linear functionalℒki:ℋ→ℝ is bounded, there is a set of sampling functions {(ϕki)k∈ℤ}i=1N in ℋ such that ℒki(𝒫h)=〈𝒫h,ϕki〉. If ℋ is an RKHS, a sampling function ϕki can be computed using the reproducing property of the kernel K as in
(29)ϕki(t)=〈ϕki,K(·,t)〉≡〈K(·,t),ϕki〉¯=ℒki(K(·,t))¯.
Finally, writing all inner products 〈ϕki,𝒫h〉=qki yields, with the notation above, a system of linear equations Φh=q and the fiter coefficients amount to h=Φ+q.


### 5.1.1. Example: Paley-Wiener Space

As an example, we consider the Paley-Wiener space which is closely related to the space of trigonometric polynomials. Specifically, the finite-dimensional spaceℋ can be thought of as a discretized version of the infinite-dimensional Paley-Wiener space
(30)Ξ={u∈𝕃2(ℝ)∣supp(ℱu)⊆[-Ω,Ω]}
in the frequency domain. An element u∈ℋ has a line spectrum at frequencies lΩ/L, l=-L,-L+1,…,L. This spectrum becomes dense in [-Ω,Ω] as L→∞. The space Ξ with the inner product 〈·,·〉:Ξ×Ξ→ℝ given by
(31)〈u,w〉=∫ℝu(t)w(t)dt
is also an RKHS with an RK [22]
(32)K(s,t)=sin(Ω(t-s))π(t-s),
with t,s∈ℝ. Defining the projection of the filter h onto Ξ as (𝒫h)(t)=∫ℝ‍h(s)K(s,t)¯ds, we find that Lemma 8 still holds with ϕk∈Ξ and we can extend Theorem 11 to the following.Proposition 18.
Let{ui∣supp(ℱui)=[-Ω,Ω]}i=1N be a collection of N linearly independent and bounded stimuli at the input of a [Filter]-[Ideal IAF] neural circuit with a dendritic processing filter h∈H. If ∑j=1N‍(b/Cδ)>Ω/π, then (𝒫h)(t) can be perfectly identified from the collection of I/O pairs {(ui,𝕋i)}i=1N as
(33)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t),
where ψki(t)=K(t,tki), i=1,2,…,N, and k∈ℤ. Finally, h=Φ+q, where h=[h1,h2,…,hN]T with [hi]k=hki, [Φij]lk=∫tlitl+1i‍ui(s-tkj)ds and q=[q1,q2,…,qN]T with [qi]l=Cδ-b(tl+1i-tli) for all i,j=1,2,…,N, and k,l∈ℤ.Proof.
As before, the spikes(tki)k∈ℤ in response to each test signal ui, i=1,2,…,N, represent distinct measurements qki=〈ϕki,𝒫h〉 of (𝒫h)(t). Thus we can think of the {(qki)k∈ℤ}i=1N's as projections of 𝒫h onto {(ϕki)k∈ℤ}i=1N, where ϕki(t)=ℒki(K(·,t))=∫tkitk+1i‍∫ℝ‍ui(z)K(s-z,t)dzds=∫tkitk+1i‍ui(s-t)ds. Since the signals are linearly independent and bounded [5], it follows that, if ∑i=1N‍(b/Cδ)>Ω/π or equivalently if the number of test signals N>ΩCδ/πb, the set of functions {(ψki)k∈ℤ}i=1N with ψki(t)=K(t,tki), is a frame for Ξ [5, 26]. Hence
(34)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t).
If the set of functions {(ϕki)k∈ℤ}i=1N forms a frame for Ξ, we can find the coefficients hki,k∈ℤ, i=1,2,…,N, by taking the inner product of (34) with each element of {ϕli(t)}i=1N: 〈ϕli,𝒫h〉=∑k∈ℤ‍hk1〈ϕli,ψk1〉+∑k∈ℤ‍hk2〈ϕli,ψk2〉+⋯+∑k∈ℤ‍hkN〈ϕli,ψkN〉≡qli, for i=1,2,…,N, l∈ℤ. Letting [Φij]lk=〈ϕli,ψkj〉, we obtain
(35)qli=∑k∈ℤ‍[Φi1]lkhk1+∑k∈ℤ‍[Φi2]lkhk2+⋯+∑k∈ℤ‍[ΦiN]lkhkN,
for i=1,2,…,N, l∈ℤ. Writing (35) in matrix form, we have q=Φh with [Φij]lk=〈ϕli,ψkj〉=〈ϕli(·),K(·,tkj)〉=ϕli(tkj)=∫tlitl+1i‍ui(s-tkj)ds. Finally, the coefficients hki, i=1,2,…,N and k∈ℤ, amount to h=Φ+q.Simulation results of a SISO CIM for a Paley-Wiener space of test signals is shown in Figure12 Input signals u1,…,u5 were bandlimited to 100 Hz and the circuit generated a total of 38 spikes. The MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −71.1 dB.Figure 12
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using signals from the Paley-Wiener spaceΞ.  (a) In contrast to Figure 6, input signals ui∈Ξ, i=1,…,5. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) The filter output in (b) is integrated by an ideal IAF neuron.  (d) The neuron generated a total of 38 spikes in response to all 5 input signals.  (e) The identified impulse response of the filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −71.1 dB. (f)–(h) Fourier amplitude spectra of K, h, and 𝒫h*. In contrast to Figure 6, K and 𝒫h* do not exhibit a discrete (line) spectrum. Again, 𝒫h*∈Ξ but h∉Ξ.


## 5.1.1. Example: Paley-Wiener Space

As an example, we consider the Paley-Wiener space which is closely related to the space of trigonometric polynomials. Specifically, the finite-dimensional spaceℋ can be thought of as a discretized version of the infinite-dimensional Paley-Wiener space
(30)Ξ={u∈𝕃2(ℝ)∣supp(ℱu)⊆[-Ω,Ω]}
in the frequency domain. An element u∈ℋ has a line spectrum at frequencies lΩ/L, l=-L,-L+1,…,L. This spectrum becomes dense in [-Ω,Ω] as L→∞. The space Ξ with the inner product 〈·,·〉:Ξ×Ξ→ℝ given by
(31)〈u,w〉=∫ℝu(t)w(t)dt
is also an RKHS with an RK [22]
(32)K(s,t)=sin(Ω(t-s))π(t-s),
with t,s∈ℝ. Defining the projection of the filter h onto Ξ as (𝒫h)(t)=∫ℝ‍h(s)K(s,t)¯ds, we find that Lemma 8 still holds with ϕk∈Ξ and we can extend Theorem 11 to the following.Proposition 18.
Let{ui∣supp(ℱui)=[-Ω,Ω]}i=1N be a collection of N linearly independent and bounded stimuli at the input of a [Filter]-[Ideal IAF] neural circuit with a dendritic processing filter h∈H. If ∑j=1N‍(b/Cδ)>Ω/π, then (𝒫h)(t) can be perfectly identified from the collection of I/O pairs {(ui,𝕋i)}i=1N as
(33)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t),
where ψki(t)=K(t,tki), i=1,2,…,N, and k∈ℤ. Finally, h=Φ+q, where h=[h1,h2,…,hN]T with [hi]k=hki, [Φij]lk=∫tlitl+1i‍ui(s-tkj)ds and q=[q1,q2,…,qN]T with [qi]l=Cδ-b(tl+1i-tli) for all i,j=1,2,…,N, and k,l∈ℤ.Proof.
As before, the spikes(tki)k∈ℤ in response to each test signal ui, i=1,2,…,N, represent distinct measurements qki=〈ϕki,𝒫h〉 of (𝒫h)(t). Thus we can think of the {(qki)k∈ℤ}i=1N's as projections of 𝒫h onto {(ϕki)k∈ℤ}i=1N, where ϕki(t)=ℒki(K(·,t))=∫tkitk+1i‍∫ℝ‍ui(z)K(s-z,t)dzds=∫tkitk+1i‍ui(s-t)ds. Since the signals are linearly independent and bounded [5], it follows that, if ∑i=1N‍(b/Cδ)>Ω/π or equivalently if the number of test signals N>ΩCδ/πb, the set of functions {(ψki)k∈ℤ}i=1N with ψki(t)=K(t,tki), is a frame for Ξ [5, 26]. Hence
(34)(𝒫h)(t)=∑i=1N∑k∈ℤhkiψki(t).
If the set of functions {(ϕki)k∈ℤ}i=1N forms a frame for Ξ, we can find the coefficients hki,k∈ℤ, i=1,2,…,N, by taking the inner product of (34) with each element of {ϕli(t)}i=1N: 〈ϕli,𝒫h〉=∑k∈ℤ‍hk1〈ϕli,ψk1〉+∑k∈ℤ‍hk2〈ϕli,ψk2〉+⋯+∑k∈ℤ‍hkN〈ϕli,ψkN〉≡qli, for i=1,2,…,N, l∈ℤ. Letting [Φij]lk=〈ϕli,ψkj〉, we obtain
(35)qli=∑k∈ℤ‍[Φi1]lkhk1+∑k∈ℤ‍[Φi2]lkhk2+⋯+∑k∈ℤ‍[ΦiN]lkhkN,
for i=1,2,…,N, l∈ℤ. Writing (35) in matrix form, we have q=Φh with [Φij]lk=〈ϕli,ψkj〉=〈ϕli(·),K(·,tkj)〉=ϕli(tkj)=∫tlitl+1i‍ui(s-tkj)ds. Finally, the coefficients hki, i=1,2,…,N and k∈ℤ, amount to h=Φ+q.Simulation results of a SISO CIM for a Paley-Wiener space of test signals is shown in Figure12 Input signals u1,…,u5 were bandlimited to 100 Hz and the circuit generated a total of 38 spikes. The MSE between the identified filter 𝒫h* (red) and the projection 𝒫h (blue) is −71.1 dB.Figure 12
Channel identification in a SISO [Filter]-[Ideal IAF] circuit using signals from the Paley-Wiener spaceΞ.  (a) In contrast to Figure 6, input signals ui∈Ξ, i=1,…,5. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) The filter output in (b) is integrated by an ideal IAF neuron.  (d) The neuron generated a total of 38 spikes in response to all 5 input signals.  (e) The identified impulse response of the filter 𝒫h* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). The MSE between 𝒫h* and 𝒫h is −71.1 dB. (f)–(h) Fourier amplitude spectra of K, h, and 𝒫h*. In contrast to Figure 6, K and 𝒫h* do not exhibit a discrete (line) spectrum. Again, 𝒫h*∈Ξ but h∉Ξ.


## 5.2. Channels with Noisy Observations

In the derivations above we implicitly assumed that the I/O system was noiseless. In practice, noise is introduced either by the channel or the sampler itself. Here we revisit thet-transform in (5) and show that the analysis/methodology employed in the previous sections can be extended within an appropriate mathematical setting to I/O systems with noisy measurements.Recall thet-transform of an ideal IAF neuron is given by ∫tktk+1‍(u*h)(t)dt=〈𝒫h,ϕk〉=qk, k=1,2,…,n-1, where n is the number of spikes generated by the neuron in an interval of length T. The measurements qk were obtained by applying a piece-wise linear operator on the channel output v=u*h. If either the channel or the sampler introduce an error, we can model it by adding a noise term εk to the t-transform [9]:
(36)〈𝒫h,ϕk〉=qk+εk.
Here we will assume that εk~𝒩(0,σ2), k=1,2,…,n-1, are i.i.d.In the presence of noise it is not possible to identify the projection𝒫h loss-free. However, we can still identify an estimate 𝒫h^ of 𝒫h that is optimal for an appropriately defined cost function. For example, we can formulate a bi-criterion Tikhonov regularization problem
(37)min𝒫h^∈ℋ∑i=1N‍∑k=1n-1‍(〈𝒫h^,ϕki〉-qki)2+λ∥𝒫h^∥ℋ2,
where the scalar λ>0 provides a trade-off between the faithfulness of the identified filter projection 𝒫h^ to measurements (qk)k=1n-1 and its norm ∥𝒫h^∥ℋ.Theorem 19.
Problem (37) can be solved explicitly in analytical form. The optimal solution is achieved by
(38)(𝒫h^)(t)=∑l=-LLhlel(t),
with h=(ΦHΦ+λI)-1ΦHq, Φ=[Φ1;Φ2;…;ΦN] and Φi, i=1,2,…,N, as defined in (13).Proof.
Since the minimizer𝒫h^ is in ℋ, it is of the form given in (38). Substituting this into (37), we obtain
(39)minh∈ℂ2L+1∥Φh-q∥Rn-12+λ∥h∥ℂ2L+12,
where Φ=[Φ1;Φ2;…;ΦN] with Φi, i=1,2,…,N, as defined in (13). This quadratic optimization problem can be solved analytically by expressing the objective as a convex quadratic function J(h)=hHΦHΦh-2qHΦh+qHq+λhHh with H denoting the conjugate transpose. A vector h minimizes J if and only if ∇J=2(ΦHΦ+λI)h-2ΦHq=0, that is, h=(ΦHΦ+λI)-1ΦHq.Remark 20.
In Section3.2, identification of the projection (𝒫h)(t)=∑l=-LL‍hlel(t) amounted to finding 𝒫h∈ℋ such that the sum of the residuals (〈𝒫h,ϕk〉-qk)2 was minimized [9]. In other words, we were solving an unconstrained convex optimization problem of the form
(40)min𝒫h∈ℋ∑i=1N∑k=1n-1(〈𝒫h,ϕki〉-qki)2⟺minh∈ℂ2L+1∥Φh-q∥ℝn-12,
where h=[h-L,…,hL] and Φ=[Φ1;Φ2;…;ΦN] with Φi, i=1,2,…,N, as defined in (13).


### 5.2.1. Example: Noisy SISO [Filter]-[Ideal IAF] Circuit

In the following example, we assume that noise is added to the measurements(qki)k=1n-1, i=1,2, by the neuron and we model that noise by introducing random thresholds that are normally distributed with a mean δ and a standard deviation 0.1δ, that is, δk~𝒩(δ,(0.1δ)2):∫tkitk+1i‍(ui*h)(t)dt=Cδk-b(tk+1i-tki)=[Cδ-b(tk+1i-tki)]+C(δk-δ)=qki+εki, where εki~𝒩(0,(0.1Cδ)2). Thus random thresholds result in additive noise εki~𝒩(0,(0.1Cδ)2), i=1,2.In Figure13(a) we show two stimuli that were used to probe the [Filter]-[Ideal IAF] circuit. Both stimuli are bandlimited to 25 Hz and have a period of T=0.2s, that is, the order of the space is L=5. The response of the neuron to a biased filter output v1(t)+b (Figure 13(b)) is shown in Figure 13(c). Note the significant deviations in thresholds δk around the mean value of δ=0.05. Although a significant amount of noise is introduced into the system, we can identify an optimal estimate 𝒫h^* that is still quite close to the true projection 𝒫h. The MSE of identification is −31.8 dB.Figure 13
Noisy channel identification in a SISO [Filter]-[Ideal IAF] circuit using multiple I/O pairs.  (a) Input signalsu1,u2 are bandlimited to 25 Hz. The order of the space L=5. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) Thresholds are random with δk~𝒩(δ,(0.1δ)2). (d) The neuron produced a total of 26 spikes in response to 2 stimuli. (e) The optimal estimate 𝒫h^* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). Note that the MSE between 𝒫h^* and 𝒫h is −31.8 dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h^*. As before, supp(ℱK)=[-Ω,Ω] = supp(ℱ𝒫h^*) but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h^*∈ℋ but h∉ℋ.


## 5.2.1. Example: Noisy SISO [Filter]-[Ideal IAF] Circuit

In the following example, we assume that noise is added to the measurements(qki)k=1n-1, i=1,2, by the neuron and we model that noise by introducing random thresholds that are normally distributed with a mean δ and a standard deviation 0.1δ, that is, δk~𝒩(δ,(0.1δ)2):∫tkitk+1i‍(ui*h)(t)dt=Cδk-b(tk+1i-tki)=[Cδ-b(tk+1i-tki)]+C(δk-δ)=qki+εki, where εki~𝒩(0,(0.1Cδ)2). Thus random thresholds result in additive noise εki~𝒩(0,(0.1Cδ)2), i=1,2.In Figure13(a) we show two stimuli that were used to probe the [Filter]-[Ideal IAF] circuit. Both stimuli are bandlimited to 25 Hz and have a period of T=0.2s, that is, the order of the space is L=5. The response of the neuron to a biased filter output v1(t)+b (Figure 13(b)) is shown in Figure 13(c). Note the significant deviations in thresholds δk around the mean value of δ=0.05. Although a significant amount of noise is introduced into the system, we can identify an optimal estimate 𝒫h^* that is still quite close to the true projection 𝒫h. The MSE of identification is −31.8 dB.Figure 13
Noisy channel identification in a SISO [Filter]-[Ideal IAF] circuit using multiple I/O pairs.  (a) Input signalsu1,u2 are bandlimited to 25 Hz. The order of the space L=5. (b) Biased output of the filter v1(t)+b in response to the stimulus u1. (c) Thresholds are random with δk~𝒩(δ,(0.1δ)2). (d) The neuron produced a total of 26 spikes in response to 2 stimuli. (e) The optimal estimate 𝒫h^* (red) is shown together with the original filter h (dashed black) and its projection 𝒫h (blue). Note that the MSE between 𝒫h^* and 𝒫h is −31.8 dB. (f)–(h) Fourier amplitude spectra of K, h and 𝒫h^*. As before, supp(ℱK)=[-Ω,Ω] = supp(ℱ𝒫h^*) but supp(ℱh)⊃[-Ω,Ω]. In other words, 𝒫h^*∈ℋ but h∉ℋ.


## 6. Conclusion

In this paper we presented a class of channel identification problems arising in the context of communication channels in [Filter]-[Asynchronous Sampler] circuits. Our results are based on a key structural conditional duality result between time decoding and channel identification. The conditional duality result shows that given a class of test signals, the projection of the filter onto the space of input signals can be recovered loss-free. Moreover, the channel identification problem can be converted into a time decoding problem. We considered a number of channel identification problems that arise both in communications and in neuroscience. We presented CIM algorithms that allow one to recover projections of both one-dimensional and multi-dimensional filters in such problems and demonstrated their performance through numerical simulations. Furthermore, we showed that under natural conditions on the impulse response of the filter, the filter projection converges to the original filter almost everywhere and in the mean-squared sense (L2 norm), with increasing bandwidth and order of the space. Thus in order to identify the impulse response of the filter with arbitrary precision, we are lead to consider a countably infinite number of time encoding problems. Finally, we generalized our results to a large class of test signal spaces and to channel models with noisy observations.



---

*Source: 209590-2012-11-14.xml*