# Maximum Likelihood Sequence Detection Receivers for Nonlinear Optical Channels

Authors: Gabriel N. Maggio; Mario R. Hueda; Oscar E. Agazzi
Journal: Journal of Electrical and Computer Engineering
 (2015)

---

## Abstract

The space-time whitened matched filter (ST-WMF) maximum likelihood sequence detection (MLSD) architecture has been recently proposed (Maggio et al., 2014). Its objective is reducing implementation complexity in transmissions over nonlinear dispersive channels. The ST-WMF-MLSD receiver (i) drastically reduces the number of states of the Viterbi decoder (VD) and (ii) offers a smooth trade-off between performance and complexity. In this work the ST-WMF-MLSD receiver is investigated in detail. We show that thespace compression of the nonlinear channel is an instrumental property of the ST-WMF-MLSD which results in a major reduction of the implementation complexity in intensity modulation and direct detection (IM/DD) fiber optic systems. Moreover, we assess the performance of ST-WMF-MLSD in IM/DD optical systems with chromatic dispersion (CD) and polarization mode dispersion (PMD). Numerical results for a 10 Gb/s, 700 km, and IM/DD fiber-optic link with 50 ps differential group delay (DGD) show that the number of states of the VD in ST-WMF-MLSD can be reduced ~4 times compared to an oversampled MLSD. Finally, we analyze the impact of the imperfect channel estimation on the performance of the ST-WMF-MLSD. Our results show that the performance degradation caused by channel estimation inaccuracies is low and similar to that achieved by existing MLSD schemes (~0.2 dB).


---

## Body

## 1. Introduction

Maximum likelihood sequence detection (MLSD) receivers for nonlinear channels have been extensively investigated in the literature (e.g., [1, 2] and references therein). Their ability to achieve optimal performance in the presence of additive white Gaussian noise (AWGN) has always been of great theoretical and practical interest. The theoretical interest lies in that it provides a performance bound for any reception scheme. The practical interest stems from its ability to actually achieve optimal or nearly optimal performance in transmissions over nonlinear channels.Much of the early work in this area has been focused on the compensation of nonlinearities in satellite communications [1–5]. A traditional architecture for the optimal nonlinear receiver consists of a matched-filter bank (MFB) followed by a maximum likelihood sequence detector (MLSD) [1]. Owing to the correlation among thespatial noise components at the MFB outputs, an MLSD with non-Euclidean metrics must be used. The use of oversampling in combination with MLSD (OS-MLSD) has also been proposed to implement the optimal receiver in the presence of nonlinearities (see [2] and references therein). Since the complexity of both MFB and OS MLSD-based receivers grows exponentially with the channel memory, their practical application in transmissions over highly dispersive channels has been limited. Despite this fact, MLSD-based receivers are still preferred over decision feedback equalizers (DFE) in applications such as multigigabit intensity modulation/direct detection (IM/DD) fiber optic systems for the two following reasons.(i)
Performance: DFE suffers from severe performance limitations inmoderate to high dispersion single-mode fiber links as a result of its inability to compensate nonlinear ISI [6, 7]. Although nonlinear DFE (NL-DFE) structures have also been considered in the literature (e.g., [8, 9]), their performances still degrade significantly at long fiber lengths (e.g., ≥500 km). On the other hand, MLSD can operate with a constant penalty around 3 dB with respect to back-to-back (B2B) at virtually any distance [10, 11].(ii)
High-speed implementation: future generation of communication systems will operate at multigigabit per second data rates on highly dispersive channels [12]. In commercial applications, the digital receiver is often implemented as a monolithic chip in CMOS technology [12]. Maximum clock frequency of state-of-the-art complex digital signal processors in 28 nm CMOS technology is limited to frequencies lower than ~1 GHz. Therefore, in order to achieve multigigabit per second data rates, parallel processing techniques are required [12]. Although the complexity of serial implementations of DFE grows linearly with channel memory, all presently known parallel processing implementations require that the bottleneck created by the feedback loop be broken using techniques such as the ones proposed by [13–15], whose complexitygrows exponentially with the channel memory. Therefore, in high-speed applications, complexity of both DFE and MLSD increases with the channel memory in a similar way.From the above it is clear that complexity reduction of MLSD is crucial for many practical applications. In IM/DD optical channels, the receiver must compensate the linear fiber dispersion as well as nonlinearities caused by lasers, optical modulators, the fiber Kerr effect, photo-detectors, and other components of the link. Chromatic dispersion (CD) and polarization-mode dispersion (PMD), in combination with the quadratic response of the photo-detector, are major factors that limit the reach and drive the complexity of optical-transmission systems at data rates ≥10 Gb/s [16, 17]. With traditional implementations based on oversampling techniques, an 8192-state MLSD is required to compensate 700 km of fiber at 10 Gb/s [10]. This is prohibitive in current CMOS technology1.A new MLSD receiver architecture for nonlinear channels has been proposed in [18]. The major breakthrough of this proposal consists in a novel representation of the received signal obtained by a Gram-Schmidt-like orthogonalization of the kernels of a Volterra series expansion of the channel. This procedure yields a special form of space-time whitened matched filter (ST-WMF) [19] whose baud-rate-sampled outputs are sufficient statistics with independent noise components in both space and time. Combined with the minimum phase property of the response of each branch, the ST-WMF provides an effective way to reduce the complexity of MLSD in nonlinear channels. The ST-WMF MLSD technique offers a smooth trade-off between performance and complexity. As complexity is progressively reduced, performance degrades in a graceful manner. Numerical results in [18] demonstrate that the number of states of the VD in ST-WMF-MLSD required on a 10 Gb/s, 700 km, and IM/DD fiber-optic link without PMD can be reduced 8 times compared to an oversampled MLSD.Further contributions on the ST-WMF-MLSD receiver and its performance on IM/DD optical links are provided in this work. First, the space channel compression achieved by the ST-WMF-MLSD is analyzed in detail. Space channel compression is important because it is the property that enables the major complexity reductions of the proposed architecture. Second, the performance evaluation of the ST-WMF-MLSD receiver is extended by addressing the combined effect of CD and PMD. Numerical results confirm that the ST-WMF-MLSD remains an attractive solution in the presence of these combined impairments. Finally, the impact of channel estimation inaccuracies on the ST-WMF-MLSD performance is assessed. Accuracy and speed of channel estimation are particularly important when tracking nonstationary channels. Nonstationarity in optical channels results from PMD and random rotations of the laser state of polarization. Our results show that the performance degradation caused by an imperfect channel estimation is low and similar to that achieved by existing MLSD schemes (~0.2 dB).This paper is organized as follows. The nonlinear channel model and the ST-WMF-MLSD architecture are described in Sections2 and 3, respectively. Performance evaluation of the ST-WMF-MLSD under different channel conditions, as well as its robustness in the presence of imperfect channel knowledge, are presented and discussed in Sections 4 and 5. Finally, conclusions are drawn in Section 6.


## 2. Nonlinear Channel Model

The noisy received signal is given by(1)r(t)=s(t)+z(t),
where s(t) is the noise-free signal and z(t) is the noise component, which is assumed to be a white Gaussian process with power spectral density N0. Component s(t) can be expressed in terms of its Volterra-series expansion [20, 21]. For example, neglecting the DC term of the expansion we get
(2)s(t)=∑k‍akf0(t-kT)+∑m=1N-1‍∑k‍akak-mfm(t-kT),
where f0(t) is the linear kernel, fm(t) with m>0 is the mth second-order kernel, ak is the kth symbol at the input of the nonlinear channel, 1/T is the symbol rate, and N is the total number of kernels2.


### 2.1. Channel Model Orthogonalization

Next we present an alternative representation of the nonlinear signals(t) [18]. Without loss of generality, we consider here the Volterra-series with a dominant linear kernel f0(t), and we select the firstpivoting response as h0(t)=f0(t). Let H0 be thesignal space spanned by the set {h0(t-kT)} [19]. We assume that signal spaces are Hilbert spaces with inner product defined as ∫-∞∞‍x(t)y*(t)dt, where superscript * denotes complex conjugate. From theprojection theorem, the nonlinear kernels fm(t) can be uniquely expressed as
(3)fm(t)=∑n‍λn(0,m)h0(t-nT)+gm(0)(t),
where gm(0)(t) isorthogonal to the signal space H0; that is,
(4)∫-∞∞‍gm(0)(t)h0*(t-jT)dt=0,m=1,…,N-1,∀j∈Z,
while ∫-∞∞‍|gm(0)(t)|2dt is minimum [19], and Z denotes the set of all integers. We highlight that the first summation in (3) is theprojection of fm(t) onto H0. Define Gm0 as the signal space spanned by {gm(0)(t-kT)}. For x(t)∈H0 and y(t)∈Gm0, from (4) note that ∫-∞∞‍x(t)y*(t)dt=0; therefore x(t) and y(t) are orthogonal signals [19]. Replacing (3) in (2) and operating, we obtain
(5)s(t)=s0(t)+s¯0(t),
where
(6)s0(t)=∑k‍ak+∑m=1N-1‍akak-m⊗λk(0,m)h0(t-kT),(7)s¯0(t)=∑k‍∑m=1N-1‍akak-mgm0t-kT,
with operator ⊗ denoting convolution. Notice that s0(t)∈H0 and s¯0t∈∪m=1N-1Gm0; therefore signals s0(t) and s¯0(t) are orthogonal (see (4)). Next we focus on s¯0(t). Without loss of generality we select the second pivoting response as h1(t)=g1(0)(t); then (7) can be rewritten as
(8)s¯0(t)=∑k‍akak-1h1(t-kT)+∑m=2N-1‍ak-mgm(0)(t-kT).
Similarly to (5)–(7), s¯0(t) can be expressed as
(9)s¯0(t)=s1(t)+s¯1(t),
where
(10)s1(t)=∑k‍akak-1+∑m=2N-1‍akak-m⊗λk(1,m)h1(t-kT),(11)s¯1(t)=∑k‍∑m=2N-1‍akak-mgm(1)(t-kT),
with λn(1,m) chosen to satisfy
(12)∫-∞∞‍gm1th1*t-jTdt=0,form=2,…,N-1,∀j∈Z.
Thus, note that s¯1(t) is orthogonal to the signal spaces H0 and H1, spanned by {h0(t-kT)} and {h1(t-kT)}, respectively. Repeating the processing on (11) and generalizing, we get
(13)s(t)=∑n=0N-1‍sn(t)=∑n=0N-1‍∑k‍bk(n)hn(t-kT),
where hn(t) is the response of the nth channelpath and with bk(n) given by
(14)bk(n)=ak+∑m=1N-1‍akak-m⊗λk(0,m)ifn=0akak-n+∑m=n+1N-1‍akak-m⊗λk(n,m)if0<n<N-1akak-N+1ifn=N-1,
with
(15)∫-∞∞‍hm(t)hn*(t-jT)dt=0m≠n,∀j∈Z.
From (13) and (15) note that signal components from different paths are orthogonal; that is,
(16)∫-∞∞‍sm(t)sn*(t)dt=0,m≠n.Figure1 shows two multiple input-single output (MISO) representations of the nonlinear channel. Figure 1(a) shows the signal represented by the traditional Volterra model of (2); Figure 1(b) depicts the orthogonal representation given by (13) and (14).MISO model of the nonlinear channel: (a) based on a Volterra representation; (b) based on the spatially compressed orthogonal representation.
(a)
(b)


### 2.2. Space Channel Compression

The model orthogonalization procedure described in Section2.1 can be performed in N! different ways with N being the number of kernels. For example, consider a Volterra model with N=3. One way to obtain the orthogonalized model may be realized by selecting the kernels as shown in Figure 2. In this case, the pulse f0(t) is selected as the pivoting h0(t) for the first orthogonalization step. Then, g1(0) and g2(0) are obtained as the components of f1(t) and f2(t) orthogonal to h0(t)=f0(t). For the second step, g1(0)(t) is selected as the pivoting response, so g2(1)(t) results as the component of g2(0)(t) orthogonal to h1(t)=g1(0)(t). This unique ordering could be identified by the following set of indexes I={0,1,2}.Figure 2
Orthogonalization procedure following the orderingI={0,1,2}.A different result is achieved by selecting the pivoting responses as illustrated in Figure3. In this situation, f1(t) is selected as the pivoting response for the first step, and g1(0)(t) and g2(0)(t) are the components of f0(t) and f2(t) orthogonal to h0(t)=f1(t). According to Figure 3, g2(0)(t) (which is different from g2(0)(t) shown in Figure 2) is selected as the pivoting response for the second orthogonalization step. Then, g2(1)(t) will be obtained as the component of g1(0)(t) orthogonal to h1(t)=g2(0)(t). The index set corresponding to the procedure depicted in Figure 3 is I={1,2,0}.Figure 3
Orthogonalization procedure following the orderingI={1,2,0}.Note that the set of responses{hi(t)}i=02 and sequences {bk(i)}i=02 resulting from the procedures described in Figures 2 and 3 are two different expansions of the same signal s(t). Any one of the N! possible sets can be selected in the orthogonalization process.Space compression of the channel can be achieved concentrating the maximum of the channel energy on a reduced set of paths, independent of the distribution of their individual energy, as explained in the following discussion.LetP be a given number of paths used to model the channel (e.g., if P=N, note that any orthogonalization process I shall represent exactly the behavior of the channel). To reduce the complexity of the receiver, it is desirable for P to be as small as possible. On the other hand, to minimize the performance degradation (or the inaccuracy of the channel representation), the orthogonalization process should be carried out in such a way that the most part of the signal energy is concentrated on these P paths. From the above,for a given value of P, theoptimum set Iop for space compression (i.e., minimal channel distortion modeling) should meet the following condition:
(17)∑i=0P-1‍EIop(i)≥∑i=0P-1‍EI^i,∀I^≠Iop,
where
(18)EI(i)=Ebki2∫-∞∞‍hit2dt
is the energy in the ith path for the set I, while E{·} denotes expectation. If P=N, notice that any orthogonalization process I will satisfy condition (17). Otherwise, if P<N the criterion guarantees that the signal energy contained inthe completePpaths shall be maximum, independent of the individual energy distribution among them.The optimum channel expansion that satisfies (17) can be found by an exhaustive search. Instead, we propose to achieve the orthogonalization process to meet the following criteria:
(19)E^(0)≥E^1≥⋯≥E^(N-1),
with E^(j)=∫-∞∞‍|hn(t)|2dt. Condition (19) can be achieved by selecting at each orthogonalization step (ith step, e.g., with i∈{1,2,…,N}) the pivoting response (hi-1(t)) as the response with highest energy among theremaining responses (those orthogonal to ∩n=0i-2Hn for t=kT, ∀k). That is, at the first step we select the pivoting response h0(t) as the Volterra kernel with the highest energy in fn(t)n=0N-1. At the second step we select the pivoting response h1(t) as the response with highest energy in the set of responses orthogonal to H0 (i.e., gm(0)(t), with m=1,2,…,N-1). This procedure is repeated in the same way at each step, where the best pivoting response at each stage is selected by an exhaustive search among all the pivot candidates at that stage. The minimization of the energy of the orthogonal responses (see (3), (4), and associated discussion) ensures that condition (19) is met. As we shall show later, condition (19) gives rise tospace channel compression, which can be exploited to reduce complexity.


### 2.3. Example

As an example, we present results for a 10 Gb/s, 700 km, and IM/DD fiber-optic link (details about the simulated system are given in Section4.1).Without loss of generality, we assume thatak∈{±1}. Let Sn be the normalized cumulative path energy of thetraditional Volterra-series expansion (see (2)) given by
(20)Sn=1Es∑m=0n‍∫-∞∞‍fmt2dt,n=1,2,…,N-1,
where Es is the total energy of the signal component s(t):
(21)Es=∑m=0N-1‍∫-∞∞‍fmt2dt.On the other hand, the normalized cumulative path energy of theorthogonalized Volterra-series expansion is defined as
(22)Sn=1Es∑m=1n‍EI(m),n=0,1,…,N-1,
where EI(m) is given by (18) with I obtained according to the criteria defined by (19). In all cases, note that Sn≤1  ∀n, with SN-1=1.Figure4 shows the normalized cumulative path energy Sn for the traditional (20) and orthogonalized (22) Volterra-series representation with N=8. For the orthogonal representation, the pivoting responses are selected according to the criteria in (19). We observe that most energy of the nonlinear signal is concentrated on the first two paths for the orthogonalized Volterra-series expansion (i.e., EI(0)+EI(1) represents ~99.4% of the total signal energy). The traditional Volterra model requires five paths to capture a similar level of nonlinear signal energy. We highlight that the increase of Sn from −0.33 dB to −0.12 dB at n=0 is due to the factor E{|bk(0)|2}≥1 in (18). Notice that this factor is a measure of the correlation between all nonlinear kernels fm(t) with m>0 and the linear kernel f0(t) (e.g., E{|bk(0)|2}=1 (N) if f0(t) and fm(t) are orthogonal (same) pulses ∀m). Thus, we conclude the following for the channel analyzed in the example.(i)
Most energy of the nonlinear kernels with the traditional Volterra model is contained in their projection onto thelinear signal space spanned by the set {h0(t-kT)}={f0(t-kT)}.(ii)
A channel model with2 paths captures practically all the signal energy.(iii)
SinceEI(0) is ~97.5% of the total signal energy (see Figure 4). Therefore, as we shall show later, a receiver that considers the first channel path should be able to achieve agood performance.Figure 4
Normalized cumulative path energy of the nonlinear IM/DD fiber-optic link withL=700km and 1/T=10Gb/s.


## 2.1. Channel Model Orthogonalization

Next we present an alternative representation of the nonlinear signals(t) [18]. Without loss of generality, we consider here the Volterra-series with a dominant linear kernel f0(t), and we select the firstpivoting response as h0(t)=f0(t). Let H0 be thesignal space spanned by the set {h0(t-kT)} [19]. We assume that signal spaces are Hilbert spaces with inner product defined as ∫-∞∞‍x(t)y*(t)dt, where superscript * denotes complex conjugate. From theprojection theorem, the nonlinear kernels fm(t) can be uniquely expressed as
(3)fm(t)=∑n‍λn(0,m)h0(t-nT)+gm(0)(t),
where gm(0)(t) isorthogonal to the signal space H0; that is,
(4)∫-∞∞‍gm(0)(t)h0*(t-jT)dt=0,m=1,…,N-1,∀j∈Z,
while ∫-∞∞‍|gm(0)(t)|2dt is minimum [19], and Z denotes the set of all integers. We highlight that the first summation in (3) is theprojection of fm(t) onto H0. Define Gm0 as the signal space spanned by {gm(0)(t-kT)}. For x(t)∈H0 and y(t)∈Gm0, from (4) note that ∫-∞∞‍x(t)y*(t)dt=0; therefore x(t) and y(t) are orthogonal signals [19]. Replacing (3) in (2) and operating, we obtain
(5)s(t)=s0(t)+s¯0(t),
where
(6)s0(t)=∑k‍ak+∑m=1N-1‍akak-m⊗λk(0,m)h0(t-kT),(7)s¯0(t)=∑k‍∑m=1N-1‍akak-mgm0t-kT,
with operator ⊗ denoting convolution. Notice that s0(t)∈H0 and s¯0t∈∪m=1N-1Gm0; therefore signals s0(t) and s¯0(t) are orthogonal (see (4)). Next we focus on s¯0(t). Without loss of generality we select the second pivoting response as h1(t)=g1(0)(t); then (7) can be rewritten as
(8)s¯0(t)=∑k‍akak-1h1(t-kT)+∑m=2N-1‍ak-mgm(0)(t-kT).
Similarly to (5)–(7), s¯0(t) can be expressed as
(9)s¯0(t)=s1(t)+s¯1(t),
where
(10)s1(t)=∑k‍akak-1+∑m=2N-1‍akak-m⊗λk(1,m)h1(t-kT),(11)s¯1(t)=∑k‍∑m=2N-1‍akak-mgm(1)(t-kT),
with λn(1,m) chosen to satisfy
(12)∫-∞∞‍gm1th1*t-jTdt=0,form=2,…,N-1,∀j∈Z.
Thus, note that s¯1(t) is orthogonal to the signal spaces H0 and H1, spanned by {h0(t-kT)} and {h1(t-kT)}, respectively. Repeating the processing on (11) and generalizing, we get
(13)s(t)=∑n=0N-1‍sn(t)=∑n=0N-1‍∑k‍bk(n)hn(t-kT),
where hn(t) is the response of the nth channelpath and with bk(n) given by
(14)bk(n)=ak+∑m=1N-1‍akak-m⊗λk(0,m)ifn=0akak-n+∑m=n+1N-1‍akak-m⊗λk(n,m)if0<n<N-1akak-N+1ifn=N-1,
with
(15)∫-∞∞‍hm(t)hn*(t-jT)dt=0m≠n,∀j∈Z.
From (13) and (15) note that signal components from different paths are orthogonal; that is,
(16)∫-∞∞‍sm(t)sn*(t)dt=0,m≠n.Figure1 shows two multiple input-single output (MISO) representations of the nonlinear channel. Figure 1(a) shows the signal represented by the traditional Volterra model of (2); Figure 1(b) depicts the orthogonal representation given by (13) and (14).MISO model of the nonlinear channel: (a) based on a Volterra representation; (b) based on the spatially compressed orthogonal representation.
(a)
(b)


## 2.2. Space Channel Compression

The model orthogonalization procedure described in Section2.1 can be performed in N! different ways with N being the number of kernels. For example, consider a Volterra model with N=3. One way to obtain the orthogonalized model may be realized by selecting the kernels as shown in Figure 2. In this case, the pulse f0(t) is selected as the pivoting h0(t) for the first orthogonalization step. Then, g1(0) and g2(0) are obtained as the components of f1(t) and f2(t) orthogonal to h0(t)=f0(t). For the second step, g1(0)(t) is selected as the pivoting response, so g2(1)(t) results as the component of g2(0)(t) orthogonal to h1(t)=g1(0)(t). This unique ordering could be identified by the following set of indexes I={0,1,2}.Figure 2
Orthogonalization procedure following the orderingI={0,1,2}.A different result is achieved by selecting the pivoting responses as illustrated in Figure3. In this situation, f1(t) is selected as the pivoting response for the first step, and g1(0)(t) and g2(0)(t) are the components of f0(t) and f2(t) orthogonal to h0(t)=f1(t). According to Figure 3, g2(0)(t) (which is different from g2(0)(t) shown in Figure 2) is selected as the pivoting response for the second orthogonalization step. Then, g2(1)(t) will be obtained as the component of g1(0)(t) orthogonal to h1(t)=g2(0)(t). The index set corresponding to the procedure depicted in Figure 3 is I={1,2,0}.Figure 3
Orthogonalization procedure following the orderingI={1,2,0}.Note that the set of responses{hi(t)}i=02 and sequences {bk(i)}i=02 resulting from the procedures described in Figures 2 and 3 are two different expansions of the same signal s(t). Any one of the N! possible sets can be selected in the orthogonalization process.Space compression of the channel can be achieved concentrating the maximum of the channel energy on a reduced set of paths, independent of the distribution of their individual energy, as explained in the following discussion.LetP be a given number of paths used to model the channel (e.g., if P=N, note that any orthogonalization process I shall represent exactly the behavior of the channel). To reduce the complexity of the receiver, it is desirable for P to be as small as possible. On the other hand, to minimize the performance degradation (or the inaccuracy of the channel representation), the orthogonalization process should be carried out in such a way that the most part of the signal energy is concentrated on these P paths. From the above,for a given value of P, theoptimum set Iop for space compression (i.e., minimal channel distortion modeling) should meet the following condition:
(17)∑i=0P-1‍EIop(i)≥∑i=0P-1‍EI^i,∀I^≠Iop,
where
(18)EI(i)=Ebki2∫-∞∞‍hit2dt
is the energy in the ith path for the set I, while E{·} denotes expectation. If P=N, notice that any orthogonalization process I will satisfy condition (17). Otherwise, if P<N the criterion guarantees that the signal energy contained inthe completePpaths shall be maximum, independent of the individual energy distribution among them.The optimum channel expansion that satisfies (17) can be found by an exhaustive search. Instead, we propose to achieve the orthogonalization process to meet the following criteria:
(19)E^(0)≥E^1≥⋯≥E^(N-1),
with E^(j)=∫-∞∞‍|hn(t)|2dt. Condition (19) can be achieved by selecting at each orthogonalization step (ith step, e.g., with i∈{1,2,…,N}) the pivoting response (hi-1(t)) as the response with highest energy among theremaining responses (those orthogonal to ∩n=0i-2Hn for t=kT, ∀k). That is, at the first step we select the pivoting response h0(t) as the Volterra kernel with the highest energy in fn(t)n=0N-1. At the second step we select the pivoting response h1(t) as the response with highest energy in the set of responses orthogonal to H0 (i.e., gm(0)(t), with m=1,2,…,N-1). This procedure is repeated in the same way at each step, where the best pivoting response at each stage is selected by an exhaustive search among all the pivot candidates at that stage. The minimization of the energy of the orthogonal responses (see (3), (4), and associated discussion) ensures that condition (19) is met. As we shall show later, condition (19) gives rise tospace channel compression, which can be exploited to reduce complexity.


## 2.3. Example

As an example, we present results for a 10 Gb/s, 700 km, and IM/DD fiber-optic link (details about the simulated system are given in Section4.1).Without loss of generality, we assume thatak∈{±1}. Let Sn be the normalized cumulative path energy of thetraditional Volterra-series expansion (see (2)) given by
(20)Sn=1Es∑m=0n‍∫-∞∞‍fmt2dt,n=1,2,…,N-1,
where Es is the total energy of the signal component s(t):
(21)Es=∑m=0N-1‍∫-∞∞‍fmt2dt.On the other hand, the normalized cumulative path energy of theorthogonalized Volterra-series expansion is defined as
(22)Sn=1Es∑m=1n‍EI(m),n=0,1,…,N-1,
where EI(m) is given by (18) with I obtained according to the criteria defined by (19). In all cases, note that Sn≤1  ∀n, with SN-1=1.Figure4 shows the normalized cumulative path energy Sn for the traditional (20) and orthogonalized (22) Volterra-series representation with N=8. For the orthogonal representation, the pivoting responses are selected according to the criteria in (19). We observe that most energy of the nonlinear signal is concentrated on the first two paths for the orthogonalized Volterra-series expansion (i.e., EI(0)+EI(1) represents ~99.4% of the total signal energy). The traditional Volterra model requires five paths to capture a similar level of nonlinear signal energy. We highlight that the increase of Sn from −0.33 dB to −0.12 dB at n=0 is due to the factor E{|bk(0)|2}≥1 in (18). Notice that this factor is a measure of the correlation between all nonlinear kernels fm(t) with m>0 and the linear kernel f0(t) (e.g., E{|bk(0)|2}=1 (N) if f0(t) and fm(t) are orthogonal (same) pulses ∀m). Thus, we conclude the following for the channel analyzed in the example.(i)
Most energy of the nonlinear kernels with the traditional Volterra model is contained in their projection onto thelinear signal space spanned by the set {h0(t-kT)}={f0(t-kT)}.(ii)
A channel model with2 paths captures practically all the signal energy.(iii)
SinceEI(0) is ~97.5% of the total signal energy (see Figure 4). Therefore, as we shall show later, a receiver that considers the first channel path should be able to achieve agood performance.Figure 4
Normalized cumulative path energy of the nonlinear IM/DD fiber-optic link withL=700km and 1/T=10Gb/s.


## 3. MLSD Receiver for Nonlinear Channels with AWGN

The MLSD receiver chooses the sequence{ak} that minimizes the metric
(23)J=∫-∞∞‍r(t)-s(t)2dt.
Using (13) and (16) in (23), we get
(24)J∝-2∑n=0N-1‍R∑k‍bk(n)∫-∞∞‍r(t)hn*(t-kT)dt+∑n=0N-1‍∫-∞∞‍snt2dt=-2∑n=0N-1‍R∑k‍bk(n)r-k(n)+∑n=0N-1‍∫-∞∞‍snt2dt,
where R denotes the real part, while
(25)r-k(n)=∫-∞∞‍rthn*t-kTdt,n=0,1,…,N-1.
From (24) notice that the computation of the cost function J for every candidate symbol sequence {ak} can be achieved from the samples at the outputs of the matched filters given by (25).From (1), equation (25) can be rewritten as
(26)r-k(n)=s-k(n)+z-k(n),n=0,1,…,N-1,
where
(27)s-k(n)=∫-∞∞‍s(t)hn*(t-kT)dt,z-k(n)=∫-∞∞‍z(t)hn*(t-kT)dt.From (15) note that Es-l(n)s-l-k(m)*=0 if m≠n while
(28)Ez-l(n)z-l-k(m)*=N0ρk(n),ifn=m0,otherwise,
where
(29)ρk(n)=∫-∞∞‍hn(t)hn*(t-kT)dt.
Since the noise is assumed Gaussian, from (28) we conclude that the noise components at the output of the proposed MFB arespatially independent. Therefore, the bank of matched filters hn*(-t)n=0N-1 followed by a baud rate sampler is called thespace-whitened matched filter (S-WMF) bank.


### 3.1. Space-Time Whitened Matched Filter MLSD Receiver

From the above, we observed that the matched filter bank derived from the new expansion of the nonlinear channel gives rise tospatially independent noise components. In order to simplify the implementation of the sequence detector, a space and time whitening filter bank is derived.Letϕn(t) be the impulse response of the filter with Fourier transform (FT) given by
(30)Φn(ω)=Hn(ω)γ(n)Mn(ejωT),n=0,…,N-1,
where Hn(ω) is the FT of hn(t) and γ(n) and Mn(z) are defined by the folded spectrum factorization Sn(z)=(γ(n))2Mn(z)Mn*(1/z*) with Sn(z) being the Z-transform of the sequence ρk(n) given by (29). The set {ϕn(t-kT)} forms an orthonormal basis for the signal space spanned by {hn(t-kT)}. Furthermore, we choose Mn(z) to be minimum phase [19]. Define r~n(t) as the projection of r(t) onto the signal space spanned by {hn(t-kT)}; that is,
(31)r~n(t)=∑k‍r~k(n)ϕn(t-kT),
where r~k(n)=∫-∞∞‍r(t)ϕn*(t-kT)dt. From (15) and following the procedure used in [19, Section 10.2.4], metric (23) can be expressed as
(32)J=∫-∞∞‍r~(t)-∑k‍H(t-kT)bk2dt,
where H(t) is an N×N diagonal matrix given by
(33)H(t)=h0(t)0⋯00h1(t)⋯0⋮⋮⋱⋮00⋯hN-1(t),
while
(34)bk=bk0bk1⋯bkN-1T,(35)r~t=r~0tr~1t⋯r~N-1(t)T,
with bk(n) and r~n(t) given by (14) and (31), respectively.On the other hand, from (30) it is possible to show that
(36)∑k‍H(t-kT)bk=∑k‍Φ(t-kT)Mk⊗bk,
where Φ(t) and Mk are N×N diagonal matrices given by
(37)Φ(t)=ϕ0(t)0⋯00ϕ1(t)⋯0⋮⋮⋱⋮00⋯ϕN-1(t),(38)Mk=γ(0)mk(0)0⋯00γ(1)mk(1)⋯0⋮⋮⋱⋮00⋯γ(N-1)mk(N-1),
with mk(n) being the inverse FT of Mn(ejωT). From (31) notice that (35) can be expressed as
(39)r~(t)=∑k‍Φ(t-kT)r~k,
where r~k=[r~k0⋯r~k(N-1)]T (see (31)). From (32), (36), and (39) it can be shown that MLSD reduces to minimize
(40)J=r~k-Mk⊗bk2.Letz~k be the N-dimensional vector with the noise components of the baud rate samples r~k. From (15) and (30), the power spectral density of z~k results in Sz~=N0I, where I is the identity matrix. Therefore, the minimization of (40) can be easily implemented by using a Viterbi detector withmultidimensional Euclidean branch metrics. Figure 5 shows a block diagram of the ST-WMF-MLSD receiver withdimensionP (i.e., P is the number of filters in the bank). If P=N, all the paths of the nonlinear channel are used by the receiver.Figure 5
Block diagram of theP-dimensional ST-WMF-MLSD receiver.


### 3.2. Complexity Considerations

The ST-WMF-MLSD consists of a filter bank followed by a Viterbi decoder (VD) and a channel estimation stage. Although the computational load of the latter may be important, its complexity is not critical since it can be implemented at a low frequency rate3. On the other hand, the dimension of the front-end (P) is expected to be very low in general as a result of the spatial energy compression achieved by the ST-WMF-MLSD. Then, P≤2 is a reasonably good estimation for applications in IM/DD optical systems and the computational complexity of the filter bank is reduced to the one of 2linear filters. Therefore,the implementation complexity of ST-WMF-MLSD is dominated by the VD. Practical aspects related to high-speed implementations of VD have been widely investigated in the past literature (e.g., see [12, 22] for more details). The computational load (usually measured in number of multiplications and comparisons) and storage requirements of the VD depend directly on the number of states. Therefore, we adopt the number of states of the VD as the measure of complexity in order to compare the MLSD architectures investigated in this work.


## 3.1. Space-Time Whitened Matched Filter MLSD Receiver

From the above, we observed that the matched filter bank derived from the new expansion of the nonlinear channel gives rise tospatially independent noise components. In order to simplify the implementation of the sequence detector, a space and time whitening filter bank is derived.Letϕn(t) be the impulse response of the filter with Fourier transform (FT) given by
(30)Φn(ω)=Hn(ω)γ(n)Mn(ejωT),n=0,…,N-1,
where Hn(ω) is the FT of hn(t) and γ(n) and Mn(z) are defined by the folded spectrum factorization Sn(z)=(γ(n))2Mn(z)Mn*(1/z*) with Sn(z) being the Z-transform of the sequence ρk(n) given by (29). The set {ϕn(t-kT)} forms an orthonormal basis for the signal space spanned by {hn(t-kT)}. Furthermore, we choose Mn(z) to be minimum phase [19]. Define r~n(t) as the projection of r(t) onto the signal space spanned by {hn(t-kT)}; that is,
(31)r~n(t)=∑k‍r~k(n)ϕn(t-kT),
where r~k(n)=∫-∞∞‍r(t)ϕn*(t-kT)dt. From (15) and following the procedure used in [19, Section 10.2.4], metric (23) can be expressed as
(32)J=∫-∞∞‍r~(t)-∑k‍H(t-kT)bk2dt,
where H(t) is an N×N diagonal matrix given by
(33)H(t)=h0(t)0⋯00h1(t)⋯0⋮⋮⋱⋮00⋯hN-1(t),
while
(34)bk=bk0bk1⋯bkN-1T,(35)r~t=r~0tr~1t⋯r~N-1(t)T,
with bk(n) and r~n(t) given by (14) and (31), respectively.On the other hand, from (30) it is possible to show that
(36)∑k‍H(t-kT)bk=∑k‍Φ(t-kT)Mk⊗bk,
where Φ(t) and Mk are N×N diagonal matrices given by
(37)Φ(t)=ϕ0(t)0⋯00ϕ1(t)⋯0⋮⋮⋱⋮00⋯ϕN-1(t),(38)Mk=γ(0)mk(0)0⋯00γ(1)mk(1)⋯0⋮⋮⋱⋮00⋯γ(N-1)mk(N-1),
with mk(n) being the inverse FT of Mn(ejωT). From (31) notice that (35) can be expressed as
(39)r~(t)=∑k‍Φ(t-kT)r~k,
where r~k=[r~k0⋯r~k(N-1)]T (see (31)). From (32), (36), and (39) it can be shown that MLSD reduces to minimize
(40)J=r~k-Mk⊗bk2.Letz~k be the N-dimensional vector with the noise components of the baud rate samples r~k. From (15) and (30), the power spectral density of z~k results in Sz~=N0I, where I is the identity matrix. Therefore, the minimization of (40) can be easily implemented by using a Viterbi detector withmultidimensional Euclidean branch metrics. Figure 5 shows a block diagram of the ST-WMF-MLSD receiver withdimensionP (i.e., P is the number of filters in the bank). If P=N, all the paths of the nonlinear channel are used by the receiver.Figure 5
Block diagram of theP-dimensional ST-WMF-MLSD receiver.


## 3.2. Complexity Considerations

The ST-WMF-MLSD consists of a filter bank followed by a Viterbi decoder (VD) and a channel estimation stage. Although the computational load of the latter may be important, its complexity is not critical since it can be implemented at a low frequency rate3. On the other hand, the dimension of the front-end (P) is expected to be very low in general as a result of the spatial energy compression achieved by the ST-WMF-MLSD. Then, P≤2 is a reasonably good estimation for applications in IM/DD optical systems and the computational complexity of the filter bank is reduced to the one of 2linear filters. Therefore,the implementation complexity of ST-WMF-MLSD is dominated by the VD. Practical aspects related to high-speed implementations of VD have been widely investigated in the past literature (e.g., see [12, 22] for more details). The computational load (usually measured in number of multiplications and comparisons) and storage requirements of the VD depend directly on the number of states. Therefore, we adopt the number of states of the VD as the measure of complexity in order to compare the MLSD architectures investigated in this work.


## 4. Performance Evaluation in IM/DD Optical Systems

Next we analyze the proposed ST-WMF-MLSD receiver in transmissions over IM/DD fiber-optic systems with on-off keying (OOK) modulation. We focus on two key aspects of ST-WMF-MLSD: its performance (in comparison with current solutions based on OS-MLSD), and its ability to reduce complexity (e.g., number of states of VD). Complexity reduction is possible owing to (i) the minimum-phase property of the equivalent channel response provided by ST-WMF and (ii) condition (19). The latter gives rise tospace compression, which reduces the ST-WMF dimension, P. This is achieved by using themost significantP paths of the nonlinear channel.Figure6 depicts the optical system under consideration. The transmitter modulates the intensity of the transmitted signal using NRZ-OOK modulation. The standard single mode fiber (SMF) introduces CD and PMD, as well as attenuation. Optical amplifiers are deployed periodically along the fiber to compensate the attenuation, also introducing amplified spontaneous emission (ASE) noise in the signal. ASE noise is modeled as AWGN in the optical domain. The received optical signal is filtered by an optical filter and then converted to a current with a PIN diode or avalanche photodetector. The resulting photocurrent is filtered by an electrical filter. The noise component after the electrical filtering is non-Gaussian and signal-dependent [16]. Therefore, the electrical signal is first processed by a memoryless nonlinear transformation. It has been found that after a square root transformation, the noise can be assumed Gaussian and signal-independent [23, 24]. Furthermore, channel nonlinearities can also be reduced by using the square root transformation [25], which improves the space compression used to reduce the receiver dimension (i.e., most of the channel energy is concentrated on the linear kernel). The split-step Fourier method [26] is used to compute the propagation of optical signals through the fiber. Oversampled linear and nonlinear kernels are extracted from the electrical signal after the square root transformation. The oversampling factor R=T/Ts depends on various parameters of the communication system (e.g., optical power, fiber length, etc.). In our case, we have found that an oversampling factor of R=16 is good enough to accurately model the system; that is, no improvement was appreciated by increasing the sampling rate for the entire set of conditions considered for this work. Then, we compute hn(kTs) and λk(n,m) according to (17), and the symbol rate channel response matrix Mk can be easily obtained from (38). Since the noise after the square root transformation is approximately Gaussian and signal-independent [27], the theory proposed in [28] is used to evaluate the bit error probability4. All the kernels of the nonlinear channel are used to compute the error probability, independent of the receiver dimension, P. Data rate is 1/T=10Gb/s and the transmitted pulse shape has an unchirped Gaussian envelope e-t2/2T02 with T0=36ps. We use a Lorentzian optical filter and a fourth-pole Butterworth electrical filter with bandwidths of 15 and 10GHz, respectively. The fiber dispersion is D=17 ps/(nm-km).Figure 6
IM/DD fiber-optic system with ST-WMF-MLSD receiver.


### 4.1. IM/DD Systems with CD and PMD

Typically, the performance of equalization stages in fiber optic transmissions is evaluated by using the optical signal-to-noise ratio (OSNR) required to achieve a given BER (e.g., see [10, 11]). The target BER is around 10-3, which corresponds to the value required at the input of the forward error correction (FEC) code to achieve the error rate expected by the application (e.g., ~10−15) [29].Figure7 depicts the OSNR penalty with respect to a B2B system at a bit-error rate (BER) of 10-3 versus the number of states of the VD for L=700 km. We present results for ST-WMF-MLSD with P=1 (i.e., only one filter in the bank). For OS-MLSD with 2 samples/bit, the reduction of states is achieved by truncation and optimization of the sampling phase in order to minimize BER (8 uniformly distributed phases in the interval T/2 were tested). From Figure 7, we verify that the number of states of the VD at a penalty of 4.6 dB can be reduced from 2048 to 256 with ST-WMF-MLSD and P=1. Notice that this performance is achieved by using a VD with one sample per bit [11]. Furthermore, we emphasize that these benefits greatly outperform the extra complexity required by the linear filter and the channel estimator (implementation details are omitted here due to space limitations). From Figure 7 we can also see that the performance degradation achieved with P=1 with respect to the ideal MLSD is only ~0.3 dB. This result can be understood from the analysis of Section 2.2, where it has been observed that 97.5% of the total signal energy is contained in the first channel path.Figure 7
OSNR penalty with respect to B2B atBER=10-3 versus number of states of the VD for L=700km and DGD=0 ps.The performance of ST-WMF-MLSD in the presence of CD and first-order PMD (i.e., differential group delay or DGD) is investigated. Figure8 depicts the OSNR penalty versus the number of states of the VD for a 700 km fiber link with two values of DGD: 25 and 50ps. We evaluate the OS-MLSD with 2 samples/bit and ST-WMF-MLSD with P=2. As expected, both receivers tend to the same performance as the number of states of the VD increases. We also note that the benefits of the ST-WMF-MLSD reduce when the DGD increases5. Nevertheless, from Figure 8 notice that the number of states of the VD at a penalty of 6 dB can be reduced 8 and 4 times with ST-WMF-MLSD at 25 and 50 ps DGD, respectively. These results show that the ST-WMF-MLSD is still an attractive solution to reduce complexity in transmissions over fiber-optic channels in the presence of PMD.OSNR penalty respect to B2B atBER=10-3 versus number of states of the VD for L=700km. DGD=25 ps (a) and 50 ps (b).
(a)
(b)


## 4.1. IM/DD Systems with CD and PMD

Typically, the performance of equalization stages in fiber optic transmissions is evaluated by using the optical signal-to-noise ratio (OSNR) required to achieve a given BER (e.g., see [10, 11]). The target BER is around 10-3, which corresponds to the value required at the input of the forward error correction (FEC) code to achieve the error rate expected by the application (e.g., ~10−15) [29].Figure7 depicts the OSNR penalty with respect to a B2B system at a bit-error rate (BER) of 10-3 versus the number of states of the VD for L=700 km. We present results for ST-WMF-MLSD with P=1 (i.e., only one filter in the bank). For OS-MLSD with 2 samples/bit, the reduction of states is achieved by truncation and optimization of the sampling phase in order to minimize BER (8 uniformly distributed phases in the interval T/2 were tested). From Figure 7, we verify that the number of states of the VD at a penalty of 4.6 dB can be reduced from 2048 to 256 with ST-WMF-MLSD and P=1. Notice that this performance is achieved by using a VD with one sample per bit [11]. Furthermore, we emphasize that these benefits greatly outperform the extra complexity required by the linear filter and the channel estimator (implementation details are omitted here due to space limitations). From Figure 7 we can also see that the performance degradation achieved with P=1 with respect to the ideal MLSD is only ~0.3 dB. This result can be understood from the analysis of Section 2.2, where it has been observed that 97.5% of the total signal energy is contained in the first channel path.Figure 7
OSNR penalty with respect to B2B atBER=10-3 versus number of states of the VD for L=700km and DGD=0 ps.The performance of ST-WMF-MLSD in the presence of CD and first-order PMD (i.e., differential group delay or DGD) is investigated. Figure8 depicts the OSNR penalty versus the number of states of the VD for a 700 km fiber link with two values of DGD: 25 and 50ps. We evaluate the OS-MLSD with 2 samples/bit and ST-WMF-MLSD with P=2. As expected, both receivers tend to the same performance as the number of states of the VD increases. We also note that the benefits of the ST-WMF-MLSD reduce when the DGD increases5. Nevertheless, from Figure 8 notice that the number of states of the VD at a penalty of 6 dB can be reduced 8 and 4 times with ST-WMF-MLSD at 25 and 50 ps DGD, respectively. These results show that the ST-WMF-MLSD is still an attractive solution to reduce complexity in transmissions over fiber-optic channels in the presence of PMD.OSNR penalty respect to B2B atBER=10-3 versus number of states of the VD for L=700km. DGD=25 ps (a) and 50 ps (b).
(a)
(b)


## 5. Impact of the Imperfect Channel Knowledge

Performance evaluation of the ST-WMF-MLSD has been achieved by assuming a perfect knowledge of the channel. In the following, we analyze the impact of the channel estimation inaccuracy on the performance of the ST-WMF-MLSD architecture in transmissions over IM/DD optical systems. This study will show that the performance degradation in ST-WMF-MLSD receivers, withP=1, caused by an imperfect channel estimation, is low (~0.2 dB) and similar to that achieved by oversampled OS-MLSD receivers in the L=700 km fiber link used in the example of Figure 7.The estimation of the oversampled linear and nonlinear kernels is required to implement both MLSD-based receivers. LetR=T/Ts be the oversampling factor. Based on the polyphase filter representation of the oversampled channel response (see Figure 9), the received samples can be expressed as
(41)rn(i)=∑k‍akf0(i)[n-k]+∑k‍∑m=1N-1‍akak-mfm(i)[n-k]+zn(i),
where rn(i)=r(nT+iTs),  zn(i)=z(nT+iTs), and fm(i)[n]=fm(nT+iTs) with i=0,…,R-1. Notice that there are R different sequences sampled at the baud rate, corresponding to R different sampling phases.Figure 9
Example of the polyphase filter representation of an oversampled linear channel withR=T/Ts.Equation (41) can be rewritten as
(42)rn(i)=∑k‍an-kf0(i)[k]+∑k‍∑m=1N-1‍an-kan-k-mfm(i)[k]+zn(i).
Since symbols ak are assumed zero-mean and i.i.d. real random variables with σa2=E|ak|2, we can verify that
(43)f0(i)k=1σa2Ernian-k,fmik=1σa4Ernian-kan-k-m,i=0,…,R-1.From (43), a simple estimator of the oversampled linear and nonlinear kernels can be implemented with an averaging filter as follows:
(44)f^0ik=1σa2LA∑n=n0n0+LA-1‍rnia^n-k,f^m(i)[k]=1σa4LA∑n=n0n0+LA-1‍rn(i)a^n-ka^n-k-m,
where LA is the length of the averaging filter, n0 is an arbitrary time index, and a^k is the detected symbol6. The accuracy of the channel estimation given by (44) depends on the precision of the decisions a^k, the length of the averaging filter LA, and the channel noise power. We consider that decisions provided by the forward error correction (FEC) decoder are available; therefore the effect of decision errors can be neglected (i.e., a^k=ak). We highlight that this assumption is still valid if pre-FEC decisions are used as a result of thelow bit error rates experienced in this link (e.g., ~10−3). From the above, we conclude that the goodness of the estimates (44) shall mainly depend on the filter length LA and the channel noise power.The precision of (44) improves as the value of LA increases. On the other hand, the maximum value of LA shall be imposed by the speed of temporal variations of the fiber optic channel. As a result of its dependence on stress and vibrations, as well as on random changes in the state of polarization of the laser, PMD is nonstationary. Fluctuations with a time scale of a hundreds of microseconds have been considered in previous works (e.g., [30]). Therefore, the response time of channel estimation algorithms for PMD mitigation must be less than 1 ms (in practice a response time less than 100 μs is required [27]). This imposes, for example, that the bandwidth of the averaging filter (~1/(4LAT)) should be ≥20 kHz in order to efficiently track the channel variation.The received signalseen by an MLSD receiver in the presence of imperfect knowledge of the channel dispersion can be expressed as
(45)r~ni=∑kan-kf0ik+∑m=1N-1an-k-mfmik+zni+z^ni=sni+zni+z^ni,
where sn(i)=s(nT+iTs) and z^n(i)=s^(nT+iTs)-s(nT+iTs) is theestimation error component, while s^(nT+iTs) is the synthesized signal obtained from (44).Figure10(a) shows the SNR penalty caused by the imperfect channel estimation as a function of LA obtained from computer simulations. We consider 1/T=10GHz, R=16, and the fiber link with L=700 km, as used in Figure 7. The SNR penalty caused by an imperfect channel estimation is computed as
(46)ΔSNR=σz2+σz^2σz2,
where σz2 is the channel noise power required to achieve a BER=10-3 with an unconstrained complexity OS-MLSD receiver (i.e., the VD uses as many states as required) and σz^2 is the variance of the estimation error component7. Notice that the penalty is ~0.4 dB for LA=105. This value of LA represents a BW of ~1/(4LAT)=25 kHz (see Figure 10(b)). Assuming that the estimation error is white Gaussian noise with power σz^2, from Figure 10(a) we infer that the SNR penalty caused by an imperfect channel knowledge in OS-MLSD receivers with LA=105 should be ≲ΔSNR~0.4 dB.SNR penalty caused by imperfect channel estimation versus length of the averaging filter (a) and the bandwidth of the averaging filter (b).
(a)
(b)Figure11 depicts the OSNR penalty at BER=10-3 versus the number of states of the Viterbi detector (VD) with L=700km. We present results with perfect knowledge of the fiber dispersion (denoted as LA=∞), and for imperfect channel estimation with LA=105. We see that the mean penalty caused by inaccuracies of channel estimation agrees with that expected from Figure 10 with LA=105 (i.e., ~0.14 dB <ΔSNR~0.4dB). Furthermore, we observe that the impact of imperfect channel knowledge on the performance is similar in both MLSD receivers (i.e., ~0.14 dB and ~0.18 dB for OS and ST-WMF, resp.). This result can be understood from the fact that the filters h0*(-t) and mk(0) are computed from the samples of the estimated linear kernel f^0(t). Taking into account that the energy of the linear component is significantly higher than the nonlinear kernels [25] (see Section 2.2), an accurate estimation of h0(t) can be achieved for the channel considered. Then, we infer that the energy loss of the signal component at the output of m^0(t) will be small. Therefore, and based on (45), notice that the performance of OS-MLSD and ST-WMF-MLSD with P=1 should degrade in a similar way.Figure 11
OSNR penalty atBER=10-3 versus number of states of the VD with L=700km.


## 6. Conclusions

New results on the recently proposed ST-WMF-MLSD nonlinear receiver have been presented in this paper. These results are the following: (1) thespace compression property of the factorization introduced in [18] has been analyzed in detail; (2) the performance of the ST-WMF-MLSD in IM/DD fiber optic systems in the combined presence of CD and PMD has been evaluated, and (3) it has been shown that the performance degradation caused by an imperfect channel estimation and tracking is low and similar to that achieved by existing MLSD schemes. These features make the ST-WMF-MLSD a good architecture for receivers for long distance IM/DD fiber-optic links.



---

*Source: 736267-2015-02-16.xml*