# Prototype Filter Design Based on Channel Estimation for FBMC/OQAM Systems

Authors: Yongjin Liu; Xihong Chen; Yu Zhao
Journal: Mathematical Problems in Engineering
 (2019)

---

## Abstract

A prototype filter design for FBMC/OQAM systems is proposed in this study. The influence of both the channel estimation and the stop-band energy is taken into account in this method. An efficient preamble structure is proposed to improve the performance of channel estimation and save the frequency spectral efficiency. The reciprocal of the signal-to-interference plus noise ratio (RSINR) is derived to measure the influence of the prototype filter on channel estimation. After that, the process of prototype filter design is formulated as an optimization problem with constraint on the RSINR. To accelerate the convergence and obtain global optimal solution, an improved genetic algorithm is proposed. Especially, the History Network and pruning operator are adopted in this improved genetic algorithm. Simulation results demonstrate the validity and efficiency of the prototype filter designed in this study.


---

## Body

## 1. Introduction

As an alternative modulation scheme for the fifth-generation (5G) communication, offset quadrature amplitude modulation-based orthogonal frequency division multiplexing (FBMC/OQAM) outperforms orthogonal frequency division multiplexing (OFDM) with respect to spectral efficiency, side lobes, and sensitivity to asynchronous transmission [1, 2]. As one of the important components of FBMC/OQAM systems, the prototype filter determines the performance of the whole system.In recent years, many effective methods about prototype filter design have been brought up. All these methods can be divided into two classes [3]: direct approaches and indirect approaches. Indirect approaches mainly include the windowing-based method [4, 5] and the frequency-sampling method [6–8]. Direct approaches optimize all coefficients of the prototype filter. Due to many more degrees of design freedom, direct approaches have the potential to achieve better performance than indirect approaches [9].However, the complexity grows rapidly along with the number of coefficients in the direct approach. In order to reduce the computational complexity, theα-based branch and bound αBB algorithm is used to design the prototype filter in [10]. In [11], the box-based branch and bound (Box-BB) method based on αBB algorithm is proposed to decrease the number of iterations. Spectral factorization is employed to design the final prototype filter in [12], but this method is unstable because the factor is chosen randomly. The existence of unknowns in the aforementioned methods makes it difficult to obtain the global optimum.The genetic algorithm (GA) is a powerful evolutionary algorithm which is able to solve this difficulty [13, 14]. GA is adopted to handle the thinning optimization problem in [15]. The Baldwin effect is added into GA via a History Network to avoid the inappropriate answer in [16], but there are so many random operators in this method. In [17], the random assignments of value during initialization and mutation operator are deleted by the pruning operator to reach the answer more quickly. In this study, an improved GA based on the History Network and pruning operator is proposed to deal with these random operators.Furthermore, accurate channel estimation (CE) at the receiver is indispensable for reliable signal detection. There have already been some studies on the CE for the FBMC/OQAM system.In [18], two modified CE methods were proposed by placing guard symbols at different sides of reference preamble symbol, the existence of guard symbols reduces spectrum efficiency. In order to increase spectrum efficiency, coded auxiliary pilot symbols that also carry information are designed in [19]. However, at least two to four pilot symbols are needed to achieve acceptable performance in practice. In [20], an efficient preamble structure is proposed to achieve a better CE with less preamble consumption. Because there is no pilot in the compressed sensing method, CE methods based on compressed sensing are proposed further to increase the spectral efficiency. In [21], when prior sparse knowledge is unknown, an adaptive, regularized, compressive sampling matching pursuit chooses the support set to achieve better CE accuracy. Inspired by the same theory as in [21], two improved CE methods are derived, respectively, based on the interference approximation method (IAM) and pair of real pilots in [22]. The key to these methods is that how to prove that the FBMC/OQAM system is sparse. However, whether the system, like the scattering channel, is sparse is debatable.Few studies have taken the CE into account in prototype filter design. In [23], the prototype filter is designed by utilizing the mean square error (MSE) of the CE. However, zero symbols inserted in the preambles will reduce the system’s spectral efficiency. Inspired by the preamble structure described in [20], preamble without guard symbols is considered in these studies to acquire higher transmission efficiency. The proposed structure will improve the accuracy of CE by increasing pseudo pilot power.The influence of noise may be also ignored in prototype filter design. For example, in [24], the weight signal-to-interference ratio is treated as the utility function to design the prototype filter under highly doubly dispersive channel without noise. However, the noise power is generally equal to or even larger than the interference power in practical systems. Therefore, the noise influence should be given sufficient attention in prototype filter design. Usually, the signal-to-interference plus noise ratio (SINR) is adopted to measure the noise influence. A bigger SINR means better system performance. Therefore, it will be difficult to set the threshold when the SINR is added to the constraints. Under this condition, in this study, the reciprocal of the signal-to-interference plus noise ratio (RSINR) is adopted to measure the influence of noise.Simulation results indicate that the proposed preamble structure achieves outperformance compared to the IAM, the improved GA shows less convergence time and average generations than the proposed GAs in [16, 17]; the designed prototype filter in this study is better than in [23, 24] when it comes to the performance of the stop-band energy and the magnitude response.The rest of the study is organized as follows: In Section2, the FBMC/OQAM model is described, the preamble is proposed, and the CE is obtained. In Section 3, the process of prototype filter design is formulated by minimizing the stop-band energy with constraints on the RSINR, and the GA with the History Network and pruning operator is utilized to solve the optimization problem. The results of the simulations are discussed in Section 4, and finally, a conclusion is given in Section 5.


## 2. System Model

As one of the contributions of this paper, the preamble structure with larger pseudo pilot power and spectrum efficiency is introduced in this section.At first, the FBMC/OQAM system model is built. Instead of transmitting complex symbolscm,n=cm,nℜ+jcm,nℑ, the FBMC/OQAM system transmits the real-valued symbols am,n which are from the constellation mapping of the real components cm,nℜ and imaginary components cm,nℑ. According to [25], the continuous-time baseband equivalent of the transmitting signal in FBMC/OQAM can be written as(1)st=∑n=−∞+∞∑m=0M−1am,nejϕm,nej2πmν0tgt−nτ0︸gm,nt,where M denotes the number of subcarriers, ν0=1/T0=1/2τ0 denotes the subcarrier spacing, T0 denotes the constellation interval, and τ0 denotes the offset between the real part and imaginary part of the OQAM symbol. gt denotes the prototype filter function. ϕm,n is an additional phase term:(2)ϕm,n=ϕ0+π2m+nmodπ,where ϕ0 can be chosen arbitrarily. For simplification, in this study, ϕ0=0.Considering the multipath channel with an additive white Gaussian noise, which is denoted asηt, when the transmitted FBMC/OQAM signal is passed through, the baseband version of the received signal is described as(3)yt=∫0Δhτst−τdτ+ηt=∑n=−∞+∞∑m=0M−1am,n∫0Δhτe−j2πmν0τejπm+n/2ej2πmν0t×gt−τ−nτ0dτ+ηt,where ht denotes the channel impulse response (CIR) and Δ presents the maximum delay spread of the channel. Especially, we assume that gt−τ−nτ0≈gt−nτ0 in the time interval τ∈0,Δ for the flat fading channel. In that case, rewrite (3) as(4)yt=∑n=−∞+∞∑m=0M−1am,nejπm+n/2ej2πmν0tgt−nτ0×∫0Δhτe−j2πmν0τdτ+ηt=∑n=−∞+∞∑m=0M−1am,ngm,ntHm+ηt,where Hm=∫0Δhte−j2πmν0tdt represents the channel impulse frequency response.The demodulation signal at them0,n0 position can be written as(5)ym0,n0=yt,gm0,n0=∑n=−∞+∞∑m=0M−1am,nHmgm,nt,gm0,n0+ηm0,n01,where x,y represents the inner product of x and y and ηm0,n01=ηt,gm0,n0.The orthogonal condition in FBMC/OQAM is(6)ℜgm,n,gm0,n0=δm,m0δn,n0,where ℜ⋅ denotes real part of the argument and δt denotes the Dirac’s delta function.Thus, the FBMC/OQAM system model has been built. After that, in order to design the new preamble structure, the relationship between preamble and CE is necessary to be analyzed.


### 2.1. Relationship between Preamble and CE

We definem=m0+p, n=n0+q, where p,q∈Z, and then, the inner product of the pulse shaping filter in equation (5) can be rewritten as(7)gm,nt,gm0,n0=∫−∞+∞gm0+p,n0+qtgm0+n0∗tdt=∫−∞+∞ejπ/2m0+n0p+qej2πm0+pvotgt−n0τ0−qτ0×e−jπ/2m0+n0e−j2πm0v0tg∗t−n0τ0dt=jp+q∫−∞+∞gt−n0τ0−qτ0g∗t−n0τ0ej2πpv0tdt,where ∗ denotes conjugate operation. Since νoτ0=1/2, t−n0τ0−qτ0=l−q/2τ0, and t−n0τ0=l+q/2τ0, equation (7) can be rewritten as(8)gm,nt,gm0,n0=jp+q∫−∞+∞gl+−q/2τ0g∗l−−q2τ0×e−j2πpν0l+n0τ0+qτ0/2dl=jp+q+pq+2pn0Ag−qτ0,pν0,where Agτ,ν denotes the ambiguity function of the filter, which can be expressed as(9)Agτ,ν=∫gt+τ2g∗t−τ2ej2πνtdt.Whengt is an even function, its instantaneous autocorrelation function γgτ,t=gt+τ/2g∗t−τ/2 is also an even conjugate. Agτ,ν is the inverse Fourier transformation of γgτ,t, and therefore, Agτ,ν is a real-valued function [26].We can get equation (10) from equation (5):(10)ym0,n0=∑p,q∈Zjp+q+pq+2pn0Ag−qτ0,pν0×am0+p,n0+qHm0+p+ηm0,n01=am0,n0Hm0+∑p,q≠0,0jp+q+pq+2pn0×Ag−qτ0,pν0am0+p,n0+qHm0+p+ηm0,n01.According to the conventional OFDM CE method, the channel frequency responseH^m0 can be estimated by H^m0,n0=ym0,n0/am0,n0 when the pilot symbol am0,n0 is transmitted at time-frequency position m0,n0.We defineCp,q=jp+q+pq+2pn0Ag−qτ0,pν0, and Δm0 and Δn0 are, respectively, the neighborhoods of m0 and n0. ΩΔm0,Δn0=p,q,p≤Δm0,q≤Δn0Hm0+p,n0+q≈Hm0,n0 and ΩΔm0,Δn0∗=ΩΔm0,Δn0−0,0, where ΩΔm0,Δn0 denotes the neighborhood of position m0,n0. With the increase in p and q, Cp,q becomes very close to zero. For example, for an isotropic orthogonal transform algorithm (IOTA) function, when p,q∉Ω1,1, we have [27](11)∑p,q∉Ω1,1Cp,q2∑p,q∉Ω1,1Cp,q2≈0.02.From equation (11), we can conclude that one column of zero symbols is enough to prevent the preamble from being interfered by adjacent unknown data symbols. However, the zero symbols affect the CE performance. The structure of the IAM is shown in Figure 1. It can be seen from Figure 1 that there are two columns of zero symbols for IAM. According to [27], when only one column of zero symbols is inserted in the preamble, its CE performance is worse than the IAM structure.Figure 1
Structure of the IAM.Sinceam0−1,n0 and am0+1,n0 are of inverse signs, equation (10) can be rewritten as(12)ym0,n0=am0,n0Hm0+C−1,0am0−1,n0Hm0−1+C1,0am0+1,n0Hm0+1+∑p,q≠0,0,p,q≠±1,0,q≠±1Cp,qam0+p,n0+qHm0+p+ηm0,n01.We assumeHm0=Hm0+1=Hm0−1. The CE based on the IAM can be obtained by(13)H^m0=ym0,n0bm0,n0=Hm0+∑p,q≠0,0,p,q≠±1,0,q≠±1Cp,qam0+p,n0+qHm0+pbm0,n0+ηm0,n01bm0,n0,where bm0,n0=am0,n0+C1,0am0+1,n0+C−1,0am0−1,n0. bm0,n0 can be treated as the pseudo pilot. Equation (13) implies that a larger pseudo pilot power leads to a better CE performance. The power of pseudo pilot bm0,n0 for the IAM is shown in the following equation:(14)E1bm0,n02=am0,n02+C−1,0am0−1,n0+C1,0am0+1,n02=σa21+4Ag+4Ag2,where σa2 denotes the variance of the transmitted FBMC/OQAM symbols am,n.Obviously, CE based on the IAM has two drawbacks: (1) the preamble duration is3τ0, which leads to the low spectrum efficiency, and (2) the second item in equation (13) cannot be ignored when both the signal-to-noise ratio and constellation mapping order are high.The fact is that the power of pseudo pilotbm0,n0 is mainly affected by the pilot symbols that surround am0,n0 for the IAM. Therefore, it can be increased by changing the structure. Additionally, the zero symbols in the IAM will be redundant if the intersymbol interference is alleviated by the preamble.In order to overcome two drawbacks of the IAM, a new preamble structure is proposed in this paper. It is shown in Figure2. For sufficient accuracy and concise description, only three-tap interference of neighborhood symbols is considered.Figure 2
Proposed structure of preamble.The pseudo pilot power of the proposed structure is(15)E2bm0,n02=am0,n02+∑p,q∈Ω3,3∗Cp,qam0+p,n0+q2>E1.From (15), we can conclude that the proposed preamble structure is able to improve CE performance for its larger pseudo pilot power. Obviously, it also increases the frequency spectrum efficiency.


### 2.2. RSINR

In order to take the influence of prototype filter on CE into account while designing the prototype filter, the RSINR is chosen to measure these influences. The expression of RSINR based on the CIR is derived in this section.First, we rewrite equation (10) as(16)ym0,n0=am0,n0Hm0+am0,n0i1Hm0+am0,n0i2Hm0+ηm0,n01,where(17)am0,n0i1=∑p,q∈Ω3,3∗,q=0Cp,qam0+p,n0+q,(18)am0,n0i2=∑p,q∈Ω3,3∗,q≠0Cp,qam0+p,n0+q,and then, the channel frequency impulse response H^m0 is estimated by(19)H^m0=ym0,n0am0,n0+am0,n0i1+am0,n0i2.Becauseam0+p,n0+q in equation (18) belongs to a random signal, its value cannot be determined during transmission. As a result, the term am0,n0i2 is unknown. Equation (19) will be invalid if am0,n0i2 is unknown. To solve this problem, the predecision method is adopted. The process of estimation based on the predecision method is shown as follows.First, the initial channel frequency responseH¯m0 can be obtained from (16):(20)H¯m0=ym0,n0am0,n0+am0,n0i1=Hm0+am0,n0i2Hm0+ηm0,n01am0,n0+am0,n0i1.Next,am0+p,n0+q in the term am0,n0i2 can be rebuilt based on zero-forcing equalization. Estimate am0+p,n0+q based on the predecision method, which is shown as follows:(21)a^m0+p,n0+q=Dym0+p,n0+qH¯m0,p,q∈Ω3,3∗,q≠0.Here,D⋅ denotes the predecision operator.Then,am0,n0i2 can be estimated by(22)a^m0,n0i2=∑p,q∈Ω3,3∗,q≠0Cp,qa^m0+p,n0+q.Finally, sinceam0,n0i2 has been obtained, the CE is obtained by substituting am0,n0i2 into equation (19). Equation (23) shows the final channel frequency response expression:(23)H^m0=ym0,n0am0,n0+am0,n0i1+a^m0,n0i2.By inverse Fourier transform, the CIR of the FBMC/OQAM system is obtained as follows:(24)h^t=12π∫0ΔH^me−j2πmν0tdt.We then discretize the CIRh^t into form h^l as follows:(25)h^t=∑i=0P−1h^iδτ−τi,h^l=h^i,l=τiTs,0,otherwise,where x denotes the smallest integer larger than or equal to x and Ts=T0/2M denotes the sampling period.The desired power of the system is closely related to the transmitted symbol. The transmitted symbol at the positionm0,n0 can be estimated as follows:(26)a^m0,n0=ℜym0,n0H^m0.The QAM symbols are reconstructed by combining(27)c^m0,n0=a^m0,2n0+ja^m0,2n0+1.We then define(28)Hm0p,q=∑l=0Lh−1h^le−jπ2m0+pl/MAg−l−qM2,−p,where Lh is the number of channel taps at the sampling rate and Agk,l=AgkTs,l/MTs.The desired power of the system is(29)Pdm0=σc2ℜHm0p,qH^m02,where σc2 denotes the variance of the complex symbol c. According to the definition of the variance, σc2 is the real number.Similarly, the interference power and the noise power are expressed as [28](30)PISI+ICIm0=2σa2∑p,q≠0,0ℜejπ/2p+q+pqHm0p,qH^m02,(31)Pnoisem0=varηm0,n01=ση2H^m02,where ση2 denotes the variance of the noise η.Thus, the expression of RSINR is obtained by equations (29)–(31):(32)RSINR=PICI+ISI+PnoisePdm0=2σa2∑p,q≠0,0ℜejπ/2p+q+pqHm0p,q/H^m02+ση2/H^m02σc2ℜHm0p,q/H^m02.


## 2.1. Relationship between Preamble and CE

We definem=m0+p, n=n0+q, where p,q∈Z, and then, the inner product of the pulse shaping filter in equation (5) can be rewritten as(7)gm,nt,gm0,n0=∫−∞+∞gm0+p,n0+qtgm0+n0∗tdt=∫−∞+∞ejπ/2m0+n0p+qej2πm0+pvotgt−n0τ0−qτ0×e−jπ/2m0+n0e−j2πm0v0tg∗t−n0τ0dt=jp+q∫−∞+∞gt−n0τ0−qτ0g∗t−n0τ0ej2πpv0tdt,where ∗ denotes conjugate operation. Since νoτ0=1/2, t−n0τ0−qτ0=l−q/2τ0, and t−n0τ0=l+q/2τ0, equation (7) can be rewritten as(8)gm,nt,gm0,n0=jp+q∫−∞+∞gl+−q/2τ0g∗l−−q2τ0×e−j2πpν0l+n0τ0+qτ0/2dl=jp+q+pq+2pn0Ag−qτ0,pν0,where Agτ,ν denotes the ambiguity function of the filter, which can be expressed as(9)Agτ,ν=∫gt+τ2g∗t−τ2ej2πνtdt.Whengt is an even function, its instantaneous autocorrelation function γgτ,t=gt+τ/2g∗t−τ/2 is also an even conjugate. Agτ,ν is the inverse Fourier transformation of γgτ,t, and therefore, Agτ,ν is a real-valued function [26].We can get equation (10) from equation (5):(10)ym0,n0=∑p,q∈Zjp+q+pq+2pn0Ag−qτ0,pν0×am0+p,n0+qHm0+p+ηm0,n01=am0,n0Hm0+∑p,q≠0,0jp+q+pq+2pn0×Ag−qτ0,pν0am0+p,n0+qHm0+p+ηm0,n01.According to the conventional OFDM CE method, the channel frequency responseH^m0 can be estimated by H^m0,n0=ym0,n0/am0,n0 when the pilot symbol am0,n0 is transmitted at time-frequency position m0,n0.We defineCp,q=jp+q+pq+2pn0Ag−qτ0,pν0, and Δm0 and Δn0 are, respectively, the neighborhoods of m0 and n0. ΩΔm0,Δn0=p,q,p≤Δm0,q≤Δn0Hm0+p,n0+q≈Hm0,n0 and ΩΔm0,Δn0∗=ΩΔm0,Δn0−0,0, where ΩΔm0,Δn0 denotes the neighborhood of position m0,n0. With the increase in p and q, Cp,q becomes very close to zero. For example, for an isotropic orthogonal transform algorithm (IOTA) function, when p,q∉Ω1,1, we have [27](11)∑p,q∉Ω1,1Cp,q2∑p,q∉Ω1,1Cp,q2≈0.02.From equation (11), we can conclude that one column of zero symbols is enough to prevent the preamble from being interfered by adjacent unknown data symbols. However, the zero symbols affect the CE performance. The structure of the IAM is shown in Figure 1. It can be seen from Figure 1 that there are two columns of zero symbols for IAM. According to [27], when only one column of zero symbols is inserted in the preamble, its CE performance is worse than the IAM structure.Figure 1
Structure of the IAM.Sinceam0−1,n0 and am0+1,n0 are of inverse signs, equation (10) can be rewritten as(12)ym0,n0=am0,n0Hm0+C−1,0am0−1,n0Hm0−1+C1,0am0+1,n0Hm0+1+∑p,q≠0,0,p,q≠±1,0,q≠±1Cp,qam0+p,n0+qHm0+p+ηm0,n01.We assumeHm0=Hm0+1=Hm0−1. The CE based on the IAM can be obtained by(13)H^m0=ym0,n0bm0,n0=Hm0+∑p,q≠0,0,p,q≠±1,0,q≠±1Cp,qam0+p,n0+qHm0+pbm0,n0+ηm0,n01bm0,n0,where bm0,n0=am0,n0+C1,0am0+1,n0+C−1,0am0−1,n0. bm0,n0 can be treated as the pseudo pilot. Equation (13) implies that a larger pseudo pilot power leads to a better CE performance. The power of pseudo pilot bm0,n0 for the IAM is shown in the following equation:(14)E1bm0,n02=am0,n02+C−1,0am0−1,n0+C1,0am0+1,n02=σa21+4Ag+4Ag2,where σa2 denotes the variance of the transmitted FBMC/OQAM symbols am,n.Obviously, CE based on the IAM has two drawbacks: (1) the preamble duration is3τ0, which leads to the low spectrum efficiency, and (2) the second item in equation (13) cannot be ignored when both the signal-to-noise ratio and constellation mapping order are high.The fact is that the power of pseudo pilotbm0,n0 is mainly affected by the pilot symbols that surround am0,n0 for the IAM. Therefore, it can be increased by changing the structure. Additionally, the zero symbols in the IAM will be redundant if the intersymbol interference is alleviated by the preamble.In order to overcome two drawbacks of the IAM, a new preamble structure is proposed in this paper. It is shown in Figure2. For sufficient accuracy and concise description, only three-tap interference of neighborhood symbols is considered.Figure 2
Proposed structure of preamble.The pseudo pilot power of the proposed structure is(15)E2bm0,n02=am0,n02+∑p,q∈Ω3,3∗Cp,qam0+p,n0+q2>E1.From (15), we can conclude that the proposed preamble structure is able to improve CE performance for its larger pseudo pilot power. Obviously, it also increases the frequency spectrum efficiency.


## 2.2. RSINR

In order to take the influence of prototype filter on CE into account while designing the prototype filter, the RSINR is chosen to measure these influences. The expression of RSINR based on the CIR is derived in this section.First, we rewrite equation (10) as(16)ym0,n0=am0,n0Hm0+am0,n0i1Hm0+am0,n0i2Hm0+ηm0,n01,where(17)am0,n0i1=∑p,q∈Ω3,3∗,q=0Cp,qam0+p,n0+q,(18)am0,n0i2=∑p,q∈Ω3,3∗,q≠0Cp,qam0+p,n0+q,and then, the channel frequency impulse response H^m0 is estimated by(19)H^m0=ym0,n0am0,n0+am0,n0i1+am0,n0i2.Becauseam0+p,n0+q in equation (18) belongs to a random signal, its value cannot be determined during transmission. As a result, the term am0,n0i2 is unknown. Equation (19) will be invalid if am0,n0i2 is unknown. To solve this problem, the predecision method is adopted. The process of estimation based on the predecision method is shown as follows.First, the initial channel frequency responseH¯m0 can be obtained from (16):(20)H¯m0=ym0,n0am0,n0+am0,n0i1=Hm0+am0,n0i2Hm0+ηm0,n01am0,n0+am0,n0i1.Next,am0+p,n0+q in the term am0,n0i2 can be rebuilt based on zero-forcing equalization. Estimate am0+p,n0+q based on the predecision method, which is shown as follows:(21)a^m0+p,n0+q=Dym0+p,n0+qH¯m0,p,q∈Ω3,3∗,q≠0.Here,D⋅ denotes the predecision operator.Then,am0,n0i2 can be estimated by(22)a^m0,n0i2=∑p,q∈Ω3,3∗,q≠0Cp,qa^m0+p,n0+q.Finally, sinceam0,n0i2 has been obtained, the CE is obtained by substituting am0,n0i2 into equation (19). Equation (23) shows the final channel frequency response expression:(23)H^m0=ym0,n0am0,n0+am0,n0i1+a^m0,n0i2.By inverse Fourier transform, the CIR of the FBMC/OQAM system is obtained as follows:(24)h^t=12π∫0ΔH^me−j2πmν0tdt.We then discretize the CIRh^t into form h^l as follows:(25)h^t=∑i=0P−1h^iδτ−τi,h^l=h^i,l=τiTs,0,otherwise,where x denotes the smallest integer larger than or equal to x and Ts=T0/2M denotes the sampling period.The desired power of the system is closely related to the transmitted symbol. The transmitted symbol at the positionm0,n0 can be estimated as follows:(26)a^m0,n0=ℜym0,n0H^m0.The QAM symbols are reconstructed by combining(27)c^m0,n0=a^m0,2n0+ja^m0,2n0+1.We then define(28)Hm0p,q=∑l=0Lh−1h^le−jπ2m0+pl/MAg−l−qM2,−p,where Lh is the number of channel taps at the sampling rate and Agk,l=AgkTs,l/MTs.The desired power of the system is(29)Pdm0=σc2ℜHm0p,qH^m02,where σc2 denotes the variance of the complex symbol c. According to the definition of the variance, σc2 is the real number.Similarly, the interference power and the noise power are expressed as [28](30)PISI+ICIm0=2σa2∑p,q≠0,0ℜejπ/2p+q+pqHm0p,qH^m02,(31)Pnoisem0=varηm0,n01=ση2H^m02,where ση2 denotes the variance of the noise η.Thus, the expression of RSINR is obtained by equations (29)–(31):(32)RSINR=PICI+ISI+PnoisePdm0=2σa2∑p,q≠0,0ℜejπ/2p+q+pqHm0p,q/H^m02+ση2/H^m02σc2ℜHm0p,q/H^m02.


## 3. Prototype Filter Design

### 3.1. Problem Formulation

The prototype filter in the FBMC/OQAM system is assumed to be real-valued, symmetric, and of unit energy. According to the previous discussion, when there is intrinsic interference and noise in the system, the prototype filter will influence the CE. Therefore, the RSINR will be chosen as an important constraint in problem formulation. At the same time, the prototype filter design should minimize the stop-band energy. Thus, the prototype filter design problem can be formulated as(33)P1mingk∫2π/MπGejω2dωs.tgk=gLg−1−k,k=0,1,…,Lg−1∑k=0Lg−1g2k=1RSINR≤TH,where Gejω=∑k=0Lg−1gke−jωk is the magnitude response of the prototype filter, TH represents a constant threshold, and Lg denotes the length of the prototype filter. According to [29], gk=Tsgk−Lg−1/2Ts.The prototype filter used to be designed without considering the interaction between prototype filter and CE. Therefore, utilizing RSINR to measure the influence of prototype filter on CE and regarding RSINR as the constraint of prototype filter design problem are also one of the contributions of this paper.There are many variables inP1. The objective function and constraint functions all have quadratic forms. Therefore, it is difficult to solve the optimization problem P1 directly. As everyone knows, orthogonal transformation does not change the nature of the original function. However, it can be employed to change the form of (33).First of all, the first constraint function in (33) implies that only half of the filter coefficients are independent. In order to reduce the number of variables, we substitute gk with a new variable x:(34)x=x1x2...xLT=g0g1...gLg−12T.The objective function can be rewritten with a vector form asf0x=xTCx, where C is a real symmetric positive-definite matrix. λ1,λ2,…,λL, respectively, denote positive eigenvalues in ascending order, and v1,v2,…,vL, respectively, denote corresponding unit orthogonal eigenvectors.Then, the orthogonal transformationx=vz is applied to the variable x, where z=z1z2...zLT denotes the new orthogonal variable.Finally, the optimization problem after the variable substitution and orthogonal transformation is expressed as(35)P2min∑j=1Lλjzj2s.tc1z=−∑k=0Lg−1g2k−1≤0c2z=RSINR≤TH.


### 3.2. Problem Solution

In this part, an improved GA with History Network and pruning operator is proposed to solve the optimization problem expressed in (35).All the potential global answers of GA exist in a search space; Gorges in GA will expand this search space. Further, the inappropriate space may be searched many times during iterations. These increase the convergence time of GA.In [16], the knowledge of search space learned by the previous generation is passed on to the next generation via an History Network to narrow the search space, and the Baldwin effect is applied to avoid searching the same space in different iterations. Figure 3 shows the flowchart of the proposed GA in [16].Figure 3
Flowchart of the proposed GA in [16].However, the influence of random operators ignored in [16] will slow down the convergence speed of GA. The random assignment operators may result in invalid iterations, early termination, and jumping out of the loop with local optimum. Inspired by [17], the pruning operator is applied in this paper to control these random operators. Pruning operator is used to take away the inappropriate amounts resulting in increment convergence time from population and reach the global optimal answer more quickly. Flowchart of improved GA in this study is shown in Figure 4.Figure 4
Flowchart of the improved GA in this paper.It can be seen from Figure4 that the improvement of the improved GA proposed in this paper mainly exists in four parts: initialization, DRS, History Network, and mutation.The pruning operator of the initialization part directly deletes the invalid subset of solution to narrow the domain at the beginning.The time order of the pruned initialization is shown as(36)initialization=ONp∗Ng∗Nc,where Ng presents the number of genomes in each chromosome, Nc presents the number of chromosomes of each generation, and Np is determined by the pruning function. It is shown as(37)Npruning=η,η≤Ng∗Nc,Ng∗Nc,η>Ng∗Nc,where η is the reduced ratio of search space.In DRS part, the range of direction change is limited by pruning operator. As a result, those individuals are discarded who are so different to avoid being too far from the last reasonable direction.There are many exemplars stored in the History Network. It is required to limit the variation of the exemplars to a certain range. Pruning operator is applied to avoid either duplication with existing exemplars or drastic changes during the updating process.Mutation is critical for reaching optimal solution. In the process of mutation, the value of genome is replaced randomly to create new generation for the next iteration. However, the random substitution likely results in inappropriate value. It requires a lot of iterations to eliminate the impact of inappropriate values on the search. As a result, iterations in the whole algorithm are increased. Pruning operator is able to help appoint appropriate value as much as possible. In all circumstances, the better the appointment of value during mutation, the faster it is to reach the best answer.The time order of the pruned search loop is shown as(38)search loop=ONsη∗Nc∗Nc∗Np,where Ns presents the solution space.Since the redundant random operators have been pruned, the computational complexity of the whole algorithm is certainly reduced.


## 3.1. Problem Formulation

The prototype filter in the FBMC/OQAM system is assumed to be real-valued, symmetric, and of unit energy. According to the previous discussion, when there is intrinsic interference and noise in the system, the prototype filter will influence the CE. Therefore, the RSINR will be chosen as an important constraint in problem formulation. At the same time, the prototype filter design should minimize the stop-band energy. Thus, the prototype filter design problem can be formulated as(33)P1mingk∫2π/MπGejω2dωs.tgk=gLg−1−k,k=0,1,…,Lg−1∑k=0Lg−1g2k=1RSINR≤TH,where Gejω=∑k=0Lg−1gke−jωk is the magnitude response of the prototype filter, TH represents a constant threshold, and Lg denotes the length of the prototype filter. According to [29], gk=Tsgk−Lg−1/2Ts.The prototype filter used to be designed without considering the interaction between prototype filter and CE. Therefore, utilizing RSINR to measure the influence of prototype filter on CE and regarding RSINR as the constraint of prototype filter design problem are also one of the contributions of this paper.There are many variables inP1. The objective function and constraint functions all have quadratic forms. Therefore, it is difficult to solve the optimization problem P1 directly. As everyone knows, orthogonal transformation does not change the nature of the original function. However, it can be employed to change the form of (33).First of all, the first constraint function in (33) implies that only half of the filter coefficients are independent. In order to reduce the number of variables, we substitute gk with a new variable x:(34)x=x1x2...xLT=g0g1...gLg−12T.The objective function can be rewritten with a vector form asf0x=xTCx, where C is a real symmetric positive-definite matrix. λ1,λ2,…,λL, respectively, denote positive eigenvalues in ascending order, and v1,v2,…,vL, respectively, denote corresponding unit orthogonal eigenvectors.Then, the orthogonal transformationx=vz is applied to the variable x, where z=z1z2...zLT denotes the new orthogonal variable.Finally, the optimization problem after the variable substitution and orthogonal transformation is expressed as(35)P2min∑j=1Lλjzj2s.tc1z=−∑k=0Lg−1g2k−1≤0c2z=RSINR≤TH.


## 3.2. Problem Solution

In this part, an improved GA with History Network and pruning operator is proposed to solve the optimization problem expressed in (35).All the potential global answers of GA exist in a search space; Gorges in GA will expand this search space. Further, the inappropriate space may be searched many times during iterations. These increase the convergence time of GA.In [16], the knowledge of search space learned by the previous generation is passed on to the next generation via an History Network to narrow the search space, and the Baldwin effect is applied to avoid searching the same space in different iterations. Figure 3 shows the flowchart of the proposed GA in [16].Figure 3
Flowchart of the proposed GA in [16].However, the influence of random operators ignored in [16] will slow down the convergence speed of GA. The random assignment operators may result in invalid iterations, early termination, and jumping out of the loop with local optimum. Inspired by [17], the pruning operator is applied in this paper to control these random operators. Pruning operator is used to take away the inappropriate amounts resulting in increment convergence time from population and reach the global optimal answer more quickly. Flowchart of improved GA in this study is shown in Figure 4.Figure 4
Flowchart of the improved GA in this paper.It can be seen from Figure4 that the improvement of the improved GA proposed in this paper mainly exists in four parts: initialization, DRS, History Network, and mutation.The pruning operator of the initialization part directly deletes the invalid subset of solution to narrow the domain at the beginning.The time order of the pruned initialization is shown as(36)initialization=ONp∗Ng∗Nc,where Ng presents the number of genomes in each chromosome, Nc presents the number of chromosomes of each generation, and Np is determined by the pruning function. It is shown as(37)Npruning=η,η≤Ng∗Nc,Ng∗Nc,η>Ng∗Nc,where η is the reduced ratio of search space.In DRS part, the range of direction change is limited by pruning operator. As a result, those individuals are discarded who are so different to avoid being too far from the last reasonable direction.There are many exemplars stored in the History Network. It is required to limit the variation of the exemplars to a certain range. Pruning operator is applied to avoid either duplication with existing exemplars or drastic changes during the updating process.Mutation is critical for reaching optimal solution. In the process of mutation, the value of genome is replaced randomly to create new generation for the next iteration. However, the random substitution likely results in inappropriate value. It requires a lot of iterations to eliminate the impact of inappropriate values on the search. As a result, iterations in the whole algorithm are increased. Pruning operator is able to help appoint appropriate value as much as possible. In all circumstances, the better the appointment of value during mutation, the faster it is to reach the best answer.The time order of the pruned search loop is shown as(38)search loop=ONsη∗Nc∗Nc∗Np,where Ns presents the solution space.Since the redundant random operators have been pruned, the computational complexity of the whole algorithm is certainly reduced.


## 4. Simulation Result

### 4.1. Problem Formulation

The CE performance based on the proposed preamble is simulated in this simulation. The truncation of an IOTA filter with4T0 length [30] is chosen as the prototype filter. The fundamental parameters are listed in Table 1.Table 1
Fundamental parameters of CE.
Parameter
Value
Subcarrier spacing
ν
0
=
10.94
kHz
Filter length
L
g
=
4
M
+
1
Channel coding
Convolution coding
Channel mode
ITU-A channel [31]
Channel path delay (us)
[0, 1.6, 3.5, 5.0]
Frame length
20 OQAM symbolsFigure5 illustrates the CE performances of the proposed structure (A) and the IAM (B) for two kinds of constellation mapping. For 16-QAM modulation, when the signal-to-noise ratio (SNR) is low (SNR < 15 dB), the CE performances of A are slightly better than those of B. Along with the SNR increasing, the CE performance of A becomes better and better and the advantages become more and more obvious. When SNR > 15 dB, the IAM performance curve tends to flatten due to the performance platform. Similarly, for 4-QAM modulation, the IAM performance curve tends to flatten when SNR > 20 dB.Figure 5
Normalized mean square error (NMSE) performance of the proposed structure (A) and the IAM structure (B) for 4-QAM and 16-QAM in CE.The CE performances of two methods with different number of subcarriers for 4-QAM modulation and 16-QAM modulation are also, respectively, compared in Figures6 and 7. It is obvious that the curve of A with 512 subcarriers is lower than the one with 256 subcarriers in Figure 6, so is B. It implies that a larger number of subcarriers mean a better CE performance. These two curves of A are both lower than those of B for the same number of subcarriers. Curves in Figure 6 show that A is better than B about 0.4 dB for 256 subcarriers and 1 dB for 512 subcarriers at BER=10−3. Modulation of curve 16-QAM in Figure 7 also illustrates the same conclusion. Therefore, the proposed preamble structure has advantages with respect to CE.Figure 6
Bit error ratio (BER) performance of the proposed structure (A) and the IAM structure (B) with different number of subcarriers in CE for 4-QAM.Figure 7
NMSE performance of the proposed structure (A) and the IAM structure (B) with different number of subcarriers in CE for 16-QAM.In order to lower the impact of predecision errors, CE with different number of iterations is simulated in Figure8. Perfect bound represents the upper bound of performance. The curves with iterations show better performance than the curve without iterations, which indicates iteration can reduce the predecision errors and improve the CE performance. The difference between curves with different number of iterations is obvious when the number is less than four. But the curve with four iterations is almost overlapped with the curve with five iterations. The reason why curves are overlapped when the number of iterations are more than four is that the value of am0+p,n0+q is close to its true value after four iterations. Therefore, there is no need to carry out more than four iterations in order to save time and reduce the complexity.Figure 8
NMSE performance with different number of iterations.


### 4.2. Improved GA performance

The performance of the improved GA is analyzed in this section. According to [16], values of parameters in History Network are assigned as follows: the Good Threshold, the Bad Threshold, and the Update Threshold are set to 10%, 25%, and 10% of the mutation radius, respectively. The mutation radius is defined as the maximum Euclidean distance from the old search space position before mutation to the new position after mutation. The rest of the parameters of the improved GA are exhibited in Table 2.Table 2
Parameters of the improved GA.
Parameters
Value
Population
Double vector
Selection
Tournament
Crossover
Two point
Mutation
Creep mutation
Crossover probability
0.8
Reject probability
0.95
Calculation accuracy
1
×
10
−
9The average generations and the convergence time of different algorithms are, respectively, shown in Figure9 and Table 3. Figure 9 implies that the generations of other three algorithms are much more than the improved GA proposed in this paper. It illustrates that the pruning operation in the History Network deletes the invalid random value assignment and avoids redundant individuals. As can been seen from Table 3, the improved GA proposed in this paper reduces the convergence time varying from 8% to 22.1% when compared to the standard GA, the accelerated GA in [16], and the pruned GA in [17]. It reveals that the improved GA proposed in this paper can reduce the time to reach the best answer and accelerate the convergence speed. In conclusion, the combination of pruning operation and History Network is more effective than just utilizing any one of them.Figure 9
Average generations of different GAs.Table 3
Convergence time of different algorithms.
Methods
The standard GA
The accelerated GA in [16]
The pruned GA in [17]
The improved GA in this paper
Convergence time
163.275 s
139.682 s
143.233 s
127.354 s


### 4.3. Performance of the Designed Prototype Filter

In this section, the validity of the designed prototype filter is verified through a series of simulations.The average BER performances of FBMC/OQAM system with different number of subcarriers for three prototype filters are, respectively, shown in Figures10 and 11. The prototype filters designed by method A and method B are, respectively, described in [23, 24], and method C is the one in this paper. The simulation results in Figure 10 reveal that the average BER performance of the system with the prototype filter designed by method C is superior to the other two methods for M = 256. It can be seen from Figure 10 that method A and method B show similar BER performance when SNR is larger than 14 dB. Meanwhile, method B and method C show similar performance when SNR is smaller than 8 dB. Although there are some intersections between these three lines in Figure 10, the outperformance of method C is obvious when SNR is larger than 10 dB. Similar conclusion can be drawn from Figure 11.Figure 10
Average BER performance of FBMC/OQAM systems with prototype filters designed by different methods forM = 256.Figure 11
Average BER performance of FBMC/OQAM systems with prototype filters designed by different methods forM = 512.Since the proposed filter is designed with a constraint on the RSINR, not only the influence of noise is considered to guarantee the effectiveness of the method but also the desired power is increased as the interference power decreases. However, the influence of noise which should not be ignored in practice was not considered in [23]. The existence of the noise destroys the efficiency of the method proposed in [23], especially when SNR of the system is at the low level. The desired symbol power was also not considered in [23], which may have obtained the solution for the optimization problem not being the minimum one.In [24], the objective of the prototype filter design is to maximize the signal-to-weighted interference radio. Large stop-band energy at high SNR level destroys the orthogonal condition, resulting in a much poorer BER performance than the method proposed in this paper.Figure12 shows the magnitude responses of IOTA, the filter designed in [23] and in [24], and that designed in this study with M = 256. The red curve in Figure 12 shows that the first side lobe of the prototype filter designed in this study is about −40 dB. It is better than the filter designed in [23] and that designed in [24] of about 8 dB and 9 dB. It is also superior to IOTA of about 1.5 dB. A lower side lobe means a better time-frequency localization property. In order to further reveal the outperformance of the prototype filter proposed in this study, Table 4 represents the stop-band energy of different filters. It shows that the stop-band energies of the IOTA filter, the filter designed in [23], and that designed in [24] are −32 dB, −43 dB, and −41 dB, respectively. The proposed prototype filter achieves the lowest stop-band energy (−51 dB). It means the proposed prototype filter outperforms three other filters in filtering clutter. Because the proposed prototype filter is designed with constraints on RSINR and the noise influence has been taken into account, the proposed prototype filter in this study shows better performance in filtering clutter.Figure 12
Magnitude response of IOTA and the filter designed in [23] and in [24] and in this study.Table 4
Stop-band energy of different filters.
Methods
IOTA
Prototype filter in [23]
Prototype filter in [24]
Proposed filter
Stop-band energy
−32 dB
−43 dB
−41 dB
−51 dB


## 4.1. Problem Formulation

The CE performance based on the proposed preamble is simulated in this simulation. The truncation of an IOTA filter with4T0 length [30] is chosen as the prototype filter. The fundamental parameters are listed in Table 1.Table 1
Fundamental parameters of CE.
Parameter
Value
Subcarrier spacing
ν
0
=
10.94
kHz
Filter length
L
g
=
4
M
+
1
Channel coding
Convolution coding
Channel mode
ITU-A channel [31]
Channel path delay (us)
[0, 1.6, 3.5, 5.0]
Frame length
20 OQAM symbolsFigure5 illustrates the CE performances of the proposed structure (A) and the IAM (B) for two kinds of constellation mapping. For 16-QAM modulation, when the signal-to-noise ratio (SNR) is low (SNR < 15 dB), the CE performances of A are slightly better than those of B. Along with the SNR increasing, the CE performance of A becomes better and better and the advantages become more and more obvious. When SNR > 15 dB, the IAM performance curve tends to flatten due to the performance platform. Similarly, for 4-QAM modulation, the IAM performance curve tends to flatten when SNR > 20 dB.Figure 5
Normalized mean square error (NMSE) performance of the proposed structure (A) and the IAM structure (B) for 4-QAM and 16-QAM in CE.The CE performances of two methods with different number of subcarriers for 4-QAM modulation and 16-QAM modulation are also, respectively, compared in Figures6 and 7. It is obvious that the curve of A with 512 subcarriers is lower than the one with 256 subcarriers in Figure 6, so is B. It implies that a larger number of subcarriers mean a better CE performance. These two curves of A are both lower than those of B for the same number of subcarriers. Curves in Figure 6 show that A is better than B about 0.4 dB for 256 subcarriers and 1 dB for 512 subcarriers at BER=10−3. Modulation of curve 16-QAM in Figure 7 also illustrates the same conclusion. Therefore, the proposed preamble structure has advantages with respect to CE.Figure 6
Bit error ratio (BER) performance of the proposed structure (A) and the IAM structure (B) with different number of subcarriers in CE for 4-QAM.Figure 7
NMSE performance of the proposed structure (A) and the IAM structure (B) with different number of subcarriers in CE for 16-QAM.In order to lower the impact of predecision errors, CE with different number of iterations is simulated in Figure8. Perfect bound represents the upper bound of performance. The curves with iterations show better performance than the curve without iterations, which indicates iteration can reduce the predecision errors and improve the CE performance. The difference between curves with different number of iterations is obvious when the number is less than four. But the curve with four iterations is almost overlapped with the curve with five iterations. The reason why curves are overlapped when the number of iterations are more than four is that the value of am0+p,n0+q is close to its true value after four iterations. Therefore, there is no need to carry out more than four iterations in order to save time and reduce the complexity.Figure 8
NMSE performance with different number of iterations.


## 4.2. Improved GA performance

The performance of the improved GA is analyzed in this section. According to [16], values of parameters in History Network are assigned as follows: the Good Threshold, the Bad Threshold, and the Update Threshold are set to 10%, 25%, and 10% of the mutation radius, respectively. The mutation radius is defined as the maximum Euclidean distance from the old search space position before mutation to the new position after mutation. The rest of the parameters of the improved GA are exhibited in Table 2.Table 2
Parameters of the improved GA.
Parameters
Value
Population
Double vector
Selection
Tournament
Crossover
Two point
Mutation
Creep mutation
Crossover probability
0.8
Reject probability
0.95
Calculation accuracy
1
×
10
−
9The average generations and the convergence time of different algorithms are, respectively, shown in Figure9 and Table 3. Figure 9 implies that the generations of other three algorithms are much more than the improved GA proposed in this paper. It illustrates that the pruning operation in the History Network deletes the invalid random value assignment and avoids redundant individuals. As can been seen from Table 3, the improved GA proposed in this paper reduces the convergence time varying from 8% to 22.1% when compared to the standard GA, the accelerated GA in [16], and the pruned GA in [17]. It reveals that the improved GA proposed in this paper can reduce the time to reach the best answer and accelerate the convergence speed. In conclusion, the combination of pruning operation and History Network is more effective than just utilizing any one of them.Figure 9
Average generations of different GAs.Table 3
Convergence time of different algorithms.
Methods
The standard GA
The accelerated GA in [16]
The pruned GA in [17]
The improved GA in this paper
Convergence time
163.275 s
139.682 s
143.233 s
127.354 s


## 4.3. Performance of the Designed Prototype Filter

In this section, the validity of the designed prototype filter is verified through a series of simulations.The average BER performances of FBMC/OQAM system with different number of subcarriers for three prototype filters are, respectively, shown in Figures10 and 11. The prototype filters designed by method A and method B are, respectively, described in [23, 24], and method C is the one in this paper. The simulation results in Figure 10 reveal that the average BER performance of the system with the prototype filter designed by method C is superior to the other two methods for M = 256. It can be seen from Figure 10 that method A and method B show similar BER performance when SNR is larger than 14 dB. Meanwhile, method B and method C show similar performance when SNR is smaller than 8 dB. Although there are some intersections between these three lines in Figure 10, the outperformance of method C is obvious when SNR is larger than 10 dB. Similar conclusion can be drawn from Figure 11.Figure 10
Average BER performance of FBMC/OQAM systems with prototype filters designed by different methods forM = 256.Figure 11
Average BER performance of FBMC/OQAM systems with prototype filters designed by different methods forM = 512.Since the proposed filter is designed with a constraint on the RSINR, not only the influence of noise is considered to guarantee the effectiveness of the method but also the desired power is increased as the interference power decreases. However, the influence of noise which should not be ignored in practice was not considered in [23]. The existence of the noise destroys the efficiency of the method proposed in [23], especially when SNR of the system is at the low level. The desired symbol power was also not considered in [23], which may have obtained the solution for the optimization problem not being the minimum one.In [24], the objective of the prototype filter design is to maximize the signal-to-weighted interference radio. Large stop-band energy at high SNR level destroys the orthogonal condition, resulting in a much poorer BER performance than the method proposed in this paper.Figure12 shows the magnitude responses of IOTA, the filter designed in [23] and in [24], and that designed in this study with M = 256. The red curve in Figure 12 shows that the first side lobe of the prototype filter designed in this study is about −40 dB. It is better than the filter designed in [23] and that designed in [24] of about 8 dB and 9 dB. It is also superior to IOTA of about 1.5 dB. A lower side lobe means a better time-frequency localization property. In order to further reveal the outperformance of the prototype filter proposed in this study, Table 4 represents the stop-band energy of different filters. It shows that the stop-band energies of the IOTA filter, the filter designed in [23], and that designed in [24] are −32 dB, −43 dB, and −41 dB, respectively. The proposed prototype filter achieves the lowest stop-band energy (−51 dB). It means the proposed prototype filter outperforms three other filters in filtering clutter. Because the proposed prototype filter is designed with constraints on RSINR and the noise influence has been taken into account, the proposed prototype filter in this study shows better performance in filtering clutter.Figure 12
Magnitude response of IOTA and the filter designed in [23] and in [24] and in this study.Table 4
Stop-band energy of different filters.
Methods
IOTA
Prototype filter in [23]
Prototype filter in [24]
Proposed filter
Stop-band energy
−32 dB
−43 dB
−41 dB
−51 dB


## 5. Conclusion

In this study, a new prototype filter was designed based on the CE for the FBMC/OQAM system in frequency-selective channels. The proposed method is able to minimize the stop-band energy and increase the CE performance. RSINR, which is utilized to measure the influence of the prototype filter on the CE, is an important constraint for prototype filter design. A new preamble structure was proposed to save the frequency spectrum before the RSINR calculation. Then, an optimization problem of prototype filter design was formulated to minimize the stop-band energy with constraint on the RSINR. Finally, an improved GA is employed to solve this optimization problem by utilizing a History Network and a pruning operation in order to obtain low convergence time and the global optimal solution. Simulation results verified that the preamble structure proposed in this paper increased the CE accuracy and spectral efficiency, the improved GA accelerated the convergence speed, and the designed prototype filter outperforms other filters.There are still many works that are worth doing in the future. On the one hand, if the preamble symbol is encoded, the spectrum efficiency can be further increased; on the other hand, the History Network is the key part of the proposed GA; how to set up the History Network and choose its exemplar more efficiently relate to the performance of the whole algorithm.



---

*Source: 3730759-2019-11-27.xml*