# Performance Evaluation of Machine Learning-Based Channel Equalization Techniques: New Trends and Challenges

Authors: Shahzad Hassan; Noshaba Tariq; Rizwan Ali Naqvi; Ateeq Ur Rehman; Mohammed K. A. Kaabar
Journal: Journal of Sensors
 (2022)

---

## Abstract

Wireless communication systems have evolved and offered more smart and advanced systems like ad hoc and sensor-based infrastructure fewer networks. These networks are evaluated with two fundamental parameters including data rate and spectral efficiency. To achieve a high data rate and robust wireless communication, the most significant task is channel equalization at the receiver side. The transmitted data symbols when passing through the wireless channel suffer from various types of impairments, such as fading, Doppler shifts, and Intersymbol Interference (ISI), and degraded the overall network performance. To mitigate channel-related impairments, many channel equalization algorithms have been proposed for communication systems. The channel equalization problem can also be solved as a classification problem by using Machine Learning (ML) methods. In this paper, channel equalization is performed by using ML techniques in terms of Bit Error Rate (BER) analysis and comparison. Radial Basis Functions (RBFs), Multilayer Perceptron (MLP), Support Vector Machines (SVM), Functional Link Artificial Neural Network (FLANN), Long-Short Term Memory (LSTM), and Polynomial-based Neural Networks (NNs) are adopted for channel equalization.


---

## Body

## 1. Introduction

In wireless communication systems [1–3], the performance may be severely degraded because of wireless channel issues. The transmitted signal passes through the communication channel and has faced various impairment issues such as Intersymbol Interference (ISI), Doppler shift, and fading effects. All these effects tend to degrade and limit the data throughput during data communication [4]. To achieve higher data rates, it is mandatory to mitigate the effects of channel-induced impairments. This requires an adaptive filter for equalization to nullify the effects of the wireless channel and recover the originally transmitted data. Recently, the use of Machine Learning (ML) [5, 6] techniques especially Artificial Neural Network- (ANN-) based methods has gained interest due to its remarkable success in the fields of Computer Vision (CV), speech recognition, and Natural Language Processing (NLP). These techniques although invented in the mid-20th century were not very popular due to the lack of required computational power. The availability of high-speed computational resources and the success of ML in various other fields have provoked its applications for the development of robust communication systems [7]. Many researchers have proposed the use of ML for designing communication systems and have demonstrated improved results in terms of Bit Error Rate (BER). However, still, there are some concerns and questions which require answers such as the following:
(i)
What will be the maximum performance gain in terms of BER by using NN and its variants such as Multilayer Perceptron (MLP), Radial Basis Functions (RBFs), Functional Link Artificial Neural Network (FLANN), Support Vector Machines (SVM), and Long-Short Term Memory (LSTM)(ii)
Is it possible to train the NN to estimate a wireless channel in real time as required by the modern-day channel equalizers to mitigate the channel in real time? Typically, an equalizer is required to train its taps in less than a few microseconds. What possible methods can be used to achieve this taskDuring data transmission, networks have experienced various types of impairments such as path loss which results in attenuation of the signal, AWGN, and multipath effects caused by the reflections of the electromagnetic waves from various obstacles. The input digital data is fed into the source encoder which effectively transforms the bitstream into the compressed form by using Huffman encoding. The input can be an audio source, text, binary, or any other sensor input, which may require A/D conversion before feeding to the source encoder block [8].The digital data at this stage can also be secured using encryption algorithms.The resulting data sequence at the output of the source encoder is passed to the channel encoder which adds redundancy in a controlled manner, to help the receiver to detect and correct the channel-induced errors. This step should make the data robust against harsh channel conditions. In the next step, the output of a channel encoder is given to a modulator that applies digital modulation methods such as BPSK, QPSK, or some variants of FSK. The output of the modulator is fed to the frequency upconverter which translates the baseband signals to passband frequency, and finally, the signal is amplified to the appropriate levels and then transmitted through the antenna. The motivation of this research work is as follows.(i)
To identify the performance metrics for the existing channel estimation and equalization techniques(ii)
To identify an improved channel equalization technique for the selected wireless channelTo critically assess the performance of various channel equalization techniques by performing simulations, the mathematical formulation is presented in [8] for the communication system where they considered st to be the transmitted signal. It is represented mathematically in Equation (1). Figure 1 shows the transmitter and receiver block:
(1)st=Rextejωct,where xt is the baseband signal and “ωc=2πfc” is the center frequency of the passband signal. The received signal is given as in
(2)rt=∑m=0N−1γmtst−τm+wt,where γmt represents the complex amplitude of the channel, τm is the delay of the mth multipath, and N represents the total number of multipaths. wt represents the AWGN. The resulting received signal can be written as in
(3)rt=∑m=0N−1γmtejωcτmxt−τm+wt,(4)rt=∫−∞∞hτ,txt−τdτ+wt,where hτ,t=∑m=0N−1γmtejωcτmδt−τ is the impulse response of the time-varying channel. It is the main goal of wireless communication systems to estimate hτ,t which is the channel impulse response for the desired level of performance.Figure 1
Transmitter and receiver block [8].


### 1.1. Performance Issues in Wireless Communications

One of the primary goals while designing a communication system is to achieve the performance as closer to Shannon’s capacity definition [9] as given in
(5)C=B1+γ,where C is the capacity of the wireless channel, B represents the bandwidth, and “γ” represents the Signal to Noise Ratio (SNR). This theorem gives the fundamental bound on the achievable capacity of the wireless channel. All communication systems tend to achieve Shannon’s capacity. As of today, this goal has not been fully achieved due to many reasons. Amongst the most notable reasons are
(i)
the wireless channel(ii)
Signal to Noise Ratio (SNR)(iii)
link budgetAssuming the availability of the required bandwidth, these three objectives must be served to achieve the desired performance in wireless communication. SNR and link budget can be improved using high gain antennas, more transmit power, and better antennas; however, the effects caused by the channel require more sophisticated handling. Its effects must be mitigated using the channel equalization method.


### 1.2. Channel Estimation Techniques

Channel estimation techniques are broadly categorized into three main types, the Pilot-Aided Channel Estimation (PACE) techniques, Blind and Semiblind Channel Estimation (BSB) techniques, and Decision-Directed Channel Estimation (DDCE) techniques [10]. In [10], an estimation technique was presented for the transmitter sending a known sequence of data symbols to the receiver called pilot symbols. The receiver estimates the channel with the help of received pilots using mathematical techniques.In [11], the receiver has no information about the input signal of the channel. This technique uses the data symbols for channel estimation by employing the precoding of the symbols at the transmitter. The receiver knows the parameters of the precoding used at the transmitter and then uses correlation-based methods to estimate the channel information [12–14]. In [15], the authors used the pilot symbols and the demodulated symbols for the channel estimation. In the absence of bit errors, the symbols can be used for estimation of the channel impairments and start acting as the pilot symbols. This technique proves to be more efficient as compared to the pilot symbol based on channel estimation techniques because it reduces the bandwidth by saving the number of pilots required in pilot-based channel estimation techniques.


### 1.3. Channel Equalization

Channel equalization and channel estimation are interdependent. The inverse of the channel estimate can be used for channel equalization. The performance of the equalizer is proportional to the accuracy of the channel estimation.The equalization mechanism can be divided into two modes including a training mode and a decision-directed mode. In the first mode, the equalizer is trained by sending a training sequence. The training sequence is known as a priori to the receiver. Equalizer weights are learned using the training sequence. In the second mode, the equalizer is operated on the channel to estimate the channel. Various types of equalizers are used in the digital communication receiver. Figure2 depicts the classification of the equalizers [16].Figure 2
The classification of equalization techniques [16].Equalization is generally divided into two categories including linear equalizers and nonlinear equalizers. The linear equalizers employ only a feedforward path and do not use the output of the equalizer in the equalization process. On the other hand, the nonlinear equalizers use the output of the equalizer in the determination of the future samples. Both the linear and nonlinear equalizers employ adaptive algorithms such as LMS, NLMS, RLS, and Kalman filtering for the adaptation of the equalizer weights. Amongst the nonlinear equalizers, it is the Maximum Likelihood Sequence Estimator (MLSE). This type of equalizer does not use the filter for equalizing the channel but instead uses the Viterbi algorithm to decode the sequence and chooses the sequence with maximum probability as the output.


## 1.1. Performance Issues in Wireless Communications

One of the primary goals while designing a communication system is to achieve the performance as closer to Shannon’s capacity definition [9] as given in
(5)C=B1+γ,where C is the capacity of the wireless channel, B represents the bandwidth, and “γ” represents the Signal to Noise Ratio (SNR). This theorem gives the fundamental bound on the achievable capacity of the wireless channel. All communication systems tend to achieve Shannon’s capacity. As of today, this goal has not been fully achieved due to many reasons. Amongst the most notable reasons are
(i)
the wireless channel(ii)
Signal to Noise Ratio (SNR)(iii)
link budgetAssuming the availability of the required bandwidth, these three objectives must be served to achieve the desired performance in wireless communication. SNR and link budget can be improved using high gain antennas, more transmit power, and better antennas; however, the effects caused by the channel require more sophisticated handling. Its effects must be mitigated using the channel equalization method.


## 1.2. Channel Estimation Techniques

Channel estimation techniques are broadly categorized into three main types, the Pilot-Aided Channel Estimation (PACE) techniques, Blind and Semiblind Channel Estimation (BSB) techniques, and Decision-Directed Channel Estimation (DDCE) techniques [10]. In [10], an estimation technique was presented for the transmitter sending a known sequence of data symbols to the receiver called pilot symbols. The receiver estimates the channel with the help of received pilots using mathematical techniques.In [11], the receiver has no information about the input signal of the channel. This technique uses the data symbols for channel estimation by employing the precoding of the symbols at the transmitter. The receiver knows the parameters of the precoding used at the transmitter and then uses correlation-based methods to estimate the channel information [12–14]. In [15], the authors used the pilot symbols and the demodulated symbols for the channel estimation. In the absence of bit errors, the symbols can be used for estimation of the channel impairments and start acting as the pilot symbols. This technique proves to be more efficient as compared to the pilot symbol based on channel estimation techniques because it reduces the bandwidth by saving the number of pilots required in pilot-based channel estimation techniques.


## 1.3. Channel Equalization

Channel equalization and channel estimation are interdependent. The inverse of the channel estimate can be used for channel equalization. The performance of the equalizer is proportional to the accuracy of the channel estimation.The equalization mechanism can be divided into two modes including a training mode and a decision-directed mode. In the first mode, the equalizer is trained by sending a training sequence. The training sequence is known as a priori to the receiver. Equalizer weights are learned using the training sequence. In the second mode, the equalizer is operated on the channel to estimate the channel. Various types of equalizers are used in the digital communication receiver. Figure2 depicts the classification of the equalizers [16].Figure 2
The classification of equalization techniques [16].Equalization is generally divided into two categories including linear equalizers and nonlinear equalizers. The linear equalizers employ only a feedforward path and do not use the output of the equalizer in the equalization process. On the other hand, the nonlinear equalizers use the output of the equalizer in the determination of the future samples. Both the linear and nonlinear equalizers employ adaptive algorithms such as LMS, NLMS, RLS, and Kalman filtering for the adaptation of the equalizer weights. Amongst the nonlinear equalizers, it is the Maximum Likelihood Sequence Estimator (MLSE). This type of equalizer does not use the filter for equalizing the channel but instead uses the Viterbi algorithm to decode the sequence and chooses the sequence with maximum probability as the output.


## 2. Machine Learning-Based Channel Equalization Technique Results

ML is a subfield of computer science that focuses on the development of algorithms to learn and solve complex problems. Unlike the traditional approach, it does not use predefined models or a set of equations to solve the given problem; instead, it learns to solve the problem. It consists of the human brain-like neurons termed “perceptrons.” A perceptron is a simple mathematical model (function) that maps the set of inputs to the set of outputs and performs three basic operations: multiplication, summation, and activation. Each input value is multiplied by its corresponding weight.The previously weighted inputs are then summed up and passed through the activation function. The activation function determines the output of the neuron concerning its input. The commonly used activation functions are threshold, linear, sigmoid, and “ReLU.” Mathematically, a perceptron can be defined in Equation (6). By subtitling the values, Equation (6) becomes (8):
(6)yx=φwTx+b.Here,w is a weight vector, b is a bias, and wTx is a dot product of w and x as represented in Equation (7):
(7)z=∑i=1Nwixi,(8)yx=φ∑i=1Nwixi+b.φ· is the activation function. A sigmoid and “ReLU” functions are defined in
(9)φx=11+e−x,(10)φx=max0,x.


## 3. Related Work

NNs are capable of processing nonlinear data and can produce complex decision regions. A new framework based on exploiting feature selection and neural network techniques has been proposed for identifing focal and nonfocal Electroencephalogram signals in TQWT domain [17]. Therefore, NNs can be employed for equalization purposes to overcome the difficulties associated with channel nonlinearities [18–20]. The performance of NN-based equalizers has been reported as superior to other conventional adaptive equalizers. In the recent past, the use of NNs has gained popularity in the design of software-defined radios where DNN, CNN, and RNN have been applied for classical radio operations [21–24]. In [25], the deep NNs have been used for the channel estimation of doubly selective channels which experience variations both in time and frequency. The deep learning-based algorithm is trained in three steps including the pretraining step, training stage, and testing stage. During the first two steps, the model is developed offline using training data. During the testing stage, the channel is estimated and equalized. The results show improved BER performance as compared to Linear Minimum Mean Square Error (MiMeSqEr). In [26], the ML and NN have been used in the Frequency Division Duplexing (FDD) system which is a double selective channel, and the results showed improvements in terms of MiMeSqEr in the prediction of the channel.In [27], the NN and DL methods have been used to predict the behavior of the Rayleigh channel, and it has been reported through simulations that the MSE performance compared with the traditional algorithms has improved. In [22, 28], DL has been thoroughly investigated and provided a review of the various ML-based techniques for wireless communication. It has been shown that traditional theories do not meet the higher data rate requirements of communication and limit the efficiency due to complex undefined channel requirements, fast processing, and limited block structure. On the other hand, AI-based communication systems face some challenges that need to be addressed. These challenges include the availability of a large amount of data and how easily it can be integrated into classical infrastructure [29]. Similarly, ML has been applied to the physical layer for modulation recognition and classification [26, 30–33].An MLP is a feedforward NN that consists of an input layer, a hidden layer, and an output layer. It has nonlinear decision-making capabilities. The training of MLP is done through the backpropagation algorithm [34]. The MLP is the first neural network used for channel equalization [19, 20, 35–38]. Gibson et al. [20] introduced an MLP-based nonlinear equalizer structure and demonstrated its superior performance over the linear equalizer (LMS). The major drawback of the MLP network is its slow convergence [39]. This is due to the backpropagation algorithm which operates based on first-order information. A genetic algorithm [40] can be used to solve this problem. The convergence can be improved by using the second-order data like the Hessian matrix, which is defined as the second-order partial derivatives of the error performance. In [41], the authors proposed an MLP-based DF equalizer with a lattice filter to overcome the convergence problem to improve the performance of MLP. However, this improvement increased the complexity of the MLP structure.The RBFNN is a three-layer network that comprises an input layer, a nonlinear hidden layer, and a linear output layer. The input layer contains the source symbols. In the hidden layer, the input space is transformed into a high-dimensional space by using nonlinear basis functions. The output layer linearly combines the output of the previous layers. RBFNN provides an appealing alternative to MLP for channel equalization.Many techniques have been developed to solve the equalization problem using RBF [42–44]. In 1991 [19], the authors used RBFNN for equalization. Similarly, an RBF-based equalizer has been reported which showed satisfactory performance [45, 46]. Another work has demonstrated the use of RBFNN for equalization and found an improvement in BER [47]. The performance of RBFNN is compared with the Maximum Likelihood Sequence Estimator (MLSE) over the Rayleigh fading channel [45, 48, 49]. Simulations have confirmed that RBFNN is a reasonable choice with low computational complexity. The authors in [50, 51] proposed a complex RBF (CRBF) network, and improved performance is observed. The drawback of RBFNN is that it is not suitable for hardware implementation. The network needs a large number of hidden nodes to achieve the desired performance.In the last few years, FLANN is very famous [52]. It is a single-layer NN that can form complex decision boundaries. FLANN provides less computational complexity and greater convergence speed than other traditional NNs. From the perspective of hardware implementation, FLANN has a simple design, less computational complexity, and higher computation performance [53, 54].The input dimension is expanded by using nonlinear functions which may lead to better nonlinear approximation. The expansion is done using three commonly used functions, i.e., trigonometric, Chebyshev expansion, and Legendre expansion. A traditional FLANN uses trigonometric functions, whereas the other two expansions are based on Legendre [55, 56] and Chebyshev [57] polynomials. Ch-FLANN is another computational efficient network. It has many applications in functional approximation [58], nonlinear dynamic system identification [59, 60], and nonlinear channel equalization [61]. In these networks, the expansion is performed using Chebyshev polynomials.RNN is a popular DL technique that was first introduced for processing sequential data [24] and gained a lot of attention in the recent past. They have been proven better than traditional signal processing methods in modeling and predicting nonlinear and time series [62] in a wide variety of applications ranging from speech processing and adaptive channel equalization [63–67].Unlike ANN, which does not have memory and cannot deal with temporal data, RNN has feedback loops which make them attractive for the equalization of nonlinear channels. This means data can be fed back to the same layers. It has been demonstrated through simulations that a reasonable size of RNN can model the inverse of the channel. RNNs are known to outperform FLANN, MLP, and RBF [68, 69]. In [70], the authors discussed that equalizers based on CNN and RNN reduce the channel’s fading effects but also increase the overall coding gain by more than 1.5 dB.RNN has one problem of exploding and vanishing gradient [71]. This problem arises when there is a long dependency in a sequence. To solve this problem, LSTM is proposed [72]. LSTM is slightly different from RNN. It has some special units in addition to standard units. These special units are called memory cells. These units can retain the information for a long period. This means that LSTM detected the patterns even in a long sequence. The sequence problems can be efficiently solved by LSTM and can also solve the channel equalization problem. In this case, future samples can be predicted by taking previous symbols into account. This means that variations in a channel can be easily tracked. We can specify the number of samples that LSTM can hold for the prediction of future sequences. If it is selected according to the delay spread of a channel, more accurate results may be observed.SVM lies in the category of supervised learning. Originally, it is developed for binary classification. Then, it has been extended to perform regression and multiclass classification problems [73–75]. It has the potential to generalize well in classification problems by maximizing the margin. The trained classifier contains support vectors on the margin boundary and summarizes the information required to separate the data. It uses the parametric learning algorithm, in which a model has fixed learnable parameters which are adapted during the training process. Once the model is trained, these parameters are then used exclusively for testing while discarding all the training examples.This makes the SVM more computationally efficient. On the other hand, NNs are nonparametric as the number of parameters increases with the number of layers. NN introduces nonlinearity by using nonlinear activation function whereas SVM uses kernel methods that implicitly transform the input space into higher dimensions. RBF kernel is the most commonly used kernel method. The SVM is suggested to address the number of digital communication issues due to its nonlinear processing capability. A DFE based on SVM is proposed, and it is observed that the performance of this equalizer is superior to MiMeSqEr DFE [76]. Similar work is done in [77].This section provides a comprehensive overview of the channel estimation and equalization techniques. Different neural network structures are discussed in the context of channel equalization. The MLP network implementation is simple, but training takes a lot of time. The main disadvantage of the FLANN structure is its computational and time complexity which gradually increases as the number of input nodes increases. The RBF-based neural network equalizer is an interesting alternative and is successfully used for blind equalization. LSTM equalizers are superior to NN feedforwards, including MLPs, RBFs, and FLANNs.


## 4. Performance Comparison of NN-Based Channel Equalization Schemes

Channel equalization methods of the respective systems are highlighted. A critical review of the methods is provided. All the methods are found to perform well in Rayleigh communication channels. However, there is a need to compare the schemes and highlight the best possible NN scheme for channel equalization. To the best of the knowledge of the authors, this work is not being carried out in the literature. In this work, the selected NNs are used for channel equalization and their performance is compared.


### 4.1. Implementation of NN-Based Equalizers

ML techniques are setting a path to replace the conventional communication techniques, and the combination of these two fields has led to a lot of successful work. NNs are capable of processing nonlinear data and can produce complex decision regions. Therefore, NNs can be employed for equalization to overcome the difficulties associated with channel nonlinearities [18–20]. The simulation setup is depicted in Figure 3.Figure 3
The simulation setup.A typical NN-based channel equalizer is depicted in Figure4. The transmitter first transmits the training symbols which are known to both the receiver and transmitter and then transmits the actual data. The equalizer uses the received training symbols to learn the equalizer weights. The optimization criterion is to minimize the MSE. Figure 4 shows the NN-based equalizer.Figure 4
NN-based equalizer.


### 4.2. Data Generation and QPSK Modulation

Data is randomly generated using the MATLAB rand function. It generates uniformly distributed data between 0 and 1. The data is QPSK-modulated and then passed through the channel filter. QPSK uses two signals I and Q, where I is an in-phase signal and Q is a quadrature signal. Both of these signals are at a 90° phase difference. This modulation is popular due to its simpler design and efficient hardware realization.The following steps are performed to produce a QPSK-modulated signal.(i)
The incoming digital data is converted into two streams. One stream contains the odd bits, and the other takes the even bits from the original stream(ii)
The streams are then pulse-shaped using root-raised cosine pulses. The duration of the pulse determines the data rate of the transmitter. In this phase, the incoming data is first upsampled by a factor “N” which corresponds to the symbol duration and then convolved with the RRC pulse. The resulting signal is termed a baseband signal(iii)
The resulting I and Q streams are then multiplied with I/Q carrier signals. In other words, these streams are amplitude-modulating using I/Q signals(iv)
Finally, the two modulated signals are summed up to form a QPSK-modulated signal. In QPSK, two bits are used in one symbolMathematically, QPSK modulation can be derived as follows.Letmk represent the message signal, where mk=xi+jyi is the complex representation of the ith message signal. This complex representation represents the group of bits together. One is represented as real, and the second one represents the imaginary bit. The message signal is QPSK-modulated as presented in Equation (11):
(11)sQPSKt=Remkej2πfot,(12)sQPSKt=Rexi+jyicoscos2πfot+jsin2πfot,(13)sQPSKt=xicoscos2πfot−yisin2πfot,where xi=0.7071A and yi=0.7071A are the amplitudes of the pulses. By substituting the values of xiandyi in Equation (11), Equation (14) becomes
(14)sQPSKt=0.7071Acoscos2πfot−0.7071Asin2πfot.Using trigonometric relations, the equation can be simplified as(15)sQPSKt=Acoscos2πfot+π4.From Equation (15), the four reference constellation points of QPSK modulation are given in
(16)mi=0.7071+j0.7071−0.7071+j0.7071−0.7071−j0.70710.7071−j0.7071.The received signal is demodulated as follows. The received QPSK signal is multiplied with the local oscillators which are at 90 degrees’ phase differences and are called I and Q. The resulting signals are low pass filtered using the RRC filters. This results in the recovery of the baseband pulses which are further downsampled byN, and the signal is received.The received signal can be expressed mathematically in(17)rt=st∗ht+ηt.Equation (17) shows that the received signal “rt” is the sum of convolution of “ht” with transmitted signal “st” and with noise “nt” added.


### 4.3. Wireless Channel Model

The wireless channel model describes the underlying communication medium. The performance of the communication system is dependent on the condition of the channel. Rayleigh and Rician fading channel models are widely used to simulate the channel in that realistic wireless environment. The Rayleigh fading channel [78–80] is the conceptual model assuming the fact that there are several objects in the atmosphere. Due to these objects, the transmitted signal may be dispersed and replicated. It is also presumed that there is no direct path between the transmitter and the receiver. On the other hand, the Rician channel [78, 79, 81] assumes that there is a direct path between the transmitter and the receiver. The received signal contains both the dispersed and scattered (or reflected) paths. In this case, the scattered (or reflected) paths appear to be weaker than the direct path.We have considered a complex-valued multipath channel mentioned in [51]. The coefficients of this channel are defined as in
(18)c=1−0.3434j0.5+0.2912j,(19)Hz=c1Xz+c2z−1Xz.


## 4.1. Implementation of NN-Based Equalizers

ML techniques are setting a path to replace the conventional communication techniques, and the combination of these two fields has led to a lot of successful work. NNs are capable of processing nonlinear data and can produce complex decision regions. Therefore, NNs can be employed for equalization to overcome the difficulties associated with channel nonlinearities [18–20]. The simulation setup is depicted in Figure 3.Figure 3
The simulation setup.A typical NN-based channel equalizer is depicted in Figure4. The transmitter first transmits the training symbols which are known to both the receiver and transmitter and then transmits the actual data. The equalizer uses the received training symbols to learn the equalizer weights. The optimization criterion is to minimize the MSE. Figure 4 shows the NN-based equalizer.Figure 4
NN-based equalizer.


## 4.2. Data Generation and QPSK Modulation

Data is randomly generated using the MATLAB rand function. It generates uniformly distributed data between 0 and 1. The data is QPSK-modulated and then passed through the channel filter. QPSK uses two signals I and Q, where I is an in-phase signal and Q is a quadrature signal. Both of these signals are at a 90° phase difference. This modulation is popular due to its simpler design and efficient hardware realization.The following steps are performed to produce a QPSK-modulated signal.(i)
The incoming digital data is converted into two streams. One stream contains the odd bits, and the other takes the even bits from the original stream(ii)
The streams are then pulse-shaped using root-raised cosine pulses. The duration of the pulse determines the data rate of the transmitter. In this phase, the incoming data is first upsampled by a factor “N” which corresponds to the symbol duration and then convolved with the RRC pulse. The resulting signal is termed a baseband signal(iii)
The resulting I and Q streams are then multiplied with I/Q carrier signals. In other words, these streams are amplitude-modulating using I/Q signals(iv)
Finally, the two modulated signals are summed up to form a QPSK-modulated signal. In QPSK, two bits are used in one symbolMathematically, QPSK modulation can be derived as follows.Letmk represent the message signal, where mk=xi+jyi is the complex representation of the ith message signal. This complex representation represents the group of bits together. One is represented as real, and the second one represents the imaginary bit. The message signal is QPSK-modulated as presented in Equation (11):
(11)sQPSKt=Remkej2πfot,(12)sQPSKt=Rexi+jyicoscos2πfot+jsin2πfot,(13)sQPSKt=xicoscos2πfot−yisin2πfot,where xi=0.7071A and yi=0.7071A are the amplitudes of the pulses. By substituting the values of xiandyi in Equation (11), Equation (14) becomes
(14)sQPSKt=0.7071Acoscos2πfot−0.7071Asin2πfot.Using trigonometric relations, the equation can be simplified as(15)sQPSKt=Acoscos2πfot+π4.From Equation (15), the four reference constellation points of QPSK modulation are given in
(16)mi=0.7071+j0.7071−0.7071+j0.7071−0.7071−j0.70710.7071−j0.7071.The received signal is demodulated as follows. The received QPSK signal is multiplied with the local oscillators which are at 90 degrees’ phase differences and are called I and Q. The resulting signals are low pass filtered using the RRC filters. This results in the recovery of the baseband pulses which are further downsampled byN, and the signal is received.The received signal can be expressed mathematically in(17)rt=st∗ht+ηt.Equation (17) shows that the received signal “rt” is the sum of convolution of “ht” with transmitted signal “st” and with noise “nt” added.


## 4.3. Wireless Channel Model

The wireless channel model describes the underlying communication medium. The performance of the communication system is dependent on the condition of the channel. Rayleigh and Rician fading channel models are widely used to simulate the channel in that realistic wireless environment. The Rayleigh fading channel [78–80] is the conceptual model assuming the fact that there are several objects in the atmosphere. Due to these objects, the transmitted signal may be dispersed and replicated. It is also presumed that there is no direct path between the transmitter and the receiver. On the other hand, the Rician channel [78, 79, 81] assumes that there is a direct path between the transmitter and the receiver. The received signal contains both the dispersed and scattered (or reflected) paths. In this case, the scattered (or reflected) paths appear to be weaker than the direct path.We have considered a complex-valued multipath channel mentioned in [51]. The coefficients of this channel are defined as in
(18)c=1−0.3434j0.5+0.2912j,(19)Hz=c1Xz+c2z−1Xz.


## 5. Experimental Setup and Modeling

### 5.1. Simulation Setup

#### 5.1.1. Simulation Parameters of NN

Different NN equalizers are plugged into the configuration of Figure4, and results are obtained. These configurations and the respective results are discussed in the sequel. The primary performance criteria are used in BER. Loss function analysis and the computational complexity are also calculated. The detailed results are compared and discussed in the later sections. The flowchart of the NN-based equalizer is depicted in Figure 5.Figure 5
The flowchart of NN-based equalizer.


#### 5.1.2. MLP-Based Equalizer

MLP is a simple three-layer network that maps the input to the output. MLP is designed using the “nntraintool” of MATLAB. It comprises an input layer, a hidden layer, and an output layer. The input layer contains two vectors. One vector is the real part of the input signal (X), and another is the complex part of the signal. The output layer generates four vectors “Y0” to Y3. The MLP is trained with these parameters as shown in Table 1.Table 1
The simulation parameters of MLP.
ParameterValueHidden nodes30Input size (X)1,000,000Training algorithmScaled conjugate gradient (SCG) [77]


#### 5.1.3. RBFNN

RBFNN is a three-layer network that comprises an input layer, a nonlinear hidden layer, and a linear output layer. Radial functions are used as an activation function. Radial functions are special functions. The output of these functions increases or decreases monotonically with distance from a center. TheK-means algorithm is used to find the centers. So first, centers of clusters are determined in an unsupervised manner, and then, classification is performed to recover the signal. We have implemented this work [51] and observed the improved BER. The simulation parameters of RBFNN are shown in Table 2.Table 2
Simulation parameters of the RBFNN.
ParameterValueData set size2000Noise variance0.01Centers16


#### 5.1.4. FLANN

FLANN is a single-layer neural network. The main concept of FLANN is to convert the input data to a higher dimension by using different functional expansions. Due to the absence of hidden layers, these networks have the following advantages: low computational complexity with very few adjustable parameters:(i)
Faster training time(ii)
Simple design that can be implemented on hardwareUsing the work in [54, 55], we have implemented the FLANN-based equalizer. The block diagram of the equalizer is shown in Figure 6.Figure 6
The block diagram of the implemented FLANN-based equalizer.The simulation parameters of FLANN, Le-FLANN, and Ch-FLANN are given in Table3.Table 3
Simulation parameters.
ParametersValueLength of input2000FLANN order30Input size4μ0.01No. of iterations10Channel noise variance0.01


#### 5.1.5. SVM-Based Channel Equalizer

SVM is a supervised algorithm used for classification problems. Channel estimation is a classification problem, so it can be used to deal with the nonlinear channel effects. In this work, we have implemented a basic SVM model equalization. Simulation parameters of the SVM are shown in Table4. The generalization error computed during simulations is 0.00001 which indicates the best performance.Table 4
Simulation parameters of SVM.
ParametersValueInput size1,000,000Kernel functionKNN


#### 5.1.6. LSTM Channel Equalizers

LSTM is a popular RNN-based DL technique. It is different from feedforward NN which does not have memory and cannot deal with temporal data. The simulation parameters are given in Table5. The training model of LSTM is illustrated in Figure 7.Table 5
Simulation parameters of LSTM model.
ParametersValueTraining SNR12 dBChannel noise variance0.01LSTM nodes16Learning rate0.01Figure 7
The LSTM model.


### 5.2. Simulation Analysis

All the simulations are executed and compared in this section. Figure8 depicts the BER comparison of all the simulated NNs. Generally, the trend verifies already established theories. As the SNR increases, the BER performance is getting better and better. The performance of FLANN is slightly worse as compared to the rest of the schemes due to its single-layer architecture. The performance of the traditional LMS algorithm is the worst. In [51], similar results are observed. All the other ML-based schemes are having the same BER performance.Figure 8
The comparison of NN techniques.In Figure9, the zoomed version of the BER graph is depicted. The LSTM is slightly bearing higher BER than SVM and RBF-based ML methods. The performance of FLANN when compared with the rest is almost 4 dB poorer than the rest. The performance of LSTM is about 0.7 dB poorer than the RBF and SVM and MLP. This may be reduced by further tuning or by increasing the size of the neural network. However, this will be at the cost of time and computational resources which can be very expensive in the communication systems.Figure 9
The zoomed comparison plot of BER.The loss function is an important parameter of the optimization and is therefore discussed. The lesser the value of the loss function, the better performance is considered. In Table6, the values of the loss function for all the algorithms used in this text are depicted. It shows that all the NNs are well trained.Table 6
Loss function values.
ANNLoss function valueRadial Basis Function0.001SVN (KNN-based)0.00001MLP0.003FLANN0.005LSTM0.003The minimum value of the loss function achieved is in the case of SVN where the value is 0.00001. The BER results depicted in Figure9 are very much in line with these results. The loss function values of RBF, FLANN, and LSTM can be further reduced by using more training data and by using better optimization algorithms.


### 5.3. Computational Complexity

Computational complexity analysis of the algorithms is presented. This presents the number of computation resources required to perform the respective ANN. Table7 presents the computational complexity of various algorithms. The number of additions, multiplications, and other computational resources such as exponentiation, powers, and trigonometric functions is enlisted [82]. This analysis is useful for the HW implementations and for estimating the computational requirements for embedded systems.Table 7
Computational complexity.
ANNComputational complexityNo. of multiplicationsNo. of additionsDivisionTanhExpRBFn0n1+2n1+n0+12n0n1+n0+n1+1n0+n1—noFLANN3n1n0+1+n02n1n0+1+n1—n1—MLP4∑i=0L−1nini+1+3∑i=0L−1ni−n0n1+2nL3∑i=0L−1nini+1+3nL−n0n1———SVM4Td3TdTdLSTM2N+2NN0N1n0 is the number of input nodes. ni is the ith node in the network. Td is the total number of Euclidean distances in the KNN-based SVM. N is the number of nodes in the convolution layer of the LSTM network. N0N1 is the number of neurons in the BiLSTM layers [82].The computational complexity analysis of the mentioned algorithms is verified by timing the MATLAB® implementations. The time of all the algorithms used in this work is measured using the MATLAB® built-in function called “timeit.” The number of iterations performed for each algorithm is 106. The machine used for the computation is DELL® 7920 running MATLAB® 2019b. The CPU is Intel® Xeon® Silver 4116 CPU running at 2.1 GHz. The time is enlisted in Table 8. The computational time computed endorses the computational complexity as given in Table 7. The minimum computational time achieved is for the SVM. SVM is running a KNN algorithm that is computationally efficient. Its BER results are also amongst the best. RBF and MLP bear good performance, but their computational time is more.Table 8
Time of various ANN algorithms.
ANNTotal time in secondsIterationsTime single iteration (seconds)SVM5.2311,000,0005.2315×10−6LSTM102.9571,000,0001.029576×10−4RBF12.27291,000,0001.2273×10−5FLANN36.34501,000,0003.6345×10−5MLP13.08591,000,0001.3086×10−5


## 5.1. Simulation Setup

### 5.1.1. Simulation Parameters of NN

Different NN equalizers are plugged into the configuration of Figure4, and results are obtained. These configurations and the respective results are discussed in the sequel. The primary performance criteria are used in BER. Loss function analysis and the computational complexity are also calculated. The detailed results are compared and discussed in the later sections. The flowchart of the NN-based equalizer is depicted in Figure 5.Figure 5
The flowchart of NN-based equalizer.


### 5.1.2. MLP-Based Equalizer

MLP is a simple three-layer network that maps the input to the output. MLP is designed using the “nntraintool” of MATLAB. It comprises an input layer, a hidden layer, and an output layer. The input layer contains two vectors. One vector is the real part of the input signal (X), and another is the complex part of the signal. The output layer generates four vectors “Y0” to Y3. The MLP is trained with these parameters as shown in Table 1.Table 1
The simulation parameters of MLP.
ParameterValueHidden nodes30Input size (X)1,000,000Training algorithmScaled conjugate gradient (SCG) [77]


### 5.1.3. RBFNN

RBFNN is a three-layer network that comprises an input layer, a nonlinear hidden layer, and a linear output layer. Radial functions are used as an activation function. Radial functions are special functions. The output of these functions increases or decreases monotonically with distance from a center. TheK-means algorithm is used to find the centers. So first, centers of clusters are determined in an unsupervised manner, and then, classification is performed to recover the signal. We have implemented this work [51] and observed the improved BER. The simulation parameters of RBFNN are shown in Table 2.Table 2
Simulation parameters of the RBFNN.
ParameterValueData set size2000Noise variance0.01Centers16


### 5.1.4. FLANN

FLANN is a single-layer neural network. The main concept of FLANN is to convert the input data to a higher dimension by using different functional expansions. Due to the absence of hidden layers, these networks have the following advantages: low computational complexity with very few adjustable parameters:(i)
Faster training time(ii)
Simple design that can be implemented on hardwareUsing the work in [54, 55], we have implemented the FLANN-based equalizer. The block diagram of the equalizer is shown in Figure 6.Figure 6
The block diagram of the implemented FLANN-based equalizer.The simulation parameters of FLANN, Le-FLANN, and Ch-FLANN are given in Table3.Table 3
Simulation parameters.
ParametersValueLength of input2000FLANN order30Input size4μ0.01No. of iterations10Channel noise variance0.01


### 5.1.5. SVM-Based Channel Equalizer

SVM is a supervised algorithm used for classification problems. Channel estimation is a classification problem, so it can be used to deal with the nonlinear channel effects. In this work, we have implemented a basic SVM model equalization. Simulation parameters of the SVM are shown in Table4. The generalization error computed during simulations is 0.00001 which indicates the best performance.Table 4
Simulation parameters of SVM.
ParametersValueInput size1,000,000Kernel functionKNN


### 5.1.6. LSTM Channel Equalizers

LSTM is a popular RNN-based DL technique. It is different from feedforward NN which does not have memory and cannot deal with temporal data. The simulation parameters are given in Table5. The training model of LSTM is illustrated in Figure 7.Table 5
Simulation parameters of LSTM model.
ParametersValueTraining SNR12 dBChannel noise variance0.01LSTM nodes16Learning rate0.01Figure 7
The LSTM model.


## 5.1.1. Simulation Parameters of NN

Different NN equalizers are plugged into the configuration of Figure4, and results are obtained. These configurations and the respective results are discussed in the sequel. The primary performance criteria are used in BER. Loss function analysis and the computational complexity are also calculated. The detailed results are compared and discussed in the later sections. The flowchart of the NN-based equalizer is depicted in Figure 5.Figure 5
The flowchart of NN-based equalizer.


## 5.1.2. MLP-Based Equalizer

MLP is a simple three-layer network that maps the input to the output. MLP is designed using the “nntraintool” of MATLAB. It comprises an input layer, a hidden layer, and an output layer. The input layer contains two vectors. One vector is the real part of the input signal (X), and another is the complex part of the signal. The output layer generates four vectors “Y0” to Y3. The MLP is trained with these parameters as shown in Table 1.Table 1
The simulation parameters of MLP.
ParameterValueHidden nodes30Input size (X)1,000,000Training algorithmScaled conjugate gradient (SCG) [77]


## 5.1.3. RBFNN

RBFNN is a three-layer network that comprises an input layer, a nonlinear hidden layer, and a linear output layer. Radial functions are used as an activation function. Radial functions are special functions. The output of these functions increases or decreases monotonically with distance from a center. TheK-means algorithm is used to find the centers. So first, centers of clusters are determined in an unsupervised manner, and then, classification is performed to recover the signal. We have implemented this work [51] and observed the improved BER. The simulation parameters of RBFNN are shown in Table 2.Table 2
Simulation parameters of the RBFNN.
ParameterValueData set size2000Noise variance0.01Centers16


## 5.1.4. FLANN

FLANN is a single-layer neural network. The main concept of FLANN is to convert the input data to a higher dimension by using different functional expansions. Due to the absence of hidden layers, these networks have the following advantages: low computational complexity with very few adjustable parameters:(i)
Faster training time(ii)
Simple design that can be implemented on hardwareUsing the work in [54, 55], we have implemented the FLANN-based equalizer. The block diagram of the equalizer is shown in Figure 6.Figure 6
The block diagram of the implemented FLANN-based equalizer.The simulation parameters of FLANN, Le-FLANN, and Ch-FLANN are given in Table3.Table 3
Simulation parameters.
ParametersValueLength of input2000FLANN order30Input size4μ0.01No. of iterations10Channel noise variance0.01


## 5.1.5. SVM-Based Channel Equalizer

SVM is a supervised algorithm used for classification problems. Channel estimation is a classification problem, so it can be used to deal with the nonlinear channel effects. In this work, we have implemented a basic SVM model equalization. Simulation parameters of the SVM are shown in Table4. The generalization error computed during simulations is 0.00001 which indicates the best performance.Table 4
Simulation parameters of SVM.
ParametersValueInput size1,000,000Kernel functionKNN


## 5.1.6. LSTM Channel Equalizers

LSTM is a popular RNN-based DL technique. It is different from feedforward NN which does not have memory and cannot deal with temporal data. The simulation parameters are given in Table5. The training model of LSTM is illustrated in Figure 7.Table 5
Simulation parameters of LSTM model.
ParametersValueTraining SNR12 dBChannel noise variance0.01LSTM nodes16Learning rate0.01Figure 7
The LSTM model.


## 5.2. Simulation Analysis

All the simulations are executed and compared in this section. Figure8 depicts the BER comparison of all the simulated NNs. Generally, the trend verifies already established theories. As the SNR increases, the BER performance is getting better and better. The performance of FLANN is slightly worse as compared to the rest of the schemes due to its single-layer architecture. The performance of the traditional LMS algorithm is the worst. In [51], similar results are observed. All the other ML-based schemes are having the same BER performance.Figure 8
The comparison of NN techniques.In Figure9, the zoomed version of the BER graph is depicted. The LSTM is slightly bearing higher BER than SVM and RBF-based ML methods. The performance of FLANN when compared with the rest is almost 4 dB poorer than the rest. The performance of LSTM is about 0.7 dB poorer than the RBF and SVM and MLP. This may be reduced by further tuning or by increasing the size of the neural network. However, this will be at the cost of time and computational resources which can be very expensive in the communication systems.Figure 9
The zoomed comparison plot of BER.The loss function is an important parameter of the optimization and is therefore discussed. The lesser the value of the loss function, the better performance is considered. In Table6, the values of the loss function for all the algorithms used in this text are depicted. It shows that all the NNs are well trained.Table 6
Loss function values.
ANNLoss function valueRadial Basis Function0.001SVN (KNN-based)0.00001MLP0.003FLANN0.005LSTM0.003The minimum value of the loss function achieved is in the case of SVN where the value is 0.00001. The BER results depicted in Figure9 are very much in line with these results. The loss function values of RBF, FLANN, and LSTM can be further reduced by using more training data and by using better optimization algorithms.


## 5.3. Computational Complexity

Computational complexity analysis of the algorithms is presented. This presents the number of computation resources required to perform the respective ANN. Table7 presents the computational complexity of various algorithms. The number of additions, multiplications, and other computational resources such as exponentiation, powers, and trigonometric functions is enlisted [82]. This analysis is useful for the HW implementations and for estimating the computational requirements for embedded systems.Table 7
Computational complexity.
ANNComputational complexityNo. of multiplicationsNo. of additionsDivisionTanhExpRBFn0n1+2n1+n0+12n0n1+n0+n1+1n0+n1—noFLANN3n1n0+1+n02n1n0+1+n1—n1—MLP4∑i=0L−1nini+1+3∑i=0L−1ni−n0n1+2nL3∑i=0L−1nini+1+3nL−n0n1———SVM4Td3TdTdLSTM2N+2NN0N1n0 is the number of input nodes. ni is the ith node in the network. Td is the total number of Euclidean distances in the KNN-based SVM. N is the number of nodes in the convolution layer of the LSTM network. N0N1 is the number of neurons in the BiLSTM layers [82].The computational complexity analysis of the mentioned algorithms is verified by timing the MATLAB® implementations. The time of all the algorithms used in this work is measured using the MATLAB® built-in function called “timeit.” The number of iterations performed for each algorithm is 106. The machine used for the computation is DELL® 7920 running MATLAB® 2019b. The CPU is Intel® Xeon® Silver 4116 CPU running at 2.1 GHz. The time is enlisted in Table 8. The computational time computed endorses the computational complexity as given in Table 7. The minimum computational time achieved is for the SVM. SVM is running a KNN algorithm that is computationally efficient. Its BER results are also amongst the best. RBF and MLP bear good performance, but their computational time is more.Table 8
Time of various ANN algorithms.
ANNTotal time in secondsIterationsTime single iteration (seconds)SVM5.2311,000,0005.2315×10−6LSTM102.9571,000,0001.029576×10−4RBF12.27291,000,0001.2273×10−5FLANN36.34501,000,0003.6345×10−5MLP13.08591,000,0001.3086×10−5


## 6. Conclusions

The communication system is an ever-evolving, well-established field of research and has shown major advances in signal estimation, equalization, and other fields such as channel coding. Channel equalization is very critical for achieving high data rates and improved spectral efficiency and has been achieved using the traditional theory of least squares estimation and minimum mean squares estimation techniques such as LMS, NLMS, RLS, and Kalman filtering. The use of NN- and SVM-based channel equalization methods is currently under research and is proving to be performing better than the conventional methods mentioned above.In this article, we have addressed the application of information theory-based methods for channel equalization comprising of neural networks and SVM techniques. It revealed that the methods used in traditional communication systems are difficult to understand and implement as compared to the ANN-based methods. Channel equalization when treated as a classification problem using ANN techniques resulted in simpler receiver structures especially in the case of OFDM. The results achieved are also found to be improved in terms of BER. Another advantage with the use of ANN-based methods is that this has resulted in a relatively simpler way to understand the communication systems, and many of the computer scientists who are not well versed with the communication system theories can also attempt to develop better communication systems by using their computer science and software development skills.This work can be extended in many ways. The following is the list of possible emerging research areas. Computational complexity analysis and computing platform optimizations of the algorithms are mandatory for efficient implementation on hardware platforms such as ARM processors and FPGA and GPUs. In this work, the preliminary computational complexity analysis has been worked out. However, this can be further extended when the implementation of these algorithms will be carried out on the FPGAs or when optimized for the implementation on the microcontrollers and DSP processors. Two-dimensional treatment of the received signal is similar to time-frequency analysis where several frames are gathered and then processed as a block. This will enable the use of advanced neural network methods such as CNN, DNN, and RNN methods. Existing frameworks such as AlexNet may also be used. Currently, performance evaluation is performed using QPSK modulation. Performance evaluation using higher-order constellations such as 16QAM, 64QAM, and 8PSK may also be carried out in the future. Validation by developing hardware may be carried out.



---

*Source: 2053086-2022-01-06.xml*