# Adaptive Step Size Gradient Ascent ICA Algorithm for Wireless MIMO Systems

Authors: Zahoor Uddin; Ayaz Ahmad; Muhammad Iqbal; Zeeshan Kaleem
Journal: Mobile Information Systems
 (2018)

---

## Abstract

Independent component analysis (ICA) is a technique of blind source separation (BSS) used for separation of the mixed received signals. ICA algorithms are classified into adaptive and batch algorithms. Adaptive algorithms perform well in time-varying scenario with high-computational complexity, while batch algorithms have better separation performance in quasistatic channels with low-computational complexity. Amongst batch algorithms, the gradient-based ICA algorithms perform well, but step size selection is critical in these algorithms. In this paper, an adaptive step size gradient ascent ICA (ASS-GAICA) algorithm is presented. The proposed algorithm is free from selection of the step size parameter with improved convergence and separation performance. Different performance evaluation criteria are used to verify the effectiveness of the proposed algorithm. Performance of the proposed algorithm is compared with the FastICA and optimum block adaptive ICA (OBAICA) algorithms for quasistatic and time-varying wireless channels. Simulation is performed over quadrature amplitude modulation (QAM) and binary phase shift keying (BPSK) signals. Results show that the proposed algorithm outperforms the FastICA and OBAICA algorithms for a wide range of signal-to-noise ratio (SNR) and input data block lengths.


---

## Body

## 1. Introduction

Independent component analysis (ICA) [1–3] is a signal processing technique used for estimation [4, 5] of the source signals from the mixed received signals without a priori knowledge of the source signals and the channel condition [3, 6–8]. Before ICA principal component analysis (PCA) [9, 10] was used as a blind source separation (BSS) technique. PCA is based on second-order statistics and makes the data uncorrelated. ICA is based on high-order statistics and makes the data independent. Independence is achieved through maximization of non-Gaussianity. Moreover, ICA has attracted attention from the last two decades due to its effective performance and applicability in practical scenarios [11]. It has various applications in different fields of engineering, that is, vibration analysis [12, 13], robotics [14], machinery fault diagnosis [15, 16], biomedical signal processing [17, 18], speech processing [19], and wireless communication [20, 21]. Its applications in wireless communication systems mainly include multiple input multiple output (MIMO) systems.The ICA algorithms can be classified into adaptive and batch ICA algorithms [22]. The adaptive algorithms perform well in time-varying mixing scenario with high-computational cost and slow convergence. The batch ICA algorithms have less computational complexity and better separation performance in case of quasistatic mixing scenario. Moreover, amongst the batch algorithms, the gradient ascent ICA (GAICA) algorithm performs well, but step size selection is critical in this algorithm. A smaller value of the step size will produce better results with slow convergence, while a larger step size will result in fast convergence with worse performance of the algorithm. An improper choice of the step size will produce worse performance or demand for large iterations [23].In the literature, various step size selection techniques are presented [24–27], where all these techniques are based on the fact that a smaller step size produces better results with slow convergence and larger step size provides fast convergence with worse performance. In [24], a so-called modified gradient-based ICA algorithm is developed. This algorithm converges for various values of the step size parameter if the objective function is properly selected. The authors in [25] presented a gradient ascent ICA algorithm, where the algorithm is terminated when the relative change in the unmixing matrices becomes less than a certain defined value for various step sizes. Furthermore, a diminishing step size gradient-based ICA technique is proposed in [26, 27], where the step size is adapted in such a way that its initial value is high and which is repeatedly reduced with increasing iterations in order to converge to the optimal value.In this paper, an adaptive step size gradient ascent ICA (ASS-GAICA) algorithm is developed. The proposed algorithm is free from the selection of the step size parameter. The objective function used for maximization of the non-Gaussianity in our algorithm is an absolute value of entropy [6]. The basic principle used in the derivation of the algorithm is that entropy of the recorded mixed signals is always less than the total entropy of the independent source signals and maximization of entropy produces the separated source signals. In addition to this, different performance evaluation criteria are used to observe the effectiveness of the proposed algorithm. These criteria include signal-to-interference ratio (SIR), symbol error rate (SER), error standard deviation (ESTD) of the signal, and norm of the product of mixing and unmixing matrices (Norm-WA). It is shown in the simulation that performance of the proposed algorithm is better than the fixed step size GAICA algorithm. Performance of the proposed ASS-GAICA algorithm is also compared with the well-known ICA algorithms: the FastICA [19] and the OBAICA algorithms [28].Notations used in this paper are.T, .′, and .″ for transpose, derivative, and second-order derivative of a vector or a matrix, respectively. Capital bold face letters represent matrices, and the small bold face letters are used for vectors. Symbols and abbreviations used in this paper are listed in Tables 1 and 2, respectively.Table 1
List of symbols.
A
Square mixing matrix of sizeM×M
h
Entropy of the extracted signals
E
Expectation operator
ln
Natural logarithm
g
CDF of the extracted signals
L
Length of the processing data block
M
Number of source signals
η
Step size
S
Matrix containing source signals of sizeM×L
s
Source signal vector of lengthL
W
Square unmixing matrix of sizeM×M
X
Mixed data matrix of sizeM×L
x
Vector representing mixed data of lengthL
Y
Unmixed data matrix of sizeM×L
y
Unmixed data vector of lengthL
g
Compensation parameter
e
Error signal
.
Norm
J
Maximum number of iterations
N
Noise matrix of sizeM×L
Δ
1
,
Δ
2
,
Δ
3
,
Δ
4
Random variables representing time-varying mixing coefficientsTable 2
List of abbreviations.
QAM
Quadrature amplitude modulation
BPSK
Binary phase shift keying
ICA
Independent component analysis
MIMO
Multiple input multiple output
SIR
Signal-to-interference ratio
SER
Symbol error rate
ESTD
Error standard deviation
BSS
Blind source separation
AWGN
Additive white Gaussian noise
SNR
Signal-to-noise ratio
CDF
Commutative density function
PDF
Probability density function
OFDM
Orthogonal frequency division multiplexing
PCA
Principal component analysisRest of the paper is organized as follows: Section2 presents the ICA signals’ model and assumptions. The ICA model is purely based on multidimensional data. This section presents the basic idea that how to model the MIMO signals for ICA algorithm. The mathematical formulation of the proposed ASS-GAICA algorithm is demonstrated in Section 3. Section 4 illustrates the simulation results. The results are demonstrated for quasistatic and time-varying mixing scenario utilizing various performance evaluation criteria. Finally, the conclusion is drawn in Section 5.


## 2. ICA Signals’ Model and Assumptions

A multiuser multiple input multiple output (MIMO) [29] wireless communication system with MT transmitting antennas and MR receiving antennas is considered. For the sake of simplicity, we have MT=MR=M. The transmitted source signal vectors are s1,s2,…,sM, where sM=sM1,sM2,…,sML and the received mixed signals are x1,x2,…,xM. The block diagram of the system model is shown in Figure 1. The ICA postprocessed estimated signals are y1,y2,…,yM. The role of the ICA algorithms is to unmix the received mixed signals in order to find the actual transmitted signals [3]. The received mixed signals can be modeled in matrix form as follows:(1)X=AS,where X is M×L mixed data matrix, A is M×M mixing matrix, and S is M×L source data matrix. Equation (1) represents the noise free model of ICA. The noisy ICA model can be represented as(2)X=AS+N,where N represents the additive white Gaussian noise with dimension M×L. The addition of noise further degrades the separation performance of the ICA algorithm. Moreover, it is known that inverse of the mixing matrix exist which is known as the unmixing matrix and denoted by W. The role of the ICA algorithms is to estimate the unmixing matrix. After estimating the unmixing matrix, the resultant unmixed signals can be modeled as follows:(3)Y=WX.Figure 1
MIMO transceiver system for ICA signal processing. This figure represents the basic ICA structure for recording multidimensional data. The transmitted signals are generated by independent sources. The receiving antennas receive linear mixtures of the transmitted independent source signals.After estimating the unmixing matrixW, the mixing source signals can be separated from the mixed signals by using (3). Accuracy of the ICA algorithms depends on proper estimation of the unmixing matrix. In case, if the mixing matrix changes dynamically within the processing data blocks, then unmixing becomes difficult.The basic assumption of ICA used in this paper is given below:(i)
The source signalssM are mutually statistically independent. The transmitting antennas transmit different source signals, and thus the independent assumption becomes true.(ii)
The source signalssM are non-Gaussian in nature. This assumption is also true because we consider the communication signals which are non-Gaussian in nature.(iii)
The mixing matrixA is assumed to be square. The validity of this assumption can be observed from the equal number of the transmitting and receiving antennas.


## 3. Proposed ASS-GAICA Algorithm

The absolute value of entropy is used as an objective function in the derivation of the algorithm. The symbols used in the formulation of the algorithm are listed in Table1. Furthermore, it is known from the ICA model that the source signals have non-Gaussian distributions. The CDF and PDF of the transmitted source signals are represented by g and g′, respectively. It is considered in [22] that the PDF of the supper-Gaussian signal can be represented by 2tanhY and sub-Gaussian by Y3. Received signals are the linear mixtures of the transmitted source signals, and the source signals can be estimated from the mixed received signals through the ICA algorithm. The ICA algorithm basically estimates the unmixing matrix W [3]. General form of the gradient ascent ICA algorithm is given in Algorithm 1. Hence, entropy of the estimated signals y1,y2,…,yM for jth iteration can be written in vector matrix form as(4)hj=Elng′WTjXj+lnWj.Algorithm 1: Gradient ascent ICA algorithm.
(1)
Initialization
Whiten the mixed received data
Choose a random initial unmixing matrixW
Select the number of maximum iterationsJ
(2)
for j=1 to J do
(3)
Y=WX
(4)
Update the unmixing matrix using the following equation:
Wj+1=Wj+η∇hj
(5)
Terminate the loop if
hj−hj−1<δ, where δ is very small
(6)
end forMaximization of entropy as given in (4) tends toward maximization of the non-Gaussianity. In order to achieve this objective, the gradient of hj with respect to each element of W can be written as(5)∇hj=CjXj+Wj−T,where(6)Cj=Eg″WTjXjg′WTjXj,∂hj∂W=∇hj.Now, general form of the update equation forW can be written as(7)Wj+1=Wj+η∇hj.After substituting (5) in (7), we will get the following updated formula for W:(8)Wj+1=Wj+ηCjXj+Wj−T.Equation (8) represents the fixed step size GAICA algorithm where the positive sign used in (8) indicates the gradient ascent nature of the algorithm. This equation updates the unmixing matrix W in such a way to further maximize the entropy while utilizing a constant step size. The optimal step size η selection is difficult in this equation. The improper value of η will result in worse performance or slow convergence of the algorithm. Moreover, in order to achieve the adaptive step size mechanism, first determine hj+1 at j+1th iteration. Then, approximate hj+1 up to the second-order terms by using Taylor’s series expansion. Finally, adjust the value of η in such a way to further maximize the entropy while processing from jth iteration to j+1th iteration. In order to achieve these objectives first, we determine the entropy for j+1th iteration as(9)hj+1=Elng′WTj+1Xj+1+lnWj+1.After applying Taylor’s series expansion over (9), the resultant equation can be written as(10)hj+1=hj+∇hjΔW+12∇h2jΔWΔW+⋅⋅⋅⋅⋅where ΔW=Wj+1−Wj.For smaller values ofΔW, hj+1 can be approximated up to second-order terms in (10), and this is indeed the case in our algorithm, which is shown in the simulation section. Using this fact in combination with (5), we can write (10) as(11)hj+1=hj+CjXj+Wj−TΔW.From (8), we have(12)ΔW=ηCjXj+Wj−T.After combining (12) and (11), we get(13)hj+1=hj+ηCjXj+Wj−TCjXj+Wj−T.In order to achieve the adaptive step size selection criterion, adjustη in such a way to further maximize hj+1. To achieve this objective, evaluate the gradient of hj+1 with respect to η as(14)∂∂ηhTj+1hj+1=0.Now after combining (13) and (14), we get the following resultant equation:(15)ηCjXj+W−T=−hjWjCjXj+I.Now substituting (15) in (8), we have(16)Wj+1=Wj−hjWjWjCjXj+I.Let(17)Q=WjWjCjXj+I,then (16) can be written as(18)Wj+1=Wj−hjQ.The updated formula (18) involves inversion of the denominator part of Q, that is, WjCjXj+I, having same dimension as matrix W. It means that the proposed algorithm is more suitable for a smaller number of the source signals because the computational complexity of the matrix inversion is OM3. In case of large M, an estimate of Q can be developed. Another factor effecting the computational complexity is the data block length L, and hence computational complexity of the proposed algorithm is OLM3 that is approximately similar to the computational complexity of the fixed step size GAICA algorithm. The proposed algorithm is summarized in Algorithm 2. We use hj−hj−1<δ as a stopping criterion in our algorithm.Algorithm 2: The ASS-GAICA algorithm.
(1)
Initialization
Whiten the mixed received data
Choose a random initial unmixing matrixW
Select the number of maximum iterationsJ
(2)
for j=1 to J do
(3)
Y=WX
(4)
Calculate entropy of the mixed received data vectors as
hj=Elng′WTjXj+lnW
(5)
Calculate matrixCj as follows:
Cj=Eg″WTjXj/g′WTjXj
(6)
Calculate matrixQ as
Q=Wj/WjCjXj+I
(7)
Update the unmixing matrix using the following equation:
Wj+1←Wj−hjQ
(8)
Terminate the loop if
hj−hj−1<δ, where δ is very small
(9)
end for


## 4. Results and Discussion

Performance of the proposed ASS-GAICA algorithm is demonstrated in this section. A dual antenna MIMO transceiver system is considered for simulations. The receiver receives linear mixtures of the transmitted signals. The channel is assumed AWGN with frequency flat fading. Simulation is performed over BPSK and 16-QAM signals with SNR ranging from −2 dB to 10 dB and input data block lengths ranging from 50 to 1000 samples using Monte–Carlo simulation. We consider the quasistatic and the time-varying wireless channels. The simulation parameters are shown in Table3.Table 3
Simulation parameters.
M
2
L
50 to 100
δ
10
−
6
SNR
−2 dB to 10 dB
Modulation type
BPSK and 16-QAM
Wireless channels
AWGN
Fadding
Frequency flate
Mixing
Quasistatic and time varyingThe order, phase, and amplitude indeterminacies are resolved by using a technique presented in [30], where a few extra samples are transmitted which are known to the receiver in advance. This technique is insensitive to the data block lengths.Different performance evaluation criteria are used to measure the effectiveness of the proposed algorithm which is given below.(1)
Signal-to-interference ratio (SIR): The SIR represents the average ratio of the desired signal power to the power of the estimation error. The SIR in dB can be written as(19)SIRdB=10log101L∑n=1Lsn2sn−yn2.(2)
Error standard deviation (ESTD) of the signal: Error signal is the difference of the source signal and the algorithm postprocessed signal. The ESTD can be defined as(20)ESTD=∑n=1Len,where en=sn−yn.(3)
Symbol error rate (SER).(4)
Norm of the product of mixing and unmixing matrices (Norm-WA): For ideal case the norm of the product of W and A will be one because A=W−1. A value of Norm-WA near to one represents efficient separation. Mathematically it can be defined as follows:(21)Norm−WA=WA.Simulation results for quasistatic wireless channels are demonstrated as follows.First, we demonstrate the convergence behavior of the proposed ASS-GAICA algorithm and the fixed step size GAICA algorithm. The GAICA is an algorithm where the step size can be adjusted experimentally. The smaller value of the step size improves the separation performance with slower convergence, while the larger value increases the convergence speed with less accurate results. In order to compare the results of the GAICA algorithm with our proposed algorithm, we consider two values of the step size parameter, that is,η=0.1and0.2. The results are shown in Figure 2 for 16-QAM signals with L=100 and SNR of 10 dB. We consider the maximum number of iterations as 50 for both the algorithms. The step size values considered for the GAICA algorithm are η=0.1and0.2. It can be observed from the results that further decrease in the step size parameter will further reduce the convergence speed. Similarly, increase in the step size will produce worse performance of the algorithm. Hence, we restrict ourselves to these two values of the step size parameter. The proposed algorithm properly converges in 22 iterations, while the GAICA algorithm converges in 40 iterations for η=0.1. If we increase the value of η to 0.2, then the GAICA algorithm converges faster with less accurate steady state results in 50 iterations. In summary, we can say that the GAICA algorithm converges faster for larger step size with less accurate steady state results and vice versa, while the proposed algorithm improves the convergence speed as well as quality of the results.Figure 2
Convergence characteristics of the proposed ASS-GAICA and GAICA algorithms. The step size values considered in GAICA algorithm are 0.1 and 0.2.Secondly, the performance of the proposed algorithm is compared with the well-known algorithm of ICA, FastICA algorithm and OBAICA algorithm using different performance measures as a cross-check. Figures3 and 4 show the effectiveness of the proposed algorithm in terms of the SIR for different block lengths and different SNRs using 16-QAM signals. In Figure 3, we consider the fixed SNR of 10 dB with different block lengths, while in Figure 4, the block length is kept 1000 samples with different values of SNR.Figure 3
Comparison of the proposed algorithm with the FastICA and OBAICA algorithm. The performance evaluation criterion used in this simulation is SIR. The results are compiled for various input data block lengths at SNR of 10 dB.Figure 4
SIR performance comparison of ASS-GAICA, FastICA, and OBAICA algorithms over various SNRs and fixed data block length of 100 samples. The SNRs considered in these results range from −2 to 10 dB.The performance of the proposed algorithm is again evaluated utilizing the error standard deviation (ESTD) of the signal. The results are shown in Figures5 and 6 for different blocks lengths and SNR using 16-QAM signals. The Norm-WA and SER performance are shown in Figures 7 and 8. All these figures show the superior performance of the proposed ASS-GAICA algorithm. From Figures 3–8 and Table 4, one can observe that the FastICA algorithm performs well than the OBAICA algorithm for the observed signals, but accuracy of the results is high for the proposed ASS-GAICA algorithm.Figure 5
ESTD performance of the ASS-GAICA, FastICA, and OBAICA algorithms for different blocks lengths and fixed SNR of 10 dB. The data block lengths utilized in these results range from 50 to 1000 samples.Figure 6
ESTD performance of the ASS-GAICA, FastICA, and OBAICA algorithms at various SNRs and fixed block length of 100 samples. SNRs considered in these results range from −2 to 10 dB.Figure 7
Norm-WA performance of all the three algorithms at various block lengths and fixed SNR of 10 dB.Figure 8
SER performance for different SNRs at a data block length of 100 samples. SNRs considered in these results range from −2 to 10 dB.Table 4
Performance comparison of the OBAICA, FastICA, and ASS-GAICA algorithms for QAM-modulated signals.
Performance measure
OBAICA
FastICA
ASS-GAICA
SIR (SNR = 10 dB,L = 1000)
33.0
36.401
38.3209
SIR (SNR = 0 dB,L = 100)
7.780
9.401
10.3209
ESTD (SNR = 10 dB,L = 1000)
0.19279
0.18043
0.1706
ESTD (SNR = 0 dB,L = 100)
0.310859
0.39004
0.40293
Norm-WA (SNR = 10 dB,L = 1000)
1.29
1.17
1.01463
Norm-WA (SNR = 0 dB,L = 100)
1.98017
1.710017
1.279
SER (SNR = 10 dB,L = 1000)
0.0084
0.0069
0.0045
SER (SNR = 0 dB,L = 100)
0.094
0.0719
0.0235Another experiment is performed while utilizing the BPSK-modulated signals. The number of source signals and the simulation environment are the same as above. The SIR is used as a performance measure. The SIR performance for different block lengths and fixed SNR is shown in Figure9. This figure shows that the proposed algorithm outperforms the FastICA and the OBAICA algorithms for quasistatic channels.Figure 9
SIR performance of all the three algorithms for BPSK-modulated signals utilizing different data block lengths and fixed SNR of 10 dB.We also investigate the proposed algorithm in comparison with the FastICA and the OBAICA algorithms for time-varying mixing scenario. The time-varying mixing matrix is represented as follows:(22)A=1+Δ12+Δ23+Δ34+Δ4,where Δ1, Δ2, Δ3, and Δ4 are the random variables representing the time-varying nature of the wireless channels. The results for time-varying mixing scenario are shown in Table 5. The results given in this table show that even at time-varying scenario our proposed algorithm outperforms.Table 5
Performance comparison of the OBAICA, FastICA, and the ASS-GAICA algorithms for QAM-modulated signals utilizing time-varying wireless channels.
Performance measure
OBAICA
FastICA
ASS-GAICA
SIR (SNR = 10 dB,L = 1000)
6.324
9.391
11.94320
ESTD (SNR = 10 dB,L = 1000)
1.99
1.390
0.91907
Norm-WA (SNR = 10 dB,L = 1000)
3.35
2.49
1.9145
SER (SNR = 10 dB,L = 1000)
0.1932
0.1099
0.099


## 5. Conclusion

The entropy-based gradient ascent ICA (GAICA) algorithm has improved accuracy of separation, but the step size selection is difficult in this algorithm. Improper step size will result in worse performance of the algorithm. In this paper, we proposed an adaptive step size gradient ascent ICA (ASS-GAICA) algorithm that selects the optimal step size adaptively. The proposed algorithm has better convergence behavior with improved accuracy of results than the fixed step size GAICA algorithm. Performance of the proposed algorithm is compared with the well-known ICA algorithms, the FastICA and the OBAICA algorithms using binary phase shift keying (BPSK) and quadrature amplitude modulation (QAM) signals in quasistatic and time-varying scenario. Different performance measures, that is, SIR, ESTD, SER, and Norm-WA, are used to evaluate the effectiveness of the proposed algorithm. The simulation results show that the proposed algorithm outperforms the FastICA and the OBAICA algorithms for various input data blocks and SNR. The proposed algorithm outperforms even for time-varying scenario.Due to matrix inversion in the update equation, the proposed algorithm is suitable for a smaller number of the mixing source signals. In case of a large number of the mixing signals, some estimate of the matrixQ is required. As a future work, we will develop an estimator of this matrix for large number of the source signals.



---

*Source: 7038531-2018-05-02.xml*