# ASEP of MIMO System with MMSE-OSIC Detection over Weibull-Gamma Fading Channel Subject to AWGGN

Authors: Keerti Tiwari; Davinder S. Saini; Sunil V. Bhooshan
Journal: Journal of Computer Networks and Communications
 (2016)

---

## Abstract

Ordered successive interference cancellation (OSIC) is adopted with minimum mean square error (MMSE) detection to enhance the multiple-input multiple-output (MIMO) system performance. The optimum detection technique improves the error rate performance but increases system complexity. Therefore, MMSE-OSIC detection is used which reduces error rate compared to traditional MMSE with low complexity. The system performance is analyzed in composite fading environment that includes multipath and shadowing effects known as Weibull-Gamma (WG) fading. Along with the composite fading, a generalized noise that is additive white generalized Gaussian noise (AWGGN) is considered to show the impact of wireless scenario. This noise model includes various forms of noise as special cases such as impulsive, Gamma, Laplacian, Gaussian, and uniform. Consequently, generalizedQ-function is used to model noise. The average symbol error probability (ASEP) of MIMO system is computed for 16-quadrature amplitude modulation (16-QAM) using MMSE-OSIC detection in WG fading perturbed by AWGGN. Analytical expressions are given in terms of Fox-H function (FHF). These expressions demonstrate the best fit to simulation results.


---

## Body

## 1. Introduction

In multiple-input multiple-output (MIMO) systems, the role of spatial multiplexing (SM) and spatial diversity is to provide high data rate and reliable communication, respectively. However, a tradeoff occurs between diversity and multiplexing for multiple access channels [1]. The demand to achieve high data rate is increasing for next generation communication systems. Using diversity, a simple detection can be formed at the cost of capacity reduction. Thus, spatial multiplexing is recommended for achieving significant capacity gain with improved transmission rate. However, demultiplexing at the receiver is still an issue for efficient system design.A variety of detection techniques exist in literature to improve the system performance [2–7]. Zero forcing (ZF) and minimum mean square error (MMSE) are simple detection techniques although they lead to the reduced error performance [3, 4]. Maximum likelihood (ML) detection provides an optimal error rate performance. However, the hardware complexity increases with the increase in transmitter antennas and modulation order. Therefore, ordered successive interference cancellation (OSIC) is recommended with MMSE for improving the error rate performance [5]. In [5–8], efficient approaches have been investigated to develop OSIC and to reduce the complexity of receiver for Vertical-Bell Laboratories Layered Space-Time (V-BLAST) system. In [9], antenna selection has been used for OSIC detection to enhance the error rate performance. Also, a near-optimal selection technique has been proposed to decrease the complexity without substantial performance reduction. Different detection ordering such as signal-to-noise ratio (SNR) based, column norm based and, signal-to-interference plus noise ratio (SINR) based ordering has been investigated. It is examined that postdetection SINR based ordering achieves the best performance among all three ordering techniques [10]. Hence, MMSC-OSIC detection is used in this paper to achieve the improved error rate performance with adaptable complexity level in composite fading scenario. Identifying an efficient detection technique is still under investigation in MIMO systems.Various small scale multipath channel models including Rayleigh, Rician, Nakagami-m, and Weibull have been recommended to analyze the wireless system performance. Large scale fading can be modeled using log-normal (LN) or Gamma distribution. Both the small scale and large scale fading effects can be observed simultaneously due to rapidly changing wireless environment. Therefore, channel modeling of composite fading which comprises multipath as well as shadowing effects is imperative to figure out. Also, it is important to resolve a number of practical problems with interference effects in MIMO wireless communications. The composite channel models such as Rayleigh-LN, shadowed-Rician, Gamma-Gamma, K, generalized-K, correlated shadowed-κ-μ [11–17], and comparatively new-fangled Weibull-Gamma (WG) [18, 19] are widely used to incorporate both small scale fading and shadowing. Gamma models are simpler and more accurate than LN models and hence preferred in recent approaches. The WG composite distribution is appropriate for MIMO system design in the present wireless scenario due to its extensive flexibility, experimental efficiency, and analytical conformity. Weibull fading model is adaptive for modeling severe and nonsevere multipath fading conditions, Gamma model is used for shadowing, and collectively WG composite fading model is formed to quantify both the effects. WG fading model is a generic model and it includes Rayleigh-Gamma or K and exponential-Gamma distributions as its special cases. Consequently, this model approximates several other fading models [11, 18].Both higher and lower modulation orders have been considered in multipath fading with the existence of AWGN [20–22]. However, the deviation of actual noise is possible; therefore, generic noise model is required. In power line communication (PLC), the system performance is extensively affected by additive and multiplicative power line noises. Further, the additive noise is categorized into background noise and impulsive noise. The background noise and impulsive noise models follow Nakagami-m and Middleton class A distribution, respectively, although the multiplicative PLC noise induces fading in the received signal power. Also, the system error rate performance has been evaluated in the presence of such noise scenarios [23–25]. The generalized Gaussian distribution (GGD) is the emerging research interest for modeling different noise effects. This generic noise model considers various forms of noise such as impulsive, Gamma, Gaussian, Laplacian [26]. Rectangular quadrature amplitude modulation (QAM) is defined by a combination of in-phase and quadrature phase pulse amplitude modulation (PAM) signals. This modulation technique has been used to compute average symbol error probability (ASEP) using Gaussian Q-function in composite fading scenario perturbed by additive white generalized Gaussian noise (AWGGN) [27].To the best of our knowledge, the MIMO system performance has not been evaluated for composite WG fading channel with the consideration of generalized noise. To improve the MIMO system performance spatial multiplexing (SM) is used with efficient detection technique, that is, MMSE-OSIC. The expressions derived for ASEP in [27] are analyzed again which were previously limited to single-input single-output (SISO) system. To achieve the high data rate of the wireless link, higher order modulation techniques are preferred, although they are less flexible to noise and interference. Exact analytical expressions are computed in terms of Fox-H function (FHF) for 16-QAM in SM-MIMO WG fading subject to AWGGN. Two special cases of AWGGN, namely, AWGN and Laplacian noise, are considered. AWGN is popularly known and Laplacian noise has also achieved attention in the signal processing and wireless systems to model impulsive noise. In addition, the variations of fading and shadowing parameters are also illustrated.The rest of the paper is organized as follows. In Section2, system model and proposed detection technique are described. Section 3 provides the simulation results and analysis for ASEP of SM-MIMO in WG fading subject to generic noise along with specified detection technique. Finally, the paper is concluded in Section 4.


## 2. System and Channel Model

Consider a MIMO system havingNr×Nt antennas, where Nt and Nr denote the number of transmit and receive antennas, respectively. X is the transmit signal modulated by 16-QAM and multiplied by a composite flat fading channel envelope H. Previously, the noise is generally considered as AWGN. However, the AWGGN noise N is assumed here with zero mean and variance σ2. MIMO system model is represented as(1)Y=HX+N.The probability density function (PDF) of the AWGGN noise is described in [28, Equation (6.2)] over n∈R as(2)PNn∣mn,σ,η=ηψ2Γ1/ηexp⁡-ψηn-mnη,where η and mn denote shaping parameter of N and mean, respectively (η∈R+,mn∈R+). Furthermore, the coefficient ψ can be defined by normalizing the noise power using normalizing coefficient ψ0 with respect to η as(3)ψ=ψ0σ=2Γ3/ηN0Γ1/η,where Γ(·) denotes Gamma function, ψ0=Γ(3/η)/Γ(1/η), σ2=EN2-mn2=N0/2, N0 is the power per positive frequency of AWGGN, and E· is expectation operator.The random variable of AWGGN distribution rigorously depends on its shaping parameterη. This distribution reveals a superior fit to the quantified noise statistics with the changing physical channel conditions and forms various noise categories as special cases of AWGGN. For η=2, η=1, η=0.5, and η=0, it characterizes the prominent Gaussian, Laplacian, Gamma, and impulsive noise, respectively. Consequently, for η=1/2 and η=1/3, the statistical properties and accurate simulation technique have been developed in the presence of AWGGN [29].The PDF of the received signal envelopeχ given in [30] is simplified to form WG distribution defined over x∈(0,∞) and represented as (4)Pχx=2Γmm+1Γ1+1/kΩmx2m-1Γ1-mk,0,m+1Γ1+1/kΩmx2,1k,where Γ(·,·,·,·) is extended incomplete Gamma function, which is given as Γξ,s,a,μ=∫s∞xξ-1exp⁡-x-ax-μdx, (s∈R+), ξ,a,μ∈C [28, Equation (6.2)]. k and m denote fading figure and shadowing shaping parameter, respectively, where 0.5≤k<∞, 0≤m<∞, and Ω=E[χ2] (0≤Ω<∞) is the average power of the received signal envelope.The SNRγ for received symbols in the presence of AWGGN follows WG PDF, which is defined over γ∈(0,∞). The average SNR per symbol is γ-=Eγ=Eχ2Ex/N0, where Ex is the energy of transmitted symbols. By exchanging variables, PDF is represented in the form of SNR as(5)Pγγ=1Γmm+1Γ1+1/kγ-mγm-1Γ1-mk,0,m+1Γ1+1/kγ-mγ,1k.Equation (5) can be expressed in terms of the FHF using [28, Equation (6.22)], [31, Equations (2.1.4), (2.1.5) and (2.1.11)] and simplifying [32, Equation (8)] as(6)Pγγ=1ΓmγH0,22,0m+1Γ1+1/kγγ-1,1/k,m,1-,where(7)Hp,qm,nzbk,Bk1,qaj,Aj1,p=12πi∫c-∏k=1mΓbk,Bks∏j=1nΓ1-aj,Ajs∏k=n+1pΓaj,Ajs∏k=m+1qΓ1-bk,Bksz-sds,where Hp,qm,n(·) is FHF defined in [31, Equation (1.1.1)], [33] and c represents Mellin-Barnes contour.


## 3. MMSE-OSIC Detection

MMSE detection technique maximizes the postdetection SINR by minimizing mean-square error (MSE). In OSIC, SINR based ordering improves the performance of linear detection technique by maximizing SINR. This technique maintains the low complexity for designing hardware. It holds number of linear receivers in which each receiver classifies one of the parallel data streams with detected signal components. These signal components are successively canceled from the received signal at each stage [34].The MMSE detection technique explained in [10] offers the 1st estimated stream with the 1st row vector of the MMSE weight matrix WMMSE which is given by(8)WMMSE=H†H+N0I-1H†,where (·)† denotes the Hermitian operator. In MMSE detection, to determine the required statistical information of N0, the ith row vector wi,MMSE of WMMSE is obtained by(9)wi,MMSE=argmaxw=w1,w2,…,wNt⁡whi2Ex∑j=1,j≠iNtExwhi2+N0w2,where hi is the ith column vector of the channel matrix and · is the Frobenius norm of matrix. x(i) denotes the ith order detected symbol which depends on the order of detection; hence, this symbol may be different from the transmit signal at the ith antenna. The sliced value of x(i) is given by x^(i). The remaining signal is represented by(10)Y~=Y-∑j=1i-1x^jhj.Assume that x^j, j=1,2,…, i-1, is accurately generated. When x(1)=x^(1) it means the interference is canceled and x(2) can be estimated. When x(1)≠x^(1) then error propagation takes place. The entire performance of OSIC technique is affected by the order of detection. The erroneous outcomes in the previous stages cause the error propagation.Primarily, signals containing a higher postdetection SINR are detected in SINR based ordering. The linear MMSE detection with the postdetection SINR is represented by(11)SINRi=Exwi,MMSEhi2∑r≠iExwi,MMSEhr+N0wi,MMSE2,i=1,2,…,Nt.Once the selection of the Nt SINR values using WMMSE is completed, the corresponding layer with the highest SINR is chosen. The second detected symbol is selected by canceling the interference due to first detected symbol from the received signals. If rth symbol is canceled first, then H of (8) is transformed into (12) by deleting the channel gain vector as per rth symbol:(12)Hr=h1h2⋯hr-1hr+1⋯hNt.Again, WMMSE is calculated after substituting H of (8) by (12). Then, Nt-1 SINR values (i.e., (SINRi)i=1,i≠rNt) are computed by selecting the symbol containing highest SINR. After canceling the next symbol with the highest SINR, the same process is continued with the remaining signal. The total number of calculated SINR values is generated by ∑j=1Ntj=Nt(Nt+1)/2.The OSIC technique can offer diversity order greater thanNr-Nt+1 for all symbols. Following the ordering approach, the diversity order of the first detected symbol is also greater than Nr-Nt+1. Nevertheless, the diversity order of remaining symbols depends on whether the previously detected symbols are exact or not. Suppose all the symbols are exact; then, the diversity order of the ith detected symbol is Nr-Nt+i. The ith detected symbol is different from the one transmitted from the ith transmit antenna. Since the ordering is based on SINR for MMSE detection, therefore, (11) is used to improve the ASEP performance.


## 4. Average Symbol Error Probability forM-QAM

The symbol error probability (SEP) has been given in [35, Equation (10)] for QAM in the presence of AWGN. The formation of M-QAM signal constellation is given by two independent in-phase and quadrature M-ary PAM signals, where MI-ary PAM and MQ-ary PAM are in-phase and quadrature signals, respectively, and M=MIMQ. Given that, GGD and Gaussian distribution demonstrate the identical symmetry properties. According to [27], the identical symmetry properties can be used to define SEP of M-QAM, given as(13)PSEP=21-1MIQηAI+21-1MQQηAQ-41-1MI1-1MQQηAIQηAQ,where AI=DI/σ, AQ=DQ/σ, and DI and DQ denote the decision distances for in-phase and quadrature phase components, respectively. Qη· is generalized-Q function for x≥0 defined in [32] as(14)Qηx=ηψ02Γ1/η∫x∞e-ψ0ηtηdt.In [32, A.5], the representation of (14) in the form of FHF using [33, Equation (8.3.2/21), (A.4)] is given as(15)Qηx=12Γ1/ηH1,22,0ψ0ηxη1/η,1,0,11,1.The ASEP is obtained by averaging the conditional SEPs in (13) under slow fading conditions over the PDF of γ. Then, Pγγ is represented by(16)PrSe=21-1MITAI+21-1MQTAQ-41-1MI1-1MQJ,where(17)Tx=∫0∞QηγxPγγdγ,(18)J=∫0∞QηγAIQηγAQPγγdγ.It is difficult to formulate T(·) and J using the conventional expressions of WG distribution and GGD. Therefore, alternative expressions (6) and (15) are used to compute simplified analytical expressions for T(·) and J and then resulting expression for the ASEP. In (17), Tx consists of an integral including the product of two FHFs which is comparable to that of [32] considering the normalized value of fading shaping factor and severity of shadowing. Unlike [32], we prefer an efficient SINR based ordering for MMSE detection to improve the error rate performance of MIMO system. Using [27] and [31, Equation (1.1.1)], Tx can be represented in the form of FHF by a closed form expression given as(19)Tx=1ηΓ1/ηΓmH2,32,2m+1Γ1+1/kx2ψ02γ-1,1/k,m,1,0,2/η1-1/η,2/η,1,2/η,(20)J=12ηΓ1/η2ΓmH2,1;0,2;1,20,2;2,0;2,0m+1Γ1+1/kAI2ψ02γ-,AQAIη1,1/k,m,1,1,1,1/η,1,0,11-1/η;2/η,1,1;2/η,1,0;2/η,1.Substituting (6) and (15) in (18), an integral which includes the product of three FHFs is used to describe (20). Then, using [36, Equation (2.3)], J is expressed in terms of the FHF of two variables known as the bivariate Fox-H function (BFHF).Substituting (19) and (20) in (16), the ASEP of M-QAM is computed. This ASEP expression is given for rectangular (MI≠MQ), square (MI=MQ) QAM in arbitrary WG fading with AWGGN. Consequently, it maintains substantial range of noise and fading parameters. The commonly considered noise cases of AWGGN in composite fading scenario are as follows.Case 1 (WG fading with Laplacian noise).
The first special case of AWGGN emerges whenη=1, and the noise is considered Laplacian. Taking η=1, Tx is represented as(21)Tx=1ΓmH2,32,2m+1Γ1+1/k2x2γ-1,1/k,m,1,0,20,2,1,2.Using [31, 36], FHF and BFHF functions are well explored and utilized to make a simplified form of (21) by reducing number of terms in H·,··,·· as (22)Tx=1ΓmH1,22,1m+1Γ1+1/k2x2γ-1,1/k,m,11,2.Similarly, J can be written as(23)J=12ΓmH2,1;0,2;1,20,2;2,0;2,0m+1Γ1+1/k2AI2γ-,AQAI1,1/k,m,1,1,1,1,1,0,10;2,1,1;2,1,0;2,1.Using [36, Equation (1.1)], for the description of BHFH and [28, Equations (6.29) and (6.42)], J is represented as(24)J=12ΓmH1,22,1m+1Γ1+1/k2AI2+AQ2γ-1,1/k,m,11,2.Equations (22) and (24) are used to calculate ASEP when Laplacian noise is present.Case 2 (WG fading with AWGN).
Forη=2, (19) and (20) can be rearranged to find the ASEP in AWGN environment. Again, the expressions for Tx and J are reduced as(25)Tx=12πΓmH2,32,2m+1Γ1+1/k2x2γ-1,1/k,m,1,0,11/2,11,1,J=114.5ΓmH2,1;0,2;1,20,2;2,0;2,0m+1Γ1+1/k2AI2γ-,AQAI21,1/k,m,1,1,1,1/2,1,0,11/2;1,1,1;1,1,0;1,1.Here, m→∞ eliminates the effect of shadowing. When m→∞, WG distribution follows Weibull distribution and m→∞, k=1 converted WG distribution into Rayleigh distribution [30, Table 1]. Thus, the fading scenario can be changed by setting the parameters m and k.


## 5. Simulation Results and Analysis

To evaluate the MIMO system performance, 16-QAM modulation is used as a function of SNR for the generalized case of noise. Therefore, distinct values ofη and arbitrary values of m and k are taken into consideration. The in-phase-to-quadrature phase decision distance ratio is represented as Rddr=DI/DQ=AQ/AI. For this case, the average total energy per symbol ET is given as ET=10.5γ-DI2+2.5γ-DQ2=0.521+5Rddr2γ-DI2 and hence ET/σ2=0.521+5Rddr2γ-AI2 [27]. Taking a fixed Rddr=(10.5)1/2, the identical average energies of the in-phase and quadrature signals are obtained.The impact of the parameterRddr is emerged to observe the system performance. When Rddr=1, the most favorable case occurs; this implies that the in-phase and quadrature distance are identical for both the Laplacian and Gaussian noise. For Rddr=(10.5)1/2, same energy is obtained between the in-phase and quadrature signal; thus the system performance is reduced with a small amount, that is, approximately 1 dB SNR reduction for large SNRs. When the quadrature signal contains 10.5 times the average energy of the in-phase signal, for this instant loss is more essential as it gets approximately 4 dB SNR loss for large SNRs, comparative to the aforementioned case, where Rddr=1.Firstly, composite WG fading is considered in Laplacian noise environment. To obtain Weibull and Rayleigh fading, the parameters are settled tom→∞ and m→∞, k=1, respectively. Figure 1 depicts the ASEP as a function of average SNR per symbol ET/σ2 for both Gaussian and Laplacian cases of noise. In addition, distinct values of m and k are chosen to determine the severity of fading. Analytical results presented in this paper by (19) and (20) demonstrate the perfect match of the simulation results. The performance of the system is improved by increasing both the parameters m and k. Results shown in Figure 1 illustrate that the ASEP performance in Laplacian noise is superior to that of Gaussian noise for lower SNR or less than 15 dB SNR. However, for high SNR, less fading (k≥2), the situation is upturned and ASEP performance improves in the Gaussian noise compared with Laplacian noise. For severe fading (m=0.5), Laplacian noise offers better results than Gaussian noise.Figure 1
ASEP of 2 × 2 MIMO system using MMSE-OSIC over WG fading channel subject to Laplacian and Gaussian noise.Afterward, Weibull and Rayleigh fading which are the special cases of WG fading are considered. In Figure2, Rayleigh fading case is taken into account. In this case, system gives superior performance by diminishing η, which validates the previous result in which the Laplacian noise gives better performance than the Gaussian noise in severe fading. It is previously mentioned that large fading parameter refers to less fading. In Figure 3, the ASEP is demonstrated as a function of the SNR per QAM symbol in Weibull fading environment (k=5) with AWGGN for which η=8,2,0.5,0.25. In this case, when the less fading condition occurs, the different two regions are investigated. At low SNR, the ASEP decreases with increasing η and at high SNR it improves by increasing η.Figure 2
ASEP of 2 × 2 MIMO system for 16-QAM using MMSE-OSIC over Rayleigh fading channel with arbitrary values ofη.Figure 3
ASEP of 2 × 2 MIMO system for 16-QAM using MMSE-OSIC detection over Weibull fading channel (k=5) with arbitrary values of η.


## 6. Conclusion

This paper evaluates the ASEP performance of MIMO system in composite WG fading environment subject to AWGGN. Analytical expressions for ASEP are derived using 16-QAM consisting of two independent in-phase and quadrature signals of PAM. The MMSE-OSIC detection is used to improve the error rate performance of MIMO system. It is concluded from the results that the ASEP performance in Laplacian noise is better than that of Gaussian noise for low SNR. However, in less fading, performance is degraded for high SNR and improved error performance is obtained in Gaussian noise compared with Laplacian noise. In severe fading, improved error rate performance can be achieved in the presence Laplacian noise compared with Gaussian noise. In Rayleigh fading case, the system gives superior performance for low noise shaping parameterη. This result again proves that the Laplacian noise gives better performance than the Gaussian noise in severe fading. In Weibull fading, the different two regions are inspected for lower amount of fading or large fading parameter. Moreover, the ASEP reduces with η at low SNR and it increases by η at high SNR. Simulation results validate the analytical results.



---

*Source: 7918456-2016-03-13.xml*