# New 3D 16-Ary Signal Constellations and Their Symbol Error Probabilities in AWGN and Rayleigh Fading Channels

Authors: Zhenxing Chen; Jiaheng Liu; Shuang Li; Seog Geun Kang
Journal: Wireless Communications and Mobile Computing
 (2018)

---

## Abstract

Two three-dimensional (3D) 16-ary signal constellations having extended lattice structures are presented in this paper. The theoretical symbol error probabilities (SEPs) of the new constellations in additive white Gaussian noise (AWGN) and Rayleigh fading channel are derived. Computer simulation confirms that the closed-form expressions for average SEPs of the constellations in the AWGN channel are very accurate. The theoretical SEP upper bounds in the Rayleigh fading channel are very tight. Since the presented constellations have larger minimum Euclidean distance (MED) than the conventional one, error performance can be improved up to 0.9 dB in an AWGN environment. Hence, the proposed constellations and their theoretical analysis can be used as a reference for the development of a wireless communication system with 3D signal constellations.


---

## Body

## 1. Introduction

A signal constellation is one of the essential components to form a digital communication system. Recently, three-dimensional (3D) signal constellations have been studied widely in the fields of wireless communications and optical communications [1–8]. Some 3D constellations and their theoretical symbol error probabilities (SEPs) in additive white Gaussian noise (AWGN) channel have been introduced in [2–7]. Four vertices of the regular tetrahedron are reported as an optimal set of the 4-ary signal constellation. The typical structure for an 8-ary signal set is the regular hexahedron, and its twisted structure is also introduced to increase the minimum Euclidean distance (MED) among symbols [4, 5].Designing a good multilevel 3D signal constellation under some limitations is a complicated issue. Repetitive search algorithms have been used to construct multilevel 3D constellations with constraints such as a constant amplitude and the maximized distance among symbols [1, 2]. As an example, the downhill simplex method generates a set of symbols that are located on the surface of a sphere and maximize the MED [2]. However, the repetitive algorithm cannot guarantee that the constellations generated are either regular or symmetric. Thus, they may not be suitable for practical implementation of a communication system. In [6], a simple method to design 3D signal constellation consisting of 32 or more symbols has been presented. Here, the multilevel constellations are created simply by extending the regular hexahedron in three dimensions. In addition, the constellations are perfectly symmetric with respect to the origin.The 3D 16-ary constellations have also been studied in [2, 5, 7, 8]. In [2], Betti used the repetitive algorithm to compute the coordinates of 16-ary constellation which was optimum in the sense of the MED. Later, Chen [5] introduced another symbol set that had larger MED than Betti’s work. Since a 16-ary phase shift keying (16-PSK) format was simply extended into 3D space, the former had much less computational load than the latter. In the meantime, the 3D 16-ary constellations based on the regular hexahedron were also presented in [7, 8]. They had a dual lattice configuration called cube-in-cube geometry. Since most of them have twisted structures, accurate SEP even in an AWGN channel has not been presented. Thus, theoretical error performance of the 16-ary constellations in the Rayleigh fading channel has never been reported yet.In this paper, we present two 3D 16-ary signal constellations having extended lattice structures for future wireless communications. Although the presented constellations are not perfectly symmetric with respect to the origin, they have larger MEDs than the dual lattice configuration reported in [8]. We also compute the theoretical SEPs of the new constellations in an AWGN channel. In particular, the theoretical upper bounds of the error probability in the Rayleigh fading channel are derived for the first time. Simulation shows that the theoretical analysis on the SEP in the AWGN environment is accurate, and the SEP upper bounds in the Rayleigh fading channel are very tight.The rest of this paper is as follows. In Section2, the structures of new 3D 16-ary constellations are introduced. The theoretical error probabilities of the proposed constellations in AWGN and Rayleigh fading channels are derived in Section 3. And the results of computer simulation are presented and analyzed in Section 4. Finally, some concluding remarks are drawn in Section 5.


## 2. The New 3D 16-Ary Signal Constellations

Repetitive search algorithms such as the downhill simplex method have been exploited to compute the coordinates of symbols in the 3D signal space [1, 2]. One of the main concerns here is to maximize the MED among symbols. The 3D 16-ary constellations generated by the algorithm do not always have a suitable structure for application to a communication system.In this paper, we consider a lattice structure obtained by extending the regular hexahedron for simple design and analysis. The new 16-ary constellations are shown in Figure1. The configuration in Figure 1(a) named L1 is an extended rectangular lattice. To reduce average power of the signal set, 2 outermost symbols (for example, N1 and N2) that are symmetric with respect to the origin among the 18 symbols are removed to form the L2 constellation in Figure 1(b).Figure 1
The new 3D 16-ary signal constellations (a)L1, (b) L2.
(a) (b)Consider thatSk=(x,y,z), k=0,1,…,15, are symbols of a 3D signal constellation, where x, y, and z are the coordinates of a symbol in the 3D signal space. To transmit the 3D signals, the Gram-Schmidt orthogonalization procedure is usually exploited. Thus, the transmitted signal s(t) can be represented as(1)st=xϕxt+yϕyt+zϕzt,where ϕx(t), ϕy(t), and ϕz(t) are the orthogonal basis functions [6, 9]. Assume that the Euclidean distance among adjacent symbols is d. The symbol coordinates of the proposed constellations in Figure 1 can be represented as a 3×16 matrix form as follows:(2)
S
L
1
=
d
2
-
d
2
-
d
2
d
2
3
d
2
-
3
d
2
-
3
d
2
3
d
2
d
2
-
d
2
-
d
2
d
2
3
d
2
-
3
d
2
-
3
d
2
3
d
2
d
2
d
2
-
d
2
-
d
2
d
2
d
2
-
d
2
-
d
2
d
2
d
2
-
d
2
-
d
2
d
2
d
2
-
d
2
-
d
2
d
2
d
2
d
2
d
2
d
2
d
2
d
2
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
,
S
L
2
=
0
d
0
-
d
0
d
-
d
-
d
0
d
0
-
d
0
d
-
d
-
d
0
0
d
0
-
d
-
d
d
-
d
0
0
d
0
-
d
-
d
d
-
d
d
2
d
2
d
2
d
2
d
2
d
2
d
2
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
-
d
2
.Then the average powers of both constellations are computed as(3)Es,L1=816d22+d22+d22+8163d22+d22+d22=74d2,Es,L2=2160+0+d22+816d2+0+d22+616d2+d2+d22=32d2,respectively.


## 3. The Theoretical SEPs of the Presented Constellations

In an AWGN environment, the received signal isr(t)=s(t)+n(t), where(4)nt=nxϕxt+nyϕyt+nzϕztis a Gaussian random process. nx, ny, and nz are independent and identically distributed Gaussian random variables with zero-mean and variance of σ2. Thus, the probability density function (PDF) of the received signals is(5)Px,y,z=12πσ23/2e-x-μx2+y-μy2+z-μz2/2σ2,where μx, μy, and μz are the mean values of 3D components. In general, correct decision of a symbol is made when the recovered symbol falls inside the corresponding decision region. Thus, the probability of correct decision for a symbol can be computed as(6)Pc=px1<x<x2py1<y<y2pz1<z<z2,where x1, x2, y1, y2, z1, and z2 are decision boundaries for a symbol. As the constellations in Figure 1 are symmetric, the projection of the decision regions on the xy plane shown in Figure 2 can be used to compute the probability of correct decision. A pair of the outermost points represented as N1 and N2 in Figure 1(b) is not included in Figure 2 because they are not the elements of L2 constellation.Figure 2
Thexy plane projection of the decision regions (a) L1, (b) L2.
(a) (b)


### 3.1. AWGN Channel

ForL1 constellation, the probabilities of correct decision of S0=(d/2,d/2,d/2) and S1=(3d/2,d/2,d/2) with the decision region COAP and PAB in Figure 2(a), respectively, can be calculated as(7)Pc0,L1=p0<x<dp0<y<∞p0<z<∞=1-2q1-q2,Pc1,L1=pd<x<∞p0<y<∞p0<z<∞=1-q3,where q=Q(d/2σ). Q(·) is the Gaussian Q-function [9]. According to (7), the average SEP of L1 can be computed as(8)PL1γs=1-1168Pc0,L1+8Pc1,L1=72q-4q2+32q3=72Q27γs-4Q227γs+32Q327γs,where γs=Es,L1/N0 represents signal-to-noise ratio (SNR). N0=2σ2 is the single-sided power spectral density of the AWGN channel.In Figure2(b), the decision regions of S0=(0,0,d/2), S1=(-d,d,d/2), and S2=(-d,0,d/2) are ABCD, GBH, and HBCI, respectively. Thus, the correct decision probabilities of S0, S1, and S2 are given as(9)Pc0,L2=p-d2<x<d2p-d2<y<d2p0<z<∞=1-q1-2q2,Pc1,L2=p-∞<x<-d2pd2<y<∞p0<z<∞=1-q3,Pc2,L2=p-∞<x<-d2p-d2<y<d2p0<z<∞=1-2q1-q2,respectively. The decision region of S3=(d,0,d/2) is PADJ which is equivalent to EADJ+FAE/2. Since the probability of EADJ is the same as Pc2,L2 in (9), the correct decision probability of S3 is(10)Pc3,L2=Pc2,L2+12pd2<x<∞pd2<y<∞p0<z<∞=1-32q1-q2.Thus, the theoretical SEP of L2 constellation is computed as(11)PL2γs=1-1162Pc0,L2+6Pc1,L2+4Pc2,L2+4Pc3,L2=298q-358q2+74q3=298Q13γs-358Q213γs+74Q313γs.


### 3.2. Rayleigh Fading Channel

In a Rayleigh fading channel, as it is well known, SNR is a random variable with PDFfR(γs)=1/Λexp⁡(-γs/Λ), where Λ is the average SNR per symbol. Then the average SEP can be calculated by integrating the error probability in AWGN over fading distribution [10] as follows:(12)PRΛ=∫0∞fRγsPγsdγs.Hence, the average SEP of the constellation in Figure 1(a) over Rayleigh fading channel is(13)PL1′Λ=∫0∞fRγsPL1γsdγs=72∫0∞1Λe-γs/ΛQ27γsdγs-4∫0∞1Λe-γs/ΛQ227γsdγs+32∫0∞1Λe-γs/ΛQ327γsdγs.The integral of the first term of the right-hand side of (13) can be calculated as(14)∫0∞1Λe-γs/ΛQ27γsdγs=∫0π/2∫0∞1πΛe-1/Λ+1/7sin2θγsdγsdθ=1π∫0π/27sin2θΛ+7sin2θdθ=12-12ΛΛ+7,where Q-function is replaced by an integral of the equivalent trigonometric function when the variable v>0 [11], that is,(15)Qv=1π∫0π/2e-v2/2sin2θdθ.And the integral equation defined in [12] is exploited.An equivalent form ofQ2(v) [13] is(16)Q2v=1π∫0π/4e-v2/2sin2θdθ.Applying (16) to the second integral in (13), we have(17)∫0∞1Λe-γs/ΛQ227γsdγs=∫0π/4∫0∞1πΛe-1/Λ+1/7sin2θγsdγsdθ=1π∫0π/47sin2θΛ+7sin2θdθ=14-1πΛΛ+7tan-1Λ+7Λ.A trigonometric function equivalent to Q3(v) in (13) is too complex to calculate an exact expression [14]. To compute the last integral term in (13), we use the Chernoff bound of Q(v)≤1/2exp⁡-v2/2. Then an upper bound of the third integral in (13) can be obtained as follows:(18)∫0∞1Λe-γs/ΛQ327γsdγs≤∫0∞1Λe-γs/Λ12e-γs/73dγs=783Λ+7.Hence, the average SEP in (13) is bounded as(19)PL1′Λ≤7212-12ΛΛ+7-414-1πΛΛ+7tan-1Λ+7Λ+32783Λ+7=34+21163Λ+7-74ΛΛ+7+4πΛΛ+7tan-1Λ+7Λ.In the case of L2 constellation, the average SEP over Rayleigh fading channel can be computed as(20)PL2′Λ=∫0∞fRγsPL2γsdγs=298∫0∞1Λe-γs/ΛQ13γsdγs-358∫0∞1Λe-γs/ΛQ213γsdγs+74∫0∞1Λe-γs/ΛQ313γsdγs.Referring to the calculation processes of (14), (17), and (18), the integral terms in (20) can be computed as(21)∫0∞1Λe-γs/ΛQ13γsdγs=12-12ΛΛ+6,∫0∞1Λe-γs/ΛQ213γsdγs=14-1πΛΛ+6tan-1Λ+6Λ,∫0∞1Λe-γs/ΛQ313γsdγs≤14Λ+2.Thus, the theoretical upper bound of the average SEP in (20) can be represented as(22)PL2′Λ≤29812-12ΛΛ+6-35814-1πΛΛ+6tan-1Λ+6Λ+7414Λ+2=2332+716Λ+2-2916ΛΛ+6+358πΛΛ+6tan-1Λ+6Λ.


## 3.1. AWGN Channel

ForL1 constellation, the probabilities of correct decision of S0=(d/2,d/2,d/2) and S1=(3d/2,d/2,d/2) with the decision region COAP and PAB in Figure 2(a), respectively, can be calculated as(7)Pc0,L1=p0<x<dp0<y<∞p0<z<∞=1-2q1-q2,Pc1,L1=pd<x<∞p0<y<∞p0<z<∞=1-q3,where q=Q(d/2σ). Q(·) is the Gaussian Q-function [9]. According to (7), the average SEP of L1 can be computed as(8)PL1γs=1-1168Pc0,L1+8Pc1,L1=72q-4q2+32q3=72Q27γs-4Q227γs+32Q327γs,where γs=Es,L1/N0 represents signal-to-noise ratio (SNR). N0=2σ2 is the single-sided power spectral density of the AWGN channel.In Figure2(b), the decision regions of S0=(0,0,d/2), S1=(-d,d,d/2), and S2=(-d,0,d/2) are ABCD, GBH, and HBCI, respectively. Thus, the correct decision probabilities of S0, S1, and S2 are given as(9)Pc0,L2=p-d2<x<d2p-d2<y<d2p0<z<∞=1-q1-2q2,Pc1,L2=p-∞<x<-d2pd2<y<∞p0<z<∞=1-q3,Pc2,L2=p-∞<x<-d2p-d2<y<d2p0<z<∞=1-2q1-q2,respectively. The decision region of S3=(d,0,d/2) is PADJ which is equivalent to EADJ+FAE/2. Since the probability of EADJ is the same as Pc2,L2 in (9), the correct decision probability of S3 is(10)Pc3,L2=Pc2,L2+12pd2<x<∞pd2<y<∞p0<z<∞=1-32q1-q2.Thus, the theoretical SEP of L2 constellation is computed as(11)PL2γs=1-1162Pc0,L2+6Pc1,L2+4Pc2,L2+4Pc3,L2=298q-358q2+74q3=298Q13γs-358Q213γs+74Q313γs.


## 3.2. Rayleigh Fading Channel

In a Rayleigh fading channel, as it is well known, SNR is a random variable with PDFfR(γs)=1/Λexp⁡(-γs/Λ), where Λ is the average SNR per symbol. Then the average SEP can be calculated by integrating the error probability in AWGN over fading distribution [10] as follows:(12)PRΛ=∫0∞fRγsPγsdγs.Hence, the average SEP of the constellation in Figure 1(a) over Rayleigh fading channel is(13)PL1′Λ=∫0∞fRγsPL1γsdγs=72∫0∞1Λe-γs/ΛQ27γsdγs-4∫0∞1Λe-γs/ΛQ227γsdγs+32∫0∞1Λe-γs/ΛQ327γsdγs.The integral of the first term of the right-hand side of (13) can be calculated as(14)∫0∞1Λe-γs/ΛQ27γsdγs=∫0π/2∫0∞1πΛe-1/Λ+1/7sin2θγsdγsdθ=1π∫0π/27sin2θΛ+7sin2θdθ=12-12ΛΛ+7,where Q-function is replaced by an integral of the equivalent trigonometric function when the variable v>0 [11], that is,(15)Qv=1π∫0π/2e-v2/2sin2θdθ.And the integral equation defined in [12] is exploited.An equivalent form ofQ2(v) [13] is(16)Q2v=1π∫0π/4e-v2/2sin2θdθ.Applying (16) to the second integral in (13), we have(17)∫0∞1Λe-γs/ΛQ227γsdγs=∫0π/4∫0∞1πΛe-1/Λ+1/7sin2θγsdγsdθ=1π∫0π/47sin2θΛ+7sin2θdθ=14-1πΛΛ+7tan-1Λ+7Λ.A trigonometric function equivalent to Q3(v) in (13) is too complex to calculate an exact expression [14]. To compute the last integral term in (13), we use the Chernoff bound of Q(v)≤1/2exp⁡-v2/2. Then an upper bound of the third integral in (13) can be obtained as follows:(18)∫0∞1Λe-γs/ΛQ327γsdγs≤∫0∞1Λe-γs/Λ12e-γs/73dγs=783Λ+7.Hence, the average SEP in (13) is bounded as(19)PL1′Λ≤7212-12ΛΛ+7-414-1πΛΛ+7tan-1Λ+7Λ+32783Λ+7=34+21163Λ+7-74ΛΛ+7+4πΛΛ+7tan-1Λ+7Λ.In the case of L2 constellation, the average SEP over Rayleigh fading channel can be computed as(20)PL2′Λ=∫0∞fRγsPL2γsdγs=298∫0∞1Λe-γs/ΛQ13γsdγs-358∫0∞1Λe-γs/ΛQ213γsdγs+74∫0∞1Λe-γs/ΛQ313γsdγs.Referring to the calculation processes of (14), (17), and (18), the integral terms in (20) can be computed as(21)∫0∞1Λe-γs/ΛQ13γsdγs=12-12ΛΛ+6,∫0∞1Λe-γs/ΛQ213γsdγs=14-1πΛΛ+6tan-1Λ+6Λ,∫0∞1Λe-γs/ΛQ313γsdγs≤14Λ+2.Thus, the theoretical upper bound of the average SEP in (20) can be represented as(22)PL2′Λ≤29812-12ΛΛ+6-35814-1πΛΛ+6tan-1Λ+6Λ+7414Λ+2=2332+716Λ+2-2916ΛΛ+6+358πΛΛ+6tan-1Λ+6Λ.


## 4. Performance Analysis

The MED of a constellation is one of the key parameters determining error performance of a communication system. The MEDs of the presented 3D 16-ary constellations are compared with Cho’s constellation [8] in Table 1. Here, all constellations are normalized to have the average power of unity. Table 1 shows that the MED of L2 constellation is increased about 10.6% as compared with Cho’s one.Table 1
The MEDs of the presented constellations.
Constellation MED Increment Cho’s 0.7385 L1 0.7559 2.4% L2 0.8165 10.6%To verify the theoretical SEP of the presented constellations in AWGN and Rayleigh fading channels, computer simulation has been carried out. Exploiting the MATLAB which is well-known simulation software, the 3D transmission system and both channel models are designed.108 3D symbols are transmitted by the form of (1), where the mathematical model of the orthogonal basis functions in a vector form is ϕx(t)=1/300T, ϕy(t)=01/30T, ϕz(t)=001/3T for 0≤t<Ts, respectively, where Ts is the symbol duration [9]. The simulation results are plotted in Figure 3. It can be observed that the theoretical SEPs of the presented constellations in the AWGN channel match exactly the simulation results. Hence, the SEP expressions in (8) and (11) seem to be accurate. When the L1 constellation is exploited, the system has almost the same error performance reported in [8]. In the case of L2 constellation, the SEP is improved about 0.9 dB as compared with Cho’s constellation at the SEP of 10-6. Such performance gain is attributed to the increased MED of the L2 constellation.Figure 3
SEPs of the 3D 16-ary signal constellations.In Figure3, we also present the theoretical SEPs of the proposed constellations and simulation results in the Rayleigh fading channel. The error performance obtained by computer simulation is very tightly bounded by the theoretical SEP upper bounds derived in (19) and (22). Especially when the average SNR is larger than 15 dB, two performance curves are almost the same.


## 5. Conclusions

Two 3D 16-ary signal constellations with extended lattice structures are introduced in this paper. In addition, their theoretical SEPs both in AWGN and Rayleigh fading channel are presented. Simulation confirms that the theoretical SEPs in the AWGN channel and SEP upper bounds in the Rayleigh fading channel are very accurate. Due to the increased MED, the new constellations provide better error performance than the conventional one. In addition, since the constellations introduced in this paper have an extended configuration of the regular hexahedron, they are much more practical than the previous ones. The proposed constellations and their exact SEPs derived can be applied for future development of wireless or optical communication systems with 3D constellations.



---

*Source: 7178631-2018-08-13.xml*