# A Nonbinary LDPC-Coded Probabilistic Shaping Scheme for a Rayleigh Fading Channel
## 3 Average Mutual Information Analysis  
The average mutual information (AMI) is the theoretical upper bound, which could be reliably transmitted for given constellation modulation formats. In this section, The regular Nyquist bit-interleaved coded modulation (BICM)-AMI and the PS BICM-AMI for Rayleigh fading channel are investigated. A Gray-labeled \(M\)-ary QAM constellation consists of two orthogonal \(\sqrt{M}\)-ASK constellations. Considering the \(\sqrt{M}\)-ASK constellations, the amplitude probability distribution \(P_{X}(x)\)is selected from Maxwell-Boltzmann (M-B) distribution [7], and \(P_{X}(x)\propto\exp(-vx^{2})\), where \(x\) denotes amplitude. \(v\) represents the non-negative factor, which can determine the system entropy. When \(v=0.0\), it means that the constellation points are subject to a uniform distribution. When \(v>0.0\), it means that the constellation points are subject to nonuniform distribution. In Figure 2, the amplitude probability distribution for 4-ASK is [3:1]=[0.3505: 0.6495], while the amplitude probability distribution for 8-ASK in Figure 3. is subject to [7:5:3:1]=[0.126429:0.213095:0.301666: 0.358810]. In Figure 4, the amplitude probability distribution for 16-ASK is satisfied to [15:13:11:9:7:5:3:1]=[ 0.0493333: 0.069: 0.0916667:0.116667:0.141333:0.163333:0.18: 0.188667]. The BICM-AMI of the proposed PS system for Rayleigh fading channel can be calculated as  
\[I_{BICM}= \sum\limits_{i=1}^{m}I(c_{i};y|h) \tag{14}\] \[= H(x)-\sum\limits_{i=1}^{m}E_{c_{i},y,h}\left[\log_{2}\frac{\sum \limits_{x\in\mathbf{\Theta}}P(y|\bar{x},h)P(\bar{x})}{\sum\limits_{x\in\mathbf{\Theta} _{c_{i}}}P(y|x,h)P(x)}\right],\]  
where \(H(x)\) represents information entropy, and \(h\) denotes the fading channel coefficients. \(\chi\) represents the constellation set. \(\mathbf{\mathcal{O}}_{c_{i}}\) denotes the constellation subset, where the \(i\)-th bit \(c_{i}\in\{0,1\}\). \(m=log_{2}M\). We give the objective function of the optimized rotation as follows. Optimization criterion: Given an SNR, select the optimal rotation angle \(\theta^{opt}\) to maximize the SE  
\[\theta^{opt}=\operatorname*{arg\,max}_{\theta\in(0,45]}\{I_{BICM}(SNR,\theta)\}. \tag{15}\]  
Figures 2-4 show the SE of different rotated angles of \(\theta\) in 16 QAM, 64 QAM and 256 QAM, respectively, which are obtained from the AMI-optimized criterion by an exhaustive search in Microsoft Visual Studio 2010 software platform. The range of the exhaustive search is from 0 to 45 degrees, and the search interval is set to 0.2 degrees. After passing through the exhaustive search, we can receive the optimal rotated angle \(\theta^{opt}\), which can also be obtained through ROC curves by estimating the cutoff points. In Figure 2, when the SNR is 13 dB, the optimal rotated angles \(\theta^{opt}\) are both 16.8 degrees for the uniform-rotated 16 QAM system and the PS-rotated 16 QAM system. In Figure 3, when the SNR is 19 dB, the optimal rotated angles \(\theta^{opt}\) are 8.6 and 12.6 degrees for the uniform-rotated 64 QAM system and the PS-rotated 64 QAM system, respectively. In Figure 2, when the SNR is 24 dB, the optimal rotated angles \(\theta^{opt}\) are both 3.6 degrees for the uniform-rotated 256 QAM system and the PS-rotated 256 QAM system. Figure 4: SE vs. \(\theta\) in 256 QAM for Rayleigh fading channel. Figure 3: SE vs. \(\theta\) in 64 QAM for Rayleigh fading channel. Figure 2: SE vs. \(\theta\) in 16 QAM for Rayleigh fading channel. 64 QAM and uniform 256 QAM constellations, respectively, which are also used in the DVB-T2 system for fading channel. In this paper, AMI and SE are equivalent. For the 16 QAM system, we compare the SE at 3.2 bits/s/Hz in Figure 5. Compared with the uniform 16 QAM with no rotation system, the uniform-rotated 16 QAM with a 16.8-degree rotation system, the PS 16 QAM with no rotation system and the PS-rotated 16 QAM with a 16.8-degree rotation system can obtain 0.65, 0.12 and 0.75 dB theoretical gains, respectively.