## 5 Performance Analysis and Optimization
### Average BEP/SEP
#### 5.3.3. ASEP for MQAM  
In this section, only the ASEP of MQAM for SSC is considered as an example for easier derivation. Hence, by using the MGF-based method, the ASEP of MQAM for SSC can be given as [1]:  
\[P_{e}(E)=\underbrace{\frac{4c}{\pi}\int_{0}^{\frac{\pi}{2}}MGF_{\gamma_{SSC}} \bigg{(}\frac{SQ_{QAM}}{\sin^{2}\theta}\bigg{)}d\theta}_{I_{5}}-\underbrace{ \frac{4c^{2}}{\pi}\int_{0}^{\frac{\pi}{2}}MGF_{\gamma_{SSC}}\bigg{(}\frac{SQ _{QAM}}{\sin^{2}\theta}\bigg{)}d\theta}_{I_{6}}, \tag{79}\]  
where \(c=1-1/\sqrt{\text{M}}\), and \(\text{g}_{\text{QAM}}=3/2(\text{M}-1)\), \(M=4\), \(16\),.... For the first integral term \(I_{5}\) in (79), we utilize the same methods provided in Appendix B. After some mathematical manipulation, we obtain:  
\[ \begin{array}{l}I_{5}\ =\sum\limits_{k=0}^{\infty}\sum\limits_{l=0}^{ \infty}\sum\limits_{l=0}^{\infty}\frac{2 cP_{k}^{2}N_{\text{P}}^{2}(\gamma_{ \text{P}}\gamma_{\text{P}}\gamma_{\text{P}})^{m+k}\mathbb{E}_{1}[\lambda,m+k;1 +m+k,-\eta_{2}\gamma_{\text{T}}]}{\mathbb{E}_{1}[\lambda,m+k;1+m+k,-\eta_{2} \gamma_{\text{T}}]}}G_{2,3}^{3,1}[\text{$g$}_{\text{QAM}}/\eta_{1}\big{|}_{ 0,n+l,0.5}^{1-(m+k),1}]\\ \frac{2 cG_{2,3}^{3,1}[\text{$g$}_{\text{QAM}}/\lambda_{AM}/\lambda_{AM}^{1-n, 1}]}{\sqrt{\pi\text{T}}(m)\Gamma(n)}-\frac{2c\Delta_{T}^{2}\gamma_{\text{P}}^ {m}}{\sqrt{\pi\text{T}}(m\Gamma(n))}G_{1,1;2;1}^{0,1;2;0,1}[\text{$g$}_{\text{QAM}} \gamma_{\text{T}},\gamma_{\text{T}}\Lambda_{2}\big{|}_{-m}^{1-m}\big{|}_{0,0, 5}\big{|}_{0}^{1-(m+n)}\big{]}.\end{array}  \tag{80}\]  
For the second integral term \(I_{6}\) in (79), similar steps are considered as in Appendix B. We use the definition of the multivariable Fox's H-function again in terms of the multiple Mellin-Barnes-type contour integral in ([47], Equation (A.1)), and provide the detailed derivative in Appendix C. Hence, the solution of \(I_{6}\) can be given as:  
\[ \begin{array}{l}I_{6}\ =\sum\limits_{k=0}^{\infty}\sum\limits_{l=0}^{ \infty}\frac{\sqrt{2c^{2}}\rho_{k}^{2}\rho_{k}^{2}(\gamma_{\text{P}}\gamma_{ \text{P}})^{m+k}\mathbb{E}_{1}[\lambda,m+k;1+m+k,-\eta_{2}\gamma_{\text{T}}]} {\mathbb{E}_{1}[\lambda,m+k;1+m+k,-\eta_{2}\gamma_{\text{T}}]}H_{1,1;2;1,1}^{0,1;2;1,1}[\frac{2\text{$g$}_{\text{QAM}}}{\lambda_{1}},-0.5]\big{|}_{(-0.5-1,1 )}^{(0.5-1,1)}\big{|}_{(0.1),(n+l,1)}^{(1-(m+k);1)}\big{|}_{(0.51)}^{(0.51)} \big{|}\\ +\frac{\sqrt{2c^{2}}}{\pi\sqrt{\pi\text{T}}(m\Gamma(n))}H_{1,1;2;1,1}^{0,1;2; 1,1}[\frac{2\text{$g$}_{\text{QAM}}}{\lambda_{AM}},-0.5]\big{|}_{(0.5-0.5-1,1 )}^{(0.5-1,1)}\big{|}_{(0.5-0.5-1,1)}^{(1-m;1)}\big{|}_{(0.51)}^{(0.51)}\\ -\frac{\sqrt{2c^{2}}\Lambda_{T}^{2}\gamma_{\text{T}}^{2}}{\pi\sqrt{\pi\text{T }}(m\Gamma(n))}H_{2,2;0,1;1,1;1}^{0,2;1,1,1}[\text{$g$}_{\text{QAM}}\gamma_{ \text{T}},\gamma_{\text{T}}\Lambda_{2},-0.5]\big{|}_{(-0.5-1,0.1),(-m;1,0)}^{(0. 5-1,1)}\big{|}_{(0.1)}^{(1-m+n)}\big{|}_{(0.1)}^{(0.51)}\big{|},\\ \text{where }\text{H}_{2,0;1,1;1,1}^{0,1;2;0,1;1,1}[\text{$g$}_{\text{QAM}} \gamma_{\text{T}},\gamma_{\text{T}}\Lambda_{2},-0.5]\big{|}_{(-0.5-1,0.1),(-m ;1,0)}^{(0.5-1,0.1)}\big{|}_{(0.1)}^{(1-m-n;1)}\big{|}_{(0.1)}^{(0.51)}\big{|} _{(0.1)}^{}\big{)},\end{array}  \tag{81}\]  
where \(\text{H}_{2,0;1,1;1,1}^{0,1;2;0,1;1,1}[\text{$g$}_{\text{QAM}}\gamma_{\text{T }},\gamma_{\text{T}}\Lambda_{2},-0.5]\big{|}_{(-0.5-1,0.1),(-m;1,0)}^{(0.5-1,1)} \big{|}_{(0.1)}^{(1-m;1)}\big{|}_{(0.1)}^{(1-m-n;1)}\big{|}_{(0.1)}^{(0.51)} \big{|}\).