## 4 System Performance Analysis  
### System Bit Error Rate Analysis  
In an FSO communication system, the average bit error rate of the received signal can be expressed in Equation (15) [16]:  
\[\overline{P_{e}}=\int_{0}^{\infty}BER(SNR)f(h)dh \tag{15}\]  
In this formula, \(BER(SNR)\) is the unconditional bit error rate when the channel signal-to-noise ratio is \(SNR\), and \(f(h)\) is the probability density function of the channel state information \(h\). The instantaneous signal-to-noise ratio of a channel can be defined as \(\gamma=(hl)^{2}/N_{0}\), and the average signal-to-noise ratio can be defined as \(\overline{\gamma}=(E[h]I)^{2}/N_{0}\), where \(I\) is the light intensity of the emission symbol \"1\", \(N_{0}\) is the variance of the channel noise, and \(E[\cdot]\) is the expectation of \(h\). In the case of fully correlated atmospheric turbulence fading, considering only the case of \(E[h]=1\)[17],\(h=\sqrt{\gamma/\overline{\gamma}}\) can be obtained, so Equation (1) can be written as:  
\[f(\gamma)=\frac{2(\alpha\beta)^{(\alpha+\beta)/2}}{\Gamma(\alpha)\Gamma(\beta )}\frac{\gamma^{\frac{\alpha+\beta}{4}-1}}{\overline{\gamma}^{\frac{\alpha+ \beta}{4}}}K_{\alpha-\beta}\bigg{(}2\sqrt{\alpha\beta\sqrt{\frac{\gamma}{ \overline{\gamma}}}}\bigg{)} \tag{16}\]Substitute Equation (16) into Equation (15) to obtain the bit error rate expression of the FSO communication system in 4/16-QAM modulation mode:  
\[\overline{P}_{e}=\frac{(\alpha\beta)^{\frac{\alpha+\beta}{2}}}{4\sqrt{\pi}\Gamma (\alpha)\Gamma(\beta)}\int_{0}^{\infty}BER\Big{(}\sqrt{SNR}\Big{)}\frac{\gamma^ {\frac{\alpha+\beta}{4}-1}}{\gamma^{\frac{\alpha+\beta}{4}}}G_{0,2}^{2,0} \Bigg{[}\alpha\beta\sqrt{\frac{\gamma}{\gamma}}\Bigg{|}_{\frac{\alpha-\beta}{2 },-\frac{\alpha-\beta}{2}}\Bigg{]}d\gamma \tag{17}\]  
In Equation (17), according to document [18], it is possible to obtain a 4/16-QAM modulation of the bit error rate expression for the constellation priority parameter \(\lambda\) as follows:  
\[BER(SNR)=erfc\bigg{(}\sqrt{G(a,b;\lambda,M)\overline{\gamma}}\bigg{)} \tag{18}\]  
\(erfc(\cdot)\) is the complementary error function, where  
\[G(a,b;\lambda,M)=\frac{(a+b\lambda)^{2}}{2\Big{[}1+\lambda\Big{[}\frac{\sqrt{ M}}{2}-1\Big{]}\Big{]}^{2}+\frac{2}{3}\Big{[}\frac{M}{4}-1\Big{]}\lambda^{2}} \tag{19}\]  
In the above equation, \(M\) is the modulation order, \(a\) is a non-negative integer, \(b\) is a positive integer, and the constellation priority parameter \(\lambda=d_{2}/d_{1}\). Substituting Equation (18) into Equation (17), the FSO communication system in 4/16-QAM hierarchical modulation under the bit error rate expression is:  
\[ \begin{array}{l}\overline{P}_{e}(a,b;\lambda,M,\gamma)=\\ \frac{(a\beta)^{\frac{\alpha+\beta}{2}}}{4\sqrt{\pi}\Gamma(a)\Gamma(\beta)} \int_{0}^{\infty}erfc\Big{(}\sqrt{G(a,b;\lambda,M)\overline{\gamma}}\Big{)} \frac{\gamma^{\frac{\alpha+\beta}{4}-1}}{\gamma^{\frac{\alpha+\beta}{4}}}G& \frac{2,0}{0,2}\ \Bigg{[}\alpha\beta\sqrt{\frac{\gamma}{\gamma}}\Bigg{|}_{ \frac{\alpha-\beta}{2},-\frac{\alpha-\beta}{2}}\Bigg{]}d\gamma\end{array}  \tag{20}\]  
In the system shown in Figure 1, the constellation diagram of each source node \(S_{i}\) using a 4/16-QAM hierarchical modulation of the noninverting subflow is shown in Figure 3,  
According to the document [18], the bit error rate of high-priority data from source node \(S_{i}\) to relay node \(R\) can be expressed as:  
\[P\Big{\{}\varepsilon_{S_{i}R}^{S_{i}}|\lambda\,\Big{\}}=\frac{1}{4}\big{(} \overline{P}_{e}(a_{1},b_{1};\lambda,M,\overline{\gamma})+\overline{P}_{e}(a_ {2},b_{2};\lambda,M,\overline{\gamma})\big{)} \tag{21}\]  
Thereinto, \(a_{1}=1\), \(b_{1}=2\) and \(a_{2}=1\), \(b_{2}=0\).