Applied Sciences  
Design and Performance Analysis of Probabilistically Shaped QAM Signals for Coherent FSO Systems with Gamma-Gamma Turbulence Channels  
[PERSON] 122  
[PERSON] 122  
[PERSON] 1  
[PERSON] 122  
[PERSON] 1\({}^{\text{\textregistered}}\)  
1 Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Dongnanhu Road 3888, Changchun 130033, China; [EMAIL_ADDRESS] (R.T.); jy018922316126.com (S.M. ); [EMAIL_ADDRESS] (Y.G. ); [EMAIL_ADDRESS] (XL.) 2 University of Chinese Academy of Sciences, Beijing 100049, China  
* Correspondence: [EMAIL_ADDRESS]; Tel: +86-139-4492-1503  
Footnote 1: [[https://www.mdpi.com/journal/applsci](https://www.mdpi.com/journal/applsci)]([https://www.mdpi.com/journal/applsci](https://www.mdpi.com/journal/applsci))  
Footnote 2: [[https://www.mdpi.com/journals/pdf/10/07](https://www.mdpi.com/journals/pdf/10/07)]([https://www.mdpi.com/journals/pdf/10/07](https://www.mdpi.com/journals/pdf/10/07))  
optimized method under the Rayleigh fading channel. In [12], [PERSON] et al. proposed two succinct PS schemes over 16 QAM FSO systems, which are based on mapping rules. Secondly, various distribution matching schemes applied to PS have been presented [13; 14; 15; 16]. For instance, constant composition distribution matching (CCDM) [13], hierarchical distribution matching (HiDM) [14], multiset-partition distribution matching (MPDM) [15] and multi-level distribution matching (MLDM) [16]. Thirdly, the design of forward error correction (FEC) codes suitable for PS schemes were proposed [17; 18]. Meanwhile, [PERSON] et al. [17] proposed a coded modulation scheme based on binary staircase codes as the FEC code, and achieved 0.56-1.55 dB gains of generalized mutual information (GMI) using only a single staircase code. In [18], [PERSON] et al. designed protograph-based low-density parity-check (LDPC) codes for PS techniques.The tailored protograph-based LDPC codes have a significant improvement over the standard schemes. Lastly, the structure of the PS implementation was discussed [19]. In [19], [PERSON] et al. proposed the probabilistic amplitude shaping (PAS) scheme, which concatenates the distribution matching and FEC code, without distorting the shaped distribution and has no additional error bursts. In this paper, we mainly focus on designing the input distributions suitable for FSO system to improve their capacity. To the best of the authors' knowledge, there are two main kinds of methods to find the optimal distribution that maximizes capacity. First, the researchers choose the objective function (the mutual information or the achievable information rate) and optimize the input distributions by convex optimization [11; 20]. The mutual information is discussed as the objective function in [11] and the alternating optimization is developed. In order to achieve the capacity and capacity-achieving distribution by the interior-point algorithm, the achievable information rate under the FSO channel is utilized in [20]. Secondly, the optimization algorithms based on machine learning are proposed [21; 22]. [PERSON] et al. adopted a known sequence, solving the optimization problem based on error counting at the receiver [21]. While [PERSON] et al. introduced a machine learning method to solve the optimization problem of PS in [22]. The Gumbel-Softmax trick is used to train a sampling mechanism for symbols drawn from the finite set. Motivated by the above methods, we propose a coded modulation scheme based on the probabilistic shaped QAM signal to achieve the capacity of the FSO channel. We choose the heuristic algorithm to optimize the probability mass function (PMF) of the discrete constellations for maximizing the achievable information rate (AIR) of bit-metric decoding (BMD) in a coherent FSO system, which is modeled as a Gamma-Gamma channel for the first time.