# Two-Dimensional Constellation Shaping in Fiber-Optic Communications
## 2 Typical Constellation Shaping Schemes
### Hybrid Probabilistic-Geometric Shaping  
When more flexible constellation formats are required, especially multi-dimensional QAM formats, HPGS may be an enabling technique, since each constellation point during the optimization process will not be limited to equal probability or uniform-grid locations. Figure 4 shows HPGS-5 QAM and HPGS-9 QAM formats based on Huffman coding [58; 59]. The probability of each symbol is predetermined by the corresponding Huffman tree, and the coordinates of the symbols can be obtained by optimizing the MI. As an illustrative example, the generation process of HPGS-9 QAM constellation can be found below:  
1. Parsing the binary source into nine blocks labeled by unique bit sets {00, 010, 110, 011, 100, 1110, 1111, 1010, 1011}. If the binary sequence is sufficiently long, the resulting blocks should be generated with the probabilities of {\(\textbf{P}(00)=1/4\), \(\textbf{P}(010)=1/8\), \(\textbf{P}(011)=1/8\), \(\textbf{P}(100)=1/8\), \(\textbf{P}(1110)=1/16\), \(\textbf{P}(1111)=1/16\), \(\textbf{P}(1010)=1/16\), \(\textbf{P}(1011)=1/16\)}. Thereafter, the entropy is 3, i.e., there is no entropy loss. 2. Mapping the 9-block sequence to any 9-QAM symbols with the constraints: (i) The 9-ary constellation points with the same probability are uniformly located in the same power layer, (ii) The constellation points with higher probabilities are located at higher power layers. In other words, such 9-ary constellation should be featured with three power layers and 1, 4, and 4 points are equally spaced in each power layer, respectively. Figure 4: Huffman coding (**a**) and the constellation format (**b**) for 5-QAM; Huffman coding (**c**) and the constellation format (**d**) for 9-QAM. Maximizing the MI by iterating over all amplitude ratios and phase differences of such 9-ary constellation. Huffman coding can be treated as a variable-length and prefix-free PS scheme, which can also be uniquely decodable, but it cannot provide flex rate. There are four major problems for this HPGS scheme. Firstly, it is quite challenging to implement Huffman coding when the amount of symbols increase. Secondly, as any other variable-length coding technology, Huffman coding will also suffer from the overflow or underflow problems. Thirdly, the higher-complexity nonbinary FEC coding is required. Fourthly, error propagation may occur if unexpected symbol errors remained after FEC decoding. A more efficient and practical way to generate HPGS-QAM is based on universal probabilistic shaping scheme and GMI-optimized GS scheme [44; 60]. Figure 5 shows the constellation formats for HPGS-32 QAM, as well as the regular/PS-32 QAM. The constellation diagram of the HPGS-32 QAM shown in Figure 5a is generated by the generalized pairwise optimization algorithm. The objective function is maximizing GMI under the constraints of zeros mean amplitude and normalized average power. The two constraints can be expressed as  
\[p_{i}x_{i}=-A-p_{j}x_{j} \tag{4}\]  
\[\left|p_{j}x_{j}+b\right|^{2}+\frac{p_{j}}{p_{i}^{2}}\left|x_{j}\right|^{2}=1-B \tag{5}\]  
where \((x_{i},~{}x_{j})\) is one pair of the M-ary constellation (\(M=32\) in this case), \(p_{i}\) is the probability of \(x_{i}\), \(A=\sum\limits_{k=1,k\
eq i\)\(\
eq k\
eq j}^{M}p_{k}x_{k}\), and \(B=\sum\limits_{k=1,k\
eq i,k\
eq j}^{M}p_{k}|x_{k}|^{2}\). The objective of maximizing GMI will be searching the \((x_{i},~{}x_{j})\) pair over a hypersphere with the center and radius determined by the other \(M-2\) constellation points. Such generalized pairwise optimization algorithm can converge to the final steady state after iterating all \(M(M-1)/2\) pairs. As a rule of thumb, we suggest start the iteration with the regular-32 QAM, and the optimal HPGS-32 QAM shown in Figure 5a can be obtained within 1000 iterations. Such HPGS constellation could provide the trade-off between the number of nearest symbols and their Hamming distance. Any constellation format shown in Figure 5 does not belong to the square QAM category.