<figure><img src="image_1.png" /></figure>

HAL
open science

# Multi-dimensional Energy Limitation in Sphere Shaping for Nonlinear Interference Noise Mitigation

Jingtian Liu, Élie Awwad, Yves Jaouën

► To cite this version:

Jingtian Liu, Élie Awwad, Yves Jaouën. Multi-dimensional Energy Limitation in Sphere Shaping for Nonlinear Interference Noise Mitigation. Asian Communications and Photonics Conference, Nov 2023, Wuhan, China. hal-04311678

HAL Id: hal-04311678

https://telecom-paris.hal.science/hal-04311678v1

Submitted on 28 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC0 1.0 - Universal - International License
# Multi-dimensional Energy Limitation in Sphere Shaping for Nonlinear Interference Noise Mitigation

Jingtian Liu

Télécom Paris, Institut Polytechnique de Paris

19 place Marguerite Perey, 91120 Palaiseau, FRANCE

jingtian.liu@telecom-paris.fr

Élie Awwad

Télécom Paris, Institut Polytechnique de Paris

19 place Marguerite Perey, 91120 Palaiseau, FRANCE

elie.awwad@telecom-paris.fr

Yves Jaouën

Télécom Paris, Institut Polytechnique de Paris

19 place Marguerite Perey, 91120 Palaiseau, FRANCE

yves.jaouen@telecom-paris.fr

**Abstract**—We propose Four-Dimensional (4D) energy limit enumerative sphere shaping (ESS) of *M*-QAM signaling to minimize rate loss and improve the transmission performance over non-linear WDM optical-fiber systems. Simulation results show that the proposed scheme outperforms the conventional ESS by 0.19 bit/4D-symbol in achievable information rate over a 205-km single-span link and a WDM transmission of five polarization-division-multiplexed channels with 400-Gbit/s net rate per channel. We also study the achieved performance over several shaping block lengths and show that the achieved gains do not scale well over multi-span systems.

**Index Terms**—Probabilistic constellation shaping, enumerative sphere shaping, coherent transmission systems, modulation and coding schemes, non-linear Kerr effect.

## I. INTRODUCTION

Probabilistic amplitude shaping (PAS) has demonstrated excellent performance over linear AWGN channels. PAS can be implemented through several approaches, among which we cite enumerative sphere shaping (ESS) [1] and constant composition [2] distribution matching (CCDM). When it comes to dealing with non-linear interference (NLI) in optical fiber transmission, PAS may be more susceptible to performance degradation [3]. Recently, several works aimed at improving the non-linear performance of PAS schemes by limiting the 1-dimensional (1D) amplitude variance or kurtosis of ESS schemes [4], [5] or by limiting the 2D energy dispersion index (EDI) of CCDM schemes [6]. Inspired by these advancements, we propose a new distribution matching approach to alleviate the distortions introduced by Kerr effects in optical fiber systems. Our design targets lowering 4D energy variations where the four dimensions refer to the in-phase and quadrature parts of two orthogonal polarization tributaries of the transmitted signal. The proposed scheme is based on polarization-division-multiplexed (PDM) quadrature-and-amplitude (QAM) signaling along with ESS for amplitude shaping. ESS is chosen for its reduced rate loss when it is applied over short shaping block lengths compared to CCDM [1].

In this paper, we propose a 4D energy limit over ESS shaping to achieve either an improved non-linear tolerance or a lower rate loss than state-of-the-art kurtosis-ESS (K-ESS) [4]

and band-limited ESS (B-ESS) [5], and this is done by tuning the design parameters of the new scheme. Simulation results show that the proposed shaping outperforms conventional ESS in two different scenarios when applied over 64-QAM. We measure a gain of 0.19 bit per 4D-symbol in the achievable information rate (AIR) over a 205-km single-span standard single-mode-fiber (SSMF) link and a WDM transmission of five channels with a net rate of 400 Gbit/s per wavelength at 50 GBaud and a block length of $N = 108$. Smaller gains are observed for larger block lengths over a 5-span 400-km link and a WDM transmission of five 110 GBaud channels.

## II. PRINCIPLE

ESS can be generated and applied over 1, 2, 4 or even more dimensions. In Fig. 3 from [7], the authors show an illustration of 1D, 2D, and 4D symbol mapping strategies of ESS schemes. For 1D mapping, 4D symbols are shaped using 4 independent amplitude sequences (one ESS per dimension). For 2D mapping, each polarization, namely *X* and *Y*, uses one ESS-shaped amplitude sequence. For 4D mapping, a single-shaped sequence is used for a wavelength channel. The latter mapping offers the advantage of reducing the probability of having equal amplitudes at the same time slot; hence, high peak power values appear less frequently and lead to lower non-linear distortions. This advantage was demonstrated in several works [3], [4], [7].

While both K-ESS [4] and B-ESS [5] (that we call 1D-BL-ESS in the following) use 4D mapping, they introduce energy constraints over a single dimension to limit the variations of each 1D amplitude. However, the strength of non-linear effects depends on the variations of energy levels of the 4D symbols in consecutive time slots. Hence, by constraining energy variations of 4D symbols, we can improve the performance and achieve a better trade-off between linear and non-linear gains. The following toy example illustrates this advantage. Consider an ESS with a block length $N = 8$ and two amplitude sequences (3, 3, 3, 3, 1, 1, 1, 1), highlighted in yellow in Fig. 1, and (1, 3, 1, 3, 1, 3, 1, 3), highlighted in green. These sequences have the same 1D average energy,
<figure><img src="image_3.png" /><figcaption>Fig. 1. 4D-Band-limited-trellis using four amplitudes {1, 3, 5, 7}, $E_{max} = 48$, and block length $N = 8$.</figcaption></figure>

<table><caption>TABLE I
METRICS FOR FIVE DIFFERENT ESS SCHEMES WITH $N = 108$, RATE = 1.5/1D AMPLITUDE</caption><thead><tr><th></th><th>ESS</th><th>1D-BL-ESS</th><th>4D-BL-ESS (Linear)</th><th>4D-BL-ESS (Nonlinear)</th><th>K-ESS</th></tr></thead><tbody><tr><td>E<sub>max</sub></td><td>860</td><td>996</td><td>948</td><td>996</td><td>1156</td></tr><tr><td>E&#773;<sub>1D</sub></td><td>7.85</td><td>9.14</td><td>8.63</td><td>9.09</td><td>8.27</td></tr><tr><td>σ<sup>2</sup><sub>1D</sub></td><td>1.75</td><td>1.49</td><td>1.52</td><td>1.45</td><td>1.16</td></tr><tr><td>μ<sub>4,2D</sub></td><td>1.87</td><td>1.71</td><td>1.69</td><td>1.64</td><td>1.57</td></tr><tr><td>EDI (W&#773;<sub>1</sub> = 42)</td><td>3.13</td><td>0.65</td><td>0.72</td><td>0.48</td><td>3.34</td></tr><tr><td>μ&#773;<sub>4,E<sub>4D</sub></sub> (W<sub>2</sub> = 54)</td><td>3.63</td><td>2.8</td><td>2.84</td><td>2.69</td><td>3.17</td></tr><tr><td>Rate loss [bit/1D]</td><td>0.03</td><td>0.13</td><td>0.097</td><td>0.13</td><td>0.07</td></tr></tbody></table>

variance, and kurtosis, and generate the two consecutive 4D-symbols: [($\pm 3 \pm 3i, \pm 3 \pm 3i)^T$, ($\pm 1 \pm 1i, \pm 1 \pm 1i)^T$] and [($\pm 1 \pm 3i, \pm 1 \pm 3i)^T$, ($\pm 1 \pm 3i, \pm 1 \pm 3i)^T$] respectively where the first symbol is attributed to the polarization X and the second symbol to the polarization Y in each pair, and $(\cdot)^T$ is the transpose of $(\cdot)$. Symbols from the first sequence show a large temporal 4D energy fluctuation from 36 to 4 while symbols from the second one have a constant 4D energy equal to 20.

In Fig. 1, we illustrate the new 4D-band-limited ESS (4D-BL-ESS) scheme through a trellis using 64-QAM that has 4 amplitudes in 1D: {1, 3, 5, 7}, a block length $N = 8$, and a maximum accumulated energy limit $E_{max} = 48$. $n$ and $L$ are respectively the column and row indices of the nodes in the trellis. Each path in the trellis correspond to a possible amplitude sequence. In each node, the number in black $e$ is the intermediate accumulated energy up to that state, and the number in red $T_n^e$ is the number of paths that lead from this state to a final state at $n = N$. The accumulated 1D energy in the 4-th and 8-th positions (red nodes) corresponds to the accumulated 4D energy in the 1-st and 2-nd time slots respectively. Sequences in grey are deleted based on a 4D-energy constraint instead of a 1D-constraint as initially done in [5]. For this example, the constraint is: $4 + 28(i - 1) \le E_{4D,i} \le 28 + 28(i - 1)$ where $E_{4D,i}$ is the accumulated 4D energy in $i$-th time slot. The construction used

in the toy example can be generalized by tuning the constraints on the accumulated energy:

$$A + iK_1 \le E_{4D,i} \le \min(B + iK_2, E_{max}) \quad (1)$$

where the parameters $A, B, K_1, K_2$ define the upper and lower limits of 4D energies in the trellis.

For ease of comparison between different schemes, we evaluate several metrics related to the linear and non-linear performance using the same signaling parameters as in [4], [5], i.e. a shaping rate of 1.5 bits/amplitude, a block length $N = 108$, an LDPC FEC code with a rate of $r_c = 5/6$ and achieve a net bit rate of 8 bits/4D symbol which is calculated as in [8]. For conventional ESS, the net bit rate for each amplitude is determined only by the chosen value of $E_{max}$. However, for all three band-limited ESS and the K-ESS, we start with a larger energy limit $E_{max}$, hence with a larger bit rate, and we then reduce the bit rate through constraints such as band limiting the energy or limiting the kurtosis until we reach the same net rate as the one of conventional ESS. In [4], [5], the average energy of 1D amplitudes $E_{1D}$ is used to assess the linear penalty, also the variance of 1D amplitudes $\sigma_{1D}^2$ and the kurtosis of 2D amplitudes $\mu_{4,2D}$ are the used metrics to assess the non-linear gain. Hence, in Tab.I, we compute these metrics for five schemes: the conventional ESS [1], the kurtosis-ESS (K-ESS) [4], the 1D-BL-ESS [5] and two proposed 4D-BL schemes. We also add two 4D
<figure><img src="image_4.png" /><figcaption>Fig. 2. Left: Electrical SNR versus Power per channel in the non-linear channel. Right: GMI versus Power per channel. Simulation of conventional ESS, K-ESS, 1D-BL-ESS, 4D-BL-ESS(Linear), and 4D-BL-ESS(Nonlinear) in 50 GBaud over 1 × 205 km SSMF link, 5 WDM channels, 55 GHz channel spacing, net rate per channel 400 Gbit/s.</figcaption></figure>

metrics: the energy dispersion index (EDI) defined in [9], and the windowed kurtosis defined in [10]. These two 4D metrics were shown to be better suited for assessing non-linear gains of non-iid (independent and identically distributed) signaling schemes such as short block-length PAS modulations [9]–[11]. The length of the averaging windows $W_1$ for the EDI and $W_2$ for the windowed kurtosis are computed as Eq. (2) in [9] and Eq. (7) in [10]. Their values in Table I are computed for the first transmission scenario over a single-span 205-km SSMF link simulated in the next section. In this paper, we design and study two 4D-BL schemes: the first one is optimized for the linear regime, is denoted 4D-BL-ESS (Linear) and has a lower accumulated energy threshold $E_{max}$ than its 1D-BL counterpart, thus a lower rate loss (linear gain) and almost the same energy variations as 1D-BL-ESS; and the second scheme is optimized for the non-linear regime, is denoted 4D-BL-ESS (Nonlinear) and has the same accumulated energy threshold $E_{max}$ as 1D-BL-ESS (hence, the same rate loss too) but it has lower 4D energy variations. The rate loss for each scheme is computed and reported in the last row of Table I. We can see that all band-limited ESS schemes show lower energy variations than the conventional ESS (first column), but have a higher rate loss due to the addition of energy constraints that reduce the number of allowed amplitude combinations. At the same net rate, K-ESS has the lowest 1D energy variation, but the highest 4D energy variation, which makes its nonlinear performance worse than the other schemes as will be shown in the next section.

### III. SIMULATION SETUP AND RESULTS

#### A. Single-span 205-km 50GBaud transmission

We evaluate the performance of the two proposed 4D-BL-ESS with $N = 108$ under the same conditions as the ones used in [4], [5]. The transmitted signals are 5 dual-polarization WDM channels, with a baud rate of 50 GBaud per channel. The raw data rate is 600 Gbit/s. For all ESS schemes, we use an LDPC code of length $n = 64800$ bits from the DVB-S2

standard with a rate of $r_c = 5/6$. The achieved net bit rate is 8 bits per 4D symbol. The channel spacing is 55 GHz. Root-raised-cosine pulse shaping is applied with a roll-off factor of 0.1. The signal propagation is simulated through the split-step Fourier method based on the nonlinear Schrödinger and Manakov equations. SSMF fiber segments are simulated with a chromatic dispersion coefficient $D = 17\text{ps/nm/km}$, non-linearity coefficient $\gamma = 1.3(\text{W}\cdot\text{km})^{-1}$, and an attenuation $\alpha = 0.2 \text{ dB/km}$ at $\lambda = 1550 \text{ nm}$. Polarization mode dispersion (PMD) is added with $0.04 \text{ ps}/\sqrt{\text{km}}$. The link is a single span of 205 km. No laser phase noise is added at this point. After propagation, the central channel is filtered. The signal undergoes matched filtering, chromatic dispersion compensation, and genie-aided multiple-input-multiple-output (MIMO) channel equalization. Finally, we use a fully-datalaided phase filter with an averaging window of 64 symbols to compensate for the phase rotation induced by cross-phase modulation (XPM), as described in [12] and we compute the electric SNR and the achievable information rate (AIR) from the equalized signal. The electric signal-to-noise ratio $\text{SNR}_{\text{elec}}$ is the SNR computed from the constellation at the end of the DSP chain at the receiver side (i.e., at the input of the decision circuit) and the AIR is computed as Eq. (10) in [13]. For each launch power, five different channel realizations are simulated.

In the left part of Fig. 2, we show the electric SNR versus the launch power per channel for the five ESS schemes. We see that the 1D-BL, 4D-BL (Linear), and 4D-BL (Nonlinear) ESS show respectively a 1 dB, 0.9 dB, and 1.1 dB increase in SNR at a power of 10 dBm compared to conventional ESS, and a 1 dB enhancement in optimal launch power. In the right part of Fig. 2, we show the results in terms of AIR computed as Eq. (1) in [14] per 4D symbol versus the optical power. The 4D-BL (linear) and (Nonlinear) show the best AIR. At a launch power of 10 dBm, the bit rate is increased by 0.07 bit/4D compared to the 1D-BL-ESS, and by 0.19 bit/4D compared to conventional ESS. 4D-BL-ESS (Nonlinear) also shows 0.13 bit/4D gain compared to the 1D-BL-ESS in highly
<figure><img src="image_5.png" /><figcaption>Fig. 3. Simulation of conventional ESS, 1D-BL-ESS, 4D-BL-ESS(Linear), and 4D-BL-ESS(Nonlinear) for a 110 GBaud transmission over 5 × 80 km SSMF link, 5 WDM channels, 112.5 GHz channel spacing, net rate per channel: 838 Gbit/s. Electrical SNR (left) and AIR (middle) vs block length at optimal power (3 dBm). EDI (right) with $W_1 = 200$ versus block length.</figcaption></figure>

non-linear regime. While K-ESS shows the lowest 1D energy variation as can be seen from Table I, its 4D energy variation is higher than those of the BL-ESS schemes, which is compliant with the measured SNR and AIR performance in Fig. 2.

### B. 5 × 80-km multi-span 110GBaud transmission

Next, we compare four schemes: conventional, 1D-BL and 4D-BL ESS (Linear and Nonlinear) in a new scenario at a higher baud rate and over a multi-span link configuration for different shaping block lengths $N$, to assess the scaling of the linear and non-linear gains. We consider a 110 GBaud transmission over 5 × 80 km SSMF link, 5 WDM channels with a 112.5 GHz channel spacing. We also consider a laser linewidth of 100 kHz at each of the transmitter side and receiver side. The net rate per channel is 838 Gbit/s (880 Gbit/s with 5% pilot overhead). We compensate the added phase noise using a pilot-aided phase estimation [15] with a 5% overhead. The other DSP blocks are the same as the ones used in the previous section. We study the impact of the block length $N$ on the performance in terms of electric SNR and AIR, both measured at the optimal launch power of 3 dBm.

In Fig. 3, we show the variation of the electric SNR, the AIR and the EDI with respect to the shaping block length $N = \{60, 108, 200, 300\}$. For each block length, five different channel realizations are simulated. First, we can see, by comparing the leftmost and the rightmost figures, that the measured SNR is inversely correlated to EDI changes. However, we also notice that EDI only gives a rough prediction of the performance as it fails to capture small differences such as the gap between the performance of 1D-BL-ESS and 4D-BL-ESS (Linear). Second, we can see that, at a short block length $N = 60$, all BL-ESS schemes fail to achieve a significant gain in SNR as the non-linear gains from energy limitation are still small compared to the conventional ESS. Third, when the rate loss decreases as the block length increases, and consequently the linear penalty decreases, the energy fluctuations of conventional ESS reduce the SNR while the SNR of the BL-ESS schemes remain stable up to $N = 300$. We did not increase the block length beyond 300 to maintain a low complexity for the distribution matching. Finally, if we assess the AIR gains through the middle figure, we see

that a net performance gain of BL-ESS schemes with respect to conventional ESS is observed beyond a block length of 200. However, all the AIR values achieved by the BL-ESS schemes remain lower than the AIR of conventional ESS for $N \le 200$ due to their higher rate loss. Our explanation of the limited gains of the BL-ESS for longer distances and higher baud rates is the following: despite the optimization of the signal characteristics at the transmitter side for low NLI distortions through the addition of energy constraints, the chromatic dispersion gradually destroys these characteristics during the transmission. Hence, the band-limited ESS can only obtain limited nonlinear gains over long-distance and high baud-rate transmission (gains achieved over the first few spans), and the gain in AIR is also limited due to the higher rate loss of the band-limited ESS.

## IV. CONCLUSION

We proposed a new design of ESS schemes that constrains energy variations in higher dimensions (four dimensions in this work) and that can be tuned to boost either linear or nonlinear gains. The new scheme achieves higher gains than state-of-the-art ESS schemes and is particularly interesting for short-distance links as the trade-off between the achieved linear and non-linear gains becomes harder to meet for longer distances because of the accumulation of chromatic dispersion that increases again the energy variations of the optical signal. Besides, our simulation results show that windowed 4D metrics of energy variations predict the non-linear performance of ESS schemes better than 1D and 2D metrics; however, their prediction is not yet accurate enough and the search for better 4D metrics is still an interesting problem to solve.

## V. ACKNOWLEDGEMENTS

This work was supported by Huawei Technologies France.

## REFERENCES

[1] Y. C. Gültekin, W. J. van Houtum, A. G. Koppelaar, F. M. Willems, J. Wim et al., "Enumerative sphere shaping for wireless communications with short packets," *IEEE Transactions on Wireless Communications*, vol. 19, no. 2, pp. 1098–1112, 2019.

[2] P. Schulte and G. Böcherer, "Constant composition distribution matching," *IEEE Transactions on Information Theory*, vol. 62, no. 1, pp. 430–434, 2015.
[3] T. Fehenberger, D. S. Millar, T. Koike-Akino, K. Kojima, K. Parsons, and H. Griesser, "Analysis of nonlinear fiber interactions for finite-length constant-composition sequences," *Journal of Lightwave Technology*, vol. 38, no. 2, pp. 457-465, 2019.

[4] Y. C. Gültekin, A. Alvarado, O. Vassilieva, I. Kim, P. Palacharla, C. M. Okonkwo, and F. M. Willems, "Kurtosis-limited sphere shaping for nonlinear interference noise reduction in optical channels," *Journal of Lightwave Technology*, vol. 40, no. 1, pp. 101-112, 2021.

[5] ——, "Mitigating nonlinear interference by limiting energy variations in sphere shaping," in *Optical Fiber Communication Conference*. Optica Publishing Group, 2022, pp. Th3F-2.

[6] K. Wu, G. Liga, A. Sheikh, Y. C. Gültekin, F. M. Willems, and A. Alvarado, "List-encoding CCDM: A nonlinearity-tolerant shaper aided by energy dispersion index," *Journal of Lightwave Technology*, vol. 40, no. 4, pp. 1064-1071, 2022.

[7] P. Skvortcov, I. Phillips, W. Forysiak, T. Koike-Akino, K. Kojima, K. Parsons, and D. S. Millar, "Huffman-coded sphere shaping for extended-reach single-span links," *IEEE Journal of Selected Topics in Quantum Electronics*, vol. 27, no. 3, pp. 1-15, 2021.

[8] I. Schmalen, "Probabilistic constellation shaping: Challenges and opportunities for forward error correction," in *2018 Optical Fiber Communications Conference and Exposition (OFC)*, 2018, pp. 1-3.

[9] K. Wu, G. Liga, A. Sheikh, F. M. Willems, and A. Alvarado, "Temporal energy analysis of symbol sequences for fiber nonlinear interference modelling via energy dispersion index," *Journal of Lightwave Technology*, vol. 39, no. 18, pp. 5766-5782, 2021.

[10] J. Cho, X. Chen, G. Raybon, D. Che, E. Burrows, S. Olsson, and R. Tkach, "Shaping lightwaves in time and frequency for optical fiber communication," *Nature communications*, vol. 13, no. 1, p. 785, 2022.

[11] Y. C. Gültekin, K. Wu, and A. Alvarado, "Temporal properties of enumerative shaping: Autocorrelation and energy dispersion index," in *Signal Processing in Photonic Communications*. Optica Publishing Group, 2022, pp. SpM2I-3.

[12] T. Fehenberger, M. P. Yankov, L. Barletta, and N. Hanik, "Compensation of xpm interference by blind tracking of the nonlinear phase in wdm systems with qam input," in *2015 European Conference on Optical Communication (ECOC)*. IEEE, 2015, pp. 1-3.

[13] G. Böcherer, "Probabilistic signal shaping for bit-metric decoding," in *2014 IEEE International Symposium on Information Theory*, 2014, pp. 431-435.

[14] A. Amari, S. Goossens, Y. C. Gültekin, O. Vassilieva, I. Kim, T. Ikeuchi, C. Okonkwo, F. M. Willems, and A. Alvarado, "Enumerative sphere shaping for rate adaptation and reach increase in WDM transmission systems," in *45th European Conference on Optical Communication (ECOC 2019)*. IET, 2019, pp. 1-4.

[15] M. S. Neves, A. Carena, A. Nespola, P. P. Monteiro, and F. P. Guiomar, "Joint carrier-phase estimation for digital subcarrier multiplexing systems with symbol-rate optimization," *Journal of Lightwave Technology*, vol. 39, no. 20, pp. 6403-6412, 2021.