## 2 Analytical Modeling  
The current coherent system for LDOTNs uses bit interleaved coded modulation [32] which performs constellation demapping and forward error correction decoding [33]. The demapper function estimates likelihood ratios after calculating multichannel statistics based on AWGN assumption. The algorithm named CPE largely removes autocorrelation nonlinearities, and phase noise is then assumed to be memoryless. Likelihood ratios are calculated by considering the presence of phase noise, potentially improving decoding outcomes without varying the receiver side. Therefore, this section expresses likelihood derivation, which suppresses nonlinearities. The block description of a channel is presented in Figure 1. The binary forward error correction encoder encodes information bits to channel bits. A block of \(N\) channel bits \(D_{n}=[D_{1,n},D_{2,n},...\,D_{N,n}]\) is mapped into \(g_{n}\) complex symbols drawn from constellation \(C\) with cardinality \(|C|=2^{N}\), where \(N\) shows the time index [33]. These symbols are transmitted over the optical medium block, consisting of the optical transmitter, the channel and the receiver. The received symbol \(h_{n}\) is obtained. Figure 1: Mechanism of digital signal processing through an optical medium. Moreover, conditional probability \(p(h/g)\) is described for CPE, which is able to mitigate the long autocorrelation phase noise. The parameters \(h\) and \(g\) denote the scalar quantities. The received symbol \(h\) is taken by the demapper, which produces a \(N\times 1\) vector \(i_{n}\) of likelihood ratios. The maximum number of positive \(i_{n}\) denotes that the estimation is reliable. On the other hand, a smaller number of positive \(i_{n}\) indicates the unreliability of the estimation. The deterministic function of \(h\) is used for calculating likelihood ratios and depends on the probability \(p(h/g)\)[16] :  
\[i_{n}=log\frac{P(D_{n}=1/h)}{P(D_{k}=0/h)}=log\frac{\Sigma_{g1}eC_{n}^{1}p(h/ g1)p(g1)}{\Sigma_{g0}eC_{k}^{0}p(h/g0)p(g0)} \tag{1}\]  
where \(n=1,2,3\)... \(m\) means the index of bit; the constellation symbol is denoted by \(C_{n}^{m}\), and \(P(g)\) defines the probability of the data being transmitted. In order to use the AWGN channel, the attained pulse \((h_{n})\)--as depicted in Figure 1--can be written as  
\[h_{n}=g_{n}+k_{n} \tag{2}\]  
In this equation, \(k_{n}\) denotes the AWGN sample and includes the variance \((\alpha_{n}^{2})\); this AWGN is the summation of amplified spontaneous emission noise and nonlinearities. The \(P(h/g)\) probability is explained considering the Gaussian probability density function, Ref. [34] given as  
\[P(g/h)=\frac{1}{\alpha_{n}^{2}}e^{\frac{-|h-g|}{\alpha_{n}^{2}}} \tag{3}\]  
By combining Equations (3) and (1), the likelihood ratio is estimated. Furthermore, in the case of the addition of NLPns, Equation (2) is upgraded as  
\[h_{n}=g_{n}e^{i\omega n}+k_{n} \tag{4}\]  
Here, \(\omega n\) denotes the NLPns bounded between \(-\pi\) and \(\pi\). According to [PERSON], the probability distribution mechanism for the bounded parameter is defined [35] as  
\[P(\omega)=\frac{e^{n\omega cos\omega}}{2\pi M_{0}(n\omega)},\omega e(-\pi,\pi) \tag{5}\]  
where \(n\omega\) describes the concentration and Bessel function denoted by \(M_{0}\). Hence, the probability of the channel is now written as  
\[P(h/g)=\frac{n\omega e^{-n}}{8\pi^{3}\alpha_{n}^{2}}e^{\frac{-|h|^{2}+|g|^{2} }{2\alpha_{n}^{2}}+\frac{gh+n\omega}{\alpha_{n}^{2}}} \tag{6}\]  
This Equation declares that AWGN \(\alpha_{n}^{2}\) and NLPns \((k\zeta)\) have an important role over the constellation of modulation. Figure 2 shows the block description of 16-QAM CS, while Figure 3 compares the CS of 16 QAM in terms of variance and different NLPn factors. Figure 3a,b explain the CS at \(\alpha_{n}^{2}=0.35\), \(k\zeta=-60\) and \(\alpha_{n}^{2}=0.35\), \(k\zeta=-70\), respectively. This shows that the NLPns significantly contribute to disturbing the signal behavior.