# Advanced DSP for Coherent Optical Fiber Communication
## 3 Nonlinearity Compensation  
Essentially, optical signals will suffer linear and nonlinear impairments during propagation along optical fibers, which limit the transmission capacity. When WDM is introduced to increase the capacity of optical fibers, the nonlinearity and dispersion can significantly affect the signal quality. Therefore, mitigating or compensating these impairments is an important part of optical fiber communication research. At present, the method of compensating linear impairments (including CD and PMD) is almost mature, so the nonlinear impairment becomes the limiting factor of increasing optical fiber system capacity. Enlarging the number of WDM channels or reducing the channel spacing will increase the nonlinear impairment. In addition, in order to use the higher-order modulation format to improve the SE and increase the transmission distance, it is necessary to improve the OSNR. Increasing the OSNR to a given level requires the increase of signal power, which in turn, results in more serious nonlinear impairments. Therefore, the reduction or compensation of nonlinear impairments can significantly increase the capacity of optical fiber channels. Figure 4: Block diagram of carrier phase estimator. Figure 5: Recovered constellation diagrams for the two polarizations after 6400 km transmission with an estimated BER = \(2.4\times 10^{-3}\). Figure 6: (a) B2B BER performance at 56 Gbaud: single polarization 112 Gbit/s quadrature-phase-shift-keying (QPSK) and 224 Gbit/s polarization-division multiplexed (PDM)-QPSK; (b) Q-factor and optical signal-to-noise ratio (OSNR) vs. transmission distance. Recent developments in coherent optical communication have enabled the complex electric field of signals to be obtained in the digital domain. Thus, it is possible to compensate for nonlinear impairments in the signal waveforms propagating along the optical fiber. Furthermore, as long as adjacent channels are received, compensation for nonlinear impairments among channels, such as cross-phase modulation (XPM) and four-wave mixing (FWM), is also possible. Among the proposed nonlinearity compensation technologies, the digital backward propagation (DBP) has proved to be the most promising approach. The recent proposed investigation has demonstrated that the nonlinear Schrodinger equation (NLSE) can be solved in the digital domain to compensate all deterministic fiber transmission impairments in a PDM WDM environment. The DBP method is based on solving the nonlinear propagation equation along the backward direction. However, the computational load of DBP is heavy. It is still very important to investigate computational efficiently nonlinearity compensation algorithms for dispersion-unmanaged coherent transmission systems. [PERSON] et al. have demonstrated approaches to reduce computational load including implementing DBP using coupled NLSEs [26; 27], XPM walk-off factorization [28] and folded DBP for dispersion-managed systems, which have the potential to reduce computational load to the same order of magnitude for dispersion compensation for a dispersion unmanaged system [29]. The schematic of DBP method is shown in Figure 7. \(A\) is the complex electric field of the received signal, \(\hat{D}\) and \(\hat{N}\) are the linear and the nonlinear operators, respectively. The signal is distorted in the real transmission fiber and is compensated through the virtual fiber. The received signal is processed by a digital model with opposite propagation parameters after it is detected at the receiver after transmission. Backward propagated NLSE can be expressed as [1]:  
\[\frac{\partial A}{\partial(-z)}=(\hat{D}+\hat{N})A \tag{18}\]  
\[\hat{D}=-j\frac{\beta_{2}}{2}\frac{\partial^{2}}{\partial t^{2}}+\frac{\beta _{3}}{6}\frac{\partial^{3}}{\partial t^{3}}-\frac{\alpha}{2} \tag{19}\]  
\[\hat{N}=j\gamma|A|^{2} \tag{20}\]  
where \(\alpha\), \(\beta_{2}\), \(\beta_{3}\) and \(\gamma\) are the attenuation, first- and second-order group velocity dispersion and fiber nonlinear coefficients, respectively.