# Equalization Methods for Out-of-Band Nonlinearity Mitigation in Fiber-Optic Communications
## 4 Equalization Algorithms
### Numerical Estimation of Equalization Performance  
We turn now to a quantitative comparison of the algorithms using a set of simulations. The simulations were performed using both the split-step method and the Virtual Lab tool [8], under the same test conditions. The test system consisted of \(10\times 100\) km spans of single mode fiber (\(D=17\) ps/nm/km, \(\gamma=1.3\) W\({}^{-1}\)km\({}^{-1}\), \(\alpha=0.2\) dB/km), separated by lumped amplifiers characterized by a noise figure of 4 dB. The transmitted signal consisted of 15 WDM channels, carrying a polarization multiplexed 32-QAM modulated constellation, operating at a baud-rate of 32 Gbaud with root-raised-cosine pulses having a roll-off coefficient of 0.02, and with channel spacing of 50 GHz. Each trace contained 21 dual-polarization symbols.  
All of the above mentioned algorithms where applied to the received signal. The algorithms' parameters (convergence parameter \(\mu\) for LMS, forget factor \(\lambda\) for RLS, window weights \(W_{l}\) and \(a_{l}\) for the window averaging algorithm) were optimized so as to obtain the best performance. The Kalman filter's parameters were found using a measurement of the statistics of NLIN, as described in [37]. Figure 4 shows the effect of each of the algorithms on the output SNR, defined as  
\[\text{SNR}_{\text{equ}}=\frac{\text{Var}\left(a_{n}\right)}{\text{Var}\left( \hat{s}_{n}-a_{n}\right)}, \tag{17}\]  
where \(\hat{s}_{n}\) is the equalized signal for each method. Figure 3(a) shows an SNR vs. power plot for each of the cases, where the black curve represents unequalized transmission, and the colored curves show different types of equalizers.  
Figure 3(b) shows the SNR improvement of each equalizer (i.e., the difference between the peaks of colored curves to that of the black curve in Figure 3(a)), vs. their computational cost. All of the algorithms described in this paper rely only on matrix multiplications and vector scalar product. The sizes of these vectors and matrices is typically fairly small, which means that an asymptotic analysis of the algorithm's complexity is not useful. For example, the RLS algorithm requires matrix multiplication, whereas the LMS algorithm requires only vector multiplications, which would suggests that the RLS algorithm is more expensive. However, because of the small sizes of these vectors and matrices, we found that the cost of these two algorithms was similar. The complexity analysis is particularly difficult for the case of the Kalman filter algorithm, which uses multiplication of sparse matrices [19].  
The circumvent this issue, we quantify the computational cost in terms of the computation time required to process a full trace of \(2^{12}\) symbols. The computation times are normalized by that of the LMS equalizer, which is used as a reference. Naturally, this metric depends on the specific implementation and type of computer used for processing, and the absolute values is likely to vary significantly between different computers. Nevertheless, it gives a useful insight as to the relative cost of different equalization algorithms. All of the methods in this paper were implemented using Matlab programing language, and executed on a desktop computer. The LMS and RLS algorithms provided similar performance, both in terms of cost and SNR gain, which was approximately 0.2 dB. The window-averaging algorithm performed significantly better, with an 0.5 dB and 0.8 dB SNR gain for 1-tap and 3-tap equalizers, respectively. In terms of computation time, the 1-tap window equalizer was twice slower than LMS, and the 3-tap equalizer was 6 times slower. The highest gain was provided by a 3-tap Kalman filter, which yielded a 1.6 dB SNR improvement, at the cost of being 50 times slower than the LMS equalizer. With such a high computational cost, real-time implementation of these algorithm may be challenging. Nevertheless, dedicated hardware (e.g., using an field programmable gate array (FPGA)), even the Kalman filter equalizer may implemented in real time.