## 3 Results and Discussion
The use of a cascaded MLSE scheme further improves the performance of the system by 1 dB for MLSE that entails 16, 64, 256, and 1024 states. As we can see, the BER performance is slightly improved by increasing the complexity up to 4096 states. Comparing these results to the results obtained for the second scheme (Figure 1b, in grey) [15], it is shown that BER performance at the 7% HD-FEC threshold can be optimized by 3.5 dB by using an FIR filter to compensate for the linear distortion of the channel, and by 4.5 dB when MLSE is used at the output of the FIR to compensate for the non-linear distortions of the system. The results for the 56 Gb/s PAM-4 transmission over 20 km for the scheme shown in Figure 1a are presented in Figure 6d, in black. The 7% HD-FEC threshold in that case can be achieved at \(-\)5.5 dBm received optical using an 81-taps FIR filter. The hybrid implementation of FIR/MLSE scheme further improves the performance of the system by 1.5 dB at the 7% HD-FEC thre  
Figure 4: Eye-diagrams before and after FIR for back-to-back, 10-km, 15-km, 20-km, 25-km, and 30-km for the experimental setup of Figure 1a. Figure 5: The system’s transfer function measured for (**a**) the back-to-back case, and after transmission over (**a**) 10-km; (**b**) 15-km; (**c**) 20-km; (**d**) 25-km; and (**e**) 30-km; (**f**) 3-dB bandwidth of the transfer functions for all scenarios under investigation. to obtain these results, the TF of the system measured in Figure 5f is used to calculate the inverse TF and pre-distort the data at the transmitter side. This leads to a significant performance improvement compared to the results of the second scheme, shown in the same Figure 6d, in grey. Specifically, the sensitivity requirements are being relaxed, and at the same time, a suppression of the BER error floors is achieved. In particular, the transmission becomes feasible by only employing an FIR filter in contrast to the results obtained with the scheme of Figure 1b (in grey), while the use of the MLSE at the latter's output leads to a 5.5, 3, 3.5, and 1.5 dB improvement if we compare the MLSE-16, -64, -256, and 1024 cases for both schemes, respectively. Figure 6: BER vs. received optical power for (**a**) 10-km; (**b**) 15-km; (**c**) 20-km; (**d**) 20-km; (**e**) 25-km; and (**f**) 30-km for the scheme of Figure 1a,b in black and in grey, respectively. The results for the 56 Gb/s PAM-4 transmission over 25 km for the scheme shown in Figure 1a are presented in Figure 6e in black. In contrast with the results based on the scheme of Figure 1b (in grey), transmission below the 7% HD-FEC threshold is now enabled by the implementation of an FIR followed by the MLSE at the receiver's DSP. As we can see, in that case, the FIR filter alone cannot compensate for the ISI induced by the fiber link, and the employment of MLSE is essential to enable transmission below the HD-FEC threshold. MLSE with 16 states can achieve a BER = 4.5 \(\times\) 10\({}^{-3}\) at \(-\)3 dBm, while further increasing the memory of the MLSE so the latter consists of 64, 256, 1024 and 4096 states further improves the performance and brings the BER error floor down to BER \(\sim\)10\({}^{-4}\). For the 30-km transmission case, the results based on the scheme of Figure 1a are presented in Figure 6f. In that case, transmission is not feasible below the 7% HD-FEC threshold if only an FIR filter or an FIR/MLSE scheme that incorporates 16, 64, or 256 states is employed. However, BER values below the 7% HD-FEC threshold are achieved by increasing the memory of the MLSE to 5 and 6 bits; i.e., to MLSE schemes that consist of 1024 and 4096 states. Lower complexity MLSE schemes could be used if a higher FEC overhead was to be employed to enable the transmission (i.e., 20% HD-FEC).