# Recent Progress on Novel DSP Techniques for Mode Division Multiplexing Systems: A Review
## 5 Novel DSP Techniques for MIMO Systems
### Artificial Intelligence (AI) Technology for MIMO Equalization
To mitigate various nonlinear effects, DSP can be utilized. Recently, deep-learning-based MIMO equalization has been adopted to tackle the interplay of FWM interference and parametric amplified noise in coherent optical communication [86]. Meanwhile, inter-carrier crosstalk effects even without frequency stochastic variations can also be compensated by this novel MIMO neural network, which has hitherto been considered impossible. In orthogonal frequency division Multiplexing (OFDM) system, the frequency dependent deterministic distortion leads to the center subcarriers suffering from higher nonlinear distortion than edge subcarriers, which leads to a lower signal-to-interference-plus-noise ratio (SINR). Consequently, as shown in Figure 25, a more complex ANN is employed in center subcarriers than edge subcarriers. Following this strategy, the SINR difference between each subcarrier is suppressed, resulting in a more robust equalization performance. Figure 26 illustrates the structure of this mentioned equalizer. The hidden layer is adaptively adjusted by a set of electronic frequency sub-carriers which represent the  
Figure 24: The BER performance of DLNN, ZF and SDR-RBR. Figure 25: Conceptual diagram of the neural network-based MIMO equalization strategy. third dimension of different frequencies. The complexity of this adaptive neural network depends on the hidden layers counts which is 3 or 4 and the modulation format level which is 4 for QPSK or 16 for 16-QAM. Therefore, compared with traditional methods, this 3D MIMO neural network will not increase the computational complexity. In order to explore the performance of the MIMO deep learning equalization algorithm, as shown in Table 3, four different cases with different number of central subcarrier blocks were used for comparison. Figure 27a shows the result in a 3200-km QPSK WDM-CO-OFDM system, it can be seen that the performance under case 2 is better than the other three cases. And compared with the commercial single-carrier/polarization QPSK system for a 3200 km multi-channel configuration, the MIMO deep learning under case 2 brings a record 3.8-dB Q factor improvement. Such performance enhancement is maintained for a 2000-km single channel 16-QAM CO-OFDM system as is shown in Figure 27b. In case 2, MIMO deep learning equalizer with traditional ANN and deterministic DBP in different transmission systems is compared: (a) WDM QPSK CO-OFDM at 3200 km and (b) Single channel 16-QAM CO-OFDM at 2000 km. It can be found from Figure 28a that compared with the ANN and DBP methods, at the optimal launched optical power (LOP) of \(-5\) dBm, the Q-factor is improved by 1.4 dB and 2.1 dB, respectively. This is due to the fact that the DBP cannot accurately know the absolute position and relative spacing of the channels and subcarrier frequencies. From Figure 28b, it can be found that compared with linear equalization and ANN, the maximum allowable LOP of this MIMO equalizer is increased by 4 dB, and at the optimal LOP of 6 dBm, the Q factor 2.1 dB is improved in comparison to DBP. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Cases** & **Block-Size for Middle Subcarriers** & **Hidden Layers** \\ \hline
1 & 50 & 3 \\
2 & 100 & 3 \\
3 & 150 & 3 \\
4 & \(2\times 54\) (middle groups), \(2\times 51\) (edge groups) & 4 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Different cases of block-size for middle subcarriers and number of hidden layers. Figure 26: The architecture of MIMO deep learning equalizer. In Table 4, different structures of neural network were summarized in terms of their respective applications in DSP techniques of MDM systems, which you could find that each type has its own speciality in mitigation of certain impairments in MDM links. Figure 28: Transmission performance comparison among MIMO deep leaning, ANN and deterministic DBP. (**a**) WDM QPSK CO-OFDM at 3200 km; (**b**) single-channel 16-QAM CO-OFDM at 2000 km. Figure 27: Transmission performance of MIMO equalizer in different cases.